tensor_prac.Self_Attention Namespace Reference

Variables
	B
	T
	C
	x = torch.randn(B, T, C)
	xbow = torch.zeros((B, T, C))
	xprev = x[b, : t + 1]
	wei = torch.tril(torch.ones(T, T))
	xbow2 = wei @ x
	tril = torch.tril(torch.ones(T, T))
	xbow3 = wei @ x
int	head_size = 16
	key = nn.Linear(C, head_size, bias=False)
	query = nn.Linear(C, head_size, bias=False)
	k = key(x)
	q = query(x)
	out = wei @ x
	a = torch.tril(torch.ones(3, 3))
	b = torch.randint(0, 10, (3, 2)).float()
	c = a @ b

Variable Documentation

a

tensor_prac.Self_Attention.a = torch.tril(torch.ones(3, 3))

Definition at line 75 of file Self_Attention.py.

B

tensor_prac.Self_Attention.B

Definition at line 9 of file Self_Attention.py.

b

tensor_prac.Self_Attention.b = torch.randint(0, 10, (3, 2)).float()

Definition at line 77 of file Self_Attention.py.

C

tensor_prac.Self_Attention.C

Definition at line 9 of file Self_Attention.py.

c

tensor_prac.Self_Attention.c = a @ b

Definition at line 79 of file Self_Attention.py.

head_size

int tensor_prac.Self_Attention.head_size = 16

Definition at line 57 of file Self_Attention.py.

k

tensor_prac.Self_Attention.k = key(x)

Definition at line 60 of file Self_Attention.py.

key

tensor_prac.Self_Attention.key = nn.Linear(C, head_size, bias=False)

Definition at line 58 of file Self_Attention.py.

out

tensor_prac.Self_Attention.out = wei @ x

Definition at line 67 of file Self_Attention.py.

q

tensor_prac.Self_Attention.q = query(x)

Definition at line 61 of file Self_Attention.py.

query

tensor_prac.Self_Attention.query = nn.Linear(C, head_size, bias=False)

Definition at line 59 of file Self_Attention.py.

T

tensor_prac.Self_Attention.T

Definition at line 9 of file Self_Attention.py.

tril

tensor_prac.Self_Attention.tril = torch.tril(torch.ones(T, T))

Definition at line 36 of file Self_Attention.py.

wei

tensor_prac.Self_Attention.wei = torch.tril(torch.ones(T, T))

Definition at line 28 of file Self_Attention.py.

x

tensor_prac.Self_Attention.x = torch.randn(B, T, C)

Definition at line 12 of file Self_Attention.py.

xbow

tensor_prac.Self_Attention.xbow = torch.zeros((B, T, C))

Definition at line 15 of file Self_Attention.py.

xbow2

tensor_prac.Self_Attention.xbow2 = wei @ x

Definition at line 30 of file Self_Attention.py.

xbow3

tensor_prac.Self_Attention.xbow3 = wei @ x

Definition at line 40 of file Self_Attention.py.

xprev

tensor_prac.Self_Attention.xprev = x[b, : t + 1]

Definition at line 18 of file Self_Attention.py.