working_gpt.Block Class Reference

Transformer block: communication followed by computation. More...

Inheritance diagram for working_gpt.Block:

Public Member Functions
	__init__ (self, n_embd, n_head)
	forward (self, x)

Public Attributes
	sa = MultiHeadAttention(n_head, head_size)
	ffwd = FeedFoward(n_embd)
	ln1 = nn.LayerNorm(n_embd)
	ln2 = nn.LayerNorm(n_embd)

Detailed Description

Transformer block: communication followed by computation.

Definition at line 142 of file working_gpt.py.

Constructor & Destructor Documentation

__init__()

working_gpt.Block.__init__	(		self,
			n_embd,
			n_head )

Definition at line 145 of file working_gpt.py.

    def __init__(self, n_embd, n_head):
        # n_embd: embedding dimension, n_head: the number of heads we'd like
        super().__init__()
        head_size = n_embd // n_head
        self.sa = MultiHeadAttention(n_head, head_size)
        self.ffwd = FeedFoward(n_embd)
        self.ln1 = nn.LayerNorm(n_embd)
        self.ln2 = nn.LayerNorm(n_embd)
 

References __init__().

Referenced by __init__().

Member Function Documentation

forward()

working_gpt.Block.forward	(		self,
			x )

Definition at line 154 of file working_gpt.py.

    def forward(self, x):
        x = x + self.sa(self.ln1(x))
        x = x + self.ffwd(self.ln2(x))
        return x
 
 

References ffwd, ln1, ln2, and sa.

Member Data Documentation

ffwd

working_gpt.Block.ffwd = FeedFoward(n_embd)

Definition at line 150 of file working_gpt.py.

Referenced by forward().

ln1

working_gpt.Block.ln1 = nn.LayerNorm(n_embd)

Definition at line 151 of file working_gpt.py.

Referenced by forward().

ln2

working_gpt.Block.ln2 = nn.LayerNorm(n_embd)

Definition at line 152 of file working_gpt.py.

Referenced by forward().

sa

working_gpt.Block.sa = MultiHeadAttention(n_head, head_size)

Definition at line 149 of file working_gpt.py.

Referenced by forward().

---

The documentation for this class was generated from the following file:

• gpt/working_gpt.py