a simple linear layer followed by a non-linearity More...

Inheritance diagram for working_gpt.FeedFoward:

Public Member Functions
	__init__ (self, n_embd)

	forward (self, x)

Public Attributes
	net

Detailed Description

a simple linear layer followed by a non-linearity

Definition at line 126 of file working_gpt.py.

Constructor & Destructor Documentation

◆ init()

working_gpt.FeedFoward.__init__	(		self,
			n_embd )

Definition at line 129 of file working_gpt.py.

    def __init__(self, n_embd):
        super().__init__()
        self.net = nn.Sequential(
            nn.Linear(n_embd, 4 * n_embd),
            nn.ReLU(),
            nn.Linear(4 * n_embd, n_embd),
            nn.Dropout(dropout),
        )
 

References __init__().

Referenced by __init__().

Member Function Documentation

◆ forward()

working_gpt.FeedFoward.forward	(		self,
			x )

Definition at line 138 of file working_gpt.py.

    def forward(self, x):
        return self.net(x)

References net.

Member Data Documentation

◆ net

working_gpt.FeedFoward.net

Initial value:

=  nn.Sequential(
            nn.Linear(n_embd, 4 * n_embd),
            nn.ReLU(),
            nn.Linear(4 * n_embd, n_embd),
            nn.Dropout(dropout),
        )

Definition at line 131 of file working_gpt.py.

Referenced by forward().

The documentation for this class was generated from the following file:

gpt/working_gpt.py

Public Member Functions

Public Attributes