GLU#

classtorch.nn.modules.activation.GLU(dim=-1)[source]#

Applies the gated linear unit function.

$GLU(a,b)=a\otimes \sigma (b)$GLU(a,b)=a⊗σ(b) where$a$a is the first halfof the input matrices and$b$b is the second half.

Parameters

dim (int) – the dimension on which to split the input. Default: -1

Shape:

• Input:$({\ast }_1,N,{\ast }_2)$(∗1​,N,∗2​) where* means, any number of additionaldimensions

• Output:$({\ast }_1,M,{\ast }_2)$(∗1​,M,∗2​) where$M=N\mathrm{/}2$M=N/2

Examples:

>>>m=nn.GLU()>>>input=torch.randn(4,2)>>>output=m(input)

extra_repr()[source]#

Return the extra representation of the module.

Return type

str

forward(input)[source]#

Runs the forward pass.

Return type

Tensor