SELU#

classtorch.nn.SELU(inplace=False)[source]#

Applies the SELU function element-wise.

$$\text{SELU}(x)=\text{scale}\ast (\max (0,x)+\min (0,\alpha \ast (\exp (x)-1)))$$SELU(x)=scale∗(max(0,x)+min(0,α∗(exp(x)−1)))

with$\alpha =1.6732632423543772848170429916717$α=1.6732632423543772848170429916717 and$\text{scale}=1.0507009873554804934193349852946$scale=1.0507009873554804934193349852946.

Warning

When usingkaiming_normal orkaiming_normal_ for initialisation,nonlinearity='linear' should be used instead ofnonlinearity='selu'in order to getSelf-Normalizing Neural Networks.Seetorch.nn.init.calculate_gain() for more information.

More details can be found in the paperSelf-Normalizing Neural Networks .

Parameters

inplace (bool,optional) – can optionally do the operation in-place. Default:False

Shape:

• Input:$(\ast )$(∗), where$\ast$∗ means any number of dimensions.

• Output:$(\ast )$(∗), same shape as the input.

Examples:

>>>m=nn.SELU()>>>input=torch.randn(2)>>>output=m(input)

extra_repr()[source]#

Return the extra representation of the module.

Return type

str

forward(input)[source]#

Runs the forward pass.

Return type

Tensor