torch.nn.functional.silu#

torch.nn.functional.silu(input,inplace=False)[source]#

Apply the Sigmoid Linear Unit (SiLU) function, element-wise.

The SiLU function is also known as the swish function.

$$\text{silu}(x)=x\ast \sigma (x),\text{where }\sigma (x)\text{ is the logistic sigmoid.}$$silu(x)=x∗σ(x),where σ(x) is the logistic sigmoid.

Note

SeeGaussian Error Linear Units (GELUs)where the SiLU (Sigmoid Linear Unit) was originally coined, and seeSigmoid-Weighted Linear Units for Neural Network Function Approximationin Reinforcement Learning andSwish:a Self-Gated Activation Functionwhere the SiLU was experimented with later.

SeeSiLU for more details.

Return type

Tensor