PReLU#

classtorch.nn.modules.activation.PReLU(num_parameters=1,init=0.25,device=None,dtype=None)[source]#

Applies the element-wise PReLU function.

$$\text{PReLU}(x)=\max (0,x)+a\ast \min (0,x)$$PReLU(x)=max(0,x)+a∗min(0,x)

or

PReLU(x)={x,ax,​ if x≥0 otherwise ​

Here$a$a is a learnable parameter. When called without arguments,nn.PReLU() uses a singleparameter$a$a across all input channels. If called withnn.PReLU(nChannels),a separate$a$a is used for each input channel.

Note

weight decay should not be used when learning$a$a for good performance.

Note

Channel dim is the 2nd dim of input. When input has dims < 2, then there isno channel dim and the number of channels = 1.

Parameters

• num_parameters (int) – number of$a$a to learn.Although it takes an int as input, there is only two values are legitimate:1, or the number of channels at input. Default: 1

• init (float) – the initial value of$a$a. Default: 0.25

Shape:

• Input:$(\ast )$(∗) where* means, any number of additionaldimensions.

• Output:$(\ast )$(∗), same shape as the input.

Variables

weight (Tensor) – the learnable weights of shape (num_parameters).

Examples:

>>>m=nn.PReLU()>>>input=torch.randn(2)>>>output=m(input)

extra_repr()[source]#

Return the extra representation of the module.

Return type

str

forward(input)[source]#

Runs the forward pass.

Return type

Tensor

reset_parameters()[source]#

Resets parameters based on their initialization used in__init__.