torchadf.nn.functional.relu

torchadf.nn.functional.relu(in_mean, in_var, mode='diag')

Applies a rectified linear unit activation.

Assumed Density Filtering (ADF) version of torch.nn.functional.relu.

Since the standard ReLU function operates component-wise there is some ambiguity in how to interpret the input dimensions if the number of leading “batch” dimensions is unknown. The following convention is used:

• if in_mean.ndim==3 the mean input is interpreted as (batch_dim, num_channels, in_dim), e.g., for 1D convolutions

• if in_mean.ndim==4 the mean input is interpreted as (batch_dim, num_channels, height, width), e.g., for 2D convolutions

• if in_mean.ndim==5 the mean input is interpreted as (batch_dim, num_channels, depth, height, width), e.g., for 3D convolutions

• otherwise the mean input is interpreted as (*,in_dim), e.g., for dense linear layers

The corresponding (co-)variance shapes are interpreted accordingly, depending on the covariance propagation mode.

Parameters:

in_meantorch.Tensor

Input mean tensor.

in_vartorch.Tensor

Input (co-)variance tensor.

mode{“diag”, “diagonal”, “lowrank”, “half”, “full”}, optional

Covariance propagation mode (Default “diag”).

Returns:

out_meantorch.Tensor

The transformed mean tensor.

out_vartorch.Tensor

The transformed (co-)variance tensor.