Module dopt.nnet.parameters

This module contains methods for initialising the parameters of neural networks.

Several of the methods implemented in this module rely on fan_in and fan_out values. These are calculated differently depending on the rank of the parameter.

For rank-2 tensors,

fan_in = shape[0], fan_out = shape[1]

For rank-4 tensors,

fan_in = shape[1] * shape[2] * shape[3], fan_out = shape[0] * shape[2] * shape[3]

Functions

Name	Description
`constantInit(val)`	Creates a parameter initialiser that sets the initial value of each element in a parameter tensor to a constant value.
`gaussianInit(mean, stddev)`	Creates a parameter initialiser that sets the initial value of each element in a parameter tensor to a different sample from a Gaussian distribution.
`glorotGaussianInit()`	Creates a parameter initialiser that uses the method of Glorot and Bengio (2010).
`glorotUniformInit()`	Creates a parameter initialiser that uses the method of Glorot and Bengio (2010).
`heGaussianInit()`	Creates a parameter initialiser that uses the method of He et al. (2015).
`heUniformInit()`	Creates a parameter initialiser that uses the method of He et al. (2015).
`lipschitz1(maxK)`	Creates a `Projection` function that can be applied to a parameter matrix/tensor to constraint the Lipschitz constant w.r.t. the L_1 vector norm.
`lipschitzInf(maxK)`	Creates a `Projection` function that can be applied to a parameter matrix/tensor to constraint the Lipschitz constant w.r.t. the L_infty vector norm.
`uniformInit(minval, maxval)`	Creates a parameter initialiser that sets the initial value of each element in a parameter tensor to a different sample from a uniform distribution.

Structs

Name	Description
`Parameter`	Encapsulates information about network parameters.

Aliases

Name	Type	Description
`ParamInitializer`	`void delegate(Operation)`	Used to initialize a parameter in the neural network.