Spectral Normalization¶

class fortuna.model.utils.spectral_norm.SpectralNormalization(layer, iteration=1, norm_multiplier=0.95, u_init=<function normal.<locals>.init>, v_init=<function normal.<locals>.init>, kernel_apply_kwargs=None, kernel_name='kernel', layer_name=None, update_singular_value_estimate=None, parent=<flax.linen.module._Sentinel object>, name=None)[source]¶

Implements spectral normalization for linear layers.

See Spectral Normalization for Generative Adversarial Networks .

layer¶

A Flax layer to apply normalization to.

Type:: nn.Module

iteration¶

The number of power iterations to estimate weight matrix’s singular value.

Type:: int

norm_multiplier¶

Multiplicative constant to threshold the normalization. Usually under normalization, the singular value will converge to this value.

Type:: float

u_init¶

Initializer function for the first left singular vectors of the kernel.

Type:: Callable[[jax.Array, Shape, Type], Array]

v_init¶

Initializer function for the first right singular vectors of the kernel.

Type:: Callable[[jax.Array, Shape, Type], Array]

kernel_apply_kwargs¶

Updated keyword arguments to clone the input layer. The cloned layer represents the linear operator performed by the weight matrix. If not specified, that operator follows SN-GAN implementation (Takeru M. et al). In particular, for Dense layers the default behavior is equivalent to using a cloned layer with no bias (by specifying kernel_apply_kwargs=dict(use_bias=False)). With this customization, we can have the same implementation (inspried by (Stephan H., 2020) for different interpretations of Conv layers. Also see SpectralNormalizationConv2D for an example of using this attribute.

Type:: Optional[Mapping[str, Any]]

kernel_name¶

Name of the kernel parameter of the input layer.

Type:: str

layer_name¶

Name of the input layer

Type:: Optional[str]

update_singular_value_estimate¶

Whether to perform power iterations to update the singular value estimate.

Type:: Optional[bool]

class fortuna.model.utils.spectral_norm.SpectralNormalizationConv2D(layer, iteration=1, norm_multiplier=0.95, u_init=<function normal.<locals>.init>, v_init=<function normal.<locals>.init>, kernel_apply_kwargs=FrozenDict({ feature_group_count: 1, padding: 'SAME', use_bias: False, }), kernel_name='kernel', layer_name=None, update_singular_value_estimate=None, parent=<flax.linen.module._Sentinel object>, name=None)[source]¶

Implements spectral normalization for Convolutional layers.See Generalizable Adversarial Training via Spectral Normalization. Implements spectral normalization for linear layers.