Spectral Normalization#
- class fortuna.model.utils.spectral_norm.SpectralNormalization(layer, iteration=1, norm_multiplier=0.95, u_init=<function normal.<locals>.init>, v_init=<function normal.<locals>.init>, kernel_apply_kwargs=None, kernel_name='kernel', layer_name=None, update_singular_value_estimate=None, parent=<flax.linen.module._Sentinel object>, name=None)[source]#
Implements spectral normalization for linear layers.
See Spectral Normalization for Generative Adversarial Networks .
- layer#
A Flax layer to apply normalization to.
- Type:
nn.Module
- iteration#
The number of power iterations to estimate weight matrix’s singular value.
- Type:
int
- norm_multiplier#
Multiplicative constant to threshold the normalization. Usually under normalization, the singular value will converge to this value.
- Type:
float
- u_init#
Initializer function for the first left singular vectors of the kernel.
- Type:
Callable[[PRNGKeyArray, Shape, Type], Array]
- v_init#
Initializer function for the first right singular vectors of the kernel.
- Type:
Callable[[PRNGKeyArray, Shape, Type], Array]
- kernel_apply_kwargs#
Updated keyword arguments to clone the input layer. The cloned layer represents the linear operator performed by the weight matrix. If not specified, that operator follows SN-GAN implementation (Takeru M. et al). In particular, for Dense layers the default behavior is equivalent to using a cloned layer with no bias (by specifying kernel_apply_kwargs=dict(use_bias=False)). With this customization, we can have the same implementation (inspried by (Stephan H., 2020) for different interpretations of Conv layers. Also see SpectralNormalizationConv2D for an example of using this attribute.
- Type:
Optional[Mapping[str, Any]]
- kernel_name#
Name of the kernel parameter of the input layer.
- Type:
str
- layer_name#
Name of the input layer
- Type:
Optional[str]
- update_singular_value_estimate#
Whether to perform power iterations to update the singular value estimate.
- Type:
Optional[bool]
- class fortuna.model.utils.spectral_norm.SpectralNormalizationConv2D(layer, iteration=1, norm_multiplier=0.95, u_init=<function normal.<locals>.init>, v_init=<function normal.<locals>.init>, kernel_apply_kwargs=FrozenDict({ feature_group_count: 1, padding: 'SAME', use_bias: False, }), kernel_name='kernel', layer_name=None, update_singular_value_estimate=None, parent=<flax.linen.module._Sentinel object>, name=None)[source]#
Implements spectral normalization for Convolutional layers.See Generalizable Adversarial Training via Spectral Normalization. Implements spectral normalization for linear layers.
See Spectral Normalization for Generative Adversarial Networks .
- layer#
A Flax layer to apply normalization to.
- Type:
nn.Module
- iteration#
The number of power iterations to estimate weight matrix’s singular value.
- Type:
int
- norm_multiplier#
Multiplicative constant to threshold the normalization. Usually under normalization, the singular value will converge to this value.
- Type:
float
- u_init#
Initializer function for the first left singular vectors of the kernel.
- Type:
Callable[[PRNGKeyArray, Shape, Type], Array]
- v_init#
Initializer function for the first right singular vectors of the kernel.
- Type:
Callable[[PRNGKeyArray, Shape, Type], Array]
- kernel_apply_kwargs#
Updated keyword arguments to clone the input layer. The cloned layer represents the linear operator performed by the weight matrix. If not specified, that operator follows SN-GAN implementation (Takeru M. et al). In particular, for Dense layers the default behavior is equivalent to using a cloned layer with no bias (by specifying kernel_apply_kwargs=dict(use_bias=False)). With this customization, we can have the same implementation (inspried by (Stephan H., 2020) for different interpretations of Conv layers. Also see SpectralNormalizationConv2D for an example of using this attribute.
- Type:
Optional[Mapping[str, Any]]
- kernel_name#
Name of the kernel parameter of the input layer.
- Type:
str
- layer_name#
Name of the input layer
- Type:
Optional[str]
- update_singular_value_estimate#
Whether to perform power iterations to update the singular value estimate.
- Type:
Optional[bool]