Probabilistic model#

We support a probabilistic classifier for classification and a probabilistic regressor for regression. Please find their references below.

class fortuna.prob_model.classification.ProbClassifier(model, prior=IsotropicGaussianPrior(), posterior_approximator=SWAGPosteriorApproximator(), output_calibrator=ClassificationTemperatureScaler(), model_editor=None, seed=0)[source]#

A probabilistic classifier class.

Parameters:

  • model (nn.Module) – A model describing the deterministic relation between inputs and outputs. The outputs must correspond to the logits of a softmax probability vector. The output dimension must be the same as the number of classes. Let \(x\) be input variables and \(w\) the random model parameters. Then the model is described by a function \(f(w, x)\), where each component of \(f\) corresponds to one of the classes.

  • prior (Prior) – A prior distribution object. The default is an isotropic standard Gaussian. Let \(w\) be the random model parameters. Then the prior is defined by a distribution \(p(w)\).

  • posterior_approximator (PosteriorApproximator) – A posterior approximation method. The default method is SWAG.

  • output_calibrator (Optional[nn.Module]) – An output calibrator object. The default is temperature scaling for classification, which rescales the logits with a scalar temperature parameter. Given outputs \(o\) of the model manager, the output calibrator is described by a function \(g(\phi, o)\), where phi are deterministic calibration parameters.

  • model_editor (ModelEditor) – A model_editor objects. It takes the forward pass and transforms the outputs.

  • seed (int) – A random seed.

model#

See model in Parameters.

Type:

nn.Module

model_manager#

This object orchestrates the model’s forward pass.

Type:

ClassificationModelManager

output_calibrator#

See output_calibrator in Parameters.

Type:

nn.Module

prob_output_layer#

This object characterizes the distribution of target variable given the calibrated outputs. It is defined by \(p(y|o)=\text{Categorical}(y|p=softmax(o))\), where \(o\) denotes the calibrated outputs and \(y\) denotes a target variable.

Type:

ClassificationProbOutputLayer

likelihood#

The likelihood function. This is defined by \(p(y|w, \phi, x) = \text{Categorical}(y|p=\text{softmax}(g(\phi, f(w, x)))\).

Type:

ClassificationLikelihood

prior#

See prior in Parameters.

Type:

Prior

joint#

This object describes the joint distribution of the target variables and the random parameters given the input variables and the calibration parameters, that is \(p(y, w|x, \phi)\).

Type:

Joint

posterior_approximator#

See posterior_approximator in Parameters.

Type:

PosteriorApproximator

posterior#

This is the posterior approximation of the random parameters given the training data and the calibration parameters, that is \(p(w|\mathcal{D}, \phi)\), where \(\mathcal{D}\) denotes the training data set and \(\phi\) the calibration parameters.

Type:

Posterior

predictive#

This denotes the predictive distribution, that is \(p(y|\phi, x, \mathcal{D})\). Its statistics are approximated via a Monte Carlo approach by sampling from the posterior approximation.

Type:

ClassificationPredictive

calibrate(calib_data_loader, val_data_loader=None, calib_config=CalibConfig())[source]#

Calibrate the probabilistic classifier.

Parameters:

  • calib_data_loader (DataLoader) – A calibration data loader.

  • val_data_loader (DataLoader) – A validation data loader.

  • calib_config (CalibConfig) – An object to configure the calibration.

Returns:

A calibration status object. It provides information about the calibration.

Return type:

Status

load_state(checkpoint_path)#

Load the state of the posterior distribution from a checkpoint path. The checkpoint must be compatible with the probabilistic model.

Parameters:

checkpoint_path (Path) – Path to a checkpoint file or directory to restore.

Return type:

None

save_state(checkpoint_path, keep_top_n_checkpoints=1)#

Save the posterior distribution state as a checkpoint.

Parameters:

  • checkpoint_path (Path) – Path to file or directory where to save the current state.

  • keep_top_n_checkpoints (int) – Number of past checkpoint files to keep.

Return type:

None

train(train_data_loader, val_data_loader=None, calib_data_loader=None, fit_config=FitConfig(), calib_config=CalibConfig(), **fit_kwargs)[source]#

Train the probabilistic model. This involves fitting the posterior distribution and calibrating the probabilistic model. Calibration is performed only if (1) calib_data_loader is passed and (2) the probabilistic model contains any calibrator.

Parameters:

  • train_data_loader (DataLoader) – A training data loader.

  • val_data_loader (DataLoader) – A validation data loader. This is used to validate both posterior fitting and calibration.

  • calib_data_loader (DataLoader) – A calibration data loader. If this is not passed, no calibration is performed.

  • fit_config (FitConfig) – An object to configure the posterior distribution fitting.

  • calib_config (CalibConfig) – An object to configure the calibration.

  • map_fit_config (Optional[FitConfig] = None) – An object to configure a preliminary posterior distribution fitting via the Maximum-A-Posteriori (MAP) method. The fit methods of several supported posterior approximations, like the ones of SWAGPosterior and LaplacePosterior, start from a preliminary run of MAP, which can be configured via this object. If the method does not start from MAP, this argument is ignored.

Returns:

Status objects for both posterior fitting and calibration.

Return type:

Dict[str, Status]

class fortuna.prob_model.regression.ProbRegressor(model, likelihood_log_variance_model, prior=IsotropicGaussianPrior(), posterior_approximator=SWAGPosteriorApproximator(), output_calibrator=RegressionTemperatureScaler(), model_editor=None, seed=0)[source]#

A probabilistic regressor class.

Parameters:

  • model (nn.Module) – A model describing the deterministic relation between inputs and outputs. It characterizes the mean model of the likelihood function. The outputs must belong to the same space as the target variables. Let \(x\) be input variables and \(w\) the random model parameters. Then the model is described by a function \(\mu(w, x)\).

  • likelihood_log_variance_model (nn.Module) – A model characterizing the log-variance of a Gaussian likelihood function. The outputs must belong to the same space as the target variables. Let \(x\) be input variables and \(w\) the random model parameters. Then the model is described by a function \(\log\sigma^2(w, x)\).

  • prior (Prior) – A prior distribution object. The default is an isotropic standard Gaussian. Let \(w\) be the random model parameters. Then the prior is defined by a distribution \(p(w)\).

  • posterior_approximator (PosteriorApproximator) – A posterior approximation method. The default method is SWAG.

  • output_calibrator (Optional[nn.Module]) – An output calibrator object. The default is temperature scaling for regression, which inflates the variance of the likelihood with a scalar temperature parameter. Given outputs \(o\) of the model manager, the output calibrator is described by a function \(g(\phi, o)\), where phi are deterministic calibration parameters.

  • model_editor (ModelEditor) – A model_editor objects. It takes the forward pass and transforms the outputs.

  • seed (int) – A random seed.

model#

See model in Parameters.

Type:

nn.Module

model_manager#

This object orchestrates the model’s forward pass. Given a mean model \(\mu(w, x)\) and a log-variance model \(\log\sigma^2\), the model manager concatenates the two into \(f(w, x)=[\mu(w, x), \log\sigma^2(w, x)]\).

Type:

RegressionModelManager

output_calibrator#

See output_calibrator in Parameters.

Type:

nn.Module

prob_output_layer#

This object characterizes the distribution of the target variable given the calibrated outputs. It is defined by \(p(y|\omega)=\text{Categorical}(p=softmax(\omega))\), where \(\omega\) denote the calibrated outputs and \(y\) denotes a target variable.

Type:

RegressionProbOutputLayer

likelihood#

The likelihood function. This is defined by \(p(y|w, \phi, x) = \text{Categorical}(p=\text{softmax}(g(\phi, f(w, x)))\).

Type:

RegressionLikelihood

prior#

See prior in Parameters.

Type:

Prior

joint#

This object describes the joint distribution of the target variables and the random parameters given the input variables and the calibration parameters, that is \(p(y, w|x, \phi)\).

Type:

Joint

posterior_approximator#

See posterior_approximator in Parameters.

Type:

PosteriorApproximator

posterior#

This is the posterior approximation of the random parameters given the training data and the calibration parameters, that is \(p(w|\mathcal{D}, \phi)\), where \(\mathcal{D}\) denotes the training data set and \(\phi\) the calibration parameters.

Type:

Posterior

predictive#

This denotes the predictive distribution, that is \(p(y|\phi, x, \mathcal{D})\). Its statistics are approximated via a Monte Carlo approach by sampling from the posterior approximation.

Type:

RegressionPredictive

calibrate(calib_data_loader, val_data_loader=None, calib_config=CalibConfig())[source]#

Calibrate the probabilistic classifier.

Parameters:

  • calib_data_loader (DataLoader) – A calibration data loader.

  • val_data_loader (DataLoader) – A validation data loader.

  • calib_config (CalibConfig) – An object to configure the calibration.

Returns:

A calibration status object. It provides information about the calibration.

Return type:

Status

load_state(checkpoint_path)#

Load the state of the posterior distribution from a checkpoint path. The checkpoint must be compatible with the probabilistic model.

Parameters:

checkpoint_path (Path) – Path to a checkpoint file or directory to restore.

Return type:

None

save_state(checkpoint_path, keep_top_n_checkpoints=1)#

Save the posterior distribution state as a checkpoint.

Parameters:

  • checkpoint_path (Path) – Path to file or directory where to save the current state.

  • keep_top_n_checkpoints (int) – Number of past checkpoint files to keep.

Return type:

None

train(train_data_loader, val_data_loader=None, calib_data_loader=None, fit_config=FitConfig(), calib_config=CalibConfig(), **fit_kwargs)[source]#

Train the probabilistic model. This involves fitting the posterior distribution and calibrating the probabilistic model. Calibration is performed only if (1) calib_data_loader is passed and (2) the probabilistic model contains any calibrator.

Parameters:

  • train_data_loader (DataLoader) – A training data loader.

  • val_data_loader (DataLoader) – A validation data loader. This is used to validate both posterior fitting and calibration.

  • calib_data_loader (DataLoader) – A calibration data loader. If this is not passed, no calibration is performed.

  • fit_config (FitConfig) – An object to configure the posterior distribution fitting.

  • calib_config (CalibConfig) – An object to configure the calibration.

  • map_fit_config (Optional[FitConfig] = None) – An object to configure a preliminary posterior distribution fitting via the Maximum-A-Posteriori (MAP) method. The fit methods of several supported posterior approximations, like the ones of SWAGPosterior and LaplacePosterior, start from a preliminary run of MAP, which can be configured via this object. If the method does not start from MAP, this argument is ignored.

Returns:

Status objects for both posterior fitting and calibration.

Return type:

Dict[str, Status]

class fortuna.prob_model.base.ProbModel(seed=0)[source]#

Abstract probabilistic model class.

load_state(checkpoint_path)[source]#

Load the state of the posterior distribution from a checkpoint path. The checkpoint must be compatible with the probabilistic model.

Parameters:

checkpoint_path (Path) – Path to a checkpoint file or directory to restore.

Return type:

None

save_state(checkpoint_path, keep_top_n_checkpoints=1)[source]#

Save the posterior distribution state as a checkpoint.

Parameters:

  • checkpoint_path (Path) – Path to file or directory where to save the current state.

  • keep_top_n_checkpoints (int) – Number of past checkpoint files to keep.

Return type:

None

train(train_data_loader, val_data_loader=None, calib_data_loader=None, fit_config=FitConfig(), calib_config=CalibConfig(), map_fit_config=None)[source]#

Train the probabilistic model. This involves fitting the posterior distribution and calibrating the probabilistic model. Calibration is performed only if (1) calib_data_loader is passed and (2) the probabilistic model contains any calibrator.

Parameters:

  • train_data_loader (DataLoader) – A training data loader.

  • val_data_loader (DataLoader) – A validation data loader. This is used to validate both posterior fitting and calibration.

  • calib_data_loader (DataLoader) – A calibration data loader. If this is not passed, no calibration is performed.

  • fit_config (FitConfig) – An object to configure the posterior distribution fitting.

  • calib_config (CalibConfig) – An object to configure the calibration.

  • map_fit_config (Optional[FitConfig] = None) – An object to configure a preliminary posterior distribution fitting via the Maximum-A-Posteriori (MAP) method. The fit methods of several supported posterior approximations, like the ones of SWAGPosterior and LaplacePosterior, start from a preliminary run of MAP, which can be configured via this object. If the method does not start from MAP, this argument is ignored.

Returns:

Status objects for both posterior fitting and calibration.

Return type:

Dict[str, Status]