gojo.deepl package

Submodules

gojo.deepl.callback module

class gojo.deepl.callback.Callback(name: str)

Bases: object

Base class (interface) used to define the callbacks to be executed in each iteration of the training loop of the neural networks defined in gojo.deepl.loop.fitNeuralNetwork(). These callbacks provide directives to modify the training of the models. A classic example would be the early stopping callback (defined in gojo.deepl.callback.EarlyStopping).

Subclasses must define the following methods:

• evaluate()

  This method will make available to the callback the following arguments used (and updated) in the current iteration of the :func:’gojo.deepl.loop.fitNeuralNetwork’ training loop:

  modelgojo.core.base.TorchSKInterface or gojo.core.base.ParametrizedTorchSKInterface

  Model to be trained.

  train_metricslist

  Train computed metrics until the last epoch.

  valid_metricslist

  Validation computed metrics until the last epoch.

  train_losslist

  Train computed loss until the last epoch.

  valid_losslist

  Validation computed loss until the last epoch.

  This method has to return a directive (as a string) that will be interpreted by the gojo.deepl.loop.fitNeuralNetwork() inner loop.

• resetState()

  This method should reset the inner state of the callback.

abstract evaluate(*args, **kwargs) → str

abstract resetState()

class gojo.deepl.callback.EarlyStopping(it_without_improve: int, track: str = 'mean', ref_metric: Optional[str] = None, smooth_n: Optional[int] = None)

Bases: gojo.deepl.callback.Callback

Callback used to perform an early stopping of the gojo.deepl.loop.fitNeuralNetwork() training loop.

it_without_improveint

Number of iterations that must be completed without the model showing a decrease in the loss value over the validation set (average of the last epochs or count of the last epochs, as defined by parameter track) to perform an early stopping ending the loop execution.

trackstr, default=’mean’

Method used to compare the latest value of the loss on the validation set with respect to the historical value. Methods currently available:

• ‘mean’: compare the current value with respect to the average of the it_without_improve epochs.

• ‘count’: compare the current value with respect to it_without_improve epochs.

ref_metricstr, default=None

Reference metric calculated on the validation set to be used as a reference. By default, the loss value will be used.

smooth_nint, default=None

Value that indicates if instead of considering the last value of the loss and comparing against the historical ones, to smooth the last value considering the average value of the last smooth_n iterations.

DIRECTIVE = 'stop'

VALID_TRACKING_OPTS = ['mean', 'count']

evaluate(valid_loss: list, **_) → str

Early stopping inner logic.

resetState()

Reset callback

class gojo.deepl.callback.SaveCheckPoint(output_dir: str, key: str, each_epoch: int, verbose: bool = True)

Bases: gojo.deepl.callback.Callback

Callback used to save the model parameters during training.

output_dirstr

Output directory used to store model parameters. If it does not exist, it will be created automatically.

keystr

Key used to identify the model.

each_epochint

Specify the number of epochs to save for each model.

verbosebool, default=True

Parameter that indicates whether to display messages on the screen when executing the early stop.

DIRECTIVE = None

evaluate(n_epoch: int, model: torch.nn.Module, **_)

static message(out_file: str)

resetState()

Reset callback

gojo.deepl.cnn module

class gojo.deepl.cnn.ResNetBlock(*args: Any, **kwargs: Any)

Bases: torch.nn.Module

Block with residual connections to build convolutional networks. Based on:

He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 770-778).

This block will perform the following operations:

where Conv1 and Conv2 will consist of:

in_channelsint

Number of input channels.

out_channelsint

Number of output channels.

kernel_sizeint

Kernel size of the convolutional layers. Only squared kernels are supported.

normalizationstr, default=’batch’

Normalization applied after convolutions. Available values are: ‘batch’ for batch normalization and ‘instance’ for instance normalization.

dimstr, default=’2d’

Type of input data (‘1d’ for one-dimensional data, ‘2d’ for two-dimensional data, and ‘3d’ for tridimensional data).

activation_fncallable, default=torch.nn.ReLU()

Activation function.

stride_conv1int, default=1

Stride applied to the first convolutional layer.

padding_conv1int, default=1

Padding applied to the first convolutional layer.

stride_conv2int, default=1

Stride applied to the second convolutional layer.

padding_conv2int ,default=1

Padding applied to the second convolutional layer.

debuggingbool, default=False

Parameter that indicates whether to print the dimensions of the inputs/outputs of each of the layers.

forward(x) → torch.Tensor

gojo.deepl.ffn module

gojo.deepl.ffn.createSimpleFFNModel(in_feats: int, out_feats: int, layer_dims: list, layer_activation: list, layer_dropout: Optional[list] = None, batchnorm: bool = False, weights_init: Optional[callable] = None, output_activation: Optional[str] = None) → torch.nn.Module

Auxiliary function that allows to easily create a simple FFN architecture from the provided input parameters. See examples for a quick overview of the posibilities of this function.

in_featsint

Number of the features in the input data.

out_featsint

Number of features in the output data.

layer_dimslist

Layer widths.

layer_activationlist or torch.nn.Module or None or str

Activation funtions. If None is provided a simple affine transformation will take place. If a string is provided, the name should match to the name of the torch.nn class (i.e., ‘ReLU’ for torch.nn.ReLU).

layer_dropoutlist or float, default=None

Layer dropouts. If an scalar is provided the same dropout rate will be applied for all the layers.

batchnormbool, default=False

Parameter indicating whether to add batch-normalization layers.

weights_initcallable or list, default=None

Function (os list of functions) applied to the generated lienar layers for initializing their weights.

output_activationstr or torch.nn.Module or None, default=None

Output activation function (similar to layer_activation).

modeltorch.nn.Module

Generated model.

>>> import torch
>>> from gojo import deepl
>>>
>>>
>>> model = deepl.ffn.createSimpleFFNModel(
>>>     in_feats=100,
>>>     out_feats=1,
>>>     layer_dims=[100, 60, 20],
>>>     layer_activation=torch.nn.ReLU(),
>>>     layer_dropout=0.3,
>>>     batchnorm=True,
>>>     output_activation=torch.nn.Sigmoid()
>>> )
>>> model
Out[0]
    Sequential(
          (LinearLayer 0): Linear(in_features=100, out_features=100, bias=True)
          (BatchNormalization 0): BatchNorm1d(100, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (Activation 0): ReLU()
          (Dropout 0): Dropout(p=0.3, inplace=False)
          (LinearLayer 1): Linear(in_features=100, out_features=60, bias=True)
          (BatchNormalization 1): BatchNorm1d(60, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (Activation 1): ReLU()
          (Dropout 1): Dropout(p=0.3, inplace=False)
          (LinearLayer 2): Linear(in_features=60, out_features=20, bias=True)
          (BatchNormalization 2): BatchNorm1d(20, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (Activation 2): ReLU()
          (Dropout 2): Dropout(p=0.3, inplace=False)
          (LinearLayer 3): Linear(in_features=20, out_features=1, bias=True)
          (Activation 3): Sigmoid()
        )
>>>
>>> model = deepl.ffn.createSimpleFFNModel(
>>>     in_feats=10,
>>>     out_feats=77,
>>>     layer_dims=[100, 60, 20],
>>>     layer_activation=[torch.nn.Tanh(), None, torch.nn.ReLU()],
>>>     layer_dropout=[0.3, None, 0.1],
>>>     batchnorm=True,
>>>     weights_init=[torch.nn.init.kaiming_uniform_] * 3 + [None],
>>>     output_activation=torch.nn.Sigmoid()
>>> )
>>> model
Out[1]
    Sequential(
          (LinearLayer 0): Linear(in_features=10, out_features=100, bias=True)
          (BatchNormalization 0): BatchNorm1d(100, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (Activation 0): Tanh()
          (Dropout 0): Dropout(p=0.3, inplace=False)
          (LinearLayer 1): Linear(in_features=100, out_features=60, bias=True)
          (BatchNormalization 1): BatchNorm1d(60, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (LinearLayer 2): Linear(in_features=60, out_features=20, bias=True)
          (BatchNormalization 2): BatchNorm1d(20, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (Activation 2): ReLU()
          (Dropout 2): Dropout(p=0.1, inplace=False)
          (LinearLayer 3): Linear(in_features=20, out_features=99, bias=True)
          (Activation 3): Sigmoid()
        )

gojo.deepl.ffn.createSimpleParametrizedFFNModel(in_feats: int, out_feats: int, n_layers: int, init_layer_dim: int, scaffold: str, min_width: int, max_width: int, beta: Optional[float] = None, alpha: Optional[float] = None, layer_activation: Optional[str] = None, layer_dropout: Optional[float] = None, batchnorm: bool = False, weights_init: Optional[callable] = None, output_activation: Optional[str] = None) → torch.nn.Module

Function that allows the creation of a simple neural network (a feed forward network, FFN) by parameterizing the number of layers and their width according to different scaffolds (selected by the scaffold parameter), and hyperparameters (alpha and beta).

in_featsint

Number of the features in the input data.

out_featsint

Number of features in the output data.

n_layersint

Number of layers.

init_layer_dimint

Dimensions of the first layer.

scaffoldstr

Model scaffold to arrange the layers. Valid scaffolds are:

• ‘exponential’: exponential decay in the number of layers. Controlled by the beta parameter.

\[n^{(l)} = (1 / beta)^{(l)} \cdot init\]

• ‘linear’: linear decay in the number of layers. Controlled by the alpha parameter.

\[n^{(l)} = init - alpha \cdot (l)\]

min_widthint

Minimum layer width.

max_widthint

Maximum layer width.

betafloat or int

Applied for exponential scaffolds.

alphafloat or int

Applied for lineal scaffolds.

layer_activationtorch.nn.Module or None or str

Activation functions. If None is provided a simple affine transformation will take place. If a string is provided, the name should match to the name of the torch.nn class (i.e., ‘ReLU’ for torch.nn.ReLU).

layer_dropoutfloat, default=None

Layer dropout. The same dropout rate will be applied for all the layers.

batchnormbool, default=False

Parameter indicating whether to add batch-normalization layers.

weights_initcallable, default=None

Function applied to the generated linear layers for initializing their weights.

output_activationstr or torch.nn.Module or None, default=None

Output activation function (similar to layer_activation).

modeltorch.nn.Module

Generated model.

>>> from gojo import deepl
>>>
>>> model = createSimpleParametrizedFFNModel(
>>>     in_feats=94,
>>>     out_feats=1,
>>>     n_layers=5,
>>>     init_layer_dim=500,
>>>     scaffold='exponential',
>>>     min_width=10,
>>>     max_width=500,
>>>     beta=1.5,
>>>     alpha=100,
>>>     layer_activation=None,
>>>     layer_dropout=None,
>>>     batchnorm=False,
>>>     output_activation='Sigmoid')
>>> model
Out [0]
    Sequential(
      (LinearLayer 0): Linear(in_features=94, out_features=500, bias=True)
      (LinearLayer 1): Linear(in_features=500, out_features=334, bias=True)
      (LinearLayer 2): Linear(in_features=334, out_features=223, bias=True)
      (LinearLayer 3): Linear(in_features=223, out_features=149, bias=True)
      (LinearLayer 4): Linear(in_features=149, out_features=99, bias=True)
      (LinearLayer 5): Linear(in_features=99, out_features=1, bias=True)
      (Activation 5): Sigmoid()
    )

gojo.deepl.loading module

class gojo.deepl.loading.GraphDataset(*args: Any, **kwargs: Any)

Bases: torch.utils.data.Dataset

Class used to generate a dataset adapted to operate with Graph Neural Networks. This class can be passed to torch.utils.data.DataLoader and subsequently used by the gojo.deepl.loops.fitNeuralNetwork() function.

Xnp.ndarray or pd.DataFrame or List[np.ndarray]

Input predictor variables used to adjust the models. If a numpy array or a pandas DataFrame is provided, entries along dimension 0 will be interpreted as instances, and the 1-axis will be interpreted as the number of nodes in the network. In the case where a list of numpy arrays is provided, each element of the list will be interpreted as an instance, the 0-axis as the number of nodes, and the remaining dimensions as node features.

ynp.ndarray or pd.DataFrame or pd.Series, default=None

Target variables to fit the models (or None).

adj_matrixnp.ndarray or pd.DataFrame or List[Union[np.ndarray, pd.DataFrame]], default=None

Adjacency matrix. If a numpy array or a pandas DataFrame is provided, it must have a shape of (n_nodes, n_nodes). In the case where a list of numpy arrays is provided, each element of the list will be interpreted as a graph, and it must have a shape of (n_nodes, n_nodes).

One of adj_matrix or edge_index must be provided.

edge_indexnp.ndarray or pd.DataFrame or List[Union[np.ndarray, pd.DataFrame]], default=None

Edge index. If a numpy array or a pandas DataFrame is provided, it must have a shape of (2, n_nodes). In the case where a list of numpy arrays is provided, each element of the list will be interpreted as a graph, and it must have a shape of (2, n_nodes).

One of adj_matrix or edge_index must be provided.

tabular_x: np.ndarray or pd.DataFrame or List[np.ndarray], default=None

Tabular characteristics that will be stored in the tabular_x attribute of the instances ( torch_geometric.data.DataBatch) returned by this dataset.

Important

Internally a dimension will be added along axis 1 to prevent torch_geometric dataloaders from flattening the data to a single dimension.

>>> import numpy as np
>>> import gojo
>>>
>>> n_samples = 10     # number of instances
>>> n_node_feats = 3   # number of node features
>>>
>>> # generate random adjacency matrices, one for each sample
>>> adj_matrices = []
>>> for _ in range(n_samples):
>>>     n_nodes = np.random.randint(5, 30)
>>>     adj_matrices.append(np.random.randint(0, 2, size=(n_nodes, n_nodes)))
>>>
>>> # generate the node features
>>> # each sample will be (n_nodes, n_node_features)
>>> node_feats = [
>>>     np.random.uniform(size=(adj_matrix.shape[0], n_node_feats))
>>>     for adj_matrix in adj_matrices
>>> ]
>>>
>>> # generate a target feature
>>> target = np.random.randint(0, 2, size=n_samples)
>>>
>>> # create the dataset
>>> graph_dt = gojo.experimental.deepl_loading.GraphDataset(
>>>     X=node_feats,
>>>     y=target,
>>>     adj_matrix=adj_matrices
>>> )
>>>

class gojo.deepl.loading.TorchDataset(*args: Any, **kwargs: Any)

Bases: torch.utils.data.Dataset

Basic Dataset class torch models. This class can be passed to torch.DataLoaders and subsequently used by the gojo.deepl.loops.fitNeuralNetwork() function or gojo.interfaces.TorchSKInterface and gojo.interfaces.ParametrizedTorchSKInterface classes.

Xnp.ndarray or pd.DataFrame or pd.Series

Input predictor variables used to fit the models.

ynp.ndarray or pd.DataFrame or pd.Series, default=None

Target variables to fit the models (or None).

x_transformslist, default=None

Transformations to be applied to the data provided in X. This parameter must be provided as a list of callables which will receive as input the X data.

y_transformslist, default=None

Transformations to be applied to the data provided in y. This parameter must be provided as a list of callables which will receive as input the y data.

x_stream_databool, default=False

Parameter indicating whether X data will be loaded in streaming. In this case the parameters of X will be passed to x_loading_fn and this function must return the data that (if provided) will then go to the transforms and subsequently be returned by the dataset.

x_loading_fncallable, default=None

Function used to load streaming data. This parameter will have no effect if ‘x_stream_data’ has not been provided.

y_stream_databool, default=False

Same logic as x_stream_data but applied to the y parameter.

y_loading_fncallable, default=None

Same logic as x_loading_fn but applied to the y parameter.

**op_instance_args

Instance-level optional arguments. This parameter should be a dictionary whose values must be np.ndarray containing the same number of elements as instances in X and y.

>>> from gojo import deepl
>>> from torch.utils.data import DataLoader
>>>
>>> # dataset loading ...
>>> X = np.random.uniform(size=(30, 10))
>>> y = np.random.uniform(size=30)
>>>
>>> # use TorchDataset for an easy use of pytorch DataLoaders
>>> dataloader = DataLoader(
>>>     deepl.loading.TorchDataset(X=X, y=y),
>>>     batch_size=16, shuffle=True)
>>>

gojo.deepl.loops module

gojo.deepl.loops.fitNeuralNetwork(iter_fn, model: torch.nn.Module, train_dl: Iterable, n_epochs: int, loss_fn: callable, optimizer_class, optimizer_params: Optional[dict] = None, lr_scheduler_class=None, lr_scheduler_params: Optional[dict] = None, valid_dl: Optional[Iterable] = None, device: Optional[str] = None, verbose: int = 1, metrics: Optional[list] = None, callbacks: Optional[List[gojo.deepl.callback.Callback]] = None, **kwargs) → dict

Main function of the gojo.deepl() module. This function is used to fit a pytorch model using the provided “iteration function” (parameter iter_fn) that defined how to run an epoch.

iter_fncallable

Function used to execute an epoch during model training. Currently available are:

• gojo.deepl.iterSupervisedEpoch()

  Used for typical supervised approaches.

modeltorch.nn.Module

Pytorch model to be trained.

train_dlIterable

Train dataloader (see torch.utils.data.DataLoader class).

n_epochsint

Maximum number of epochs for training a model.

loss_fncallable

Loss function used to fit the model. This loss function must follow the pytorch guideliness.

IMPORTANTE: be carreful with this function does not break the Pytorch gradient calculation.

optimizer_classtype

Optimizer class used to adjust model weights (see torch module).

optimizer_paramsdict, default=None

Parameters used to initialize the optimizer provided using optimizer_params.

lr_scheduler_classtype, default=None

Class used to construct a learning rate schedule as defined in torch.optim.lr_scheduler().

lr_scheduler_paramsdict, default=None

Parameters used to initialize the learning rate scheduler as defined based on lr_scheduler_class.

valid_dlIterable, default=None

Validation dataloader (see torch.utils.data.DataLoader class).

devicestr, default=None

Device used to optimize the input model. Commonly devices are: ‘cpu’, ‘cuda’, ‘mps’.

verboseint, default=1

Verbosity level.

metricslist, defualt=None

Metrics to compute in each epoch during model training across the train and validation datasets.

callbacksList[Callback], default=None

Callbacks used to modify the training loop (for more information see gojo.deepl.callback)

fitting_historydict

History with the model metrics (if provided) and loss for each epoch for the training (‘train’ key) and validation (‘validation’ key) datasets.

gojo.deepl.loops.getAvailableIterationFunctions() → list

Function that returns a list with all the available iteration functions used as iter_fn argument in gojo.deepl.loops.fitNeuralNetwork() callings.

gojo.deepl.loops.iterSupervisedEpoch(model: torch.nn.Module, dataloader: Iterable, optimizer, loss_fn: callable, device: str, training: bool, metrics: list, scheduler=None, **kwargs) → tuple

Basic function applied to supervised problems that executes the code necessary to perform an epoch.

This function will return a tuple where the first element correspond to dictionary with the loss-related parameters, and the second element to a dictionary with the calculated metrics.

>>> import torch
>>> from gojo import deepl
>>> from gojo import core
>>>
>>> # ... previous dataloader creation and model definition
>>> history = deepl.fitNeuralNetwork(
>>>     iter_fn=deepl.iterSupervisedEpoch,    # function used to perform an epoch
>>>     model=model,
>>>     train_dl=train_dl,
>>>     valid_dl=valid_dl,
>>>     n_epochs=50,
>>>     loss_fn=torch.nn.BCELoss(),
>>>     optimizer_class=torch.optim.Adam,
>>>     optimizer_params={'lr': 0.001},
>>>     device='cuda',
>>>     metrics=core.getDefaultMetrics('binary_classification', bin_threshold=0.5)
>>> )
>>>

NOTE: the input dataloader is required to return at least two arguments where the first parameter must correspond to the predictor variables and the second parameter to the target variable.

gojo.deepl.loops.iterUnsupervisedEpoch(model: torch.nn.Module, dataloader: Iterable, optimizer, loss_fn: callable, device: str, training: bool, metrics: list, scheduler=None, **kwargs) → tuple

Basic function applied to supervised problems that executes the code necessary to perform an epoch.

This function will return a tuple where the first element correspond to dictionary with the loss-related parameters, and the second element to a dictionary with the calculated metrics.

gojo.deepl.loss module

class gojo.deepl.loss.BCELoss(*args: Any, **kwargs: Any)

Bases: torch.nn.Module

Weighted binary cross-entropy.

weightfloat, default = 1.0

Weight applied to the positive class.

allow_nansbool, default = False

Boolean parameter indicating whether the true values contain missing values. If the value is indicated as False this class will use gojo.deepl.loss.weightedBCE() as internal function, if the value is indicated as True, the class will use gojo.deepl.loss.weightedBCEwithNaNs() as internal function.

forward(y_hat: torch.Tensor, y_true: torch.Tensor) → torch.Tensor

class gojo.deepl.loss.ELBO(*args: Any, **kwargs: Any)

Bases: torch.nn.Module

Evidence lower bound (ELBO) loss function as described in “Auto-Encoding Variational Bayes” from Kigma and Welling (2014).

kld_weightfloat, default=1.0

Weight applied to the Kullback-Leibler divergence term.

forward(x_hat: torch.Tensor, x_true: torch.Tensor, mu: torch.Tensor, logvar: torch.Tensor) → Tuple[torch.Tensor, dict]

Forward pass.

x_hattorch.Tensor

Reconstructed model input.

x_truetorch.Tensor

True model input.

mutorch.Tensor

Mean projection vector.

logvartorch.Tensor

Log-var projection vector.

outputTuple[torch.Tensor, dict]

This function will return a two element tuple where the first element will correspond to the loss while the second element will be a dictionary containing other loss function related parameters.

gojo.deepl.loss.huberLossWithNaNs(y_hat: torch.Tensor, y_true: torch.Tensor, delta: float) → torch.Tensor

Calculate the Huber loss allowing for missing values in the y_true argument.

y_hattorch.Tensor

Model predictions.

y_truetorch.Tensor

Ground true values.

deltafloat

Specifies the threshold at which to change between delta-scaled L1 and L2 loss. The value must be positive.

losstorch.Tensor

Averaged loss value.

gojo.deepl.loss.mseLossWithNaNs(y_hat: torch.Tensor, y_true: torch.Tensor) → torch.Tensor

Calculate the mean squared loss error (MSE) allowing for missing values in the y_true argument.

y_hattorch.Tensor

Model predictions.

y_truetorch.Tensor

Ground true values.

losstorch.Tensor

Averaged loss value.

gojo.deepl.loss.weightedBCE(y_hat: torch.Tensor, y_true: torch.Tensor, weight: float) → torch.Tensor

Calculate the binary cross-entropy by weighting the positive class.

y_hattorch.Tensor

Model predictions.

y_truetorch.Tensor

Ground true values.

weightfloat

Weight applied to the positive class.

losstorch.Tensor

Averaged loss value.

gojo.deepl.loss.weightedBCEwithNaNs(y_hat: torch.Tensor, y_true: torch.Tensor, weight: float) → torch.Tensor

Similar to gojo.deepl.loss.weightedBCE() but allowing the incorporation of missing values in y_true.

y_hattorch.Tensor

Model predictions.

y_truetorch.Tensor

Ground true values.

weightfloat

Weight applied to the positive class.

losstorch.Tensor

Averaged loss value.

gojo.deepl.models module

class gojo.deepl.models.FusionModel(*args: Any, **kwargs: Any)

Bases: torch.nn.Module

Model designed to allow the merging of information from several models that receive different input values.

encoderstorch.nn.ModuleList

Models associated with each of the inputs. The models will be executed in order by receiving as argument the input parameters of the model after eliminating the entries defined in the indexes of parameter ignore_inputs.

fusion_modeltorch.nn.Module

Fusion model that will receive all the merged data internally (either by the function defined in concat_fn or concatenated along dimension 1 by default) and will generate the final model output.

concat_fncallable, default=None

Function used to concatenate the outpus of the input models. This function will receive as input a list of tensors and must return a unified tensor. By default, function torch.cat will be called by concatenating the outputs along dimension 1.

ignore_inputslist, default=None

List specifying the input items to be ignored.

encode(*inputs) → torch.Tensor

Processes the input elements through the models defined in parameter encoders and returns a unified vector as specified by argument concat_fn.

forward(*inputs) → torch.Tensor

class gojo.deepl.models.MultiTaskFFN(*args: Any, **kwargs: Any)

Bases: torch.nn.Module

Model adapted to perform multi-task classification with a layer specialized in extracting features (accessible through feature_extractor()) and, as is typical in multi-task model training, layers specialized in performing the different tasks. The model will return a tensor with the outputs of each of the layers concatenated, where the first n_clf_task classification tasks will go first, followed by the n_reg_task regression tasks.

in_featsint

(feature extractor) Number of input features.

emb_featsint

(feature extractor) Number of output features for the feature extractor feed forward network (FFN).

layer_dimslist

(feature extractor) Layer dims for the feature extractor feed forward network (FFN).

n_clf_taskint

(feature extractor) Number of classification task. If n_clf_task = 0, then n_reg_task must be greater than 1.

n_reg_taskint

(feature extractor) Number of regression task. If n_reg_task = 0, then n_clf_task must be greater than 1.

multt_layer_dimslist

(multi-task layers) Architecture used for the task’s specialized layers.

multt_dropoutlist or float, default=None

(multi-task layers) Dropout for the task’s specialized layers.

multt_layer_activationtorch.nn.Module or str or None, default=’ELU’

(multi-task layers) Activation function for the task’s specialized layers.

multt_batchnormbool, default=False

(multi-task layers) Indicates whether to used batch normalization in the task’s specialized layers.

multt_clf_activationstr or torch.nn.Module or list or None, default=’Sigmoid’

(multi-task layers, classification) Output activation function for the classification task. If a list is provided this must match the length of the parameter n_clf_task.

multt_reg_activationstr or torch.nn.Module or list or None, default=None

(multi-task layers, regression) Output activation function for the regression task. If a list is provided this must match the length of the parameter n_reg_task.

layer_dropoutlist or float or None, default=None

(feature extractor) Dropout rate for the feature extractor feed forward network (FFN).

layer_activationtorch.nn.Module or str or None, default=’ELU’

(feature extractor) Activation function for the feature extractor feed forward network (FFN).

batchnormbool, default=False

(feature extractor param)Indicates whether to used batch normalization in the feature extractor feed forward network (FFN).

>>> from gojo import deepl
>>>
>>>
>>> model = deepl.models.MultiTaskFFN(
>>>     in_feats=100,
>>>     emb_feats=20,
>>>     layer_dims=[250, 50],
>>>     n_clf_task=2,
>>>     n_reg_task=3,
>>>     multt_layer_dims=[20, 10],
>>>     multt_dropout=0.2,
>>>     multt_layer_activation='ELU',
>>>     multt_batchnorm=False,
>>>     multt_clf_activation='Sigmoid',
>>>     multt_reg_activation=['TanU', None, None],
>>>     layer_dropout=0.4,
>>>     layer_activation='ELU',
>>>     batchnorm=True
>>> )

forward(X: torch.Tensor, **_) → torch.Tensor

class gojo.deepl.models.MultiTaskFFNv2(*args: Any, **kwargs: Any)

Bases: torch.nn.Module

(Simplified version of gojo.deepl.models.MultiTaskFFN) Model adapted to perform multi-task classification with a layer specialized in extracting features (accessible through feature_extractor()) and, as is typical in multi-task model training, layers specialized in performing the different tasks (accessible through multitask_projection()). The model will return a tensor with the outputs of each of the layers from the input parameter multitask_projection concatenated in the same order as declared in the input parameter.

feature_extractortorch.nn.Module

Layer that will take the input from the model and generate an embedded representation that will be subsequently used by the layers defined in multitask_projection.

multitask_projectiontorch.nn.ModuleList

Layers specialized in different tasks. Their outputs will be concatenated along dimension 1.

>>> import torch
>>> from gojo import deepl
>>>
>>>
>>> X = torch.rand(10, 40)    # (batch_size, n_feats)
>>>
>>> multitask_model = deepl.models.MultiTaskFFNv2(
>>>     feature_extractor=torch.nn.Sequential(
>>>         torch.nn.Linear(40, 20),
>>>         torch.nn.ReLU()
>>>     ),
>>>     multitask_projection=torch.nn.ModuleList([
>>>         torch.nn.Sequential(
>>>             torch.nn.Linear(20, 2),
>>>             torch.nn.Tanh()
>>>         ),
>>>         torch.nn.Sequential(
>>>             torch.nn.Linear(20, 1),
>>>             torch.nn.Sigmoid()
>>>         ),
>>>     ])
>>> )
>>>
>>> with torch.no_grad():
>>>     mtt_out = multitask_model(X)
>>>     emb = multitask_model.feature_extractor(X)
>>>
>>>
>>> mtt_out[:, :2].min(), mtt_out[:, :2].max()
    Out[0]: (tensor(-0.2965), tensor(0.1321))
>>>
>>> mtt_out[:, 2].min(), mtt_out[:, 2].max()
    Out[1]: (tensor(0.3898), tensor(0.4343))
>>>
>>> emb.shape
    Out[2]: torch.Size([10, 20])

forward(X, **_) → torch.Tensor

class gojo.deepl.models.VanillaVAE(*args: Any, **kwargs: Any)

Bases: torch.nn.Module

Basic variational autoencoder model as presented in (https://arxiv.org/abs/1312.6114).

encodertorch.nn.Module

Encoder model. The encoder will model P(Z|X) during training.

encoder_out_dimint

Output shape of the encoder.

decodertorch.nn.Module

Decoder model. The decoder will model P(X|Z) where P(Z) is assumed to follow a multivariate Gaussian distribution.

decoder_in_dimint

Expected input shape for the decoder.

latent_dimint

Latent dimensions of Z.

decode(Z: torch.Tensor) → torch.Tensor

Decode the latent representation

Ztorch.Tensor

Latent representation generated by the :meth`reparametrize` function.

decoder_outtorch.Tensor

Decoder output.

encode(X: torch.Tensor) → Tuple[torch.Tensor, torch.Tensor]

Generate the latent representation

Xtorch.Tensor

Input data

mu_stdTuple[torch.Tensor, torch.Tensor]

Mean and standard deviation of the latent dimensions.

forward(X: torch.Tensor, *args, **kwargs) → Tuple[torch.Tensor, dict]

Forward function. This function will do the following operations:

X -> encoder -> projection -> [mu, std] -> reparametrization -> decoder

Xtorch.Tensor

Input data to be codified.

outputTuple[torch.Tensor, dict]

This function will return a two element tuple where the first element will correspond to the reconstructed input and the second element to a dictionary with the mean and logvar vectors of the latent representations generated by the encoder.

reparametrize(mu: torch.Tensor, logvar: torch.Tensor) → torch.Tensor

Reparametrization trick as described in “Auto-Encoding Variational Bayes” from Kigma and Welling.

mutorch.Tensor

Mean of the distribution of the latent variables.

logvartorch.Tensor

Logarithm of the standard deviation of the latent variables.

sampletorch.Tensor

Sample from the latent variable distribution.

sample(n_samples: int, current_device: str = 'cpu') → torch.Tensor

Sample from the latent space.

n_samplesint

Number of samples

current_devicestr, default=’cpu’

Device to run the model

samplestorch.Tensor

Tensor of shape (n_samples, *)

Module contents