lisbet.modeling.backbones.transformer

Transformer Backbone for Lisbet.

Classes

TransformerBackbone(feature_dim, ...)

Transformer backbone for sequence modeling.

class lisbet.modeling.backbones.transformer.TransformerBackbone(feature_dim, embedding_dim, hidden_dim, num_heads, num_layers, max_length)

Transformer backbone for sequence modeling.

A transformer-based backbone that processes input sequences using self-attention mechanisms. The backbone includes frame embedding, positional embedding, transformer encoder layers, and layer normalization.

Parameters:

• feature_dim (int) – Dimension of the input features.

• embedding_dim (int) – Dimension of the output embeddings.

• hidden_dim (int) – Dimension of the feedforward network inside transformer layers.

• num_heads (int) – Number of attention heads in the multi-head attention mechanism.

• num_layers (int) – Number of transformer encoder layers.

• max_length (int) – Maximum sequence length for positional embeddings.

frame_embedder

Linear layer for embedding input frames.

Type:

nn.Linear

pos_embedder

Positional embedding module.

Type:

PosEmbedding

transformer_encoder

Stack of transformer encoder layers.

Type:

nn.TransformerEncoder

layer_norm

Layer normalization applied to the output.

Type:

nn.LayerNorm

__init__(feature_dim, embedding_dim, hidden_dim, num_heads, num_layers, max_length)

Initialize internal Module state, shared by both nn.Module and ScriptModule.

forward(x)

Forward pass through the transformer backbone.

Parameters:

x (Tensor) – Input tensor of shape (batch_size, sequence_length, feature_dim).

Returns:

Output tensor of shape (batch_size, sequence_length, embedding_dim).

Return type:

Tensor

get_config()

Get the configuration dictionary for this backbone.

Returns:

Configuration dictionary containing all parameters needed to recreate this backbone instance.

Return type:

dict[str, Any]

classmethod from_config(config)

Create a TransformerBackbone instance from a configuration dictionary.

Parameters:

config (dict[str, Any]) – Configuration dictionary containing all parameters needed to create the backbone instance.

Returns:

New TransformerBackbone instance created from the configuration.

Return type:

TransformerBackbone