langtest.modelhandler.transformers_modelhandler.PretrainedModelForSensitivity#

class PretrainedModelForSensitivity(model, *args, **kwargs)#

Bases: ModelAPI

A class for handling a pretrained model for sensitivity testing.

This class wraps a pretrained transformer model for performing sensitivity testing.

Parameters:

model (tuple) – A tuple containing the model and tokenizer.

Raises:

ValueError – If the input model is not a tuple.

model#

The pretrained transformer model.

Type:

Any

tokenizer#

The tokenizer associated with the model.

Type:

Any

__init__(model, *args, **kwargs)#

Initialize a PretrainedModelForSensitivity instance.

Parameters:

model (tuple) – A tuple containing the model and tokenizer.

Raises:

ValueError – If the input model is not a tuple.

Methods

__init__(model, *args, **kwargs)

Initialize a PretrainedModelForSensitivity instance.

load_model(path)

Load the model into the model attribute.

predict(text, prompt, test_name, **kwargs)

Perform predictions on the input text.

Attributes

model_registry

classmethod load_model(path: str)#

Load the model into the model attribute.

Parameters:

path (str) – Path to model or model name.

Returns:

An instance of the class containing the loaded model and tokenizer.

Return type:

PretrainedModelForSensitivity

predict(text: str, prompt, test_name: str, **kwargs)#

Perform predictions on the input text.

Parameters:

  • text (str) – Input text to perform sensitivity testing on.

  • test_name (str) – Name of the sensitivity test (e.g., “add_negation”, “add_toxic_words”).

  • **kwargs – Additional keyword arguments.

Returns:

A dictionary containing the following keys:

  • ’loss’ (float): Difference in loss between transformed and original text (for “add_negation” test).

  • ’result’ (str): Decoded result from the model.

Return type:

dict