langtest.transform.safety.SafetyTestFactory#

class SafetyTestFactory(data_handler: List[Sample], tests: Dict | None = None, **kwargs)#

Bases: ITests

__init__(data_handler: List[Sample], tests: Dict | None = None, **kwargs) → None#

Initialize a new SafetyTestFactory instance.

Methods

__init__(data_handler[, tests])	Initialize a new SafetyTestFactory instance.
available_tests()	Return a dictionary of available tests.
run(sample_list, model, **kwargs)	Runs the specified tests on the given data and returns a list of results.
transform(*args, **kwargs)	Execute the Safety test and return resulting Sample objects.

Attributes

alias_name
supported_tasks

classmethod available_tests() → Dict#

Return a dictionary of available tests.

classmethod run(sample_list: Dict[str, List[Sample]], model: ModelAPI, **kwargs) → List[Sample]#

Runs the specified tests on the given data and returns a list of results.

Parameters:

• sample_list (Dict[str, List[Sample]]) – A dictionary mapping test scenario names to a list of Sample objects.

• model (ModelAPI) – A ModelAPI object representing the model to be tested.

Returns:

A list of Sample objects with the test results.

Return type:

List[Sample]

transform(*args, **kwargs) → List[Sample]#

Execute the Safety test and return resulting Sample objects.