langtest.transform.legal.LegalTestFactory#

class LegalTestFactory(data_handler: List[Sample], tests: Dict | None = None, **kwargs)#

Bases: ITests

Factory class for the legal

__init__(data_handler: List[Sample], tests: Dict | None = None, **kwargs) → None#

Initializes the legal tests

Methods

__init__(data_handler[, tests])	Initializes the legal tests
async_run(model, *args, **kwargs)	Runs the legal tests
available_tests()	Returns the empty dict, no legal tests
run(sample_list, model, **kwargs)	Runs the legal tests
transform()	Nothing to use transform for no longer to generating testcases.

Attributes

alias_name
supported_tasks

async async_run(model: ModelAPI, *args, **kwargs)#

Runs the legal tests

Parameters:

• sample_list (List[Sample]) – The input data to be transformed.

• model (ModelAPI) – The model to be used for evaluation.

• **kwargs – Additional arguments to be passed to the legal tests

Returns:

The transformed data based on the implemented legal tests

Return type:

List[Sample]

classmethod available_tests() → Dict[str, str]#

Returns the empty dict, no legal tests

Returns:

Empty dict, no legal tests

Return type:

Dict[str, str]

async classmethod run(sample_list: List[Sample], model: ModelAPI, **kwargs) → List[Sample]#

Runs the legal tests

Parameters:

• sample_list (List[Sample]) – The input data to be transformed.

• model (ModelAPI) – The model to be used for evaluation.

• **kwargs – Additional arguments to be passed to the wino-bias tests

Returns:

The transformed data based on the implemented legal tests

Return type:

List[Sample]

transform() → List[Sample]#

Nothing to use transform for no longer to generating testcases.

Returns:

Empty list