langtest.transform.stereotype.StereoTypeTestFactory#

class StereoTypeTestFactory(data_handler: List[Sample], tests: Dict | None = None, **kwargs)#

Bases: ITests

Factory class for the crows-pairs or wino-bias tests

__init__(data_handler: List[Sample], tests: Dict | None = None, **kwargs) → None#

Initializes the crows-pairs or wino-bias tests

Methods

__init__(data_handler[, tests])	Initializes the crows-pairs or wino-bias tests
async_run(sample_list, model, *args, **kwargs)	Runs the crows-pairs or wino-bias tests
available_tests()	Returns the empty dict, no crows-pairs or wino-bias tests
run(sample_list, model, **kwargs)	Runs the crows-pairs or wino-bias tests
transform()	Nothing to use transform for no longer to generating testcases.

Attributes

alias_name
supported_tasks

async static async_run(sample_list: Dict[str, List[Sample]] | List[Sample], model: ModelAPI, *args, **kwargs)#

Runs the crows-pairs or wino-bias tests

Parameters:

• sample_list (List[Sample]) – The input data to be transformed.

• model (ModelAPI) – The model to be used for evaluation.

• **kwargs – Additional arguments to be passed to the crows-pairs or wino-bias tests

Returns:

The transformed data based on the implemented crows-pairs or wino-biastests

Return type:

List[Sample]

classmethod available_tests() → Dict[str, str]#

Returns the empty dict, no crows-pairs or wino-bias tests

Returns:

Empty dict, no crows-pairs or wino-bias tests

Return type:

Dict[str, str]

async classmethod run(sample_list: List[Sample], model: ModelAPI, **kwargs) → List[Sample]#

Runs the crows-pairs or wino-bias tests

Parameters:

• sample_list (List[Sample]) – The input data to be transformed.

• model (ModelAPI) – The model to be used for evaluation.

• **kwargs – Additional arguments to be passed to the crows-pairs or wino-bias tests

Returns:

The transformed data based on the implemented crows-pairs or wino-bias tests

Return type:

List[Sample]

transform() → List[Sample]#

Nothing to use transform for no longer to generating testcases.

Returns:

Testcases List