langtest.transform.stereotype.StereoTypeTestFactory#

class StereoTypeTestFactory(data_handler: List[Sample], tests: Dict | None = None, **kwargs)#

Bases: ITests

Factory class for the crows-pairs or wino-bias tests

__init__(data_handler: List[Sample], tests: Dict | None = None, **kwargs) None#

Initializes the crows-pairs or wino-bias tests

Methods

__init__(data_handler[, tests])

Initializes the crows-pairs or wino-bias tests

async_run(sample_list, model, *args, **kwargs)

Runs the crows-pairs or wino-bias tests

available_tests()

Returns the empty dict, no crows-pairs or wino-bias tests

run(sample_list, model, **kwargs)

Runs the crows-pairs or wino-bias tests

transform()

Nothing to use transform for no longer to generating testcases.

Attributes

alias_name

supported_tasks

async static async_run(sample_list: Dict[str, List[Sample]] | List[Sample], model: ModelAPI, *args, **kwargs)#

Runs the crows-pairs or wino-bias tests

Parameters:
  • sample_list (List[Sample]) – The input data to be transformed.

  • model (ModelAPI) – The model to be used for evaluation.

  • **kwargs – Additional arguments to be passed to the crows-pairs or wino-bias tests

Returns:

The transformed data based on the implemented crows-pairs or wino-biastests

Return type:

List[Sample]

classmethod available_tests() Dict[str, str]#

Returns the empty dict, no crows-pairs or wino-bias tests

Returns:

Empty dict, no crows-pairs or wino-bias tests

Return type:

Dict[str, str]

async classmethod run(sample_list: List[Sample], model: ModelAPI, **kwargs) List[Sample]#

Runs the crows-pairs or wino-bias tests

Parameters:
  • sample_list (List[Sample]) – The input data to be transformed.

  • model (ModelAPI) – The model to be used for evaluation.

  • **kwargs – Additional arguments to be passed to the crows-pairs or wino-bias tests

Returns:

The transformed data based on the implemented crows-pairs or wino-bias tests

Return type:

List[Sample]

transform() List[Sample]#

Nothing to use transform for no longer to generating testcases.

Returns:

Testcases List