langtest.transform.stereoset.StereoSetTestFactory#

class StereoSetTestFactory(data_handler: List[Sample], tests: Dict | None = None, **kwargs)#

Bases: ITests

Factory class for the StereoSet tests

__init__(data_handler: List[Sample], tests: Dict | None = None, **kwargs) → None#

Initializes the stereoset tests

Methods

__init__(data_handler[, tests])	Initializes the stereoset tests
async_run(sample_list, model, *args, **kwargs)	Runs the StereoSet tests
available_tests()	Returns the empty dict, no stereoset tests
run(sample_list, model, **kwargs)	Runs the stereoset tests
transform()	Nothing to use transform for no longer to generating testcases.

Attributes

alias_name
supported_tasks

async static async_run(sample_list: List[Sample], model: ModelAPI, *args, **kwargs)#

Runs the StereoSet tests

Parameters:

• sample_list (List[Sample]) – The input data to be transformed.

• model (ModelAPI) – The model to be used for evaluation.

• **kwargs – Additional arguments to be passed to the StereoSet tests

Returns:

The transformed data based on the implemented crows-pairs tests

Return type:

List[Sample]

classmethod available_tests() → Dict[str, str]#

Returns the empty dict, no stereoset tests

Returns:

Empty dict, no stereoset tests

Return type:

Dict[str, str]

async classmethod run(sample_list: List[Sample], model: ModelAPI, **kwargs) → List[Sample]#

Runs the stereoset tests

Parameters:

• sample_list (List[Sample]) – The input data to be transformed.

• model (ModelAPI) – The model to be used for evaluation.

• **kwargs – Additional arguments to be passed to the stereoset tests

Returns:

The transformed data based on the implemented stereoset tests

Return type:

List[Sample]

transform() → List[Sample]#

Nothing to use transform for no longer to generating testcases.

Returns:

List of testcase