langtest.transform.fairness.FairnessTestFactory#

class FairnessTestFactory(data_handler: List[Sample], tests: Dict, **kwargs)#

Bases: ITests

A class for performing fairness tests on a given dataset.

__init__(data_handler: List[Sample], tests: Dict, **kwargs) → None#

Methods

__init__(data_handler, tests, **kwargs)
available_tests()	Get a dictionary of all available tests, with their names as keys and their corresponding classes as values.
get_gendered_data(data)	Split list of samples into gendered lists.
run(sample_list, model, raw_data, **kwargs)	Runs the fairness tests on the given model and dataset.
transform()	Runs the fairness test and returns the resulting Sample objects.

Attributes

alias_name
model_result

static available_tests() → dict#

Get a dictionary of all available tests, with their names as keys and their corresponding classes as values.

Returns:

A dictionary of test names and classes.

Return type:

dict

static get_gendered_data(data: List[Sample]) → Dict[str, List[Sample]]#

Split list of samples into gendered lists.

classmethod run(sample_list: Dict[str, List[Sample]], model: ModelAPI, raw_data: List[Sample], **kwargs)#

Runs the fairness tests on the given model and dataset.

Parameters:

• sample_list (Dict[str, List[Sample]]) – A dictionary of test names and corresponding Sample objects.

• model (ModelAPI) – The model to be tested.

• raw_data (List[Sample]) – The raw dataset.

transform() → List[Sample]#

Runs the fairness test and returns the resulting Sample objects.

Returns:

A list of Sample objects representing the resulting dataset after running the fairness test.

Return type:

List[Sample]