langtest.transform.TestFactory#

class TestFactory#

Bases: object

A factory class for creating and running different types of tests on data.

__init__()#

Methods

__init__()
async_run(samples_list, model_handler, **kwargs)	Runs the specified tests on the given data and returns a list of results.
run(samples_list, model_handler, **kwargs)	Runs the specified tests on the given data and returns a list of results.
test_categories()	Returns a dictionary mapping test category names to the corresponding test classes.
test_scenarios()	Returns a dictionary mapping test class names to the available test scenarios for each class.
transform(data, test_types, *args, **kwargs)	Runs the specified tests on the given data and returns a list of results.

Attributes

is_augment

async classmethod async_run(samples_list: List[Sample], model_handler: ModelFactory, **kwargs)#

Runs the specified tests on the given data and returns a list of results.

Parameters:

• samples_list – List[Sample]

• model_handler – ModelFactory

static run(samples_list: List[Sample], model_handler: ModelFactory, **kwargs)#

Runs the specified tests on the given data and returns a list of results.

Parameters:

• samples_list – List[Sample]

• model_handler – ModelFactory

classmethod test_categories() → Dict#

Returns a dictionary mapping test category names to the corresponding test classes.

Returns:

A dictionary mapping test category names to the corresponding test classes.

Return type:

Dict

classmethod test_scenarios()#

Returns a dictionary mapping test class names to the available test scenarios for each class.

Returns:

A dictionary mapping test class names to the available test scenarios for each class.

Return type:

Dict

static transform(data: List[Sample], test_types: dict, *args, **kwargs) → List[Result]#

Runs the specified tests on the given data and returns a list of results.

Parameters:

• data – List[Sample] The data to be tested.

• test_types – dict A dictionary mapping test category names to lists of test scenario names.

• model – ModelFactory Model to be tested.

Returns:

List[Results]

A list of results from running the specified tests on the data.