langtest.transform.performance.PerformanceTestFactory#

class PerformanceTestFactory(data_handler: List[Sample], tests: Dict | None = None, **kwargs)#

Bases: ITests

Factory class for the model performance

This class implements the model performance The robustness measure is the number of test cases that the model fails to run on.

__init__(data_handler: List[Sample], tests: Dict | None = None, **kwargs) → None#

Initializes the model performance

Methods

__init__(data_handler[, tests])	Initializes the model performance
available_tests()	Returns the available model performance
run(sample_list, model, **kwargs)	Runs the model performance
transform()	Transforms the sample data based on the implemented tests measure.

Attributes

alias_name

classmethod available_tests() → Dict[str, str]#

Returns the available model performance

Returns:

The available model performance

Return type:

Dict[str, str]

async classmethod run(sample_list: List[Sample], model: ModelAPI, **kwargs) → List[Sample]#

Runs the model performance

Parameters:

• sample_list (List[Sample]) – The input data to be transformed.

• model (ModelAPI) – The model to be used for evaluation.

• **kwargs – Additional arguments to be passed to the model performance

Returns:

The transformed data based on the implemented model performance

Return type:

List[Sample]

transform() → List[Sample]#

Transforms the sample data based on the implemented tests measure.

Parameters:

• sample (Sample) – The input data to be transformed.

• **kwargs – Additional arguments to be passed to the tests measure.

Returns:

The transformed data based on the implemented tests measure.

Return type:

Sample