langtest.transform.factuality.FactualityTestFactory#

class FactualityTestFactory(data_handler: List[Sample], tests: Dict | None = None, **kwargs)#

Bases: ITests

Factory class for factuality test

__init__(data_handler: List[Sample], tests: Dict | None = None, **kwargs) → None#

Initializes the FactualityTestFactory

Methods

__init__(data_handler[, tests])	Initializes the FactualityTestFactory
async_run(model, *args, **kwargs)	Runs factuality tests
available_tests()	Retrieves available factuality test types.
run(sample_list, model, **kwargs)	Runs factuality tests
transform()	Nothing to use transform for no longer to generating testcases.

Attributes

alias_name
supported_tasks

async async_run(model: ModelAPI, *args, **kwargs)#

Runs factuality tests

Parameters:

• sample_list (list[Sample]) – A list of Sample objects to be tested.

• model (ModelAPI) – The model to be used for testing.

• *args – Additional positional arguments.

• **kwargs – Additional keyword arguments.

Returns:

A list of Sample objects with test results.

Return type:

list[Sample]

classmethod available_tests() → Dict[str, str]#

Retrieves available factuality test types.

Returns:

A dictionary mapping test names to their corresponding classes.

Return type:

dict

async classmethod run(sample_list: List[Sample], model: ModelAPI, **kwargs) → List[Sample]#

Runs factuality tests

Parameters:

• sample_list (list[Sample]) – A list of Sample objects to be tested.

• model (ModelAPI) – The model to be used for evaluation.

• **kwargs – Additional keyword arguments.

Returns:

A list of Sample objects with test results.

Return type:

list[Sample]

transform() → List[Sample]#

Nothing to use transform for no longer to generating testcases.

Returns:

Empty list