nlptest.testrunner.TestRunner#

class TestRunner(load_testcases: List[Sample], model_handler: ModelFactory, data: List[Sample])#

Bases: BaseRunner

Class for running robustness tests on models. Subclass of BaseRunner.

__init__(load_testcases: List[Sample], model_handler: ModelFactory, data: List[Sample]) None#

Initialize the BaseRunner class.

Parameters:
  • load_testcases (List) – List containing the testcases to be evaluated.

  • model_handler (spark, spacy, transformer) – Object representing the model handler, either spaCy, SparkNLP or transformer.

Methods

__init__(load_testcases, model_handler, data)

Initialize the BaseRunner class.

evaluate()

Evaluate the testcases and return the evaluation results.

evaluate()#

Evaluate the testcases and return the evaluation results.

Returns:

all containing the predictions for both the original text and the perturbed one

Return type:

List[Sample]