Multiple Perturbations

The multiple_perturbations test combines multiple tests into a single test by applying a sequence of perturbations to transform the given sentences. These perturbations are applied in a specific sequence.

Please note that this test is only supported for the text-classification, question-answering, and summarization tasks.

alias_name: multiple_perturbations

Config YAML format :

multiple_perturbations:
    min_pass_rate: 0.60
    prob: 0.5 # Defaults to 1.0, which means all words will be transformed.
    perturbations1:
        lowercase
        add_ocr_typo
        titlecase

The perturbation set perturbations1 follows the transformation order: lowercase ➜ add_ocr_typo ➜ titlecase

You can adjust the level of transformation in the sentence by using the “prob” parameter, which controls the proportion of words to be changed during multiple_perturbations test.

• min_pass_rate (float): Minimum pass rate to pass the test.
• prob (float): Controls the proportion of words to be changed.

Examples