Robustness

Robustness testing aims to evaluate the ability of a model to maintain consistent performance when faced with various perturbations or modifications in the input data.

How it works:

test_type	original	test_case	expected_result	actual_result	pass
add_ocr_typo	” He ended the World Cup on the wrong note , “ Coste said .	” He ended t^e woiUd Cup on t^e v/rong notc , “ Coste said .	World Cup: MISC, Coste: PER	woiUd Cup: MISC, Coste: PER	True
uppercase	Despite winning the Asian Games title two years ago , Uzbekistan are in the finals as outsiders .	DESPITE WINNING THE ASIAN GAMES TITLE TWO YEARS AGO , UZBEKISTAN ARE IN THE FINALS AS OUTSIDERS .	Asian Games: MISC, Uzbekistan: LOC	ASIAN: MISC, UZBEKISTAN: LOC	False

• Perturbations, such as lowercase, uppercase, typos, etc., are introduced to the original text, resulting in a perturbed test_case.
• It’s important to note that when we add perturbations to the original text, we also track the span of the words that are perturbed. This allows us to determine the indices of those words, simplifying the process of realigning the original text with the perturbed text.
• The model processes both the original and perturbed inputs, resulting in expected_result and actual_result respectively.
• During evaluation, the predicted entities in the expected and actual results are compared to assess the model’s performance