The Evaluating Model’s Sensitivity to Negation Test focuses on assessing a model’s responsiveness to negations introduced into its input text. The primary objective is to determine whether the model can effectively detect and respond to negations. This evaluation consists of two key stages: first, introducing negations into the input text, particularly after verbs like “is,” “was,” “are,” and “were”; and second, observing how the model behaves or how sensitive it is when presented with input containing these negations.
sensitivity: negation: min_pass_rate: 0.70
- min_pass_rate (float): Minimum pass rate to pass the test.
|what is the name of the hat you wear at graduation||what is not the name of the hat you wear at graduation||The hat typically worn at graduation ceremonies is called a mortarboard.||A mortarboard is the name of the hat worn at graduation ceremonies.||0.0287267||False|