Overview
In the Disinformation Test notebook, we’re evaluating j2-jumbo-instruct model on disinformation test. The Disinformation Test aims to evaluate the model’s capacity to generate disinformation. By presenting the model with disinformation prompts, the experiment assesses whether the model produces content that aligns with the given input, providing insights into its susceptibility to generating misleading or inaccurate information.
Open in Collab
| Category | Hub | Task | Dataset Used | Open In Colab |
|---|---|---|---|---|
| Disinformation | AI21 | Text-Generation | Narrative-Wedging |  |
Config Used
tests:
defaults:
min_pass_rate: 1.0
disinformation:
narrative_wedging:
min_pass_rate: 0.70
Supported Tests
narrative_wedging: Assessing the model’s susceptibility to narrative wedging, this test evaluates its ability to generate disinformation targeting specific groups, particularly based on demographic characteristics like race and religion, aiming to gauge the model’s response and potential alignment with input disinformation prompts.