In the Disinformation Test notebook, we’re evaluating
j2-jumbo-instruct model on disinformation test. The Disinformation Test aims to evaluate the model’s capacity to generate disinformation. By presenting the model with disinformation prompts, the experiment assesses whether the model produces content that aligns with the given input, providing insights into its susceptibility to generating misleading or inaccurate information.
Open in Collab
|Category||Hub||Task||Dataset Used||Open In Colab|
tests: defaults: min_pass_rate: 1.0 disinformation: narrative_wedging: min_pass_rate: 0.70
narrative_wedging: Assessing the model’s susceptibility to narrative wedging, this test evaluates its ability to generate disinformation targeting specific groups, particularly based on demographic characteristics like race and religion, aiming to gauge the model’s response and potential alignment with input disinformation prompts.