The following table gives an overview of the different tutorial notebooks. We have test specific (Accuracy, Fairness, Robustness, Representation, Bias etc.) notebooks listed below.
| Tutorial Description | Hub | Task | Open In Colab |
|---|---|---|---|
Accuracy test : In this notebook we are evaluating ner.dl model on accuracy tests. |
John Snow Labs | NER |  |
Bias test : In this notebook we are evaluating ner.dl model on bias tests. |
John Snow Labs | NER | |
Fairness test : In this notebook we are evaluating ner.dl model on fairness tests. |
John Snow Labs | NER | |
Representation test : In this notebook we are evaluating ner.dl model on representation tests. |
John Snow Labs | NER | |
Robustness test: In this notebook we are evaluating ner.dl model on robustness tests. |
John Snow Labs | NER | |
| Performace test : In this notebook we are testing time taken to complete the tests in LangTest on the datasets with Models. | Hugging Face/John Snow Labs/Spacy | NER | |
| Translation test : In this section, we dive into testing translation models. We will use the Hugging Face Transformers library/John Snow Labs to load the translation models. | Hugging Face/John Snow Labs | Translation | |
| CrowS Pairs test : In this notebook we are measuring the degree to which stereotypical biases are present in masked language models using Crows Pairs dataset | Hugging Face | Fill-Mask | |
| Stereoset test : In this notebook we are evaluating Hugging Face models on StereoSet. StereoSet is a dataset and a method to evaluate the bias in LLM’s. This dataset uses pairs of sentences, where one of them is more stereotypic and the other one is anti-stereotypic. | Hugging Face | Question-Answering | |
| Wino-Bias test : In this tutorial, we assess the model on gender occupational stereotype statements using Hugging Face fill mask models. | Hugging Face | Fill-Mask | |
Grammar test : In this notebook we are evaluating lvwerra/distilbert-imdb model on grammar test. |
Hugging Face | Text-Classification | |