The following table gives an overview of the different tutorial notebooks. In this section we have miscellaneous notebooks to enhance user experience with different features.
Miscellaneous Notebooks
| Tutorial Description | Hub | Task | Open In Colab |
|---|---|---|---|
| Add Custom Data: In this tutorial, we explore the process of incorporating custom data for bias and representation testing from JSON files. | Spacy/Hugging Face | NER/Text-Classification |  |
| Augmentation Control: In this tutorial, we explore the process of how to contol the augmentations. | John Snow Labs | NER | |
| Multiple Model Comparison: In this tutorial, we compared different language models on various taks. | Hugging Face/John Snow Labs/Spacy | NER/Text-Classification | |
| Custom Hub: In this tutorial, we compared different language models on various taks. | Custom | Text-Classification | |
| Report Exportation: In this tutorial, we explored different ways in which user can export their report. | Spacy | NER | |
| Editing Testcases: In this section, we discussed how to edit test cases in the Harness class. | Hugging Face | NER | |
| Evaluation Metrics: In this section, we discussed different evatuation metrics for evauate Quetion-Answering models. | OpenAI | Question-Answering | |
| HuggingFace Datasets: In this section, we dive into testing of HuggingFace Models for different HuggingFace Datasets. | Hugging Face/Spacy/OpenAI | NER/Text-Classification/Question-Answering/Summarization | |
| Custom Column Loading: In this section, we discussed how to load a csv data for different task such as QA, Text-Classification, NER, Summarization. | Hugging Face/OpenAI | NER/Text-Classification/Question-Answering/Summarization | |
Multiple Variations: In this section, we discussed Multiple variations for a perturbation. Some of the robustness tests take a parameter count which specifies how many instances/variations of a sentence to produce. |
John Snow Labs | NER | |
| Templatic Augmentation: In this section, we discussed about Templatic Augmentation which is a technique that allows you to generate new training data by applying a set of predefined templates to the original training data. | John Snow Labs | NER | |
| LangTestCallback: In this section, we discussed how to utilize the LangTestCallback funtion while training an NER transformers model. | Hugging Face | NER | |
| LangTestCallback: In this section, we discussed how to utilize the LangTestCallback funtion while training an Text Classification transformers model. | Hugging Face | Text-Classification | |
| Multiple_dataset: In this section, we discussed how to evaluate multiple datasets for a particular model. | OpenAI | Question-Answering | |
| Generic API-Based Model: In this section, we discussed how to test API-based models hosted using Ollama, vLLM, and other tools. | Web | Question-Answering | |
| Data Augmenter: In this Notebook, we can allows for streamlined and harness-free data augmentation, making it simpler to enhance your datasets and improve model robustness. | - | NER | |
| Multi-Dataset Prompt Configs: In this Notebook, we discussed about optimized prompt handling for multiple datasets, allowing users to add custom prompts for each dataset, enabling seamless integration and efficient testing. | OpenAI | Question-Answering | |