Miscellaneous Notebooks

 

The following table gives an overview of the different tutorial notebooks. In this section we have miscellaneous notebooks to enhance user experience with different features.

Miscellaneous Notebooks

Tutorial Description Hub Task Open In Colab
Add Custom Data: In this tutorial, we explore the process of incorporating custom data for bias and representation testing from JSON files. Spacy/Hugging Face NER/Text-Classification Open In Colab
Augmentation Control: In this tutorial, we explore the process of how to contol the augmentations. John Snow Labs NER Open In Colab
Multiple Model Comparison: In this tutorial, we compared different language models on various taks. Hugging Face/John Snow Labs/Spacy NER/Text-Classification Open In Colab
Custom Hub: In this tutorial, we compared different language models on various taks. Custom Text-Classification Open In Colab
Report Exportation: In this tutorial, we explored different ways in which user can export their report. Spacy NER Open In Colab
Editing Testcases: In this section, we discussed how to edit test cases in the Harness class. Hugging Face NER Open In Colab
Evaluation Metrics: In this section, we discussed different evatuation metrics for evauate Quetion-Answering models. OpenAI Question-Answering Open In Colab
HuggingFace Datasets: In this section, we dive into testing of HuggingFace Models for different HuggingFace Datasets. Hugging Face/Spacy/OpenAI NER/Text-Classification/Question-Answering/Summarization Open In Colab
Custom Column Loading: In this section, we discussed how to load a csv data for different task such as QA, Text-Classification, NER, Summarization. Hugging Face/OpenAI NER/Text-Classification/Question-Answering/Summarization Open In Colab
Multiple Variations: In this section, we discussed Multiple variations for a perturbation. Some of the robustness tests take a parameter count which specifies how many instances/variations of a sentence to produce. John Snow Labs NER Open In Colab
Templatic Augmentation: In this section, we discussed about Templatic Augmentation which is a technique that allows you to generate new training data by applying a set of predefined templates to the original training data. John Snow Labs NER Open In Colab
LangTestCallback: In this section, we discussed how to utilize the LangTestCallback funtion while training an NER transformers model. Hugging Face NER Open In Colab
LangTestCallback: In this section, we discussed how to utilize the LangTestCallback funtion while training an Text Classification transformers model. Hugging Face Text-Classification Open In Colab
Multiple_dataset: In this section, we discussed how to evaluate multiple datasets for a particular model. OpenAI Question-Answering Open In Colab
Generic API-Based Model: In this section, we discussed how to test API-based models hosted using Ollama, vLLM, and other tools. Web Question-Answering Open In Colab
Data Augmenter: In this Notebook, we can allows for streamlined and harness-free data augmentation, making it simpler to enhance your datasets and improve model robustness. - NER Open In Colab
Multi-Dataset Prompt Configs: In this Notebook, we discussed about optimized prompt handling for multiple datasets, allowing users to add custom prompts for each dataset, enabling seamless integration and efficient testing. OpenAI Question-Answering Open In Colab
Multi-Model, Multi-Dataset: In this Notebook, we discussed about testing on multiple models with multiple datasets, allowing users to allows for comprehensive comparisons and performance assessments in a streamlined manner. OpenAI Question-Answering Open In Colab
Evaluation_with_Prometheus_Eval: In this Notebook, we disscussed about integrating the Prometheus model to langtest brings enhanced evaluation capabilities, providing more detailed and insightful metrics for model performance assessment. OpenAI Question-Answering Open In Colab
Misuse_Test_with_Prometheus_evaluation: In this Notebook, we discussed about new safety testing features to identify and mitigate potential misuse and safety issues in your models OpenAI Question-Answering Open In Colab
Visual_QA: In this Notebook, we discussed about the visual question answering tests to evaluate how models handle both visual and textual inputs, offering a deeper understanding of their versatility. OpenAI Visual-Question-Answering (visualqa) Open In Colab
Add_New_Lines_and_Tabs_Tests: In this Notebook, we discussed about new tests like inserting new lines and tab characters into text inputs, challenging your models to handle structural changes without compromising accuracy. Hugging Face/John Snow Labs/Spacy Text-Classification/Question-Answering/Summarization Open In Colab
Safety_Tests_With_PromptGuard: In this Notebook, we discussed about evaluating prompts before they are sent to large language models (LLMs), ensuring harmful or unethical outputs are avoided with PromptGuard. Hugging Face/John Snow Labs/Spacy Text-Classification/Question-Answering/Summarization Open In Colab
Last updated