task: Union[str, dict]
Task for which the model is to be evaluated
Supported Tasks
| Task | Description | Supported Model Categories |
|---|---|---|
| ner | Named Entity Recognition | All NER models available through the different hubs |
| text-classification | Text Classification | All text classification models available through the different hubs |
| question-answering | Question Answering | Large Language Models available through the different hubs |
| summarization | Summarization | Large Language Models available through the different hubs |
| fill-mask | Fill Mask | Fill mask models and Large Language Models available through the different hubs |
| translation | Translation | Translation models available through the different hubs |
| text-generation | Text Generation | Large Language Models available through the different hubs |