In the Custom Hub notebook, we’re evaluating our very own trained model independent from hubs. The main focus is showing we can run the tests with any possible model/framework easily if we have a predict function in the similar format. The notebook showcases an implementation of an LSTM model for text classification task trained using pytorch. After creating the harness object with
custom parameter, we can continue to use it as always.
Open in Collab
|Category||Hub||Task||Open In Colab|
tests: defaults: min_pass_rate: 1.0 robustness: add_typo: min_pass_rate: 0.7 american_to_british: min_pass_rate: 0.7 accuracy: min_micro_f1_score: min_score: 0.7 bias: replace_to_female_pronouns: min_pass_rate: 0.7 replace_to_low_income_country: min_pass_rate: 0.7 fairness: min_gender_f1_score: min_score: 0.6 representation: min_label_representation_count: min_count: 50