langtest.utils.report_utils.model_report#

model_report(summary: Dict, min_pass_dict: Dict, default_min_pass_dict: float, generated_results: List) → DataFrame#

Generate a report summarizing the performance of a model based on provided results.

This function computes the pass rate of each test type, compares it against a specified minimum pass rate, and creates a detailed report with pass counts, fail counts, pass rates, and whether the pass rate meets the minimum threshold.

Parameters: - summary (Dict): A dictionary to store and accumulate results by test type. - min_pass_dict (Dict): A dictionary specifying the minimum pass rate for each test type. - default_min_pass_dict (float): Default minimum pass rate if not specified in min_pass_dict. - generated_results (List): A list of objects where each object should have a test_type attribute indicating the type,

a category attribute indicating the category (e.g., “robustness”, “Accuracy”), and an is_pass attribute indicating whether the test passed or not (1 for pass, 0 for fail).

Returns: - pd.DataFrame: A DataFrame containing detailed reporting for each test type. The columns include “test_type”,

“category”, “fail_count”, “pass_count”, “pass_rate”, “minimum_pass_rate”, and “pass”.