langtest.utils.report_utils.multi_model_report#

multi_model_report(summary: Dict, min_pass_dict: Dict, default_min_pass_dict: float, generated_results: Dict, model_name: str) → DataFrame#

Generate a report summarizing the performance of a specific model from multiple models based on provided results.

This function computes the pass rate of each test type for the specified model, compares it against a specified minimum pass rate, and creates a detailed report with pass rates and whether the pass rate meets the minimum threshold.

Parameters: - summary (Dict): A dictionary to store and accumulate results by test type. - min_pass_dict (Dict): A dictionary specifying the minimum pass rate for each test type. - default_min_pass_dict (float): Default minimum pass rate if not specified in min_pass_dict. - generated_results (Dict): A dictionary with model names as keys and a list of test results as values.

Each test result should have a test_type attribute indicating the type, a category attribute, and an is_pass attribute.

• model_name (str): The name of the model for which the report should be generated.

Returns: - pd.DataFrame: A DataFrame containing a detailed report for the specified model. The columns include “test_type”,

“model_name”, “pass_rate”, “minimum_pass_rate”, and “pass”.