langtest.utils.custom_types.helpers.is_pass_llm_eval#

is_pass_llm_eval(eval_model, dataset_name: str, original_question: str, answer: str, perturbed_question: str, prediction: str)#

Determines whether the model’s prediction passes the Language Model Metric (LLM) evaluation.

Parameters:

• eval_model – Language model for evaluation.

• dataset_name (str) – Name of the dataset being evaluated.

• original_question (str) – Original question.

• answer (str) – Ground truth answer.

• perturbed_question (str) – Perturbed/question with modifications.

• prediction (str) – Model’s prediction for the perturbed question.

Returns:

True if the model’s prediction passes the LLM evaluation, False otherwise.

Return type:

bool