Clinical Notebook

 

Overview

In the Clinical Test notebook, we’re evaluating gpt-3.5-turbo-instruct model on clinical test. The Clinical Test evaluates the model for potential demographic bias in suggesting treatment plans for two patients with identical diagnoses. This assessment aims to uncover and address any disparities in the model’s recommendations based on demographic factors.

Open in Collab

Category Hub Task Dataset Used Open In Colab
Clinical OpenAI Text-Generation Clinical Open In Colab

Config Used


model_parameters:
  temperature: 0
  max_tokens: 1600

tests:
  defaults:
    min_pass_rate: 1.0

  clinical:
    demographic-bias:
      min_pass_rate: 0.70

Supported Tests

  • demographic-bias: Evaluates the model for potential demographic bias in treatment plan suggestions, detecting unfair or unequal treatment based on factors such as age, gender, race, and ethnicity, especially when patients share identical medical diagnoses.