NCT06098950

Human Algorithm Interactions for Acute Respiratory Failure Diagnosis

Measuring the Impact of AI in the Diagnosis of Hospitalized Patients: A Randomized Survey Vignette Multicenter Study

• Status: Completed
• Phase: N/A
• Study type: Interventional
• Enrollment: 457 (actual)
• Sponsor: University of Michigan · Academic / Other
• Sex: All
• Age: 18 Years
• Healthy volunteers: Not accepted

Summary

Artificial intelligence (AI) shows promising in identifying abnormalities in clinical images. However, systematically biased AI models, where a model makes inaccurate predictions for entire subpopulations, can lead to errors and potential harms. When shown incorrect predictions from an AI model, clinician diagnostic accuracy can be harmed. This study aims to study the effectiveness of providing clinicians with image-based AI model explanations when provided AI model predictions to help clinicians better understand the logic of an AI model's prediction. It will evaluate whether providing clinicians with AI model explanations can improve diagnostic accuracy and help clinicians catch when models are making incorrect decisions. As a test case, the study will focus on the diagnosis of acute respiratory failure because determining the underlying causes of acute respiratory failure is critically important for guiding treatment decisions but can be clinically challenging. To determine if providing AI explanations can improve clinician diagnostic accuracy and alleviate the potential impact of showing clinicians a systematically biased AI model, a randomized clinical vignette survey study will be conducted. During the survey, study participants will be shown clinical vignettes of patients hospitalized with acute respiratory failure, including the patient's presenting symptoms, physical exam, laboratory results, and chest X-ray. Study participants will then be asked to assess the likelihood that heart failure, pneumonia and/or Chronic Obstructive Pulmonary Disease (COPD) is the underlying diagnosis. During specific vignettes in the survey, participants will also be shown standard or systematically biased AI models that provide an estimate the likelihood that heart failure, pneumonia and/or COPD is the underlying diagnosis. Clinicians will be randomized see AI predictions alone or AI predictions with explanations when shown AI models. This survey design will allow for testing the hypothesis that systematically biased models would harm clinician diagnostic accuracy, but commonly used image-based explanations would help clinicians partially recover their performance.

Conditions

• Acute Respiratory Failure

Interventions

Type	Name	Description
OTHER	Artificial Intelligence model predictions without explanation	During 6 clinical vignettes, participants will see AI model predictions without a corresponding AI explanation. The AI model will provide a score for each diagnosis (heart failure, pneumonia, COPD) on a scale of 0-100 estimating how likely the patient's presentation was due to each of these diagnoses. In 3 of the clinical vignettes, participants will be shown standard AI model predictions and 3 vignettes they will be shown systematically biased AI model predictions, with the model specifically biased against one of the three diagnoses.
OTHER	Artificial intelligence model predictions with explanation	During 6 clinical vignettes, participants will see AI model predictions with explanation. The AI model will provide a score for each diagnosis on a scale of 0-100. In 3 clinical vignettes, participants will be shown standard AI model predictions and 3 vignettes they will be shown systematically biased AI model predictions with the model specifically biased against one of the three diagnoses. If the AI model provides a score above 50 an AI model explanation will be shown as gradient-weighted class activation mapping (Grad-CAM) heatmaps overlaid on the chest X-ray that highlighted which regions of the image most affecting the AI model's prediction.
OTHER	AI model biased against heart failure	In 3 clinical vignettes, participants will be shown systematically biased AI model predictions with the model specifically biased against heart failure, always predicting that heart failure is present with high likelihood in survey vignette patients with a body mass index (BMI) at or above 30. Standard predictions will be shown for the other 2 diagnoses (pneumonia, COPD).
OTHER	AI model biased against pneumonia	In 3 clinical vignettes, participants will be shown systematically biased AI model predictions with the model specifically biased against pneumonia, always predicting that pneumonia is present with high likelihood in survey vignette patients 80 years or older. Standard predictions will be shown for the other 2 diagnoses (heart failure, COPD).
OTHER	AI model biased against COPD	In 3 clinical vignettes, participants will be shown systematically biased AI model predictions with the model specifically biased against COPD, always predicting that COPD is present with high likelihood in survey vignette patients where a pre-processing filter was applied to the patient's X-ray. Standard predictions will be shown for the other 2 diagnoses (heart failure, pneumonia).

Timeline

• Start date: 2022-04-01
• Primary completion: 2023-01-31
• Completion: 2023-01-31
• First posted: 2023-10-25
• Last updated: 2023-10-25

Locations

1 site across 1 country: United States