NCT07374926

The Big Unknown: A Journey Into Generative AI's Transformative Effect on Medical Professions

The Big Unknown: A Journey Into Generative AI's Transformative Effect on Meical Professions

• Status: Completed
• Phase: N/A
• Study type: Interventional
• Enrollment: 249 (actual)
• Sponsor: Maastricht University · Academic / Other
• Sex: All
• Age: —
• Healthy volunteers: Accepted

Summary

A parallel group randomized controlled trial using a superiority framework. Clinical vignettes will be used to assess the impact of a large language model on the clinical reasoning of physicians. Quantitative analyses will be performed on graded vignette responses.

Detailed description

This study is a multi-country, parallel-group randomized controlled trial designed to evaluate whether access to a large language model (LLM) improves physician clinical decision-making. The trial uses a superiority framework and compares physicians randomized to either complete standardized clinical vignettes with access to GPT-4o or without any AI assistance. Clinical vignettes simulate common primary care conditions such as cardiovascular, respiratory, musculoskeletal, fatigue-related, and infectious diseases. Each vignette includes multiple steps in the clinical reasoning process, from initial history-taking to diagnosis, treatment, and follow-up. Physician responses are graded using rubrics developed from evidence-based, context-specific best-practice guidelines. The study is conducted across three countries-Indonesia, Kenya, and the Netherlands-representing different income levels and health system contexts. The primary outcome is performance on clinical vignettes, defined as adherence to best-practice guidelines. Secondary objectives include examining cross-country variation in physician performance, variation in performance distributions, and the role of engagement with the LLM in shaping outcomes.

Conditions

• Diagnosis
• Vignette of Fictional Patients

Interventions

Type	Name	Description
OTHER	GPT-4o	GPT-4o provided via an iFrame in the online Qualtrics environment

Timeline

• Start date: 2024-08-01
• Primary completion: 2025-01-17
• Completion: 2025-01-17
• First posted: 2026-01-29
• Last updated: 2026-01-29

Locations

3 sites across 3 countries: Indonesia, Kenya, Netherlands