NCT06774612

The Impact of Large Language Models on Diagnostic Reasoning Among LLM-Trained Medical Doctors

Diagnostic Reasoning With and Without AI Support: A Randomized Controlled Trial of LLM-Trained Medical Doctors

• Status: Completed
• Phase: N/A
• Study type: Interventional
• Enrollment: 60 (actual)
• Sponsor: Lahore University of Management Sciences · Academic / Other
• Sex: All
• Age: —
• Healthy volunteers: Accepted

Summary

This study aims to evaluate whether large language model-trained medical doctors demonstrate enhanced diagnostic reasoning performance when utilizing ChatGPT-4o alongside conventional resources compared to using conventional resources alone.

Detailed description

Diagnostic errors are a major source of preventable patient harm. Recent advances in Large Language Models (LLM), particularly ChatGPT-4o, have shown promise in enhancing medical decision-making. However, little is known about their impact on medical doctors' (e.g., physicians' and surgeons') diagnostic reasoning. Diagnostic accuracy relies on complex clinical reasoning and careful evaluation of patient data. While AI assistance could potentially reduce errors and improve efficiency, ChatGPT-4o lacks medical validation and could introduce new risks through incorrect information generation (also known as hallucinations). To mitigate these risks, doctors need adequate training in understanding ChatGPT-4o's capabilities, limitations, and proper usage. Given these uncertainties and the importance of proper AI training, systematic evaluation is essential before clinical implementation. This randomized study will assess whether ChatGPT-4o access improves LLM-trained medical doctors' diagnostic performance compared to conventional resources (e.g., textbooks, online medical databases) alone. All participating doctors will have completed at least a 10-hour training program covering ChatGPT-4o usage, prompt engineering techniques, and output evaluation strategies. Participants will provide differential diagnoses with supporting evidence and recommended next steps for clinical cases, with responses evaluated by blinded reviewers.

Conditions

• Diagnosis

Interventions

Type	Name	Description
OTHER	ChatGPT-4o	OpenAI's ChatGPT-4o large language model with chat interface.

Timeline

• Start date: 2025-01-10
• Primary completion: 2025-05-17
• Completion: 2025-05-17
• First posted: 2025-01-14
• Last updated: 2025-07-17

Locations

1 site across 1 country: Pakistan