A multi-reader multi-case study of clinician performance with, and without artificial intelligence assistance for wrist fracture detection, with embedded qualitative study
Wrist fracture
This is a multi-reader multi-case (MRMC) study evaluating clinician interpretation of wrist radiographs with and without artificial intelligence (AI) decision support. Participating clinicians will interpret a fixed set of anonymised wrist radiograph cases using a secure, computer-based research application. For each case, clinicians will first provide an unaided diagnostic assessment and confidence rating. They will then be shown an AI output for the same case and asked to confirm or revise their diagnostic assessment and confidence. AI assistance will be presented in one of two predefined formats at the case level: a low-information display (diagnostic label and visual heatmap) or a high-information display (structured report including diagnostic label, model confidence, image quality feedback, and heatmap). Each case will be assigned to one AI display format, and all readers will view the same cases under the same format. Updated 20/03/2026: AI assistance will be presented in one of two predefined formats at the case level: a low-information display (diagnostic label) or a high-information display (structured report including diagnostic label, model confidence, image quality feedback, and heatmap). Each case will be assigned to one AI display format, and all readers will view the same cases under the same format. No clinical treatment or patient intervention is administered. All cases are retrospective and de-identified, and participant responses do not influence patient care. Quantitative data collected will include diagnostic scores, categorical decisions, confidence ratings, time-to-decision measures, and agreement between clinician decisions and AI outputs. Diagnostic performance will be assessed using area under the receiver operating characteristic curve (AUC), with comparisons made between unaided and AI-assisted interpretation, and between low- and high-information AI formats, accounting for the MRMC study design. An embedded qualitative component w
Wrist imaging: 1. Adult patients (18 years or older) who have had a x-rays of the hand/wrist for suspected fracture at Oxford University Hospitals NHS Foundation Trust, 2010-2022. Readers: 1. Clinicians who interpret wrist and/or hand radiographs as part of routine clinical practice. 2. Healthcare professionals (advanced nurse practitioners and doctors) who currently work, have worked, or will work (within 1 year of the study) in emergency departments, minor injuries units, radiology, musculoskeletal trauma services in the United Kingdom. 3. Able to provide informed consent. 4. Able to attend the study session and use the study software.
1. Clinicians who do not routinely interpret wrist or hand radiographs. 2. Inability to provide informed consent. 3. Prior involvement in the development, training, validation, or evaluation of the AI system used in the study. 4. Prior access to, or knowledge of, the reference standard labels for the study cases. 5. Any conflict of interest or role that could reasonably compromise independent interpretation of cases (e.g. involvement in case selection or ground-truth annotation).