Systematic assessment of the medical utility of radiology and diagnostic artificial intelligence - retrospective analysis
Evaluation of artificial intelligence algorithms for detecting various pathologies across multiple diagnostic modalities including medical imaging (X-ray, CT, MRI, ultrasound) and other diagnostic tools (e.g., electrocardiograms). Specific conditions will be defined at the sub-study level.
Data Collection: Anonymised medical diagnostic test datasets (imaging and other diagnostic investigations) collected from routine clinical care via Electronic Patient Records (EPR) and clinical IT systems Ground Truth Establishment: Reference standard determined within each sub-study (e.g., through sub-specialist consultant reports or expert arbitration methodology using two independent experts with arbitrator) AI Algorithm Application: CE-approved or late-stage development AI algorithms applied to anonymised datasets, either locally or via secure data transfer to vendors Performance Analysis: AI outputs compared against ground truth reference standard to calculate diagnostic accuracy metrics (sensitivity, specificity, accuracy, area under the curve, positive predictive value, negative predictive value) Statistical Analysis: Statistical tests applied to identify differences between AI algorithms and across subgroups (using methods such as McNemar's test, Cochran's Q test, one-way ANOVA) Data Handling: Full compliance with GDPR and Data Protection Act 2018 Advanced pseudonymisation/anonymisation techniques applied Secure, password-protected, encrypted databases National Data Opt-Out respected
Since this is a data-only retrospective study, there are no direct participant inclusion criteria. Rather, the inclusion criteria apply to the imaging/diagnostic data: 1. Anonymised medical diagnostic investigations (imaging or diagnostic tests) obtained as part of routine clinical care 2. Diagnostic data that can be anonymised without compromising data integrity 3. Images/tests meeting quality standards for AI algorithm analysis
1. Imaging or diagnostic investigations where data cannot be anonymised or where anonymisation compromises data integrity 2. National Data Opt-Out: Data from patients listed in the National Data Opt-Out database who have formally opted out of having their data shared for research purposes 3. Substudy-specific exclusions: Additional exclusion criteria (e.g., age restrictions, specific imaging modality requirements, or pathology-specific criteria) that are dependent on individual substudies