Evaluation of a deep learning model applied to chest X-rays of patients with suspected pneumonia presenting to the emergency department designed to predict admission risk: a retrospective training and prospective non-intervention validation

Objectives To develop, train and test a deep learning model Image-based PROgnostication applied to Chest X Rays (IPRO-X) tool that predicts the inpatient (IP) admission risk in patients with suspected pneumonia presenting to the emergency department (ED).Design The study consists of a retrospective (training) and prospective non-interventional shadow deployment (validation) of a deep learning model.Setting Three-hospital tertiary care system with two EDs.Participants Consecutive adult patients (18 years old or older) who presented to the ED from December 2022 to February 2023 with a chief clinical complaint potentially related to pneumonia were reviewed for eligibility (n=5567), with a final number of 3677 included in the validation study.Primary and secondary outcome measures IPRO-X was developed using standard two-dimensional convolutional neural network InceptionNet architecture and processes chest radiographs (CXRs) acquired at admission generating a continuous value from 0 (IP-negative) to 1 (IP-positive). We examined IPRO-X’s ability to predict IP admission using accuracy, area under the curve (AUC), receiver operating characteristic, sensitivity, specificity, positive predictive value and negative predictive value. IPRO-X scores were compared against observed outcomes (admission and discharge) to determine clinical utility in an ED setting. Four thresholds were defined from the retrospective phase: Youden optimal, highest specificity and thresholds equivalent to 30-day mortality rates of Pneumonia Severity Index (PSI) Risk class IV and Confusion, Respiratory Rate, Blood Pressure and Age 65 or older Score (CRB-65) scores 1 and 2. Performance was also analysed per group of chief complaints.Results In the validation set, 3677 patients (1777 (48%) female, median age 56 years (min 18, max 99)) were included. IPRO-X predicted IP admissions with an AUC of 0.795, and in the pneumonia-specific chief complaints, AUC increased to 0.828, whereas in non-related pneumonia chief complaints, the AUC decreased to 0.755. IPRO-X score was significantly higher in admitted patients compared with discharged patients (p<0.001). The accuracy of IPRO-X when using the PSI-anchored threshold was 0.751 and with the CRB-anchored threshold was 0.729, with a specificity of 0.803 and 0.968, respectively.Conclusions IPRO-X applied to CXR of patients with signs and symptoms commonly related to pneumonia in the ED can accurately predict IP admission.