Introduction
In emergency departments, rapid and accurate diagnosis of patients presenting with pulmonary symptoms is critical. Differentiating between COVID-19 pneumonia and typical bacterial pneumonia is paramount for effective patient management. Artificial intelligence (AI) offers a promising tool to aid radiologists in this challenging Chest X Ray Differential Diagnosis, potentially speeding up the diagnostic process and improving accuracy. This study investigates the diagnostic performance of an AI system in distinguishing between these two types of pneumonia, as well as identifying healthy individuals, using chest X-ray (CXR) images.
Methods
This research evaluated an AI system’s ability to detect COVID-19 pneumonia and bacterial pneumonia from chest X-rays acquired in an emergency setting. The study utilized a dataset comprising three groups: patients confirmed positive for COVID-19 pneumonia (n = 1140), patients diagnosed with typical bacterial pneumonia (n = 500), and healthy control subjects (n = 1000). To ensure unbiased assessment, two experienced radiologists independently reviewed the CXRs, blinded to patient demographics, clinical history, and laboratory results. Subsequently, the AI system analyzed the same CXRs, classifying each into one of three categories: COVID-19 pneumonia, bacterial pneumonia, or healthy. Interrater reliability between the radiologists and the AI system was assessed using Cohen’s κ statistic. The diagnostic accuracy of the AI system was rigorously evaluated using a confusion matrix, with 95% confidence intervals (CIs) calculated to determine the precision of the results.
Results
The interrater reliability analysis revealed a high degree of agreement between the expert radiologist and the AI system. For COVID-19 pneumonia detection, the κ value was 0.822, indicating almost perfect agreement. For bacterial pneumonia detection, the agreement was even stronger, with a κ of 0.913, also signifying almost perfect agreement. In detecting COVID-19 pneumonia, the radiologist achieved a sensitivity of 96% (95% CIs = 94.9-96.9%) and a specificity of 79.8% (76.4-82.9%). Remarkably, the AI system demonstrated comparable performance, with a sensitivity of 94.7% (93.4-95.8%) and a slightly improved specificity of 80.2% (76.9-83.2%). For bacterial pneumonia detection, both the radiologist and AI system showed excellent sensitivity: 97.9% (98-99.3%) and 97.5% (96.5-98.3%), respectively. The radiologist’s specificity was 88% (83.5-91.7%), while the AI system achieved a specificity of 83.9% (79-87.9%). Overall, when classifying CXRs into COVID-19 pneumonia, bacterial pneumonia, or healthy, the AI system attained an impressive accuracy of 93.8%, with a misclassification rate of only 6.2% and a weighted-F1 score of 93.8%.
Conclusion
This study demonstrates the excellent diagnostic capabilities of the AI system in performing chest x ray differential diagnosis to identify both COVID-19 and typical bacterial pneumonia from chest X-rays in an emergency department context. The AI system’s performance is on par with, and in some aspects slightly exceeds, that of experienced radiologists. These findings suggest that AI can be a valuable asset in emergency settings, facilitating quicker and more accurate diagnoses, ultimately contributing to improved patient care and efficient resource allocation during surges in pulmonary infections.
