A study of the use of artificial intelligence in detecting acute illnesses in neuroradiological scans
Artificial intelligence (AI) is advancing rapidly and becoming increasingly integrated into healthcare, particularly in radiology. In the field of neuroradiology, which deals with brain and spine imaging, AI tools hold significant promise. However, there are notable challenges. Many AI applications in this field are complex and not well understood, often functioning as a "black box" where their processes and decision-making are unclear. Furthermore, many of these AI tools are tested using data from only one institution, which can lead to performance issues when applied in broader clinical settings.
To address these concerns, the American Society of Functional Neuroradiology organized an AI Competition to evaluate AI models in a more realistic manner. The competition used 1,201 anonymized non-contrast CT scans from five different institutions. The AI models were tasked with three main objectives: detecting pathologies, assessing if findings were normal for the patient's age, and classifying the urgency of any abnormalities. This approach aimed to test the AI tools in a setting that mimics real-world clinical conditions more closely.
The results of the competition were revealing. Most AI models performed poorly across all tasks. For example, their ability to detect acute ischemic stroke was only slightly better than random guessing. Similarly, the detection of intracranial hemorrhage and traumatic brain injury showed modest results, with performance metrics that indicated only marginal improvements over chance. When it came to evaluating normality relative to age, the AI models also struggled, further underscoring their current limitations.
These findings point to significant issues with the current AI models used in neuroradiology. Despite their potential, these tools are not yet reliable enough for routine clinical use. They often lack the necessary accuracy and adaptability when faced with diverse and external datasets, highlighting the need for continued research and development. Without more robust validation and refinement, these AI tools cannot be fully trusted in real-world medical settings.
For the layperson, this means that a human radiologist will be reading any scans you get for for a long while now. While AI in healthcare is an exciting and promising development, it is not yet at a stage where it can be relied upon for critical diagnostic decisions. Patients should understand that AI tools are still being improved and are not yet a substitute for experienced medical professionals. The end purpose of all the ongoing research and future advancements is for the enhancement of patient care and outcomes via accurate and dependable AI tools.
Assessed and Endorsed by the MedReport Medical Review Board
Comments