A recent real-world study found that an FDA-cleared AI algorithm helped identify more brain aneurysms on CT scans, and suggests that combining AI with physician interpretation could improve detection, reports Medical News Today.
The algorithm identified 55 true-positive brain aneurysms that radiologists had missed, representing a 39% relative increase in detection, the researchers said.
AI and radiologists had different strengths: AI was more sensitive (84.6% vs 71.8%), whereas radiologists were more accurate at identifying an aneurysm (92.7% vs 78.2%). The study found that AI was particularly useful in higher-acuity settings, with the greatest benefit seen among inpatients and emergency department patients, while the benefit was more modest in outpatient settings.
The scientists involved said further research would be necessary to determine whether increased detection improves patient outcomes.
A brain aneurysm is a weakened area in the wall of a blood vessel in the brain that can bulge outward. Unruptured brain aneurysms are relatively common, although many cause no symptoms and are discovered incidentally.
However, when an aneurysm ruptures, it can cause bleeding around the brain – a subarachnoid haemorrhage, which is a medical emergency.
Radiologists can often detect brain aneurysms on CT imaging, particularly with CT angiography (CTA), which allows assessment of the aneurysm’s location, size, shape and relationship to nearby blood vessels, helping determine appropriate clinical management.
Accurate detection of brain aneurysms on CT scans is crucial because early identification can help guide timely treatment and reduce the risk of potentially life-threatening complications.
The latest findings, published in the Journal of the American College of Radiology, now suggest that AI could complement – rather than replace – radiologists when interpreting brain imaging.
The prospective study involved 3 856 CTA examinations performed across the Northwell Health system in New York. Researchers evaluated an AI algorithm developed by Aidoc, called aiOS, which is designed to detect intracranial aneurysms.
Importantly, the AI analysed scans in parallel with routine clinical care, but radiologists did not have access to its results when initially interpreting the examinations.
This allowed the researchers to assess how the technology performed in a real-world clinical environment, gauge how many additional aneurysms it could potentially identify, and determine the added value of combining AI with physician interpretation.
Overall, radiologists and the AI algorithm agreed in more than 96% of examinations.
“AI performance in clinical practice requires pre-deployment validation to understand not only the accuracy of the tool itself, but to test the expected operational utility in clinical practice,” said lead study author Shlomit Stein, MD, FACR, Professor of Radiology at the Zucker School of Medicine at Hofstra/Northwell and Director of Artificial Intelligence in the Department of Radiology at Northwell Health.
“We found that AI identified 55 aneurysms not detected by initial radiologists, and that the operational metrics of the AI tool were favourable, overall. AI tools can enhance the detection of findings by radiologists. In our study, the AI tool achieved a 39% relative enhanced detection rate, suggesting radiologists using AI will outperform those who do not.”
Algorithm identified small aneurysms missed by radiologists
The AI was more sensitive than radiologists alone, detecting 84.6% of aneurysms compared with 71.8% for radiologists, meaning it found more aneurysms that were truly present.
However, radiologists were more likely to be correct when they reported that an aneurysm was present. Their positive predictive value was 92.7%, compared with 78.2% for the AI.
Both radiologists and AI had similar strengths in ruling out aneurysms when none was present and in avoiding false alarms.
Notably, some of the additional aneurysms identified by the AI were among the smallest lesions. As such, this could provide clinicians with an opportunity to assess the patient’s risk and determine whether monitoring or treatment is appropriate, before a life-threatening haemorrhage occurs.
“The AI tool demonstrated high sensitivity for aneurysm detection which in fact surpassed that of radiologists, implying a potentially high Radiologist-AI collaborative detection rate,” Stein told Medical News Today.
“AI was able to identify mostly small aneurysms missed by radiologists. Despite their small size, they may nevertheless be clinically important since risk is not only a function of aneurysm size, but also shape, location, and other patient-related risk factors,” she noted.
AI and radiologists may have complementary strengths
The researchers emphasised that the findings do not suggest that AI should replace radiologists. Instead, the results indicate that the two approaches may identify different aneurysms.
The AI detected 55 aneurysms that radiologists missed, while radiologists identified 30 that the AI did not detect.
Of 101 findings identified only by the AI, 46 were ultimately determined to be false positives. Despite this, the researchers found that the number of additional true aneurysm detections enhanced overall detection performance and outweighed the false-positive findings. Additionally, radiologists identified important aneurysms that the algorithm missed.
They suggest that this illustrates the complementary strengths of combining radiologists and AI.
The researchers also found that the usefulness of the AI differed depending on where patients were receiving care.
The strongest performance was observed among inpatient examinations. In this setting, the AI identified 18 additional aneurysms while producing seven false-positive alerts. The technology also performed favourably in the emergency department.
However, its benefit was more limited in outpatient care, where the AI identified four additional aneurysms but generated more false-positive than true-positive findings.
The researchers suggest that differences in patient populations and examination complexity could help explain these findings. Higher-acuity inpatient and emergency settings may involve more clinically complex cases, offering more opportunities for AI to serve as a complementary detection tool.
Could AI provide an additional safety net in radiology?
The study provides evidence that evaluating medical AI in routine clinical settings may reveal strengths and weaknesses that are not apparent during initial testing.
An algorithm can perform well in a controlled validation study but behave differently when exposed to the variety of patients, imaging equipment, clinical indications, and workflows found in everyday healthcare. As such, it is important to assess AI according to whether it improves physician performance and patient care in real-world settings.
The researchers also emphasised the importance of continued monitoring after AI systems are introduced into clinical practice.
The findings suggest that AI could serve as an additional layer of review when radiologists interpret brain CTA examinations.
While the technology was not perfect, radiologists also missed aneurysms that the AI identified. Rather than indicating that one approach is superior, the results point toward a potential benefit from combining the two.
For patients, the potential advantage is that an aneurysm that might otherwise go unnoticed could receive further clinical attention.
“Aneurysms carry the risk of rupture and potentially catastrophic brain haemorrhage. Some of these detected aneurysms will therefore require surveillance or preventative intervention. The first step is aneurysm detection, which we have shown can be aided by the use of AI,” Stein said.
However, the study does not establish that AI-assisted detection ultimately leads to better patient outcomes. Further research is still necessary to determine whether increased detection translates into meaningful reductions in aneurysm rupture or other complications.
For now, the research adds to evidence that AI may be most useful in radiology when it works alongside clinicians, helping identify findings that might otherwise be overlooked while leaving final interpretation and clinical decision making to physicians.
Study details
Prospective Shadow-Mode Evaluation of an Artificial Intelligence Tool for Intracranial Aneurysm Detection on CT Angiography: Incremental Yield and Operational Impact
Shlomit Goldberg-Stein, Maria Sanmartin, Rachel Saks et al.
Published in Journal of the American College of Radiology on 15 September 2026
Abstract
Objective
To evaluate the operational performance of an FDA-cleared artificial intelligence (AI) algorithm for brain CT angiography (CTA) aneurysm detection and the incremental value of combining AI with radiologists.
Methods
Prospective shadow-mode study of consecutive brain CTAs (November 7, 2023, to December 19, 2023) was performed with radiologists blinded to AI. Aidoc’s (Tel Aviv, Israel) AI algorithm processed CTAs, and natural language processing extracted radiology report results. Discordances underwent neuroradiologist adjudication with AI unblinding. Performance metrics included AI: radiologist incremental detection ratio, relative enhanced detection rate (rEDR), gain-to-pain ratio (GPR), and number-needed-to-examine (NNE). Performance metrics are estimated under a hybrid reference standard wherein only discordances are adjudicated.
Results
Among 3,856 CTAs, examination positive rate was 5.1% (195 of 3,856). Radiologist-AI concordance was 96.3% (3,714 of 3,856). Sensitivity of AI alone (0.846, 0.787-0.894) exceeded radiologist alone (0.718, 0.649-0.779), with similar specificity (radiologist: 0.985, 0.981-0.989; AI: 0.987, 0.983-0.991). AI surfaced additional aneurysms not identified by radiologists, yielding an rEDR of 39% (55 AI-only true-positives per 140 radiologist true-positives) and projecting a higher combined sensitivity for radiologists with AI. AI:radiologist incremental detection ratio was 1.83 (55 of 30), favorable for AI. Operationally, GPR was favorable at 1.20 (55 of 46), and NNE was 70.1 (3,856 of 55). Operational metrics were most favourable in inpatients (rEDR 78.3%, GPR 2.57, NNE 28.9) and emergency (rEDR 37.1%, GPR 1.0, NNE 85.3) and were unfavorable in outpatients (rEDR 14.1%, GPR 0.67, NNE 130.3). Most AI-only aneurysms were <3 mm (31 of 55, 56.4%); radiologist-only were mostly 3 to 5 mm (18 of 33, 54.5%).
Conclusions
The AI tool demonstrated favourable operational performance, supporting clinical deployment in real-world practice, although setting-specific metrics were variable.
See more from MedicalBrief archives:
AI changing radiology, but not replacing human input
Will AI replace radiologists, or make them even better?
Research shows aneurysms can develop recently