Problems of assessing the clinical efficiency of artificial intelligence systemsin diagnosing ischemic stroke
https://doi.org/10.15829/1560-4071-2025-6357
EDN: CBNZRM
Abstract
Artificial intelligence (AI) is actively used in the diagnosis of ischemic stroke, allowing speeding up the decision-making process and improving the accuracy of diagnosis. Machine learning models are able to identify ischemic areas based on computed tomography and magnetic resonance imaging data, as well as indicate the volume of damage and calculate the ASPECTS score. Modern AI systems demonstrate high diagnostic accuracy, comparable to the accuracy of radiologists. According to clinical studies, these systems significantly reduce the time from the patient admission to the vascular center to treatment, but their impact on clinical outcomes remains unclear. The review discusses the problems of assessing the clinical effectiveness of AI in the diagnosis of ischemic stroke, including systematic bias in model training and choice of study design, as well as publication bias. To integrate AI into clinical practice, randomized controlled trials with clinically relevant endpoints, as well as standardization of data and methods for assessing effectiveness, are needed. Despite significant AI advances in diagnosing ischemic stroke, their effectiveness in real-world practice requires further study and validation.
About the Author
I. O. Balunov
Pirogov Russian National Research Medical University
Russian Federation
Russian Federation
Moscow
References
1. Wolcott ZC, English SW. Artificial intelligence to enhance prehospital stroke diagnosis and triage: a perspective. Front Neurol. 2024;15:1389056. doi:10.3389/fneur.2024.1389056.
2. McCarthy JJ, Minsky M, Rochester N, Shannon CE. A Proposal for the Dartmouth Summer Research Project on Artificial Intelligence. Ai Magazine. 2006;27(4):12. doi:10.1609/aimag.v27i4.1904.
3. Adlung L, Cohen Y, Mor U, et al. Machine learning in clinical decision making. Med. 2021;2(6):642-65. doi:10.1016/j.medj.2021.04.006.
4. Lindroth H, Nalaie K, Raghu R, et al. Applied Artificial Intelligence in Healthcare: A Review of Computer Vision Technology Application in Hospital Settings. J Imaging. 2024;10(4):81. doi:10.3390/jimaging10040081.
5. Esteva A, Chou K, Yeung S, et al. Deep learning-enabled medical computer vision. NPJ Digit Med. 2021;4(1):5. doi:10.1038/s41746-020-00376-2.
6. Maegerlein C, Fischer J, Mönch S, et al. Automated Calculation of the Alberta Stroke Program Early CT Score: Feasibility and Reliability. Radiology. 2019;291(1):141-8. doi:10.1148/radiol.2019181228.
7. Guberina N, Dietrich U, Radbruch A, et al. Detection of early infarction signs with machine learning-based diagnosis by means of the Alberta Stroke Program Early CT score (ASPECTS) in the clinical routine. Neuroradiology. 2018;60(9):889-901. doi:10.1007/s00234-018-2066-5.
8. Adamou A, Beltsios ET, Bania A, et al. Artificial intelligence-driven ASPECTS for the detection of early stroke changes in non-contrast CT: a systematic review and meta-analysis. J Neurointerv Surg. 2023;15(e2):e298-e304. doi:10.1136/jnis-2022-01944.
9. Amukotuwa SA, Straka M, Smith H, et al. Automated Detection of Intracranial Large Vessel Occlusions on Computed Tomography Angiography: A Single Center Experience. Stroke. 2019;50(10):2790-8. doi:10.1161/STROKEAHA.119.026259.
10. Yahav-Dovrat A, Saban M, Merhav G, et al. Evaluation of Artificial Intelligence-Powered Identification of Large-Vessel Occlusions in a Comprehensive Stroke Center. AJNR Am J Neuroradiol. 2021;42(2):247-54. doi:10.3174/ajnr.A6923.
11. Martinez-Gutierrez JC, Kim Y, Salazar-Marioni S, et al. Automated Large Vessel Occlusion Detection Software and Thrombectomy Treatment Times: A Cluster Randomized Clinical Trial. JAMA Neurol. 2023;80(11):1182-90. doi:10.1001/jamaneurol.2023.3206.
12. Delora A, Hadjialiakbari C, Percenti E, et al. Viz LVO versus Rapid LVO in detection of large vessel occlusion on CT angiography for acute stroke. J Neurointerv Surg. 2024;16(6):599-602. doi:10.1136/jnis-2023-020445.
13. Weyland CS, Papanagiotou P, Schmitt N, et al. Hyperdense Artery Sign in Patients With Acute Ischemic Stroke-Automated Detection With Artificial Intelligence-Driven Software. Front Neurol. 2022;13:807145. doi:10.3389/fneur.2022.807145.
14. Kasasbeh AS, Christensen S, Parsons MW, et al. Artificial Neural Network Computer Tomography Perfusion Prediction of Ischemic Core. Stroke. 2019;50(6):1578-81. doi:10.1161/STROKEAHA.118.022649.
15. Vilela P. Acute stroke differential diagnosis: Stroke mimics. Eur J Radiol. 2017;96:133-44. doi:10.1016/j.ejrad.2017.05.008.
16. Bojsen JA, Elhakim MT, Graumann O, et al. Artificial intelligence for MRI stroke detection: a systematic review and meta-analysis. Insights Imaging. 2024;15(1):160. doi:10.1186/s13244-024-01723-7.
17. Miceli G, Basso MG, Rizzo G, et al. Artificial Intelligence in Acute Ischemic Stroke Subtypes According to Toast Classification: A Comprehensive Narrative Review. Biomedicines. 2023;11(4):1138. doi:10.3390/biomedicines11041138.
18. Newbury-Chaet I, Mercaldo SF, Chin JK, et al. Evaluation of an Artificial Intelligence Model for Identification of Mass Effect and Vasogenic Edema on CT of the Head. AJNR Am J Neuroradiol. 2024;45(10):1528-35. doi:10.3174/ajnr.A8358.
19. Zatcepin A, Kopczak A, Holzgreve A, et al. Machine learning-based approach reveals essential features for simplified TSPO PET quantification in ischemic stroke patients. Z Med Phys. 2024;34(2):218-30. doi:10.1016/j.zemedi.2022.11.008.
20. Cross JL, Choma MA, Onofrey JA. Bias in medical AI: Implications for clinical decision-making. PLOS Digit Health. 2024;3(11):e0000651. doi:10.1371/journal.pdig.0000651.
21. Hernandez-Boussard T, Bozkurt S, Ioannidis JPA, et al. MINIMAR (MINimum Information for Medical AI Reporting): Developing reporting standards for artificial intelligence in health care. J Am Med Inform Assoc. 2020;27(12):2011-5. doi:10.1093/jamia/ocaa088.
22. Liu X, Cruz Rivera S, Moher D, et al. Reporting guidelines for clinical trial reports for interventions involving artificial intelligence: the CONSORT-AI extension. Nat Med. 2020;26(9):1364-74. doi:10.1038/s41591-020-1034-x.
23. Cruz Rivera S, Liu X, Chan AW, et al. Guidelines for clinical trial protocols for interventions involving artificial intelligence: the SPIRIT-AI extension. Lancet Digit Health. 2020;2(10):e549-e560. doi:10.1016/S2589-7500(20)30219-3.
24. Tejani AS, Klontzas ME, Gatti AA, et al. Checklist for Artificial Intelligence in Medical Imaging (CLAIM): 2024 Update. Radiol Artif Intell. 2024;6(4):e240300. doi:10.1148/ryai.240300.
25. Akay EMZ, Hilbert A, Carlisle BG, et al. Artificial Intelligence for Clinical Decision Support in Acute Ischemic Stroke: A Systematic Review. Stroke. 2023;54(6):1505-16. doi:10.1161/STROKEAHA.122.041442.
26. Galanty M, Luitse D, Noteboom SH, et al. Assessing the documentation of publicly available medical image and signal datasets and their impact on bias using the BEAMRAD tool. Sci Rep. 2024;14(1):31846. doi:10.1038/s41598-024-83218-5.
27. Garin SP, Parekh VS, Sulam J, et al. Medical imaging data science competitions should report dataset demographics and evaluate for bias. Nat Med. 2023;29(5):1038-9. doi:10.1038/s41591-023-02264-0.
28. Obermeyer Z, Powers B, Vogeli C, et al. Dissecting racial bias in an algorithm used to manage the health of populations. Science. 2019;366(6464):447-53. doi:10.1126/science.aax2342.
29. Larrazabal AJ, Nieto N, Peterson V, et al. Gender imbalance in medical imaging datasets produces biased classifiers for computer-aided diagnosis. Proc Natl Acad Sci U S A. 2020;117(23):12592-4. doi:10.1073/pnas.1919012117.
30. Chilamkurthy S, Ghosh R, Tanamala S, et al. Development and validation of deep learning algorithms for detection of critical findings in head CT scans. arXiv preprint. 2018. doi:10.48550/arXiv.1803.05854.
31. Pearce FJ, Cruz Rivera S, Liu X, et al. The role of patient-reported outcome measures in trials of artificial intelligence health technologies: a systematic evaluation of ClinicalTrials.gov records (1997-2022). Lancet Digit Health. 2023;5(3):e160-e167. doi:10.1016/S2589-7500(22)00249-7.
32. van Leeuwen KG, Schalekamp S, Rutten MJCM, et al. Artificial intelligence in radiology: 100 commercially available products and their scientific evidence. Eur Radiol. 2021;31(6):3797-804. doi:10.1007/s00330-021-07892-z.
33. Kim DW, Jang HY, Kim KW, et al. Design Characteristics of Studies Reporting the Performance of Artificial Intelligence Algorithms for Diagnostic Analysis of Medical Images: Results from Recently Published Papers. Korean J Radiol. 2019;20(3):405-10. doi:10.3348/kjr.2019.0025.
34. Yu AC, Mohajer B, Eng J. External Validation of Deep Learning Algorithms for Radiologic Diagnosis: A Systematic Review. Radiol Artif Intell. 2022;4(3):e210064. doi:10.1148/ryai.210064.
35. Mikhail P, Le MGD, Mair G. Computational Image Analysis of Nonenhanced Computed Tomography for Acute Ischaemic Stroke: A Systematic Review. J Stroke Cerebrovasc Dis. 2020;29(5):104715. doi:10.1016/j.jstrokecerebrovasdis.2020.104715.
36. Soun JE, Zolyan A, McLouth J, et al. Impact of an automated large vessel occlusion detection tool on clinical workflow and patient outcomes. Front Neurol. 2023;14:1179250. doi:10.3389/fneur.2023.1179250.
37. Le NM, Iyyangar AS, Kim Y, et al. Machine Learning–Enabled Automated Large Vessel Occlusion Detection Improves Transfer Times at Primary Stroke Centers. Stroke: vascular and interventional neurology. 2024;4:e001119. doi:10.1161/svin.123.001119.
38. Westwood M, Ramaekers B, Grimm S, et al. Software with artificial intelligence-derived algorithms for analysing CT brain scans in people with a suspected acute stroke: a systematic review and cost-effectiveness analysis. Health Technol Assess. 2024;28(11):1-204. doi:10.3310/RDPA1487.
39. Medvedeva NA, Kazieva MYu, Sidorova AA, et al. Modern capabilities of an integrated artificial intelligence algorithm in the diagnosis of ischemic stroke and intracranial hemorrhages. Zhurnal Diagnosticheskaya i interventsionnaya radiologiya. 2024;18(2.1):152-9. (In Russ.)
40. Tyrov IA, Vasil’ev YuA, Arzamasov KM, et al. Assessment of the maturity of artificial intelligence technologies for healthcare: methodology and its application based on the materials of the Moscow experiment on computer vision in radiation diagnostics. Vrach i informatsionnye tekhnologii. 2022;4:76-92. (In Russ.) doi:10.25881/18110193_2022_4_76.
Supplementary files
Review
For citations:
Balunov I.O. Problems of assessing the clinical efficiency of artificial intelligence systemsin diagnosing ischemic stroke. Russian Journal of Cardiology. 2025;30(9S):6357. (In Russ.) https://doi.org/10.15829/1560-4071-2025-6357. EDN: CBNZRM
Views:
232
Email this article 