Personalization of the Radiology Residency Training Process Using Deep Learning and Interactive Extraction of Diagnostic Error Patterns

Abstract :

Radiological imaging is an essential diagnostic tool widely utilized by physicians for the diagnosis and monitoring of various medical conditions. Accurate interpretation of these images demands training of specialized residents and medical experts to ensure high diagnostic precision. This study presents an intelligent system that leverages machine learning to provide a personalized radiology residency training process through the interactive extraction of false-positive and false-negative diagnostic error patterns. The proposed approach is designed as an online personalized training platform that employs an iterative process to evaluate residents through adaptive tests generated by a question-selection algorithm based on each resident's performance, with the aim of extracting diagnostic error models of them independently. The proposed method employs a deep learning-based model to evaluate the difficulty level of educational radiographic images. Test questions for residents are then systematically designed to ensure an optimal distribution of disease classes and a diverse range of image difficulty levels. In subsequent tests, by analyzing each resident's mistakes in previous tests and finding their error patterns, the evaluation weight of each test image is determined according to each resident's mistakes in previous similar images. During this iterative process, each resident's training period is conducted taking into account their strengths and weaknesses in previous tests. Experimental results indicate significant improvements in diagnostic accuracy of residents across different disease classes and various difficulty levels, accompanied by remarkable reductions in false-positive and false-negative error rates. 

References:

[1] A. P. Brady, "Error and discrepancy in radiology: inevitable or avoidable?," (in eng), Insights Imaging, vol. 8, no. 1, pp. 171-182, 2017, doi: 10.1007/s13244-016-0534-1.
[2] S. Waite et al., "A Review of Perceptual Expertise in Radiology-How it develops, How we can test it, and Why humans still matter in the era of Artificial Intelligence," Academic Radiology, vol. 27, no. 1, pp. 26-38, 2020/01/01/ 2020, doi: https://doi.org/10.1016/j.acra.2019.08.018.
[3] J. Irvin et al., "CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparison," Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, pp. 590-597, 07/17 2019, doi: 10.1609/aaai.v33i01.3301590.
[4] M. Perumal, A. Nayak, R. P. Sree, and M. Srinivas, "INASNET: Automatic identification of coronavirus disease (COVID-19) based on chest X-ray using deep neural network," ISA Transactions, 2022/03/03/ 2022, doi: https://doi.org/10.1016/j.isatra.2022.02.033.
[5] B. Qi et al., GREN: Graph-Regularized Embedding Network for Weakly-Supervised Disease Localization in X-ray images. 2021.
[6] V. Sorin, Y. Barash, E. Konen, and E. Klang, "Deep Learning for Natural Language Processing in Radiology—Fundamentals and a Systematic Review," Journal of the American College of Radiology, vol. 17, no. 5, pp. 639-648, 2020/05/01/ 2020, doi: https://doi.org/10.1016/j.jacr.2019.12.026.
[7] L. Guo, A. Tahir, D. Zhang, Z. Wang, and R. Ward, Automatic Medical Report Generation: Methods and Applications. 2024.
[8] M. Mazurowski, J. Baker, H. Barnhart, and G. Tourassi, "Individualized computer-aided education in mammography based on user modeling: Concept and preliminary experiments," Medical physics, vol. 37, pp. 1152-60, 03/01 2010, doi: 10.1118/1.3301575.
[9] M. Wang, M. Wang, L. J. Grimm, and M. A. Mazurowski, "A computer vision-based algorithm to predict false positive errors in radiology trainees when interpreting digital breast tomosynthesis cases," Expert Systems with Applications, vol. 64, pp. 490-499, 2016/12/01/ 2016, doi: https://doi.org/10.1016/j.eswa.2016.08.023.
[10] M. Wang et al., "Predicting false negative errors in digital breast tomosynthesis among radiology trainees using a computer vision-based approach," Expert Systems with Applications, vol. 56, pp. 1-8, 2016/09/01/ 2016, doi: https://doi.org/10.1016/j.eswa.2016.01.053.
[11] S. Burti, A. Zotti, and T. Banzato, "Role of AI in diagnostic imaging error reduction," Frontiers in Veterinary Science, vol. 11, 08/30 2024, doi: 10.3389/fvets.2024.1437284.
[12] M. A. Mazurowski and G. D. Tourassi, "Exploring the potential of collaborative filtering for user-adaptive mammography education," in Proceedings of the 2011 Biomedical Sciences and Engineering Conference: Image Informatics and Analytics in Biomedicine, 15-17 March 2011 2011, pp. 1-4, doi: 10.1109/BSEC.2011.5872325.
[13] S. Voisin, F. Pinto, G. Morin-Ducote, K. B. Hudson, and G. D. Tourassi, "Predicting diagnostic error in radiology via eye-tracking and image analytics: preliminary investigation in mammography," (in eng), Med Phys, vol. 40, no. 10, p. 101906, Oct 2013, doi: 10.1118/1.4820536.
[14] H. Lin, X. Yang, and W. Wang, "A content-boosted collaborative filtering algorithm for personalized training in interpretation of radiological imaging," Journal of digital imaging, vol. 27, pp. 449-456, 2014.
[15] M. A. Mazurowski, J. Zhang, J. Y. Lo, C. M. Kuzmiak, S. V. Ghate, and S. Yoon, "Modeling resident error-making patterns in detection of mammographic masses using computer-extracted image features: preliminary experiments," in Medical Imaging 2014: Image Perception, Observer Performance, and Technology Assessment, 2014, vol. 9037: SPIE, pp. 197-202.
[16] J. Zhang, J. Y. Lo, C. M. Kuzmiak, S. V. Ghate, S. C. Yoon, and M. A. Mazurowski, "Using computer‐extracted image features for modeling of error‐making patterns in detection of mammographic masses among radiology residents," Medical physics, vol. 41, no. 9, p. 091907, 2014.
[17] J. Zhang, J. I. Silber, and M. A. Mazurowski, "Modeling false positive error making patterns in radiology trainees for improved mammography education," Journal of biomedical informatics, vol. 54, pp. 50-57, 2015.
[18] Z. Gandomkar, K. Tay, W. Ryder, P. Brennan, and C. Mello-Thoms, Predicting radiologists' true and false positive decisions in reading mammograms by using gaze parameters and image-based features. 2016, p. 978715.
[19] B. Ibragimov and C. Mello-Thoms, "The Use of Machine Learning in Eye Tracking Studies in Medical Imaging: A Review," IEEE journal of biomedical and health informatics, vol. PP, 02/29 2024, doi: 10.1109/JBHI.2024.3371893.
[20] H. Wainer, N. J. Dorans, R. Flaugher, B. F. Green, and R. J. Mislevy, Computerized Adaptive Testing: A Primer. Taylor & Francis, 2000.
[21] W. A. Sands, B. K. Waters, and J. R. McBride, Computerized adaptive testing: From inquiry to operation. American Psychological Association, 1997.
[22] E. E. Roskam and P. G. Jansen, "A new derivation of the Rasch model," in Advances in Psychology, vol. 20: Elsevier, 1984, pp. 293-307.
[23] A. D. Mead, "An introduction to multistage testing," Applied Measurement in Education, vol. 19, no. 3, pp. 185-187, 2006.
[24] Y. Zhuang, Q. Liu, Z. Huang, Z. Li, S. Shen, and H. Ma, "Fully adaptive framework: Neural computerized adaptive testing for online education," in Proceedings of the AAAI conference on artificial intelligence, 2022, vol. 36, no. 4, pp. 4734-4742.
[25] D. F. Mujtaba and N. R. Mahapatra, "Artificial intelligence in computerized adaptive testing," in 2020 International Conference on Computational Science and Computational Intelligence (CSCI), 2020: IEEE, pp. 649-654.
[26] D. Kermany. Large dataset of labeled optical coherence tomography (OCT) and Chest X-Ray images, Mendeley, 2018, doi: 10.17632/RSCBJBR9SJ.3.
[27] D. S. Kermany et al., "Identifying medical diagnoses and treatable diseases by image-based deep learning," Cell, vol. 172, no. 5, pp. 1122-1131.e9, 2018/2 2018, doi: 10.1016/j.cell.2018.02.010.
[28] J. Cohen, P. Morrison, L. Dao, K. Roth, T. Duong, and M. Ghassemi, COVID-19 Image Data Collection: Prospective Predictions Are the Future. 2020.
[29] A. Chung, "Actualmed COVID-19 chest x-ray data initiative," 2020. [Online]. Available: https://github.com/agchung/Actualmed-COVID-chestxray-dataset.
[30] A. Chung, "Figure 1 COVID-19 chest x-ray data initiative," 2020. [Online]. Available: https://github.com/agchung/Figure1-COVID-chestxray-dataset.
[31] P. Garg, M. Gautam, B. Chugh, and K. Dwivedi, "Employing transfer learning techniques for COVID-19 detection using chest X-ray," International Journal of Advances in Applied Sciences, vol. 13, p. 680, 09/01 2024, doi: 10.11591/ijaas.v13.i3.pp680-688.
[32] G. Huang, Z. Liu, L. V. D. Maaten, and K. Q. Weinberger, "Densely Connected Convolutional Networks," in 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 21-26 July 2017 2017, pp. 2261-2269, doi: 10.1109/CVPR.2017.243.
[33] J. Yu et al., "A Unified Adaptive Testing System Enabled by Hierarchical Structure Search," in Forty-first International Conference on Machine Learning.
[34] Q. Liu et al., "Survey of computerized Adaptive Testing: A machine learning perspective," 2024 2024, doi: 10.48550/ARXIV.2404.00712.
[35] S. J. Chen, A. Choi, and A. Darwiche, "Computer Adaptive Testing Using the Same-Decision Probability," in BMA@ UAI, 2015, pp. 34-43.
[36] J.-J. Vie, F. Popineau, É. Bruillard, and Y. Bourda, "A Review of Recent Advances in Adaptive Assessment," in Learning Analytics: Fundaments, Applications, and Trends: A View of the Current State of the Art to Enhance e-Learning, A. Peña-Ayala Ed. Cham: Springer International Publishing, 2017, pp. 113-142.