Evaluating machine learning models for triage in the emergency room: A case study at Bang Khla Hospital

Abstract

Emergency room triage is a critical process that significantly impacts patient outcomes and resource allocation. The application of Machine Learning (ML) in this domain has garnered substantial interest due to its potential to enhance decision-making accuracy and efficiency. This study aims to develop and evaluate an ML model for classifying patient urgency levels in an emergency room setting. We conducted a retrospective analysis using data from patients admitted to Bang Khla Hospital, Chachoengsao, Thailand, between October 1, 2021, and August 30, 2024. From an initial dataset of 61,602 records, 29,389 were retained after data cleaning and preprocessing. These records were categorized into four urgency levels according to the Emergency Severity Index (ESI): resuscitate, emergent, urgent, and semi-urgent. The Random Forest model developed in this study achieved an overall accuracy of 98.67%, with an F1-Score of 1.00 for both resuscitate and emergent levels. However, the model faced challenges in classifying the urgent level, achieving an F1-Score of 0.86. These findings demonstrate the potential of ML in improving triage accuracy in emergency rooms, which could lead to more efficient patient prioritization and resource management. Nonetheless, further research is necessary to enhance the model's performance in classifying complex cases and to evaluate its implementation in real-world clinical settings. This study contributes to the growing body of evidence supporting the integration of ML technologies in emergency healthcare services.

References

C. B. Soster, F. Anschau, N. H. Rodrigues, L. G. A. da Silva, and A. Klafke, "Advanced triage protocols in the emergency service: A systematic review and meta-analysis," Rev. Latino-Am. Enfer magem, vol. 30, no. e3511, 2022, doi: 10.1590/1518- 8345.5479.3511.

N. Gilboy, P. Tanabe, D. Travers, and A. M. Rosenau, "Emergency Severity Index (ESI): A Triage Tool for Emergency Department Care, Version 4," Implementation Handbook. Agency for Healthcare Research and Quality, 2012.

Department of Health & Human Services (USA), Emergency Severity Index (ESI) A Triage Tool for Emergency Department Care, 4th ed. USA: AHRQ Publishing, 2011.

Y. Raita, T. Goto, M. Faridi, D. F. M. Brown, C. A. Camargo, and K. Hasegawa, "Emergency department triage prediction of clinical outcomes using machine learning models," Critical Care, vol. 23, no. 1, p . 64, 2019.

O. Salman, Z. Taha, M. Alsabah, Y. Soubhi, et al., "A review on utilizing machine learning technology for emergency department triage," Computer Methods and Programs in Biomedicine, vol. 203, pp. 106357, 2021.

T. Goto, C. Camargo, M. Faridi, R. Freishtat, et al., "Machine Learning–Based Prediction of Clinical Outcomes for Emergency Department Triage," JAMA Network Open, vol. 2, no. 9, p. e187354, 2019.

S. Tyler, M. Olis, N. Aust, L. Patel, L. Simon, C. Triantafyllidis, V. Patel, D. W. Lee, B. Ginsberg, H. Ahmad, and R. J. Jacobs, “Use of Artificial Intelligence in Triage in Hospital Emergency Departments: A Scoping Review,” Cureus, vol. 16, no. 5, p. e59906, May 2024.

N. Shafaf and H. Malek, “Applications of Machine Learning Approaches in Emergency Medicine: A Review Article,” Archives of Academic Emergency Medicine, vol. 7, no. 1, p. e34, July 2019.

F. Gao, B. Boukebous, M. Pozzar, E. Alaoui, B. Sano, and S. Bayat-Makoei, “Predictive Models for Emergency Department Triage using Machine Learning: A Review,” Obstetrics and Gynecology Research, vol. 5, no. 2, pp. 107–121, June 2022.

G. Feretzakis, G. Karlis, E. Loupelis, D. Kalles, R. Chatzikyriakou, N. Trakas, E. Karakou, A. Sakagianni, L. Tzelves, S. Petropoulou, A. Tika, I. Dalainas, and V. Kaldis, “Using Machine Learning Techniques to Predict Hospital Admission at the Emergency Department,” The Journal of Critical Care Medicine, vol. 8, no. 2, pp. 107–116, 2022.

J. Y. Yu, G. Y. Jeong, O. S. Jeong, D. K. Chang, and W. C. Cha, “Machine Learning and Initial Nursing Assessment-Based Triage System for Emergency Department,” Healthc. Inform. Res., vol. 26, no. 1, pp. 13–19, Jan. 2020

Levin S, Toerper M, Hamrock E, Hinson JS, Barnes S, Gardner H, Dugas A, Linton B, Kirsch T, Kelen G. Machine-learning-based electronic triage more accurately differentiates patients with respect to clinical outcomes compared with the emergency severity index. Ann Emerg Med 2018;71:565–574. e2. https://doi.org/ 10.1016/j.annemergmed.2017. 08.005.

A. Wright, M. McCoy, M. Henkin, et al., "Use of Machine Learning for Predicting Hospital Admissions and ED Resource Use," Acad. Emerg. Med., vol. 24, no. 8, pp. 893-902, 2017.

R. W. Handel, J. A. Ma, P. A. Hacke, K. B. Heller, and S. J. Kellermann, "Impact of a Machine Learning Model on Emergency Department Triage Accuracy: A Systematic Review," J. Emergency Med., vol. 56, no. 3, pp. 324-330, 2021.

M. J. Aljubran, H. J. Aljubran, M. Aljubran, M. Alkhalifah, M. Alkhalifah, and T. Alabdullah, "The utilizing of machine learning algorithms to improve triage in emergency departments: a retrospective observational study," Saudi J. Emerg. Med., vol. 4, no. 2, pp. 112–119, 2023. [Online]. Available: https:// doi.org/10.24911/SJEMed/72-1673885230&#8203