(O-L4) Is Simple Clinical Judgment Superior to the Most Used Early Warning Scores for the Prediction of Death and ICU Admission in the Emergency Department? A Retrospective Study on 225,369 Cases

Objectives: This study aimed to compare the accuracy of the most used Early Warning Scores (EWS) to find the most accurate in the general ED population, and to compare their added value compared to the sole clinical judgment.

Background: Early warning scores (EWS) are physiological scoring systems based on a rapid and quantitative assessment of changes in vital signs. EWSs were initially developed to identify and track hospitalized patients at risk of deterioration outside critical care areas . However, it is still unknown if the application of EWS could provide better patient stratification compared to triage nurse clinical judgment .
To answer these questions, we conducted a study comparing the accuracy of the six most used EWS in a large ED patient dataset, evaluating the score's performance in addition to triage nurse clinical judgment.

Methods: This is a single-center, retrospective observational study. The digital records of all consecutive ED admissions ≥18 years from 2010 to 2019 were evaluated and six EWSs were calculated: NEWS, NEWS2, MEWS, RAPS, REMS, and SEWS. The discrimination performance of each EWS was calculated by ROC analysis. The primary endpoint was death/ICU admission within 24 hours. Secondary endpoints were the assessment of the added value of clinical judgment to each EWS, measured by the net reclassification improvement, and the evaluation of the relative weight of each EWS item on the prediction of the primary outcome, measured by neural network analysis.

Results: The study cohort consisted of 225,369 patients (median age was 49 [IQ range 35-68] years 47% male. Overall 1941 (0.9%) patients were admitted to ICU or died within 24 hours. NEWS was the most accurate predictor, with an AUROC of 0.904 [95% CI 0.805-0.913] similar to NEWS2 (AUROC 0.901). The remaining EWS had a significantly lower accuracy.
The NRI analysis confirmed that clinical judgment could add significant risk stratification to the baseline EWS. However, the discrimination increased by about 15/20% for NEWS and NEWS2, and just about 5% for REMS. NRI value was for NEWS 0.156 [0.14 – 0.17], NEWS2 0.178 [0.16 – 0.20], MEWS 0.116 [0.10 – 0.13], RAPS 0.085 [0.07 – 0.09], REMS 0.055 [0.04 – 0.07], and SEWS 0.109 [0.09 – 0.12].
Once divided into two groups by clinical judgment, the EWS accuracy in discriminating the events was different in nurse-alert patients, compared to non-alert ones (Figure).

Conclusions: All the EWS demonstrated good accuracy for the prediction of short-term poor outcomes, with the sole exception of RAPS. The NEWS had the better balance in discrimination and calibration both in patients selected by clinical judgment and in the general ED population. Although clinical judgment proved to be accurate for the prediction of adverse events, the EWS evaluation could improve risk stratification by up to 20%.

References (Optional): 1. Fu, L.H.; Schwartz, J.; Moy, A.; Knaplund, C.; Kang, M.J.; Schnock, K.O.; Garcia, J.P.; Jia, H.; Dykes, P.C.; Cato, K.; Albers, D.; Rossetti, S.C., Development and validation of early warning score system: A systematic literature review. Journal of biomedical informatics, 2020, 105, 103410.
2. Chapman, S.M.; Maconochie, I.K., Early warning scores in paediatrics: an overview. 2019, 104, (4), 395-399.
3. Mhyre, J.M.; D'Oria, R.; Hameed, A.B.; Lappen, J.R.; Holley, S.L.; Hunter, S.K.; Jones, R.L.; King, J.C.; D'Alton, M.E., The Maternal Early Warning Criteria: A Proposal from the National Partnership for Maternal Safety. Journal of Obstetric, Gynecologic & Neonatal Nursing, 2014, 43, (6), 771-779.
4. Nannan Panday, RS; Minderhoud, T.C.; Alam, N.; Nanayakkara, PWB, Prognostic value of early warning scores in the emergency department (ED) and acute medical unit (AMU): A narrative review. European journal of internal medicine, 2017, 45, 20-31.
5. Iversen AKS, Kristensen M, Østervig RM, Køber L, Sölétormos G, Lundager Forberg J, Eugen-Olsen J, Rasmussen LS, Schou M, Iversen KK. A simple clinical assessment is superior to systematic triage in prediction of mortality in the emergency department. Emerg Med J. 2019 Feb;36(2):66-71. doi: 10.1136/emermed-2016-206382. Epub 2018 Oct 16. PMID: 30327415.
6. Churpek, M.M.; Snyder, A.; Han, X.; Sokol, S.; Pettit, N.; Howell, M.D.; Edelson, D.P., Quick Sepsis-related Organ Failure Assessment, Systemic Inflammatory Response Syndrome, and Early Warning Scores for Detecting Clinical Deterioration in Infected Patients outside the Intensive Care Unit. American journal of respiratory and critical care medicine, 2017, 195, (7), 906-911.
7. Candel BGJ, de Groot B, Nissen SK, Thijssen WAMH, Lameijer H, Kellett J. The prediction of 24-h mortality by the respiratory rate and oxygenation index compared with National Early Warning Score in emergency department patients: an observational study. Eur J Emerg Med. 2022 Dec 5. doi: 10.1097/MEJ.0000000000000989. Epub ahead of print. PMID: 36729955.
8. Arévalo-Buitrago P, Morales-Cané I, Olivares Luque E, Godino-Rubio M, Rodríguez-Borrego MA, López-Soto PJ. Early detection of risk for clinical deterioration in emergency department patients: validation of a version of the National Early Warning Score 2 for use in Spain. Emergencias. 2022 Dec;34(6):452-457. English, Spanish. PMID: 36625695.
9. Covino, M.; Sandroni, C.; Santoro, M.; Sabia, L.; Simeoni, B.; Bocci, M.G.; Ojetti, V.; Candelli, M.; Antonelli, M.; Gasbarrini, A.; Franceschi, F., Predicting intensive care unit admission and death for COVID-19 patients in the emergency department using early warning scores. Resuscitation, 2020, 156, 84-91.

Funding (Optional): None