Tülin Akarsu Ayazoglu*
* Chief Assistant, Department of Anesthesiology and Reanimation, SB High Graduate Education and Training Hospital, Kartal /Kosuyolu, İstanbul (Türkiye)
Correspondence: Dr.Tülin Akarsu Ayazoglu, SB High Graduate Education and Training Hospital, Department of Anesthesiology and Reanimation, denizer caddesi cevizli kavşağı no:2 cevizli / kartal/Türkiye posta kodu: 34846, İstanbul (Türkiye); e-mail: akarsu.dr@gmail.com
ABSTRACT
Introduction: Stroke is the second major cause of death worldwide. APACHE IV is a successful scoring system assessing severity of illness and prognosis of ICU patients. The objective of this study was to compare APACHE IV scoring system for patients admitted with stroke with APACHE II scoring system Methodology: We included all patients with the diagnosis of stroke, who were admitted to intensive care unit of our hospital for tracheal intubation and mechanical ventilation, between 1 January 2008 and 1 February 2009 from prospectively collected ICU database. Observed mortality rates were compared with predicted mortality rates for both the APACHE IV and APACHE II scoring systems, SMR, sensitivity and specificity were determined. The mortality percentages were predicted using the APACHE IV system and were compared with the observed data. The statistical analysis was carried out using SPSS for Windows version 15.0. The qualitative variables were compared to a χ
2 (chi-squared) test.
Results: Fifty five patients were included in the study, with an average age of 76.5±11.5 years for male patients and 72±5 years for females. The overall mortality observed was 34.54% in all the patients (19/55 patients). Apache IV predicted mortality rate sensititivity and specificity were 94.7% and 94.4% respectively, SMR of 0.95 and diagnostics value was 94.5%. Apache II predicted mortality rate sensitivity and specificity were 100% and 86.1% respectively, SMR of 0.79 and diagnostics value was 90.9%.
Conclusion: Predicting outcome in stroke patients is difficult due to variability in etiology, presentation and underlying patho-physiology. We conclude that APACHE IV scoring system is better than the APACHE II system in predicting mortality rate in ICU stroke patients. APACHE IV (score of >84) gives probably a more reliable prediction of high risk of death in patients with stroke than APACHE II (score >24).
Key Words: Intensive care unit; mortality prediction; APACHE IV; APACHE II; stroke
Citation: Ayazoglu TA. A comparison of APACHE II and APACHE IV scoring systems in predicting outcome in patients admitted with stroke to an intensive care unit. Anaesth Pain & Intensive Care 2011;15(1):7-12.
INTRODUCTİON
Stroke is a major health problem and the second major cause of death worldwide. As the population ages, its significance will grow
1,2. The Oxford Vacular Study reported that the incidence of cerebrovascular events was 1.2-fold higher than coroner events
3. There are about 5.5 million deaths yearly and an estimated loss of 49 million disability adjusted life years worldwide
4,5. Stroke can occur at any age, but half of all strokes occur in people aged over 70 years. About 80% of all acute strokes are ischaemic, usually resulting from thrombotic or embolic occlusion of a cerebral artery
6.
The survival, recovery and final outcome in stroke patients depends on various variables such as neurological damage, age, hypertension, diabetes, smoking, atrial fibrillation (AF)
7,8 and social factors. Intensive care units (ICUs) have played a vital role in the practice of stroke patients. The ICU provides advanced and resource-intensive treatment for the sickest hospitalized patients. Critically ill patients frequently require mechanical ventilation, circulatory support, and other assist devices; but it is still not clear whether intensive care treatment does provide any help to patients with stroke, since most of them have a very poor prognosis despite intensive care treatment.
9-13
The use of scoring systems to predict risk of mortality and evaluating outcome in critically ill patients is important in modern evidence-based medicine. Clinicians can predict the outcome for patients, who are severely ill and for those who have a good prognosis. Measuring the severity of disease and prognosis in patients in the ICU is very important, because it effects the quality of patient care across ICUs, but this cannot be done without some objective index of disease severity. Predictive scoring systems can provide a stable fundamental principle and help clinical decision making. The other objective is to identify ICUs requiring longer or shorter length of stay (LOS). Accurate prediction of LOS of stroke patients in ICUs is critical to ICU outcome assessment, its resource management and floor management.
APACHE (Acute Physiology and Chronic Health Evaluation) scoring system
11 takes into consideration various parameters like physiological variables, vital signs, urine output, neurological score, along with age related parameters and comorbid conditions, which may have a significant impact on the outcome of these critically ill patients.
APACHE II has been used worldwide for measuring ICU performance
14,15. The system, outlined by Knaus
16 et al. in 1985, has been validated in many clinical trials, and is a commonly used ICU severity of illness estimation. APACHE II estimates risk, based on data available within the first 24 h of ICU stay.
APACHE III was developed in 1991
17 and this system was designed to predict an individual's risk of dying in a hospital. Disease-specific scoring systems have been developed for several important subgroups treated in the ICU. APACHE IV is the newest standardized scoring system to assess the severity of illness and prognosis in the ICU and new variables have been added to APACHE III like mechanical ventilation, thrombolysis, impact of sedation on Glasgow Coma Scale, rescaled Glasgow Coma Scale, PaO
2: FiO
2 and disease-specific subgroups.
18-20
We compared the performance of the APACHE IV system with APACHE II in stroke patients admitted to our ICU.
METHODOLOGY
This study was carried out at an 11 bedded ICU. Fifty five patients, ≥65 yrs of age, who had been admitted with stroke into the ICU, were included in the study. These patients were either admitted from emergency room or transfered from another hospital; evaluated clinically and CT scans were performed to confirm the diagnosis. The necessity of tracheal intubation and mechanical ventilation was the leading cause of admission to ICU. We included all ICU patients with the diagnosis of stroke between 1 January 2008 and 1 February 2009 from a prospectively collected ICU database.
The patients aged under 65 years or readmitted during the study period and those transferred from other ICUs or with a stay of less than 24 h were excluded.
On the day after ICU admission the worst values on APACHE IV and APACHE II variables(worst measurements observed during 24 h following ICU admission) were abstracted from clinical and laboratory records and APACHE scores were calculated using an online APACHE score calculator. Observed mortality rates were compared with predicted mortality rates for both the scoring systems and standardized mortality ratios (SMR) and sensitivity and specificity were determined. APACHE -IV predicted ICU-LOS of stroke patients were compared with observed ICU-LOS and days on mechanical ventilation.
Statistical analysis was carried out using a software package (SPSS for Windows; version 15.0) and p values less than 0.05 were considered significant. All data were tested for normal distribution with the Kolmogorov-Smirnov test before further statistical analysis. Differences between study groups were assessed using the Mann Whitney U test. The Wilcoxon signed rank test was used for paired comparisons of abnormal distribution variables into the groups. The qualitative variables was compared to a χ
2 (chi-squared) test.
Receiver operating characteristic (ROC)
21 curve is a graphical plot of the sensitivity, or true positive rate vs false positive rate (1−specificity or 1-true negative rate), for a binary classifier system as its discrimination threshold is varied. The ROC can also be represented equivalently by plotting the fraction of true positives out of the positives (TPR = true positive rate) vs. the fraction of false positives out of the negatives (FPR = false positive rate). The area under the ROC curve was measured to test discrimination.
The SMR with 95% confidence intervals were calculated and the differences between observed and predicted numbers of ICUs deaths were analyzed.
RESULTS
In this study the average age of male patients was 76.5±11.5 years and of female patients was 72±5 years. There was no difference between gender (p>0,05); but the age was the most significant factor for stroke associated mortality in both sexes (p=0,000) (Table 1).
Table 1. Demographic variables
| Gender |
Non-survivors |
Survivors |
p |
| N |
% |
N |
% |
| Female |
9 |
47,4 |
10 |
27,8 |
0,146 |
| Male |
10 |
52,6 |
26 |
72,2 |
| Total |
19 |
100 |
36 |
100 |
|
Mean±SD |
Range |
Mean±SD |
Range |
|
| Age (years) |
77,1±6,5 |
65–88 |
69,1±4,3 |
65–79 |
0,000 |
Twenty three patients had hemorrhagic infarction (41.8%) and thirty two had ischemic infarction (58.1%). Twelve patients out of 23 of the hemorrhagic group (52.17%) and seven out of 32(21.8%) in the ischemic infarction group died. The overall mortality observed was 34.54% in all the patients (19/55 patients) (Table 2).
Table 2. Stroke subtypes
| Subtypes |
N |
Non-survivors |
Survivors |
| Ischemic |
32 |
7/32 (21.8%) |
24 /32(78.2%) |
| Hemorrhagic |
23 |
12/23 (52.17%)** |
11/23 (47.83%) |
| Total |
55 |
19/55 |
36/55 |
**p < 0.01
Mean observed ICU-LOS (19±8 days) for non-survivors, and (16±6) for survivors was significantly greater than APACHE -IV predicted ICU-LOS. Length of ventilation period was18±8 days in ICU for non-survivors, and 13±7 days for survivors (p<0.05)(Table 3).
Table 3. APACHE-IV LOS ICU and ventilation period: Comparison of non-survivors and survivors
-
|
Non-survivors |
Survivors |
P |
| N |
Mean±SD |
Range |
N |
Mean±SD |
Range |
| LOS ICU day |
19 |
19±8 |
7-39 |
36 |
16±6 |
9-45 |
0.037 |
| LOS Vent.D |
19 |
18±8 |
7-39 |
36 |
13±7 |
6-45 |
0.012 |
| Predicted ICU LOS |
19 |
5.5±0.8 |
3.9-7.5 |
36 |
6±0.8 |
4.7-8 |
0.021 |
LOS: lenght of stay Vent.D: Ventilated Day
Predicted ICU lenght of stay was significantly short both non-survivors and survivors group p<0.05
Table 4. Comparison of non-survivors and survivors scoring systems
-
|
Non-survivors |
Survivors |
P |
| N |
Mean±SD |
Range |
N |
Mean±SD |
Range |
| APS score |
19 |
89.6±13.7 |
74.0-115.0 |
36 |
68.1±10.6 |
45.0-91.0 |
0.000 |
| AP II score |
19 |
28.9±3.7 |
25.0-40.0 |
36 |
21.4±3.1 |
14.0-27.0 |
0.000 |
AP II Pred
M.Rate |
19 |
0.66±0.10 |
0.53-0.91 |
36.00 |
0.41±0.10 |
0.19-0.61 |
0.000 |
| AP IV score |
19 |
105.4±14.9 |
84.0-139.0 |
36 |
79.9±11.6 |
50.0-103.0 |
0.000 |
AP IV Pred
M.Rate |
19 |
0.65±0.11 |
0.50-0.89 |
36.00 |
0.38±0.09 |
0.17-0.52 |
0.000 |
APACHE-IV, APS and APACHE-II scores were significantly elevated in non-survivors groups (p=0,000).
APS = Acute Physiology Score *APS is the acute physiology score derived from APACHE IV
AP II = Acute Physiology and Chronic Health Evaluation-II
AP IV= Acute Physiology and Chronic Health Evaluation-IV
Pred. M.Rate = Predicted mortality rate
APACHE IV, APS and APACHE II scores were significantly elevated in non-survivors groups (p=0.000)(Table4).
Tables 4, 5 and 6 provide patient data in relation to APACHE IV and II scores, observed deaths and predicted mortality rates.
The mean APACHE IV score was 88.7 (±17.6), sensitivity was 94.7%, specifity was 94.4%, diagnostics value was 94.5% and was SMR of 0.95. Mean APACHE II score was 24 (±4.9), sensitivity was 100%, specifity was 86.1, diagnostics value was 90.9% and was SMR of 0.79 .
Table 5: Apache-IV predicted mortality rate * Situation crosstabulation
|
Situation |
Total |
| Non-survivors |
Survivors |
Predicted
Apache IV |
deaths |
18 |
2 |
20 |
%36.3 |
| discharged |
1 |
34 |
35 |
|
| Observed |
19 |
36 |
55 |
%34.5 |
Sensitivity = 18 / 19 = 94,7% Specificity = 34 / 36 = 94,4% Diagnostics value ( 18+34) + 55= 94,5%
SMR 19/20=0.95
APACHE IV predicted deaths were 36.36% and APACHE II was 43.63%. Observed mortality rate was 34.54 ( Table 5 and 6). Observed and predicted mortality rate is shown figure 1.
Table 6: APACHE-II predicted mortality rate * Situation Crosstabulation
|
Situation |
Total |
| Non-survivors |
survivors |
Predicted
Apache II |
death |
19 |
5 |
24 |
43.6% |
| discharged |
0 |
31 |
31 |
|
| Observed |
19 |
36 |
55 |
34.5% |
The sensitivity = 19/19 = 100% Specificity = 31 / 36 = 86.1% Diagnostics value ( 19+31) / 55= 90.9%
SMR=19/24=0.79
The area under ROC curve was 93% for APACHE IV and 98% for APACHE II (Fig 2, 3)(Table 7). The predictability of APACHE II was more sensitive than APACHE IV but APACHE IV predictions were more selector and more reliable than APACHE II.
Table 7. Area Under the Curve; Test Result Variable(s):
|
Area |
Std. Error(a) |
Asymptotic Sig.(b) |
Asymptotic 95% Confidence Interval |
| APACHE IV Score |
.935 |
.033 |
.000 |
.871 |
.999 |
| APACHE II Score |
0.981 |
0.014 |
0.000 |
0.95 |
1.00 |
The square under the curve 93% (confidence interval 0.87 – 0.99; p<.001) was found APACHE IV. The distinction of non-survivors situation was 93%. The square under the curve 98% (confidence interval 0.95 – 1.00; p<.001) was found APACHE II. The distinction of non-survivors situation was 98%
Acute Physiology Score (APS) was derived from APACHE IV. Mean APS score was 75.5 (±15.5). APACHE IV and APACHE II APS were significantly different betweeen survivors and non-survivors groups (p=0.000). All scores were significantly higher in non-survivors. It was also observed that the likelihood of mortality increased as the score increased
DISCUSSION
In the ICU, risk adjustment and mortality prediction has usually been performed using severity score taxonomies such as the APACHE score, the Simplified Acute Physiology Score (SAPS) or the Mortality Prediction Model (MPM) and their updated derivatives
22. APACHE IV model is the most recent version and it used the same variables as APACHE III
21 but new variables added and disease-specific subgroups.
The results from our study demonstrate that the APACHE IV prognostic scoring system better predicts mortality rate than APACHE II scoring system.
Stroke severity at onset and patient age are the most important factors for predicting prognosis1. Burtin et al emphasized that age was the most significant independent risk factor for stroke-associated mortality in both sexes.11 In this study there was no difference between gender (p>0.05) but the age of non-survivors was seen to be more than the survivors (p=0.000, Table 1.)
The total mortality observed was 34.54%. The patients with hemorrhagic infarction group had a higher mortality (52.17%vs21.8%) than those with ischemic infarction (Table 2). Per Thorvaldsen et al reported that the case-fatality rates for stroke at 28 days varied from 15% to 49% among men and from 18% to 57% among women18. Bhalla A. et al. reported an overall mortality, of 34% in all stroke patients due to all causes 23. In our study mortality rate is similar to other studies.
The APACHE system is the only validated ICU risk-adjustment model that provides performance information about two separate outcomes of care, e.g. mortality and ICU length-of-stay (LOS). Prediction of duration of a patient’s stay in the ICU, however, is difficult and less studied than the prediction of mortality
24. Prolonged stay in the ICU not only increases the overall costs and consumes more resources, but also limits the number of beds available for use.
Kakar et al experienced that the predictive ICU length of stay and mortality percentage did not correlate in severe acute pancreatitis
25. We found that APACHE IV predicted ICU lenght of stay was not correlated and significantly short for both non-survivors and survivors groups (p<0.05)(Table 3).
APACHE IV, APS and APACHE II scores were elevated in non-survivors groups. It was observed that the likelihood of mortality increased as the score increased (Table 4).
Daley et al pointed out that APACHE II has been widely used for measuring ICU performance but this scoring system was not disease spesific26. Bhattacharyya et al found it to overestimate ICU performance and suggested that APACHE IV might be more relevant to estimate ICU performance27.
The SMR of 0.95 and predicted mortality rate sensitivity was 94.7% and the specificity was 94.4% for APACHE IV. SMR of 0.79 and predicted mortality rate sensitivity was 100% and the specificity was 86.1% for APACHE II . The correctness was 94.5% for APACHE IV and 90.9% for APACHE II.
We found that APACHE IV was more sensitive than APACHE II (Table 5-7, Figures 2,3)
APACHE IV scoring system better predicts mortality rate than APACHE II scoring system in our study, which may be the result of having disease-specific subgroups and including a specific reason for ICU admission in its risk prediction. Thus, this may be a better alternative and a good, effective predictor of short term outcome in elderly stroke patients in ICU.
CONCLUSION
Predicting outcome in stroke patients is difficult due to the variability in etiology, presentation and underlying patho-physiology. In this study, APACHE IV (score of >84.5) is probably a more reliable prediction of high risk of death in patients with stroke than APACHE II (score >25.5). APACHE IV score is a valid mode of predicting outcome in stroke patient, further comprehensive studies are needed to supplement our finding.
REFERENCES
1. Ingall T. Stroke-incidence, mortality, morbidity and risk. J. Insur. Med 2004;36(2):143-52
2. M Kelly-Hayes. Influence of age and health behaviours on stroke risk:lessons from longitudinal studies. Journal of
the American Geriatrics Society 2010;58:325-8.
3. Rothwell PM, Coull AJ, Silver LE, Fairhead JF, Giles MF et al. Population-based study of event-rate, incidence, case fatality, and mortality for all acute vascular events in all arterial territories (Oxford Vascular Study). Lancet 2005;366:1773–83
4. Feigin VL. Stroke epidemiology in the developing world. Lancet 2005; 25;365:2160-61
5. Cox AM, McKevitt C, Rudd AG, Wolfe CD. Socioeconomic status and stroke. Lancet Neurol 2006;5:181-8
6. Warburton E, Alawneh JA, Clatworthy PL, Morris RS. Stroke Managenent. Clin Evidence(Online). 2008;16:pii:0201(Abstract).
7. Weimar C, Ziegler A, König IR, Diener HC. Predicting functional outcome and survival after acute ischemic stroke. J Neurol. 2002;249(7):888-95.
8. König IR, Andreas Ziegler A, Bluhmki E, Hacke W, Bath PMW. Predicting long-term outcome after acute ischemic stroke: a simple index works in patients from controlled clinical trials. Stroke. 2008;39(6):1821-6.
9. Foerch C,Kessler KR,Steckel DA,Steinmetz H, Sitzer M. Survival and quality of life outcome after mechanical ventilation in elderly stroke patients. J Neurol Neurosurg Psychiatry 2004;75:988-93
10. R G. Holloway, CG. Benesch, WS. Burgin, JB. Zentner. Prognosis and Decision Making in Severe Stroke JAMA 2005;294:725-33
11. Burtin P, Bollaert PE, Feldmann L, Lelarge P, Bauer P, Larcan A. Prognosis of patients with stroke undergoing mechanical ventilation. Intensive Care Med. 1994;20:32-36.
12. Strand T, Asplund K, Eriksson S, Hägg E, Lithner F, Wester PO. Stroke unit care: who benefits? Comparisons with general medical care in relation to prognostic indicators on admission. Stroke.1986;17:377-81
13. RS.Howard, DM Kullmann, NP Hirsch. Admission to neurological intensive care: who, when, and why? J Neurol Neurosurg Psychiatry 2003;74:1112- 9
14. Gupta R, Arora VK. Performance evaluation of APACHE II score for an Indian patient with respiratory problems. Indian J Med Res 2004;119:273-82.
15. Ludwigs U, Csatlos M, Hulting J. Predicting in.hospital mortality in acute myocardial infarction: Impact of thrombolytic therapy on APACHE II performance. Scand Cardiovasc J 2000:34;371-6.
16. Knaus WA, Draper EA, Wagner DP, et al. APACHE II: A severity of disease classification system. Crit Care Med 1985;13:818-29
17.Knaus WA, Wagner DP, Draper EA et al. The APACHE III prognostic system. Risk prediction of hospital mortality for critically ill hospitalized adults. Chest 1991;100:1619–36.
18. Zimmerman JE, Kramer AA, McNair AA, Douglas S, FernR. Acute Physiology and Chronic Health Evaluation (APACHE). IV: ICU length of stay benchmarks for today’s critically ill patients. Crit Care Med 2006;34:2517-29
19. Zimmerman JE, Kramer AA, McNair DS, Malila FM. Crit Care Med. 2006;34:1297-310
20.Zimmerman JE, Kramer AA, McNair DS, Malila FM, Schaffer VL. Crit Care Med 2006 Oct;34(10):2517-29
21. Hanley JA, McNeil BJ. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology 1982;143(1):29-36
22 .PA Mendez Tellez, D.Todd. Predicting Patient Outcomes, Futility, and Resource Utilization in the Intensive Care Unit: The Role of Severity Scoring Systems and General Outcome Prediction Models.Mayo Clinic Proceedings 2005;80(2):161-323.
23. Bhalla A, Gupta OP, Gupta SB. Predicting mortality in stroke. Neurology India 2002;50(3): 279-81
24.Copeland.Fields L, Griffin T, Jenkins T, Buckley M, Wise L.C. Comparison of outcome prediction made by physicians, by nurses and by using the mortality prediction model. Am. J.Critical Care 2001;10:313–19
25.Kakar P,Govil D, Gupta S, Srinivasan S, Mehta P. Validation of APACHE IV in patients with severe acute pancreatitis. Critical Care 2008;12(Suppl 2):P500
26. Daley J, Jencks S, Draper D. Predicting hospital.associated mortality for medicare patients. A method for patients with stroke, pneumonia, acute myocardial infarction, and congestive heart failure. JAMA 1988;260: 3617-24
27. Bhattacharyya M, Todi S. APACHE IV: benchmarking in an Indian ICU. Critical Care 2009;13(Suppl 1):P510