Mohammad Nizam Mokhtar1, Farah Hanim Abdullah2, Mohd Ramzisham Abdul Rahman3, Joanna Su Min Ooi4
Author affiliations:
Background: The original EuroSCORE was proposed in 1995, with a predicted mortality rate of 3.9%. The study aimed to assess the validity of the European System for Cardiac Operative Risk Evaluation-II (EuroSCORE II), presented in 2012, in predicting 30-day in-hospital mortality with the observed mortality in cardiac surgical patients operated in Universiti Kebangsaan Malaysia Medical Centre (UKMMC).
Methodology: It was a retrospective study on patients who had undergone coronary artery bypass graft surgery during 6 y from January 01, 2011 to December 31, 2016, in UKMMC. Comparison between the output of the EuroSCORE II obtained and compared with the patients’ actual postoperative outcome.
Results: The actual in-hospital mortality rate was 6.8%. In comparison, the predicted mortality rate by the median EuroSCORE II was 1.23%. Receiver operating characteristics (ROC) curve analysis showed an excellent discriminatory power with the area under the curve (AUC) of 0.844 (95% CI 0.705 - 0.983, P < 0.001) between the survivors and non-survivors.
Conclusion: This single-center retrospective study of validation showed that the overall observed mortality was under-predicted by EuroSCORE II. However, it demonstrated a good calibration with excellent discriminatory power in predicting 30 days in-hospital mortality risk among patients undergoing coronary artery bypass graft surgery.
Abbreviations: BMI: Body Mass Index; CCS: Canadian Cardiovascular Society; LVEF: Left Ventricular Ejection Fraction; NYHA: New York Heart Association; PA: Pulmonary Artery
Key words: Risk assessment; Hospital mortality; Cardiac surgery
Citation: Mokhtar MN, Abdullah FH, Rahman MRA, Ooi JSM. Evaluation of cardiac operative risk and outcome using EuroSCORE II for coronary artery bypass graft surgery. Anaesth. pain intensive care 2023;27(3):308−314; DOI: 10.35975/apic.v27i3.2228
Received: November 15, 2022; Reviewed: April 17, 2023; Accepted: April 19, 2023
Cardiac surgery has always been associated with a higher peri-operative risk compared to other types of surgery. Crude mortality rates have been used as an indicator of the quality of care but their value is limited without considering the patients’ pre-morbid conditions and risk profiles.1 Performing risk stratification on patients undergoing cardiac surgery is important not only to identify high-risk cases for the selection of appropriate treatment options to improve surgical outcomes, and but
also in the allocation of healthcare resources. The cost of performing particularly resource-intense and high-risk operations such as coronary artery bypass graft (CABG) and heart valve surgery can vary enormously between patients with an uncomplicated recovery and those who suffer from complications.2, 3
To predict the risks of cardiac surgery, several risk stratification models have been developed over the past few decades. These models have played an important role in guiding clinical decision-making, when obtaining informed consent and benchmarking clinical services.4 Combining the most important pre-operative risk factors, the EuroSCORE model (owned by Royal Papworth Hospital) was developed in 1999.5 The accuracy of this initial additive EuroSCORE model tends to underestimate some very high-risk groups of patients which led to the development of the logistic EuroSCORE version in 2003.6 Although this version has been used as the risk prediction model for European cardiac surgery for many years, it was also found to over-predict the risk of hospital mortality as the outcome of cardiac surgery have substantially improved with better current surgical practices.5−7 An updated risk stratification model, EuroSCORE II, was launched in 2011 with additional preoperative risk variables as shown in Table 1. Its online calculator (www.euroscore.org) has also been updated to be used in this new stratification model.8, 9
However, it is important to realize that majority of the data obtained in the development of EuroSCORE II was from the European countries.10 Therefore, cautious clinical considerations are required for the application of EuroSCORE II in other populations as there are other interrelated factors such as the genetic background of the population, different healthcare systems as well as different social and cultural practices.
Based on the above, the primary aim of this study was to assess the validation of EuroSCORE II in patients undergoing isolated CABG in Universiti Kebangsaan Malaysia Medical Centre (UKMMC) and to assess the differences in the observed and predicted mortality for patients undergoing isolated CABG and to verify the calibration and discrimination power of EuroSCORE II.
This study was submitted and approved by the Research Committee of the Department of Anesthesiology & Intensive Care, UKMMC and the Medical Research & Ethics Committee, UKMMC (Approval/Reference Num.: UKM PPI/111/8/JEP-2017-339). Permission to assess patients’ records was obtained from the Health Information Department, UKMMC.
This was a retrospective study whereby records of all patients who had undergone CABG surgery from January 01, 2011 to December 31, 2016 in UKMMC were reviewed. A total of 235 patients who underwent cardiac surgery during this period were identified through our local hospital electronic database [Caring Hospital Enterprise System (C-HEtS)] and patients medical records from the Health Information Department, UKMMC. Out of this, a total of 205 patients who fulfilled our inclusion criteria; isolated CABG surgery on cardiopulmonary bypass with cardioplegia, were identified. Fifteen patients were further excluded due to incomplete or missing data making the final number of patients to be 190 patients (Figure 1). Data collected included age, gender, race, ASA status, body mass index, history of smoking, hypertension, diabetes mellitus, chronic obstructive pulmonary disease, renal dysfunction, NYHA class, left ventricular function, history of pulmonary hypertension, recent acute myocardial infarction and duration of hospital stay as well as the outcome at 30 days (survived or otherwise). All information gathered for every patient was entered into the EuroSCORE II calculator system obtained from the website www.euroscore.org). Comparison between the obtained output of EuroSCORE II and patients’ actual outcomes were analyzed. Thirty-day mortality was taken as the primary outcome (hospital mortality or death within 30 days postoperatively).
Statistical Analysis
Statistical analysis was performed using SPSS software version 23 (IBM Corp, Chicago, USA). Descriptive statistics were used to summarize the data. Continued variables were presented as mean and standard deviation. The difference between observed and predicted mortality was tested with Fisher’s exact test. A P < 0.05 was considered statistically significant. The association between the overall EuroSCORE II score and in-hospital mortality were assessed through the ROC curve for discrimination analysis. The area under the ROC curve (from 0 to 1.0) correlated with the discriminatory capability of the model. In general, the larger the area under the curve, the better the discriminatory power. The calibration of the EuroSCORE II model was analyzed using the Hosmer-Lemeshow goodness-of-fit test.
The demographic and pre-operative characteristics of patients are shown in Table 2. The mean age was 59.2 ± 8.8 y, predominantly males (88.9%) and Malays being the majority ethnic group (60.0%), which corresponded to the race distribution in Malaysia. The mean BMI was 26.5 ± 3.4 kg/m2, and 67.9% of the patients were overweight or obese. Hypertension was found to be the major co-morbidity (88.9%) followed by diabetes mellitus (64.7%). Pre-operatively, most of the patients were from NYHA Class II (76.3%) with good left ventricular function with an ejection fraction of > 50% (71.6%). The majority of the patients had moderate renal function based on creatinine clearance (46.3%). Out of the 190 patients, more than half suffered from a recent myocardial infarction (57.4%) before surgery (Table 2).
The actual in-hospital mortality rate was 6.8% (13 out of 190 patients). Most of our non-survivors had multiple co-existing diseases such as hypertension (69.2%), diabetes (53.8%) and renal failure (61.5%). In comparison, the predicted mortality rate by the median EuroSCORE II was 1.23% (1st quartile: 0.810, 3rd quartile: 2.315). This showed that the predicted in-hospital mortality rate was lower compared to the observed mortality rate. The actual mortality rate by quartiles of EuroSCORE II was 2.1% in the 1st and the 2nd quartile, 2.0% in the 3rd quartile and 21.3% in the 4th quartile as shown in Table 3. EuroSCORE II data were further grouped according to the predicted risk; low risk (< 2%), moderate risk (2−5%) and high-risk groups (> 5%). The risk category between observed and predicted mortality is further tabulated in Table 4.
Calibration is important to determine the agreement between the real observed and the predicted mortality. The Hosmer-Lemeshow goodness-of-fit test showed there was no difference between predicted and observed mortality according to the EuroSCORE II model (Chi-square = 4.903, P = 0.768), thus indicating good calibration of this model in predicting the overall in-hospital mortality.
The ROC curve analysis was performed to estimate the discrimination ability of the risk-scoring model in predicting 30 days of in-hospital mortality. The discrimination performance is important to determine how the model distinguishes between survivors and non-survivors during the in-hospital period. EuroSCORE II showed an excellent discriminatory power with an AUC of 0.844 (95% CI 0.705-0.983, P < 0.001) to discriminate between the non-survivors and survivors (Figure 2).
Although no single statistical test can be used to validate a risk prediction model, various tests can be used
together to describe model performance, which in turn indicates how useful the model is.
In our study, we included the calibration power using Hosmer-Lemeshow goodness-of-fit test as well as the discriminatory power showed by the ROC curve (AUC = 0.844) for our local patients who underwent CABG surgery. Our results showed that EuroSCORE II has a good calibration as well as excellent discriminatory power in this group of patients who underwent CABG surgery. The discriminative power is thought to be excellent if AUC is > 0.80, very good if > 0.75 and good (acceptable) if > 0.70.11 With good calibration power, this shows agreement between the observed and predicted mortality and thus reflects the accuracy of the model. Our study showed excellent discrimination power and this shows that the EuroSCORE II was able to distinguish between low-risk and high-risk patients. This meant that most deaths occurred in patients where EuroSCORE II correctly identified as high risk.
Various studies have been conducted to assess the validity of EuroSCORE II in predicting in-hospital mortality post-CABG surgery. Based on multiple validated studies conducted worldwide, the observed mortality ranged from 5.4%, and 6.3% in Egypt and Turkey respectively.12, 13 In our study, the observed in-hospital mortality rate was 6.8% which was similar and consistent to these previous studies as most of our non-survivors had multiple co-existing diseases thus, putting them at higher risk of mortality.
In comparison with other validation studies across Europe in terms of calibration and discrimination power, most of the studies have an AUC of more than 0.8. Koszta et al.12 demonstrated an excellent discrimination power with an AUC of 0.802 even though their cohort included patients who underwent valvular and aortic surgical procedures as well.12 In a large series involving 23,740 patients from a multi-center clinical audit data from the Society for Cardiothoracic Surgery in Great Britain and Ireland database, they found that EuroSCORE II showed excellent discrimination (AUC of 0.808, 95% CI 0.793 – 0.824) in all sub-groups analyzed. However, the calibration was found to be poor for isolated CABG surgery (Hosmer-Lemeshow, P <0.001).4 A retrospective EuroSCORE II validation study which was performed on patients who underwent isolated CABG surgery in Finland involving over 1,027 patients, also showed an excellent discrimination power with an AUC of 0.852.14 However, Amr et al. assessed the accuracy of EuroSCORE II in 580 patients who underwent mitral valve replacement which showed poor calibration and discriminative power with an AUC of 0.52. They attributed these differences to demographic-related factors in the Egyptian population who were younger, more prevalent of the female gender and had higher BMI.15
Our study demonstrated that EuroSCORE II underestimated our observed mortality rate (1.23% vs 6.8%), which was consistent with other similar studies. Koszta et al. found that the observed mortality for CABG surgery was 2.1% while the predicted mortality was 2.8%.12 Among the Indian population, Borde et al. similarly demonstrated an underestimation of risk by EuroSCORE II with the actual observed mortality for CABG surgery which was 6.6% while the predicted mortality was 3.1%.16 Another recent comparable study with similar ethnic background conducted in an East Java tertiary hospital revealed similar results with predicted and actual mortalities of 1.74% and 8.9% respectively.17
In our cohort, when the risk was further classified into low, moderate and high-risk groups, we found that mortality was over-predicted in the low-risk group and under-predicted in the high-risk group (Table 4). A local study done at the National Heart Institute of Kuala Lumpur involving 1718 patients, who underwent CABG surgery also demonstrated an over-prediction of the low-risk group (observed: 4.1%, predicted: 4.7%) and under-prediction of the high-risk group (observed: 38%, predicted: 4.8%).18 The similarity of patient demographics, dietary intake, lifestyle and age at onset of disease makes these comparable to ours and thus consistent with our findings. The over-prediction and under-prediction of lower scores and higher scores respectively is one of the limitations that we noticed for EuroSCORE II. One of the possible explanations why the score over-predicted mortality at lower score is because the model was developed using data from patients who underwent surgery before 2011. With the advancement of medical technology and treatment modalities over time, the outcome of cardiac surgery may have improved which may affect the accuracy of the model in predicting mortality at lower scores. Furthermore, EuroSCORE II does not take into account all relevant factors that could affect mortality such as socioeconomic status or frailty status. Patients who have a high EuroSCORE II with poor premorbids is expected to have higher frailty index. As frailty index is not part of the EuroSCORE II, this might has have contributed to why patients with higher score were underpredicted of their mortality.
A valid risk model should exhibit both robust discrimination and calibration to accurately predict mortality outcomes.19 The original authors of EuroSCORE II had estimated their ability to discriminate with a ROC curve AUC of 0.81 across all procedures.9 Despite good calibration and discriminatory power in our study, we have only focused on isolated CABG compared to the original EuroSCORE II study. This result questions the validity of using the EuroSCORE II outside of isolated CABG surgery in our local population. Thus, further analysis outside of isolated CABG surgery would yield more information on the validity of EuroSCORE II and also the need for a Malaysia-specific cardiac surgical risk scoring system.
The main limitation in our study was its retrospective model and the data was obtained from a single-center, therefore results may not be representative of whole od our country or region. Our sample size was also small as it was confined only to patients who underwent CABG surgery and did not include the other subgroups of other cardiac surgical procedures.
This single-center retrospective study of validation showed that the overall mortality was under-estimated by the EuroSCORE II. On the other hand, we demonstrated a good calibration and have an excellent discriminatory power in predicting 30 days in-hospital mortality risk among patients undergoing CABG surgery. Based on our finding we agree EuroSCORE II is a good predictive tool for predicting mortality.
7. Availability of data
The numerical data generated in the conduct of this study is available with the corresponding author.
8. Funding
No financial support for the research, authorship and publication of this article.
9. Conflict of interest
No potential conflict of interest relevant to this article.
10. Authors’ contribution
JSMO: Principle investigator, supervising the project and final editor of the manuscript
MNM: Preparation, creation and writing the initial draft
MRAR: Provision of study materials, patients and analysis tools
FHA: Conceptualisation with ideas, formulation of research gaols and aims
Author affiliations:
- Mohammad Nizam Mokhtar, MBChB, Department of Anesthesiology and Intensive Care, Universiti Kebangsaan Malaysia Medical Centre, Kuala Lumpur, Malaysia; E-Mail: drnizam@ukm.edu.my
- Farah Hanim Abdullah, MBBS, Department of Anesthesiology and Intensive Care, Universiti Kebangsaan Malaysia Medical Centre, Kuala Lumpur, Malaysia; E-Mail: farahha0301@gmail.com
- Mohd Ramzisham Abdul Rahman, MS, Division of Cardiothoracic Surgery, Heart and Lung Centre, Department of Surgery, Universiti Kebangsaan Malaysia Medical Centre, Kuala Lumpur, Malaysia; E-Mail: drramzi@ymail.com
- Joanna Su Min Ooi, MMed, Department of Anesthesiology and Intensive Care, Universiti Kebangsaan Malaysia Medical Centre, Kuala Lumpur, E-Mail: joanna@ppukm.ukm.edu.my
ABSTRACT
Background: The original EuroSCORE was proposed in 1995, with a predicted mortality rate of 3.9%. The study aimed to assess the validity of the European System for Cardiac Operative Risk Evaluation-II (EuroSCORE II), presented in 2012, in predicting 30-day in-hospital mortality with the observed mortality in cardiac surgical patients operated in Universiti Kebangsaan Malaysia Medical Centre (UKMMC).
Methodology: It was a retrospective study on patients who had undergone coronary artery bypass graft surgery during 6 y from January 01, 2011 to December 31, 2016, in UKMMC. Comparison between the output of the EuroSCORE II obtained and compared with the patients’ actual postoperative outcome.
Results: The actual in-hospital mortality rate was 6.8%. In comparison, the predicted mortality rate by the median EuroSCORE II was 1.23%. Receiver operating characteristics (ROC) curve analysis showed an excellent discriminatory power with the area under the curve (AUC) of 0.844 (95% CI 0.705 - 0.983, P < 0.001) between the survivors and non-survivors.
Conclusion: This single-center retrospective study of validation showed that the overall observed mortality was under-predicted by EuroSCORE II. However, it demonstrated a good calibration with excellent discriminatory power in predicting 30 days in-hospital mortality risk among patients undergoing coronary artery bypass graft surgery.
Abbreviations: BMI: Body Mass Index; CCS: Canadian Cardiovascular Society; LVEF: Left Ventricular Ejection Fraction; NYHA: New York Heart Association; PA: Pulmonary Artery
Key words: Risk assessment; Hospital mortality; Cardiac surgery
Citation: Mokhtar MN, Abdullah FH, Rahman MRA, Ooi JSM. Evaluation of cardiac operative risk and outcome using EuroSCORE II for coronary artery bypass graft surgery. Anaesth. pain intensive care 2023;27(3):308−314; DOI: 10.35975/apic.v27i3.2228
Received: November 15, 2022; Reviewed: April 17, 2023; Accepted: April 19, 2023
1. INTRODUCTION
Cardiac surgery has always been associated with a higher peri-operative risk compared to other types of surgery. Crude mortality rates have been used as an indicator of the quality of care but their value is limited without considering the patients’ pre-morbid conditions and risk profiles.1 Performing risk stratification on patients undergoing cardiac surgery is important not only to identify high-risk cases for the selection of appropriate treatment options to improve surgical outcomes, and but
| Table 1: EuroSCORE II | ||
| Patient-related factors | ||
| Age (y) | ||
| Gender | Male | Female |
| Chronic lung disease | Yes | No |
| Extracardiac arteriopathy | Yes | No |
| Poor mobility | Yes | No |
| Previous cardiac surgery | Yes | No |
| Active endocarditis | Yes | No |
| Critical preop state | Yes | No |
| Renal impairment | Normal (CC > 85 ml/min) | |
| Moderate (CC 50−85 ml/min) | ||
| Severe (CC < 50 ml/min) | ||
| Dialysis (Regardless of CC) | ||
| Diabetes on insulin | Yes | No |
| Cardiac-related factors | ||
| CCS angina class 4 | Yes | No |
| LV function | Good (LVEF > 50%) | |
| Moderate (LVEF 31−50%) | ||
| Poor (LVEF 21−30%) | ||
| Very Poor (LVEF ≤ 20%) | ||
| Recent MI | Yes | No |
| Pulmonary hypertension | No | |
| Moderate (PA systolic 32−55 mmHg) | ||
| Severe (PA systolic > 55 mmHg) | ||
| NYHA class | Class I | |
| Class II | ||
| Class III | ||
| Class IV | ||
| Operation-related factors | ||
| Surgery on thoracic aorta | Yes | No |
| Urgency of operation | Elective | |
| Urgency | ||
| Emergency | ||
| Salvage | ||
| Weight of operation | Isolated CABG | |
| Single non-CABG | ||
| 2 procedures | ||
| 3 procedures | ||
| Abbreviations: CC: Creatinine clearance; CCS: Canadian Cardio-
vascular Society; LV: Left ventricular; LVEF: Left ventricular ejection fraction, MI: Myocardial infarction; NYHA: New York Heart Association. |
||
also in the allocation of healthcare resources. The cost of performing particularly resource-intense and high-risk operations such as coronary artery bypass graft (CABG) and heart valve surgery can vary enormously between patients with an uncomplicated recovery and those who suffer from complications.2, 3
To predict the risks of cardiac surgery, several risk stratification models have been developed over the past few decades. These models have played an important role in guiding clinical decision-making, when obtaining informed consent and benchmarking clinical services.4 Combining the most important pre-operative risk factors, the EuroSCORE model (owned by Royal Papworth Hospital) was developed in 1999.5 The accuracy of this initial additive EuroSCORE model tends to underestimate some very high-risk groups of patients which led to the development of the logistic EuroSCORE version in 2003.6 Although this version has been used as the risk prediction model for European cardiac surgery for many years, it was also found to over-predict the risk of hospital mortality as the outcome of cardiac surgery have substantially improved with better current surgical practices.5−7 An updated risk stratification model, EuroSCORE II, was launched in 2011 with additional preoperative risk variables as shown in Table 1. Its online calculator (www.euroscore.org) has also been updated to be used in this new stratification model.8, 9
However, it is important to realize that majority of the data obtained in the development of EuroSCORE II was from the European countries.10 Therefore, cautious clinical considerations are required for the application of EuroSCORE II in other populations as there are other interrelated factors such as the genetic background of the population, different healthcare systems as well as different social and cultural practices.
Based on the above, the primary aim of this study was to assess the validation of EuroSCORE II in patients undergoing isolated CABG in Universiti Kebangsaan Malaysia Medical Centre (UKMMC) and to assess the differences in the observed and predicted mortality for patients undergoing isolated CABG and to verify the calibration and discrimination power of EuroSCORE II.
2. METHODOLOGY
This study was submitted and approved by the Research Committee of the Department of Anesthesiology & Intensive Care, UKMMC and the Medical Research & Ethics Committee, UKMMC (Approval/Reference Num.: UKM PPI/111/8/JEP-2017-339). Permission to assess patients’ records was obtained from the Health Information Department, UKMMC.
This was a retrospective study whereby records of all patients who had undergone CABG surgery from January 01, 2011 to December 31, 2016 in UKMMC were reviewed. A total of 235 patients who underwent cardiac surgery during this period were identified through our local hospital electronic database [Caring Hospital Enterprise System (C-HEtS)] and patients medical records from the Health Information Department, UKMMC. Out of this, a total of 205 patients who fulfilled our inclusion criteria; isolated CABG surgery on cardiopulmonary bypass with cardioplegia, were identified. Fifteen patients were further excluded due to incomplete or missing data making the final number of patients to be 190 patients (Figure 1). Data collected included age, gender, race, ASA status, body mass index, history of smoking, hypertension, diabetes mellitus, chronic obstructive pulmonary disease, renal dysfunction, NYHA class, left ventricular function, history of pulmonary hypertension, recent acute myocardial infarction and duration of hospital stay as well as the outcome at 30 days (survived or otherwise). All information gathered for every patient was entered into the EuroSCORE II calculator system obtained from the website www.euroscore.org). Comparison between the obtained output of EuroSCORE II and patients’ actual outcomes were analyzed. Thirty-day mortality was taken as the primary outcome (hospital mortality or death within 30 days postoperatively).
Statistical Analysis
Statistical analysis was performed using SPSS software version 23 (IBM Corp, Chicago, USA). Descriptive statistics were used to summarize the data. Continued variables were presented as mean and standard deviation. The difference between observed and predicted mortality was tested with Fisher’s exact test. A P < 0.05 was considered statistically significant. The association between the overall EuroSCORE II score and in-hospital mortality were assessed through the ROC curve for discrimination analysis. The area under the ROC curve (from 0 to 1.0) correlated with the discriminatory capability of the model. In general, the larger the area under the curve, the better the discriminatory power. The calibration of the EuroSCORE II model was analyzed using the Hosmer-Lemeshow goodness-of-fit test.
3. RESULTS
The demographic and pre-operative characteristics of patients are shown in Table 2. The mean age was 59.2 ± 8.8 y, predominantly males (88.9%) and Malays being the majority ethnic group (60.0%), which corresponded to the race distribution in Malaysia. The mean BMI was 26.5 ± 3.4 kg/m2, and 67.9% of the patients were overweight or obese. Hypertension was found to be the major co-morbidity (88.9%) followed by diabetes mellitus (64.7%). Pre-operatively, most of the patients were from NYHA Class II (76.3%) with good left ventricular function with an ejection fraction of > 50% (71.6%). The majority of the patients had moderate renal function based on creatinine clearance (46.3%). Out of the 190 patients, more than half suffered from a recent myocardial infarction (57.4%) before surgery (Table 2).
| Table 2. Demographic and pre-operative characteristics. | ||
| Variables
|
Study population
(n = 190) |
|
| Age (y) | 59.2 ± 8.8 [31−80] | |
| Gender |
Male Female |
169 (88.9) 21 (11.1) |
| Race | Malay Chinese Indian Others |
114 (60.0) 57 (30.0) 15 (7.9) 4 (2.1) |
| BMI, kg/m2 | 26.5 ± 3.4 [17.9−39.2] | |
| BMI class |
Underweight Normal Overweight or obese |
3 (1.6) 58 (30.5) 129 (67.9) |
| Co-morbids | Hypertension | 169 (88.9) |
| Diabetes mellitus (oral therapy only) | 123 (64.7) | |
| Diabetes on insulin | 40 (21.1) | |
| NYHA |
NYHA I NYHA II NYHA III NYHA IV |
14 (7.4) 145 (76.3) 30 (15.8) 1 (0.5) |
| LVEF |
LVEF > 50% LVEF 31-50% LVEF 21-30% LVEF < 21% |
136 (71.6) 48 (25.3) 5 (2.6) 1 (0.5) |
| Creatinine clearance (ml/min) |
Normal (>85) Moderate (50-85) Severe (<50) On dialysis |
64 (33.7) 88 (46.3) 34 (17.9) 4 (2.1) |
| Other co-morbids | Extracardiac arteriopathy | 4 (2.1) |
| Poor mobility | 9 (4.7) | |
| Pulmonary disease | 4 (2.1) | |
| Critical pre-operative state | 5 (2.6) | |
| CCS class IV angina | 108 (56.8) | |
| Recent myocardial infarction | 109 (57.4) | |
| Pulmonary hypertension |
PA systolic <31mmHg, PA systolic 31-55 mmHg PA systolic >55 mmHg |
190 (100) 0 (0.0) 0 (0.0) |
| Urgency |
Elective Urgent Emergency Salvage |
140 (73.7) 41 (21.6) 9 (4.7) 0 (0.0) |
| EuroSCORE II (median & interquartile range) | 1.23 [0.5−25.2] | |
| Mortality | 13 (6.8) | |
| Abbreviations: CCS: Canadian Cardiovascular Society, LVEF: Left ventricular ejection fraction, NYHA: New York Heart Association, PA: Pulmonary artery.
Values are expressed in mean ± SD [range] or numbers (percentage). |
||
The actual in-hospital mortality rate was 6.8% (13 out of 190 patients). Most of our non-survivors had multiple co-existing diseases such as hypertension (69.2%), diabetes (53.8%) and renal failure (61.5%). In comparison, the predicted mortality rate by the median EuroSCORE II was 1.23% (1st quartile: 0.810, 3rd quartile: 2.315). This showed that the predicted in-hospital mortality rate was lower compared to the observed mortality rate. The actual mortality rate by quartiles of EuroSCORE II was 2.1% in the 1st and the 2nd quartile, 2.0% in the 3rd quartile and 21.3% in the 4th quartile as shown in Table 3. EuroSCORE II data were further grouped according to the predicted risk; low risk (< 2%), moderate risk (2−5%) and high-risk groups (> 5%). The risk category between observed and predicted mortality is further tabulated in Table 4.
Calibration is important to determine the agreement between the real observed and the predicted mortality. The Hosmer-Lemeshow goodness-of-fit test showed there was no difference between predicted and observed mortality according to the EuroSCORE II model (Chi-square = 4.903, P = 0.768), thus indicating good calibration of this model in predicting the overall in-hospital mortality.
The ROC curve analysis was performed to estimate the discrimination ability of the risk-scoring model in predicting 30 days of in-hospital mortality. The discrimination performance is important to determine how the model distinguishes between survivors and non-survivors during the in-hospital period. EuroSCORE II showed an excellent discriminatory power with an AUC of 0.844 (95% CI 0.705-0.983, P < 0.001) to discriminate between the non-survivors and survivors (Figure 2).
4. DISCUSSION
Although no single statistical test can be used to validate a risk prediction model, various tests can be used
| Table 3: Quartiles of EuroSCORE II. Values are expressed in numbers (percentages). | ||||
| Mortality | Quartiles of EuroSCORE II | |||
| (0−0.81) | (0.82−1.23) | (1.24−2.32) | (> 2.32) | |
| No | 47 (98.0) | 46 (97.9) | 47 (98.0) | 37 (78.7) |
| Yes | 1 (2.1) | 1 (2.1) | 1 (2.0) | 10 (21.3) |
| Total | 48 | 47 | 48 | 47 |
| Table 4: Cross-tabulation analysis based on EuroSCORE II predicted risk category. Values are expressed in numbers (percentages). | |||||
| Mortality
Predicted Risk |
Yes | No | Total patients | ||
| Observed | Expected | Observed | Expected | ||
| Low (< 2%) | 3 (2.3) | 8.8 | 126 (97.6) | 120.2 | 129 |
| Moderate (2-5%) | 1 (2.3) | 3 | 43 (97.7) | 41.5 | 44 |
| High (> 5%) | 9 (52.9) | 1.2 | 8 (47.1) | 15.8 | 17 |
| Total | 13 (6.8) | 177 (93.2) | 190 | ||
together to describe model performance, which in turn indicates how useful the model is.
In our study, we included the calibration power using Hosmer-Lemeshow goodness-of-fit test as well as the discriminatory power showed by the ROC curve (AUC = 0.844) for our local patients who underwent CABG surgery. Our results showed that EuroSCORE II has a good calibration as well as excellent discriminatory power in this group of patients who underwent CABG surgery. The discriminative power is thought to be excellent if AUC is > 0.80, very good if > 0.75 and good (acceptable) if > 0.70.11 With good calibration power, this shows agreement between the observed and predicted mortality and thus reflects the accuracy of the model. Our study showed excellent discrimination power and this shows that the EuroSCORE II was able to distinguish between low-risk and high-risk patients. This meant that most deaths occurred in patients where EuroSCORE II correctly identified as high risk.
Various studies have been conducted to assess the validity of EuroSCORE II in predicting in-hospital mortality post-CABG surgery. Based on multiple validated studies conducted worldwide, the observed mortality ranged from 5.4%, and 6.3% in Egypt and Turkey respectively.12, 13 In our study, the observed in-hospital mortality rate was 6.8% which was similar and consistent to these previous studies as most of our non-survivors had multiple co-existing diseases thus, putting them at higher risk of mortality.
In comparison with other validation studies across Europe in terms of calibration and discrimination power, most of the studies have an AUC of more than 0.8. Koszta et al.12 demonstrated an excellent discrimination power with an AUC of 0.802 even though their cohort included patients who underwent valvular and aortic surgical procedures as well.12 In a large series involving 23,740 patients from a multi-center clinical audit data from the Society for Cardiothoracic Surgery in Great Britain and Ireland database, they found that EuroSCORE II showed excellent discrimination (AUC of 0.808, 95% CI 0.793 – 0.824) in all sub-groups analyzed. However, the calibration was found to be poor for isolated CABG surgery (Hosmer-Lemeshow, P <0.001).4 A retrospective EuroSCORE II validation study which was performed on patients who underwent isolated CABG surgery in Finland involving over 1,027 patients, also showed an excellent discrimination power with an AUC of 0.852.14 However, Amr et al. assessed the accuracy of EuroSCORE II in 580 patients who underwent mitral valve replacement which showed poor calibration and discriminative power with an AUC of 0.52. They attributed these differences to demographic-related factors in the Egyptian population who were younger, more prevalent of the female gender and had higher BMI.15
Our study demonstrated that EuroSCORE II underestimated our observed mortality rate (1.23% vs 6.8%), which was consistent with other similar studies. Koszta et al. found that the observed mortality for CABG surgery was 2.1% while the predicted mortality was 2.8%.12 Among the Indian population, Borde et al. similarly demonstrated an underestimation of risk by EuroSCORE II with the actual observed mortality for CABG surgery which was 6.6% while the predicted mortality was 3.1%.16 Another recent comparable study with similar ethnic background conducted in an East Java tertiary hospital revealed similar results with predicted and actual mortalities of 1.74% and 8.9% respectively.17
In our cohort, when the risk was further classified into low, moderate and high-risk groups, we found that mortality was over-predicted in the low-risk group and under-predicted in the high-risk group (Table 4). A local study done at the National Heart Institute of Kuala Lumpur involving 1718 patients, who underwent CABG surgery also demonstrated an over-prediction of the low-risk group (observed: 4.1%, predicted: 4.7%) and under-prediction of the high-risk group (observed: 38%, predicted: 4.8%).18 The similarity of patient demographics, dietary intake, lifestyle and age at onset of disease makes these comparable to ours and thus consistent with our findings. The over-prediction and under-prediction of lower scores and higher scores respectively is one of the limitations that we noticed for EuroSCORE II. One of the possible explanations why the score over-predicted mortality at lower score is because the model was developed using data from patients who underwent surgery before 2011. With the advancement of medical technology and treatment modalities over time, the outcome of cardiac surgery may have improved which may affect the accuracy of the model in predicting mortality at lower scores. Furthermore, EuroSCORE II does not take into account all relevant factors that could affect mortality such as socioeconomic status or frailty status. Patients who have a high EuroSCORE II with poor premorbids is expected to have higher frailty index. As frailty index is not part of the EuroSCORE II, this might has have contributed to why patients with higher score were underpredicted of their mortality.
A valid risk model should exhibit both robust discrimination and calibration to accurately predict mortality outcomes.19 The original authors of EuroSCORE II had estimated their ability to discriminate with a ROC curve AUC of 0.81 across all procedures.9 Despite good calibration and discriminatory power in our study, we have only focused on isolated CABG compared to the original EuroSCORE II study. This result questions the validity of using the EuroSCORE II outside of isolated CABG surgery in our local population. Thus, further analysis outside of isolated CABG surgery would yield more information on the validity of EuroSCORE II and also the need for a Malaysia-specific cardiac surgical risk scoring system.
5. LIMITATIONS
The main limitation in our study was its retrospective model and the data was obtained from a single-center, therefore results may not be representative of whole od our country or region. Our sample size was also small as it was confined only to patients who underwent CABG surgery and did not include the other subgroups of other cardiac surgical procedures.
6. CONCLUSION
This single-center retrospective study of validation showed that the overall mortality was under-estimated by the EuroSCORE II. On the other hand, we demonstrated a good calibration and have an excellent discriminatory power in predicting 30 days in-hospital mortality risk among patients undergoing CABG surgery. Based on our finding we agree EuroSCORE II is a good predictive tool for predicting mortality.
7. Availability of data
The numerical data generated in the conduct of this study is available with the corresponding author.
8. Funding
No financial support for the research, authorship and publication of this article.
9. Conflict of interest
No potential conflict of interest relevant to this article.
10. Authors’ contribution
JSMO: Principle investigator, supervising the project and final editor of the manuscript
MNM: Preparation, creation and writing the initial draft
MRAR: Provision of study materials, patients and analysis tools
FHA: Conceptualisation with ideas, formulation of research gaols and aims
11. REFERENCES
- Hansen LS, Hjortdal VE, Andreasen JJ, Mortensen PE, Jakobsen CJ. 30-day mortality after coronary artery bypass grafting and valve surgery has greatly improved over the last decade, but the 1-year mortality remains constant. Ann Card Anaesth. 2015 Apr-Jun;18(2):138-42. [PubMed] DOI: 4103/0971-9784.154462
- Kurki TS, Häkkinen U, Lauharanta J, Rämö J, Leijala M. Evaluation of the relationship between preoperative risk scores, postoperative and total length of stays and hospital costs in coronary bypass surgery. Eur J Cardiothorac Surg. 2001;20(6):1183-7. [PubMed] DOI: 1016/s1010-7940(01)00988-5
- Au W, Sun M, Lam K, Cheng L, Chiu S, Das S. Mortality prediction in adult cardiac surgery patients: comparison of two risk stratification models. Hong Kong Med J. 2007;13(4):293-7. [PubMed]
- Grant SW, Hickey GL, Dimarakis I, Trivedi U, Bryan A, Treasure T, et al. How does EuroSCORE II perform in UK cardiac surgery; an analysis of 23 740 patients from the Society for Cardiothoracic Surgery in Great Britain and Ireland National Database. Heart. 2012;98(21):1568-72. [PubMed] DOI: 1136/heartjnl-2012-302483
- Nashef SA, Roques F, Michel P, Gauducheau E, Lemeshow S, Salamon R, et al. European system for cardiac operative risk evaluation (Euro SCORE). Eur J Cardiothorac Surg. 1999;16(1):9-13. [PubMed] DOI: 1016/s1010-7940(99)00134-7
- Roques F, Michel P, Goldstone A, Nashef S. The logistic euroscore. Eur Heart J. 2003;24(9):882-3. [PubMed] DOI: 1016/s0195-668x(02)00799-6
- Bhatti F, Grayson AD, Grotte G, Fabri BM, Au J, Jones M, et al. The logistic EuroSCORE in cardiac surgery: how well does it predict operative risk? Heart. 2006;92(12):1817-20. [PubMed] DOI: 1136/hrt.2005.083204
- Noyez L, Kievit P, Van Swieten H, de Boer MJ. Cardiac operative risk evaluation: The EuroSCORE II, does it make a real difference? Neth Heart J. 2012;20(12):494-8. [PubMed] DOI: 1007/s12471-012-0327-1
- Nashef SAM, Roques F, Sharples LD, Nilsson J, Smith C, Goldstone AR, et al. EuroSCORE II. Eur J Cardiothorac Surg. 2012;41(4):734-44. [PubMed] DOI: 1093/ejcts/ezs043
- Granton J, Cheng D, eds. Risk stratification models for cardiac surgery. Seminars in cardiothoracic and vascular anesthesia. SAGE Publications Sage CA: Los Angeles, CA; 2008.
- Nezic D, Borzanovic M, Spasic T, Vukovic P. Calibration of the EuroSCORE II risk stratification model: is the Hosmer–Lemeshow test acceptable any more? Eur J Cardiothorac Surg. 2013;43(1):206. [PubMed] DOI: 1093/ejcts/ezs346
- Koszta G, Sira G, Szatmári K, Farkas E, Szerafin T, Fülesdi B. Performance of EuroSCORE II in Hungary: a single-centre validation study. Heart Lung Circ. 2014;23(11):1041-50. [PubMed] DOI: 1016/j.hlc.2014.04.005
- Iyem H. Evaluation of the reliability of the EuroSCORE risk analysis prediction in high-risk older patients undergoing CABG. Cardiovasc J Afr. 2009;20(6):340-3. [PubMed]
- Biancari F, Vasques F, Mikkola R, Martin M, Lahtinen J, Heikkinen J. Validation of EuroSCORE II in patients undergoing coronary artery bypass surgery. Ann Thorac Surg. 2012;93(6):1930-5. [PubMed] DOI: 1016/j.athoracsur.2012.02.064
- Amr MA, El-shorbagy AA. Evaluation of accuracy of EuroSCORE II in prediction of in-hospital mortality in patients underwent mitral valve replacement in Egypt. J Egypt Soc Cardiothorac 2016;24(2):135-42. DOI: 10.1016/j.jescts.2016.08.008
- Borde D, Gandhe U, Hargave N, Pandey K, Khullar V. The application of European system for cardiac operative risk evaluation II (EuroSCORE II) and Society of Thoracic Surgeons (STS) risk-score for risk stratification in Indian patients undergoing cardiac surgery. Ann Card Anaesth. 2013 Jul-Sep;16(3):163-6. [PubMed] DOI: 4103/0971-9784.114234
- Sembiring YE, Ginting A, Puruhito, Budiono. Validation of EuroSCORE II to predict mortality in post-cardiac surgery patients in East Java tertiary hospital. Med J Indonesia. 2021;30(1):54-9. DOI: https://doi.org/10.13181/mji.oa.204536
- Musa AF, Cheong XP, Dillon J, Nordin RB. Validation of EuroSCORE II in patients undergoing coronary artery bypass grafting (CABG) surgery at the National Heart Institute, Kuala Lumpur: a retrospective review. 2018 May 2;7:534. [PubMed] DOI: 10.12688/f1000research.14760.2
- Alba AC, Agoritsas T, Walsh M, Hanna S, Iorio A, Devereaux P, et al. Discrimination and calibration of clinical prediction models: users’ guides to the medical literature. JAMA. 2017 Oct 10;318(14):1377-1384. [PubMed] DOI: 10.1001/jama.2017.12126