Introduction
The practice of Medicine at present demands quality care. Multiple indicators are utilized to measure the latter; however, the clinical results are, in the end, the most relevant.1 For some investigators, mortality is the first and most important indication of quality in the clinical practice.1 Crude or non-adjusted mortality has its limitations in that it does not consider, among other aspects, the clinical conditions of the sick person nor his/her comorbilities.1,2 In heart surgery, statistical programs have been developed from a group of risk factors; these factors are capable of predicting mortality.2 For comparing the mortality rate among diverse institutions or for temporary comparison at a same institution, it is recommended to employ some of the mathematical models developed that evaluate mortality adjusted to the clinical conditions and risk profile of the population cared for.1,2 During the last decade, the Parsonnet score, that of the U.S. Society of Thoracic Surgeons (STS), and the EuroSCORE3 score have been those most frequently utilized. The latter was obtained from the study of 19,030 adult patients submitted to heart surgery (ischemic, valvular, and congenital) with extracorporeal circulation pump in 128 hospitals in eight European countries (Germany, France, the U.K., Italy, Spain, Finland, Sweden, and Switzerland). In this study, based on its objectivity, reliability, and prevalence, the authors analyzed 97 variables or risk factors (68 preoperatory and 29 operatory) were analyzed, among which 18 remained with prognostic value.2-4 The EuroSCORE model has two variants: additive and logistic. The former awards a numerical value to each variable that the patient presents, and the sum of these values provides the probability of death.4 It entertains the advantage of being a simple and uncomplicated instrument that can be applied at the patient’s bedside. The logistic version is more sophisticated. The formula for obtaining the latter is the following: estimated mortality = e (β0+∑βiXi)/ 1+e(β0+∑βiXi), in which β0 is the constant of the logistic regression model (-4.789594) and βi is the coefficient of variable Xi of the logistic-regression information.5 The definition, the odds ratio (OR), the value of each variable of the additive model, and coefficient β of each variable of the logistic model appear in Table 1. To utilize a risk score or a predictive model developed to predict mortality in patients submitted to heart surgery, which has shown good performance in the group of patients where it was generated, thus proving its internal validity, it is recommended that it be evaluated at the hospitals where its use is intended to confirm or discard external validity, therefore using it or not.1,2,4 The objective of the present study was to know the risk profile of the adult population submitted to cardiac surgery and to validate its additive as well as in its logistic version in the population of adult patients submitted to cardiac surgery with and without extracorporeal circulation pump at the Hospital Regional de Alta Especialidad del Bajío (HRAEB) in the city of León, Guanajuato, Mexico.
Variable | Definition | OR | Points | β |
---|---|---|---|---|
Patient factors | ||||
Age | Every 5 years from the age of 60 years | 1.1 | 1 | 0.066635 |
Gender | Female | 1.4 | 1 | 0.330405 |
Serum creatinine | > 200 μmol/L | 1.9 | 2 | 0.652165 |
Extracardiac arteriopathy | Claudication of the lower limbs Carotid stenosis ≥ 50%, prior or planned vascular surgery on the abdominal aorta, carotids o peripheral arteries |
1.9 | 2 | 0.655892 |
Chronic lung disease | Requires prolongad treatment with bronchodilators or steroids | 1.6 | 1 | 0.493134 |
Neurological dysfunction | Neurological damage that severely affects walking or daily activity | 2.3 | 2 | 0.841626 |
Previous cardiac surgery | Cardiac surgery that required opening of the pericardium | 2.6 | 3 | 1.002625 |
Active endocarditis | The patient is under antibiotic treatment at the time of surgery | 2.5 | 3 | 1.101265 |
Critical preoperative state | One of the following conditions: ventricular tachycardia/ventricular fibrillation or sudden death recovered, preoperative cardiac massage, mechanical ventilation prior to anesthesia, preoperative inotropic, acute Preoperatory kidney failure (oliguria ≤ 10 mL/hour) |
2.2 | 3 | 0.905813 |
Cardiac factors | ||||
Unstable angina | Resting angina requiring intravenous nitrates until arrival at the operating room | 1.5 | 2 | 0.567708 |
Recent myocardial infarction | Previous myocardial infarction in the last 90 days | 1.6 | 2 | 0.546022 |
Expulsion fraction of the VI | Expulsion fraction less than 30% | 2.5 | 3 | 1.09443 |
Expulsion fraction of the VI | 30-50% | 1.5 | 1 | 0.419643 |
Pulmonary artery systolic | > 60% | 2.0 | 2 | 0.767692 |
Factors of surgery | ||||
Emergency surgery | Required before next work day | 2.8 | 2 | 0.712795 |
Rupture of Septum IV | 3.8 | 4 | 1.462009 | |
Surgery different from coronary revascularization | Cardiac surgery other than coronary revascularization or in addition to it | 1.6 | 2 | 0.542036 |
Thoracic aortic surgery | Surgery in ascending aorta arch or descending aorta | 3.2 | 3 | 1.159787 |
Material and methods
We conducted an observational, transversal, and retrospective study by means of the review of the clinical files of patients aged of ≥ 16 years submitted to cardiac surgery with and without extracorporeal circulation pump that were effected from January 1, 2008 to December 31, 2013 at the HRAEB. We did not include incomplete files, or those with doubtful information on the variables-of-interest, nor the files of patients who died due to causes not related to the cardiac surgery index. The following variables were obtained: mortality not adjusted up to the patient’s hospital discharge, defined as death occuring during the hospitalization index; type of surgery, defined as the procedure or procedures carried out during the index surgery, whether a) valvular surgery, b) aortic-coronary bypass surgery, c) corrective surgery for some congenital malformation(s), and d) surgery of another different type, including valvular + coronary bypass surgery, aortic surgery, closing of post-infarct interventricular communication, traumatic cardiac lesions, pericardium resection). The required variables were collected and the mortality-risk score was calculated of the EuroSCORE employing the EuroSCORE-program on-line calculator in its additive as well as its logistic version (www.euroscore.org/calsp.html ).
Statistical treatment
The qualitative variables are presented in percentages of frequency and are compared with the χ2 test, the Cochran Q test, and the Friedman ranges. The numerical variables are presented as averages and standard deviations (SD). Comparison of the averages between the two groups was performed with the student t test, and comparison among three or more averages was carried out with ANalysis Of VAriance (ANOVA) and the Bonferroni post-hoc test. A significance level of p ≤ 0.05 was accepted. To validate the EuroSCORE model, we evaluated discrimination and calibration. Discrimination consists of the capacity of the model to identify the patients who will survive those who will die. This was evaluated by means of the area under the receiver operating characteristics (ROC) Curve. Values of ≤ 0.5 speaks to the model that does not discriminate better than chance, and values of 1 indicate perfect discrimination. Values greater than 0.75 identify systems with a good capacity of discrimination of the model.3 Calibration consists of the comparison of the expected episodes against those observed along the entire risk range. We carried out calibration with the χ2 test of goodness-of-fit of the Hosmer and Lemeshow (H-L) adjustment, which calculates a statistical C, with which the difference can be evaluated between the values of mortality predicted by the model and the mortality values observed in the distinct groups of the population studied. The lower the value of this statistic, the better the calibration of the model (the predicted and the observed mortality becomes closer). A p value greater than 0.05 suggests that the model fits well, and consequently, it will predict well the probability of the patients dying.
Results
From the list of cardiac-surgery and cardiovascular-surgery procedures that were carried out from January 1, 2008 to December 31, 2013, which were collected by the HRAEB Department of Statistics, we identified 350 surgical procedures. Among the latter, we eliminated eight as follows: two for not having complete data-of-interest in the file; one for having been triplicated and with different information, and four due to death unrelated to the cardiac-surgical-procedure index. A total of 342 procedures remained that complied with the selection criteria and brought together and that are those that comprise the material of the present report. The general characteristics of the study group appear in Table 2. The frequency of each of the variables of the population studied and its comparison with that obtained in the EuroSCORE study are depicted in Table 3. As we can observe, there are notable differences in the prevalence of the majority of the risk factors. The population of the EuroSCORE study is 12 years older than that of our study and presents higher rates of extracardiac arteriopathy and of the left ventricle expulsion fraction (LVEF) of < 30%. On the other hand, our population has the following: a greater percentage of women; of chronic pulmonary disease; neurological dysfunction; prior cardiac surgery; creatinine values above 200 μmol/L (2.26 mg/dL); active endocarditis; critical preoperatory status; unstable angina; pulmonary arterial hypertension; emergent surgeries, and surgeries different from those of coronary bypass and thoracic aorta surgeries. In Table 4, we can appreciate the distribution of patients according to risk. As can be observed, only 16.9% of patients were low risk. The remainder were medium (44.4%) or high risk (38.6%). This is in agreement with the greater prevalence of more than 50% of the risk factors of our studied population in comparison with the EuroSCORE study population. There were 27 deaths, which yielded a crude mortality rate of 10.8%. In Table 4, we may observe the risk-adjusted mortality as follows: low-risk group 1/58 (expected, < 2%; observed, 1.7%), intermediate-risk group 7/152 (expected, <5%; observed, 4.6%), and the high-risk group 29/132 (expected, > 6%; observed, 22.0%). Table 5 presents the comparison of the variables of the following: the EuroSCORE model; type of surgery; the additive and logistic EuroSCORE score; and time of extracorporeal circulation between the group of patients who died and the group who survived. We observed a higher age in the group of patients who died (53.11 ± 13.87 vs. 49.72 ± 17.5 years, p = ns), as well as a notably greater prevalence in 10 variables of the model (extracardiac arteriopathy, neurological dysfunction, prior cardiac surgery, serum creatinine of > 2.2 mg/dL, active endocarditis, critical preoperatory status, LVEF 30-50%, LVEF of < 30%, PAH > 60 mmHg, and emergent surgery). Consequently, the additive EuroSCORE model (7.92 ± 3.63 vs. 5.11 ± 3.04, p = 0.0005) and the logistic EuroSCORE model (15.62 ± 15.99 vs. 6.70 ± 8.65, p = 0.0005) were greater in the group of patients who died. The extracorporeal circulation pump time (ECPT) was also higher in the group of patients who died (106.33 ± 73.76 vs. 82.91 ± 48, p = 0.0005).
N | 342 |
---|---|
Age | |
Mean ± SD | 50.02 ± 16.6 |
% ≤ 50 years | 40.6 |
51-60 years | 29.8 |
61-70 years | 21.6 |
70 years | 7.9 |
Gender male/female | 181/161 |
% Diabetes | 26.6 |
Hypertension | 45.3 |
Hypercholesterolemia | 21.6 |
Smoking | 30.1 |
% Type of surgery | |
Valvular | 34.2 |
Coronary bypass | 29.5 |
Congenital | 14.6 |
Another type | 21.6 |
LVEF (mean ± SD) | 54.4 ± 11.6 (321/342) |
Systolic pulmonary pressure (mean ± SD) | 42.6 ± 17 (298/342) |
Additive EuroSCORE (mean ± SD) | 5.3 ± 3.2 |
Logistic EuroSCORE (mean ± SD) | 7.6 ± 10.6 |
Extracorporeal circulation time (mean ± SD) | 118.8 ± 54.4 (179/342) |
Aortic clamping time (mean ± SD) | 89.5 ± 44.4 (175/342) |
Dead patients | 37/342 (10.8%) |
LVEF = left ventricular ejection fraction (mmHg).
Variable | EuroSCORE (n = 19,030) |
HRAEB (n = 342) |
---|---|---|
Age (mean ± SD) | 62.5 ± 10.7 | 50.02 ± 16.6 |
< 60 years (%) | 33.2 | 65.2 |
60-64 | 17.8 | 14.3 |
65-69 | 20.7 | 9.3 |
70-74 | 17.9 | 6.7 |
≥ 75 | 9.6 | 4.3 |
Female gender | 27.8 | 47.0 |
Pulmonary disease | 3.9 | 12.86 |
Extracardiac arteriopathy | 11.3 | 2.63 |
Neurological dysfunction | 1.4 | 2.04 |
Previous cardiac surgery | 7.3 | 10.23 |
Serum creatinine > 200 μmol/L | 1.8 | 8.47 |
Active endocarditis | 1.1 | 4.97 |
Critical preoperative status | 4.1 | 14.61 |
Unstable angina | 8.0 | 11.69 |
LVEF 30-50% | 25.6 | 24.26 |
LVEF < 30% | 5.8 | 2.04 |
Recent myocardial infarct (< 90 days) | 9.7 | 9.06 |
Systolic pulmonary pressure > 60 mmHg | 2.0 | 9.64 |
Emergency operation | 4.9 | 16.9 |
Non-coronary surgery | 36.4 | 71.05 |
Thoracic aortic surgery | 2.4 | 3.80 |
Ventricular septal rupture | 0.2 | 0.29 |
LVEF = left ventricular ejection fraction (%).
Risk groups | Patients (n) | Mortality | Observed (IC 95%) | Expected (IC 95%) |
---|---|---|---|---|
Low (0-2) | 58 | 1 (1.7%) | 1.91-3.51 | 1.01-3.51 |
Medium (3-5) | 152 | 7 (4.6%) | 2.51-9.78 | 2.87-11.17 |
High (≥ 6) | 132 | 29 (22.0%) | 9.44-39.40 | 9.79-38.04 |
Variables | Survivor group n = 305 | Deceased group n = 37 |
---|---|---|
Age (mean ± SD) | 49.72 ± 17.5 | 53.11 ± 13.87 |
% Female | 46.22 | 54.05 |
Pulmonary disease | 12.78 | 13.51 |
Extracardiac arteriopathy | 1.96 | 8.10 |
Neurological dysfunction | 0.98 | 10.81 |
Previous cardiac surgery | 7.54 | 32.40 |
Serum creatinine > 200 μmol/L | 6.88 | 21.62 |
Active endocarditis | 4.59 | 8.10 |
Critical preoperative state | 12.45 | 32.42 |
Unstable angina | 11.80 | 10.80 |
LVEF 30-50% | 22.95 | 35.13 |
LVEF < 30% | 1.31 | 8.10 |
Recent myocardial infarct (< 90 days) | 9.18 | 8.10 |
Systolic pulmonary pressure > 60 mmHg | 8.85 | 16.21 |
Emergency operation | 15.40 | 29.72 |
Non-coronary surgery | 70.16 | 75.67 |
Thoracic aortic surgery | 3.60 | 5.40 |
Ventricular septal rupture | 0.32 | 0 |
% Valve surgery (n = 117) | 84.61 | 15.39 |
Isolated coronary surgery (n = 101) | 94.05 | 5.94 |
Congenital surgery (n = 50) | 96 | 4 |
Others surgery (n = 74) | 85.13 | 14.86 |
Additive EuroSCORE (mean ± SD) | 5.11 ± 3.04 | 7.92 ± 3.63 |
Logistic EuroSCORE (mean ± SD) | 6.70 ± 8.65 | 15.62 ± 15.99 |
Extracorporeal circulation time (mean ± SD) | 82.91 ± 48.00 | 106.33 ± 73.76 |
Validation of the EuroSCORE model in our studied population yielded the following results: for the additive version, the area under the receiver operating curve (ROC) was 0.763 (Figure 1), and the χ2 of the H-L test was 5.30, with p = 0.62 (Figure 2). For the logistic version, the area under the ROC curve was 0.761 (Figure 3) and H-L test was 8.78, with p = 0.36 (Figure 4).
Discussion
The additive version of the EuroSCORE model was evaluated in a sample of 1,497 patients obtained from the population in which the model was developed, noting discrimination with an area under the ROC curve of 0.76 and a calibration measured with the H-L test of 7.5 (p < 0.68), confirming good internal validation.4 Later, the equation employed for the calculation of its logistic version was revealed.5 The goodness-of-it of the additive and logistic EuroSCORE model has been evaluated in multiple and different populations, in general cardiac surgery and in its subgroups, with and without the extracorporeal circulation pump. Perhaps this is the probabilistic model that is most evaluated to predict mortality in heart surgery worldwide, with studies conducted in Europe, Asia, Australia, and in America, with different results.6-18 In Europe, in a study that evaluated the performance of the score in populations of the six countries that contributed more than 500 patients to develop the EuroSCORE model, the authors found that, despite the notable epidemiological differences among these populations, the adjustment of the score for predicting mortality was good, with a discrimination evaluated with the area under the ROC curve of 0.82 and calibration evaluated with the H-L goodness-of-fit test of 0.59 (p ≥ 0.05).6 In the U.K., the logistic version of the model was evaluated in nearly 10,000 patients, 67.5% with coronary bypass, 15.33% with isolated valvular surgery, and 15.77% valvular aortic surgery + coronary bypass, observing good discrimination, with an area under the ROC curve of 0.79 for the whole group, of 0.77 for the coronary-bypass sub-group, and of 0.79 for valvular surgery; however, calibration was not good. The authors concluded that the logistic version model possesses good discrimination, but that calibration varies among the different risk sub-groups, with overestimation of the mortality observed. These authors recommended recalibration of the model.7 In Spain, in a study that included nearly 500 patients submitted to coronary bypass, adjustment of the model was evaluated and the additive and logistic versions were compared. The model’s discrimination was very good, with an area under the ROC curve of 0.830 for the logistic version and of 0.839 for the additive version. The value for the statistical C of the χ2 equation of H-L was 11.51 with p = 0.32 for the logistic version. The authors concluded that the logistic model is in greater approximation to the real mortality in the high-risk patient sub-group.8 Other studies have also compared the additive and logistic versions, observing that both versions possess satisfactory discrimination, with areas under the ROC curve of 0.80 (logistic) and 0.79 (additive). However, calibration was better with the logistic version (p = 0.12) than with the additive version (p = 0.001).9 In the population with highest risk, better performance of the logistic version of the model was observed.5 The results obtained in other studies have also found limitations in the model’s additive version. For example, in a study that included six large international samples, the result observed strongly suggest that the additive version overestimates mortality in low-risk groups (EuroSCORE ≤ 6 points) and underestimates mortality in high-risk groups (EuroSCORE ≥ 13 points).10
In Australia, in a study conducted with 8,331 patients (valvular, coronary bypass, and aortic surgery) to assess the goodness-of-fit of the model in its population, it was found that the discriminatory power of both variants of the model were very good, with an area under the ROC curve for the whole cohort of 0.83, and for the coronary-bypass sub-group, one of 0.82. However, calibration of both variants was poor in terms of predicting mortality in risk sub-groups, in that this was underestimated in nearly all of the risk deciles (χ2 of H-L with a p < 0.05).11 In the U.S., based on the STS data of the EuroSCORE model in its additive version, it revealed good to very good discrimination in general cardiac surgery and in coronary-bypass surgery, with areas under the ROC curve of 0.75 and 0.78, respectively. Calibration was evaluated in five risk sub-groups, noting that the mortality observed was nearly identical to that foreseen for the model in all of the sub-groups. The authors concluded that, despite the epidemiological and demographic differences between Europe and North America, the EuroSCORE model can be recommended as a simple and uncomplicated instrument for application on both side of the Atlantic.12 In Latin America, the EuroSCORE model has been evaluated in diverse studies. In Brazil, Moraes de Carvalho et al., in a study of four hospitals of the city of Rio de Janeiro with a randomized sample of 546 patients of a population submitted to coronary bypass, the model demonstrated poor discriminatory power, with an ROC curve of 0.62.13 In Argentina, Scaro and collaborators conducted a study with 123 patients to test the discriminatory power of the EuroSCORE model, concluding that the model was not useful for predicting mortality, in particular in sub-groups of intermediate and high-risk.14 In Colombia, Parga-Gómez et al., in a study of 498 patients, found good discrimination for the additive as well as for the logistic version (area under the ROC curve of 0.85 in both versions), in addition to good calibration (statistic C of χ2 of H-L 3.39 additive version, and 9.99 for the logistic version with p of 0.8 and 0.2, respectively).15 In Cuba, Chao-García and collaborators, in a study that included 158 patients submitted to mitral valve surgery, reported an area under the ROC curve of 0.97, that is, excellent discrimination.16 In Mexico, Careaga et al., in a study of 206 patients submitted to valvular surgery, the authors found an area under the ROC curve of 0.77 for the additive version and of 0.97 for the logistic version, values that are compatible with good and excellent discrimination, respectively. The χ2 of H-L was 6.7 (p = 0.034) and 2.86 (p = 0.99), the latter compatible with very good calibration.17 Rodríguez Chávez et al., in another study in Mexico, evaluated the model in 1,188 patients submitted to valvular surgery and found an Area Under the ROC Curve of 0.707 and of 0.694 for the additive and logistic versions, respectively, while the value of the χ2 of H-L was 63.15 (p < 0.001) and 45.6 (p < 0.001) for the corresponding additive and logistic versions. These authors concluded that prediction of mortality in this population of patients, in the additive as well as in the logistic version, was inadequate and that discrimination was scarcely borderline with the additive and poor for the logistic version.18
In our study, we observed that, while the population evaluated was 12 years younger than the European population from which the model arose, the prevalence of diabetes mellitus (DM) was greater, as well as of the majority of the prognostic variables of the score already cited in the results section. However, despite the epidemiological differences between the EuroSCORE development group and those of our population, the results obtained in the present study are compatible with good discrimination and good calibration of the model, in its additive as well as in its logistic version, and are similar to the results reported in Europe,6,8 the U.S.,12 Colombia,15 and Careaga in our environment.17 It is noteworthy that our evaluated population included vascular surgery, coronary bypass, and congenital surgery in adults, and also other types of surgery that included infrequent cardiac pathologies such as those produced by non-shooting weapons and/or by firearms, that is, it was a population submitted to cardiac surgery for all of the pathologies.
Why are there discrepancies in the results in different studies? The explanations can be several. On the one hand, the continuous progress in cardiovascular surgery (technological, better organization of medical groups, acquisition of experts of surgical groups and of the groups charged with perioperatory management, among others) and, on the other hand, improvement in the coverage of the health systems in countries on various continents, as well as a population with a higher degree of education in health, have yielded as results the progressive diminution of perioperatory mortality in these patients (despite their being older and having more comorbidities), more notorious in developed countries, which has originated that a determined score that has shown a good prognostic performance during the time in which it was developed, can, a few years later, result in poor discrimination and/or calibration, as suggested in some studies.19 From this arises the suggestion to update these instruments, as has already been done in some models (in the Parssonnet in its different versions and in the EuroSCORE model, which has already developed model II, and in the STS score, which performs this periodically and in a dynamic manner).3 Perhaps it would also be convenient to develop models for some surgical groups in particular, such as valvular surgery20 and infectious endocarditis surgery,21-23 as has been carried out in some places, with the purpose of possessing more reliable instruments for utilization in these specific pathologies.