Assessment of the Accuracy of Surgical Risk Scales in Predicting Postoperative Morbidity and Mortality in Patients with Gastric Cancer

1. Introduction

Gastric cancer (GC) is currently the fifth most incident neoplasm and the fourth cause of cancer mortality worldwide. It accounted for over one million new cases in 2020, with approximately 769.000 people dying due natural history disease’s [1]. In Portugal, GC incidence is approximately four times higher than the European average. [2,3]

GC staging is based on the American Joint Commitment on Cancer TMN classification. GC prognosis, as well as the type of treatment to be performed, depend on this staging. [4,5] In early stages, endoscopic resection is potentially curative, however the majority of patients with GC do not fit into this profile. Thus, surgical resection remains the best curative option for early-stage tumors. Surgery is also recommended for the palliation of symptoms in cases of advanced disease that, despite having surpassed the possibility of cure, present formal surgical indication due to hemorrhage or obstruction. [3,6]. The surgical treatment with curative intent generally involves a major procedure with total or subtotal gastrectomy, eventual resection of adjacent organs and extensive lymphadenectomy, so postoperative complications inherent to this type of intervention still constitute an important challenge to clinical practice. [3,6]. Postoperative mortality and morbidity are often used for evaluating outcomes and comparing the quality of surgical care. [3] But the absolute results of morbimortality doesn´t reflect que quality of the realized treatment, so models have been used, namely the Physiological and Operative Severity Score for the enumeration of mortality and morbidity (POSSUM), the Portsmouth physiologic and operative severity score for the enumeration of mortality and morbidity (P-POSSUM) and the Oesophagogastric Physiologic and Operative Severity Score for the enumeration of mortality and morbidity (O-POSSUM), which represent an accessible tool that allows determining preoperative surgical risk, in order to optimize patients with a higher risk of developing postoperative morbimortality.

2. Results

2.1. Sample Characterization

The sample consisted of 131 patients who underwent gastrectomy for GC in Hospital de Braga between January 1, 2010 and December 31, 2013. The characterization of the population is described in Table 1.

2.1.1. Morbidity and Mortality

Of the 131 patients under analysis, 31 (23.7%) present postoperative mobility. With regard to local complications, it was observed that 11 patients had anastomotic leak (8.4%), 2 patients had a fistula (1.5%) and 2 patients had surgical wound infection (1.5%). Regarding to systemic complications, 12 patients had pulmonary complications (9.2%), 5 patients had sepsis (3.8%), 5 patients had abdominal complications (3.8%), 1 patient had cardiovascular complications (0. 8%) and 1 patient had hypovolemic shock (0.8%). Table 2 describes observed morbility. The mortality observed in the 30 days following surgery was 6.1%, corresponding this mortality to 8 deaths. Until January 31, 2018, there were 46 deaths (34.8%), and 38 of the deaths were related to GC (28.8%).

2.1.2. Tumor recurrence

Of the 131 patients, 33 had tumor recurrence (25%) during the follow-up, evaluated until January 31, 2018 (5-year follow-up).

2.1.3. Survival time and disease-free survival

The average survival time was 70.237 months (95% CI 63.315-77.160). (Figure 1) The average disease-free time was 17.569 months (95% CI 12.404-22.735) and the median was 14 months (95% CI 9.510-8.490). (Figure 2)

2.2. Results of the application of the scales

2.2.1. POSSUM

The predicted mortality risk by POSSUM ranged between 1.43% and 23.51%, with a median value of 4.19 and an interquartile range of 3.76. Regarding the calculated morbidity risk by POSSUM, the values ranged between 7.59% and 77.56%, with a median value of 23.87 and an interquartile range of 20.45.

2.2.2. P-POSSUM

The predicted mortality risk by P-POSSUM ranged between 0.31% and 12.82%, with a median value of 1.18 and an interquartile range of 1.52. Regarding the calculated morbidity risk by P-POSSUM, the values varied between 7.59% and 77.56%, with a median value of 23.87 and an interquartile range of 20.45.

2.2.3. O-POSSUM

The predicted mortality risk by O-POSSUM ranged between 0.94% and 52.30%, with a median value of 10.25 and an interquartile range of 10.88.

2.3. Assessment of surgical risk scales

2.3.1. Comparison of results obtained from the scales regarding postoperative outcome

Mortality

Regarding patients who died within 30 days postoperatively, all scales yielded significantly different results between the group of patients who died and the group of patients who did not die. (Table 3)

Morbidity

Regarding patients who experienced complications within 30 days postoperatively, the POSSUM and P-POSSUM scales showed significantly different results between the group of patients who developed morbidity and the group of patients who did not develop morbidity. (Table 4)

2.3.2. Calibration

As for the calibration of the scales, it is measured by the O:E ratio and the Hosmer-Lemeshow test, which was applied to the POSSUM, P-POSSUM and O-POSSUM scales for both mortality and morbidity.

Mortality

Hosmer-Lemeshow test was applied to all evaluated parameters and, in all cases, non-significant values were obtained, demonstrating that the POSSUM, P-POSSUM and O-POSSUM scales exhibited good calibration. The O:E ratio was simultaneously assessed revealing that the POSSUM and P-POSSUM scales demonstrated the best calibration in assessing the overall patient population. When divided into subgroups, POSSUM and P-POSSUM also exhibited the best calibration. In the subgroups, these scales showed better performance in patients with Stages II and III and when total gastrectomy was performed. (Table 5 and Table 6)

Morbidity

The Hosmer-Lemeshow test was applied to all evaluated parameters and in all cases non-significant values were obtained, demonstrating that the POSSUM and P-POSSUM scales exhibited good calibration. The O:E ratio was simultaneously assessed, both for overall patient population and for the various subgroups. Thus, the POSSUM and P-POSSUM scales showed better performance in patients with Stage III and when total gastrectomy was performed. (Table 7 and Table 8)

2.3.3. Discrimination

The discriminative capacity of the POSSUM, P-POSSUM and O-POSSUM scales, in terms of mortality and morbidity, was tested using the AUC of the measured ROC curves.

Mortality

Regarding mortality, all evaluated scales exhibited significant results and all showed good discrimination, as the AUC was greater than 0.8 in all cases. The POSSUM scale showed the best discrimination. (Table 9 and Figure 3)

Morbidity

Regarding morbidity, the POSSUM and P-POSSUM scales exhibited significant results and showed good discrimination, as the AUC was greater than 0.8. (Table 10 and (Figure 4)

2.4. Correlation among possum, p-possum and o-possum scales

Table 11 presents the studied correlations among the parameters and demonstrates a very strong correlation between the POSSUM and P-POSSUM scales, along with weak correlations between the POSSUM and O-POSSUM scales, as well as between the P-POSSUM and O-POSSUM scales.

2.5. Survival analysis

To assess the impact of these scales on overall mortality (not just within 30 days postoperatively), patients were divided into 2 groups for each scale using the median value (one group with a value below the median and another group with a value equal to or above the median). Subsequently, Kaplan-Meier curves were constructed for each group, and these were compared using the Log Rank test. Figure 5, Figure 6 and Figure 7 represents Kaplan-Meier curves for POSSUM, P-POSSUM and O-POSSUM, respectively.

Regarding the POSSUM and P-POSSUM scales, patients with values below the median obtained from the scale exhibited significant differences in survival when compared to patients with values equal to or above the median. However, the same was not observed with the O-POSSUM scale, as while survival differed between the two groups of patients, this difference was not considered significant. (Table 12 and Table 13)

3. Discussion

Gastric cancer (GC) continues to be one of the most incident and lethal conditions worldwide, and Portugal remains one of the European countries with the highest incidence of this neoplasia. [1,2] Surgery is the best curative option, however, despite advancements in perioperative care leading to a reduction in morbidity and mortality associated with GC surgery, it still remains a clinically significant issue. [3,6] Additionally, surgical morbidity and mortality are often used to assess performance and enable comparison among individual surgeons and hospitals. [7] Therefore, preoperative surgical risk assessment is essential as it allows for the optimization of patients at higher risk of developing postoperative morbidity and mortality, ensuring the quality control of surgical procedures. [3,6] The observed 30-day postoperative mortality was 6.1%, and the observed 30-day postoperative morbidity was 23.7%, values similar to those studied in the literature, particularly ranging from 5% to 12% for mortality and 30% to 43% for morbidity. [3,8,9,10] Intra-abdominal and pulmonary complications were more frequent, which is also consistent with the literature. [3,11] The study aimed to assess the accuracy of surgical risk prediction scales in GC, specifically the POSSUM, P-POSSUM, and O-POSSUM, among patients undergoing surgical interventions for curative purposes at Hospital de Braga. The ideal model would be one that could predict postoperative morbidity and mortality as accurately as possible, being rapid, easy to apply, and adaptable to all types of surgical procedures. Among these models, the most widely validated for surgical patients was POSSUM, followed by P-POSSUM, and specifically for gastric surgery, O-POSSUM. [10] The POSSUM model, along with its subsequent modifications like P-POSSUM and O-POSSUM, employs a scoring system based on patient physiological variables and the severity of the surgical procedure performed. This is done to evaluate the outcomes of surgical interventions and establish the relationship between predicted and observed mortality and morbidity within each risk category of the study population. [3,7] Regarding the scores obtained from the different scales, they were significantly different between patients who died or experienced morbidity and patients who did not die or did not develop morbidity within 30 days postoperatively, as observed in the literature. [12,13] This implies that patients who experienced morbidity or mortality had distinct scoring patterns compared to those who did not experience such outcomes. In terms of calibration, all scales demonstrated good ability to predict the occurrence of morbidity and mortality. Regarding mortality, both POSSUM and P-POSSUM exhibited good performance in most subgroups, particularly in patients with stages II, III, and when total gastrectomy was performed, with an O:E ratio of 1 for both scales. However, in the overall patient population and among patients with intestinal histological type of GC, both scales underestimated mortality, with O:E ratios of 1.25 and 1.34, respectively. These findings contrast with those described in the literature, where POSSUM and P-POSSUM are reported as good predictors of mortality. However, when evaluating O:E, it varies between 0.08 and 0.17 for POSSUM and between 0.21 and 0.7 for P-POSSUM, indicating an overestimation of mortality. [7,9,14,15,16,17] O-POSSUM underestimated mortality both in the overall patient population and in all subgroups, with O:E ratios ranging from 1.5 to 8.04. In the literature, this scale is described as a poor predictor of mortality, with O:E ratios ranging from 0.29 to 0.9, which means it overestimates mortality. This is the opposite of what was observed in the present study. [7,15] Regarding morbidity, both POSSUM and P-POSSUM underestimated morbidity in the overall patient population and when divided into subgroups, with O:E ratios ranging from 1.21 to 1.75. They showed better performance in patients with stage III and when total gastrectomy was performed, with an O:E ratio of 1.2 in both subgroups. In several studies, these scales are described as good predictors of morbidity. However, when evaluating O:E ratios, they ranged from 0.4 to 0.9, indicating an overestimation of morbidity. [16,17,18] Concerning discrimination, all evaluated scales exhibited a good ability to distinguish between patients who survived and patients who died. The POSSUM scale showed the best discrimination, followed by P-POSSUM, and finally O-POSSUM. All scales had AUC values above 0.8. [7,19] However, these results contradict other studies where O-POSSUM was found to be the best discriminator of patients. [8,9] Concerning morbidity, the POSSUM and P-POSSUM scales also displayed good discrimination, with AUC values exceeding 0.8. In the literature, these scales are described as reasonable predictors of morbidity. [8] Thus, overall, in this case series, POSSUM and P-POSSUM managed to predict the occurrence of morbidity and mortality more accurately, presenting overlapping calibration values and very close discrimination values. To understand the reason behind this similarity, potential correlations between the different evaluated scales were tested. It was concluded that there is a strong correlation between the POSSUM and P-POSSUM scales. In other words, the values obtained from both scales tend to move in the same direction and at a consistent rate, thereby increasing together. Additionally, the correlation between the POSSUM and O-POSSUM scales, as well as between P-POSSUM and O-POSSUM, was evaluated and found to be weak. In the long term, the POSSUM and P-POSSUM scales served as predictors of survival. Patients scoring below the median values of these scales, namely 4.19 for POSSUM and 1.18 for P-POSSUM, exhibited distinct survival rates compared to those with scores equal to or higher than the aforementioned values. The same outcome was not observed with the O-POSSUM scale, as although there was a survival difference between the two patient groups, this difference was not considered significant. The present study has several limitations. Primarily, being a retrospective study, the collection of specific data on physiological status and intraoperative course was constrained. Additionally, the fact that it involves a small sample size may limit the accuracy of the results, particularly in predicting mortality.

4. Materials and Methods

This is an observational, retrospective, descriptive and analytical.

The target population of the study corresponds to patients with a histological diagnosis of gastric adenocarcinoma, who underwent surgical treatment with curative intent at the General Surgery Department at Hospital de Braga, in the period between January 1, 2010 and December 31, 2013. Patients who did not underwent surgical resection or those who received surgical treatment with palliative intent were excluded from the study. Also, patients with a histological diagnosis different from the one mentioned earlier, patients without anatomopathological staging results, or patients with incomplete clinical data necessary for determining the prognosis scale score were excluded. Thus, from an initial population of 180 patients, patients who met all the aforementioned inclusion criteria and who did not have any exclusion criteria were selected, resulting in a final study sample of 131 patients. Data collection and processing: The data were collected by consulting clinical files, using the Glintt® computer system. The information was registered in a hospital database and anonymized. A code was assigned to each clinical file analyzed that was different from the hospital file number. No data that could enable patient identification were ever recorded, thus ensuring confidentiality and anonymity. The clinical and anatomopathological data collected from each patient in the sample were as follows: age, gender, tumor location and histological type, depth of tumor invasion [T), number of lymph nodes involved [N), presence or absence of metastases [M), tumor recurrence, including site and time of recurrence, 30-day postoperative mortality, 30-day postoperative complications, overall mortality, cancer-related survival, and overall survival. By complications, we mean any type of event that lead to a deviation from the normal postoperative course, and in this study the assessment of complications was carried out in accordance with the initial definition by Copeland et al. [20] For statistical purposes, only the presence or absence of operative complications was recorded, regardless of the number or type of complication. To calculate the different scales, the following parameters were collected: age, cardiac signs, respiratory signs, electrocardiogram, systolic blood pressure, heart rate, hemoglobin, leukocytes, urea, sodium, potassium and the Glasgow Coma Scale. Regarding to surgical parameters, the following data were collected: operative severity, number of surgical procedures performed in the last 30 days, amount of blood loss, presence of peritoneal contamination, status of tumor dissemination, surgical approach and type of surgery. In this study, the calculation of physiological and operative scores of the POSSUM, P-POSSUM and O-POSSUM scales were obtained through the arithmetic sum of the scores assigned to each parameter defined according to Copeland et al [20]. The risk of morbidity and mortality in the 30 days after the operation predicted by the POSSUM, P-POSSUM and O-POSSUM scales were calculated using the calculation equations shown in Table 14. [7,9,21]

Statistical analysis: Statistical analysis of the data was performed using the Statistical Package for Social Science® [SPSS®) software, version 27. Initially, an exploratory analysis of the data was conducted, for the quantitative variables to assess whether the assumption of normal distribution was met. This analysis was based on measures of skewness, kurtosis, the Kolmogorov-Smirnov test, and histograms. Subsequently, a descriptive analysis of the studied variables was carried out. For the quantitative variables with normal distribution, means and standard deviations were calculated. For those that did not show normality, means, standard deviations, medians and interquartile range were calculated. As for the qualitative variables, absolute and relative frequencies are shown. The normality assumption was not met for all variables; therefore, in these cases, non-parametric tests were conducted. Next, the Mann Whitney test was performed with the aim of comparing the means of the predicted mortality rate by the assessed scales between the group of patients who died and the group of patients who did not die within the first 30 days after surgery. Additionally, the means of the predicted morbidity rate by the assessed scales were compared between the group of patients who experienced complications and the group of patients who did not experience complications within the first 30 days after surgery. The scales were analyzed for calibration and discrimination, both globally and by subgroups. For all statistical analyses, a p-value less than 0.05 was considered significant. Calibration was defined as the capacity of the scale to correctly predict the probability of an event occurrence in a population [3,19]. This was assessed by comparing the observed number of events with the expected number based on the Observed-to-Expected [O:E) ratio and the Hosmer-Lemeshow test . In this test, p<0.05 indicates poor calibration, meaning there are significant differences between the observed and expected values [11). An O:E ratio <1 indicates that the scale overestimates morbidity and mortality, while, an O:E> 1 indicates that the scale underestimates morbidity and mortality [7]. Discrimination was defined as the ability of the scale to distinguish between patients who will survive and those who will die [3,19], measured by the area under the Receiver Operating Characteristic [ROC) curve. The discrimination levels of the scale depend on the Area Under the Curve [AUC), therefore an AUC below than 0.70 indicates low discrimination, between 0.70-0.80 indicates moderate discrimination and above 0.80 indicates good discrimination [7,19]. To analyze the relationship between the various scales, Spearman's correlation coefficient was also used. Finally, Kaplan-Meier curves were used to analyze survival associated with the Long-Rank test to compare the survival of different groups of patients.

5. Conclusions

GC surgery is associated with considerable morbidity and mortality, making it essential to determine surgical risk preoperatively in order to optimize patients at higher risk of postoperative morbidity and mortality. Several scales have been developed with the aim of predicting the morbidity and mortality associated with this type of surgical intervention, but none have satisfactorily met this need. The present study evaluated the POSSUM, P-POSSUM, and O-POSSUM scales.

In this study, for mortality estimation, both POSSUM and P-POSSUM demonstrated better overall calibration as well as in various subgroups. POSSUM overestimated mortality, while O-POSSUM underestimated it. Regarding discrimination, POSSUM exhibited the best discrimination, followed by P-POSSUM, and finally O-POSSUM. Concerning morbidity, the POSSUM scale underestimated the number of complications but demonstrated good discriminative capacity.

Thus, overall, POSSUM and P-POSSUM more accurately predicted the occurrence of morbidity and mortality. These scales should be applied to different populations to test their accuracy. In the future, conducting a prospective study involving other surgical risk scales would be important.