Explaining the Elusive Nature of a Well-Defined Threshold for Blood Transfusion in Advanced Epithelial Ovarian Cancer Cytoreductive Surgery

1. Introduction

Institutional transfusion protocols are not universal, and a variety of transfusion policies may exist across participating institutions. The recently published European Society of Gynaecological Oncology (ESGO) guidelines for the peri-operative management of advanced epithelial ovarian cancer (EOC) patients clearly stated the absence of a well-defined threshold for BT in cytoreductive surgery [1]. Indeed, since many patients will require adjuvant chemotherapy, more liberal transfusion thresholds may be considered. Equally, the BT requirements have long been debated because they are not without risk [2]. Intra-operative decisions are critical in the surgical management of advanced EOC. In such a dynamic theatre environment, harmonic coordination amongst team members including surgeons, anaesthetists and theatre staff can lead to procedural success. Surgical decision- making is layered upon situational awareness and the impact of human factors [3]. While clinical judgments are enhanced over years of training, risk prediction tools still provide the opportunity to augment decision-making. Nevertheless, traditional models fall short to address the complex interactions amongst discrete prediction variables. Machine Learning- based prediction tools can potentially rise above the peak of inflated expectations [4]. Identifying the optimal BT threshold for patients undergoing cytoreduction for advanced EOC becomes crucial, considering the potential risks associated with the surgery.

We hypothesized that pre-operative and intra-operative risk factors for planned cytoreductive procedures can help predict anticipated blood requirements, use of haemostatic sealants and, subsequently anaesthetic interventions in the peri-operative period. We aimed to devise an ML-driven algorithm for transfusion prediction that could trigger a BT communication alert based on anticipated intra-operative events.

2. Methods

We analyzed prospectively collected data from 560 patients with advanced epithelial ovarian cancer (EOC) who underwent cytoreductive surgery at a UK tertiary center between January 2014 and December 2019. The internally developed database was integrated with the Electronic Health Records (EHRs). Institutional research ethics board approval was obtained through the Leeds Teaching Hospitals Trust (MO20/133163/18.06.20), and informed written consent was obtained. The study was added to the UMIN/CTR Trial Registry (UMIN000049480). Surgical management was agreed at the weekly central MDT meeting prior to patient pre-operative review. The cohort has been previously described [5]. Exclusion criteria included patients aged < 18 years; those with non-epithelial histology; synchronous primary malignancy; those undergoing secondary cytoreduction for recurrent disease. We also excluded those with pre-operative anaemia (defined as a haemoglobin level < 120 g/dl) and confirmed coagulopathy (by reviewing medical history or routinely requested coagulation results). The intra-operative fluid protocol consisted of both maintenance therapy (1 mL/kg/h of crystalloid) and replacement of blood loss (1 mL of crystalloid for each mL of directly measured blood loss). If necessary, vasoactive agents were titrated to obtain a mean blood pressure >65 mmHg. The EBL was estimated by subtracting the intra-operative fluid losses suctioned through suctioned drains into the surgical canister from the total blood volume contained in the surgical canister at the end of the procedure and empirically adding the blood loss on the required surgical gauzes. The dataset included the following features: Pre-operative features: age, Eastern Co-operative Oncology Group (ECOG) performance status (PS), histology type (serous and non-serous), grade (low and high), stage (FIGO 3 or 4), and pre-treatment and pre-surgery CA-125. Operative/tumor factors: timing of surgery (PDS or IDS, operative time (OT), estimated blood loss (EBL), site and size of main disease bulk, Peritoneal Carcinomatosis Index (PCI) [6], intra-operative mapping of ovarian cancer (IMO) [7], Surgical Complexity Score (SCS) [8], and residual disease. Residual disease was documented by a comprehensive visual assessment of all the abdominopelvic regions. Human factors: Age of consultant surgeon, and years of experience. Surgical sub-procedures: We employed an adaptation of the structured ESGO Ovarian Cancer Operative Report following the publication of quality indicators for ovarian cancer surgery [9]. This provided detailed information on the patterns of tumor dissemination (https://guidelines.esgo.org/, accessed on 10 Jun 2023). Descriptive statistics were displayed by frequency and percentages for binary and categorical variables and by means and standard deviation (SD) or medians (with lower or upper quartiles for continuous variables). The t test for continuous variables and the Chi- square test for categorical variables were performed. The Fischer’s exact tests were used for binary variables. Statistical significance was set at p < 0.05. Overall survival was analyzed using the Kaplan–Meier and the Mantel–Cox log-rank tests for comparison.

We employed the eXtreme Gradient Boosting (XGBoost) algorithm to model the features [10]. This boosts the performance of weak learning algorithms by combining all the generated hypotheses into a single hypothesis. We investigated the combined effect of eight parameters to maximize model performance by evaluating a grid of 7680 combinations of values using Scikit-learn’s GridSearchCV function. In the absence of standardized definitions for intra-operative bleeding, the study consisted of a concept phase, in which we calculated the estimated blood volume (EBV) using the formula EBV= weight x 80 and set off 10%EBV as threshold for individual intervention [11]. The 10% EBV threshold was used to formulate a binary prediction classification problem. Based on the known EBL, we identified two subgroups. The dataset was split into training and test cohorts (70%:30% ratio). There was no significant difference (p > 0.20) between the two cohorts, with respect to all variables. A five-fold stratified cross-validation (CV) was performed and imbalance was overcome by creating stratified folds to ensure equal distribution between both above and below threshold groups. The CV was iterated 100 times to decrease both variance and bias, hence creating and evaluating 500 models with each round. Model performance was assessed by measuring the total area under the receiver-operating curve (AUC). Receiver operating characteristic (ROC) and Precision-Recall curves and state-of-art scores were used for performance metrics. We used the artificial intelligence SHapley Additive exPlanations (SHAP) framework to explain the cohort-level risk estimates to define novel surgical risk phenotypes. This methodology enhances interpretability by explaining how much the presence of a feature contributes to the model’s overall prediction [12]. To demonstrate the value of the model prediction, we produced (a) SHAP summary plots for the global (cohort) explanation of the results; (b) SHAP dependence plots of the critical risk features pertinent to the prediction. The Python’s SciPy library (version 2.7) (Python Software Foundation) Python Language Reference, version 2.7. available at http://www.python.org) was used for the analyses.

3. Results

Initially, 560 EOC patients were enrolled in the study. One hundred seventeen patients and 26 patients were excluded due to pre-operative anaemia and abnormal coagulation system, respectively. Four patients were further excluded: one patient with a synchronous primary malignancy and three patients who underwent emergency cytoreduction with no intention to treat. Ten patient cases were omitted due to incomplete data. Finally, 403 EOC patients participated in the final analysis (Table 1). The patients were followed-up until April 2022. The mean age of patients in the entire cohort was 65.31 + 11.2 years. This was comparable between the train and test sets; 63.2 ± 11, 64,6 ± 12, respectively (p:NS). Histology, PS, tumour grade and stage, timing of surgery, residual disease, SCS, IMO score, PCI, pre-surgery Ca 125 and occurrence of ascites were similar in the training and test sets (P:NS). All but pre-treatment Ca 125 and age were statistically significant between the < and > 10%EBV groups. The descriptive of surgical sub-procedures has been previously published [13].

The model performance for the above threshold prediction was moderate-to-high (AUC 0.82, 95% CI 0.78-0.86; AP 0.72, 95% CI 0.67-0.76) (Figure 1). To promote reproducibility, the optimal set of model parameters are available upon reasonable request.

The importance of features in this model based on SHAP values is shown in Figure 2. The order of the features reflects their weighted importance, i.e., the sum of the SHAP value magnitudes across all the samples. Each point on the summary plot is a Shapley value for a feature. The position on the y-axis is determined by the feature and on the x-axis by the Shapley value. The color represents the value of the feature from low (blue =< 10% EBV) to high (red => 10% EBV). The top feature commonly shared between both interrogators was operative time (OT). The top-5 list of important features included pre-treatment CA125, logCa125/PCI, a marker of biological tumor aggressiveness, followed by the SCS and the size of the largest disease bulk. Long tails were observed mainly for operative time and size of the largest tumor bulk, which suggests these features can be equally important for specific if not all patients (local explainability).

The SHAP dependence plots reveal the impact of each feature on the prediction by plotting the value of the feature on the x-axis and the SHAP feature value on the y-axis. While Figure 3A fails to show a clear inflection point, inflection points are demonstrated for high grade (Figure 3B), PS > 1 (Figure 3C) and serous histology (Figure 3D). When overall SHAP values are positive, the likelihood for BT intervention is higher. For OT < 150 min, the overall SHAP values are negative, which makes BT intervention unlikely (Figure 4A). Then, there is no clear inflection point for OT ranging between 150 min and 250 min. Subsequently, for OT > 250 min, it becomes more likely to administer BT. If PCI is selected as a feature to determine its impact, an increasing PCI up to eight results in a lower likelihood for BT (Figure 4D). Similarly, the inflection points for IMO score and SCS are four and three, respectively. Clearly, delayed surgery and R0 resection do not favor BT intervention (Figure 4B and 4F). From the list of surgical sub-procedures clearly reflecting radical or ultra-radical cytoreductive efforts, an intra-operative blood loss of at least 10%EBV was associated with the performance of (A) stoma formation (Figure 5A), bladder peritonectomy (Figure 5B), para-aortic lymphadenectomy (Figure 5C), ileo-caecal resection/right hemicolectomy (Figure 5D), mesenteric resection (Figure 5E), upper abdominal peritonectomy (Figure 5F), and large bowel resection (Figure 5G).

The 5-year OS for the entire cohort was 61 months (95% CI 58-63). The median OS was 60 months for the <10%EBV group (95% CI 58–61) and 61 months (95% CI 60–63) for the >10%EBV group (p:NS). In the predicted >10%EBV group, only 10% (n= 23) received BT (Figure 6).

4. Discussion

We developed and explained an ensemble ML algorithm to predict intra-operative interventional risk for advanced EOC cytoreduction (Figure 7). Equally, we quantified the contribution of independent patient- disease- and operation-specific features to intra- operative blood loss to understand its impact on surgery. While we urge for standardized definitions of intra-operative bleeding in surgical practice, we conceptually identified a threshold for potential personalized intervention, irrespective of BT policies. This EBV- and EBL-based simple tool could improve the language and communication surrounding blood loss. Including percentage EBV triggers at the surgical safety list can potentially minimize the requirement for further interventions. Such mental preparation can be an important factor for surgical success [14]. To our knowledge, this is the first attempt to identify an objective threshold that could proactively result in interventions inclusive of BT or usage of haemostatic agents at EOC cytoreductive surgery. The study explains how EBL can be divided into a single contributing component, which is the surgical time. As such, the main study finding is that precise prediction of blood requirements or strategies to address haemostasis is not possible unless a rough estimate of OT is known in advance. An inflection point was not identified but OT >250 min clearly increases the risk for BT administration. Blood transfusion and OT are inter-related events prompting for quality assurance of all intra-operative events to be the focus of quality assurance in the future [15]. Interestingly, the top-5 list of important features included not just markers of surgical aggressiveness but also inherent tumor biology, such as the pre-treatment Ca125 or the Ca125/PCI ratio. This information is clinically relevant to be used in studies investigating haemostatic agents [11]. Previous studies identified age and low baseline hemoglobin level as risk factors for transfusion in this population [16,17]. Malignant ascites >500 mL can be a risk factor as it implies increased fluid demands and substantial alterations in circulatory blood flow during cancer surgery [2], but this was not shown in our study. Lately, we read with interest the development of a new tool that can predict the need for BT during or after primary debulking surgery in EOC patients. The tool, called the “BLOODS score” was developed to help identify patients who may benefit from pre-operative planning and blood-saving techniques [18]. The score was created using data from a similar practice setting. In a cohort totaling 1289 patients, and for a median estimated blood loss of 650 cc, the seven features associated with BT included: the American Society of Anesthesiologists physical score of 3 or higher, the pre-operative serum albumin level of 3.5 g/dL or lower, presence of ascites, pre-operative CA-125 level of 600 U/mL or higher, carcinomatosis, moderate or high SCS compared to low SCS, (h), pre-operative creatinine level greater than 0.90 mg/dL , and upper abdominal disease of 1 cm or greater. The features were assigned a score of 1 if present. The sum of the BLOODS score corresponded to the need for BT. The authors claim that a BLOODS score of 3 or higher is estimated to identify 80% of patients who require a BT, including 85% of patients who need a transfusion intra-operatively and 78% who need one post-operatively. Notably, the rate of full clearance was just above 50%. In our study, we clearly demonstrated that the risk for BT is associated with primary surgery- owing to a complete cytoreduction rate of 67.3%- too. It is likely that with expansion of our surgical cohort, we may encounter similar results in the future. At our tertiary center, consideration of patients for cytoreductive surgery is based on pre- operative evaluation with contrast-enhanced cross-sectional imaging but, more importantly stringent selection through MDT discussion. Surgeons are requested to estimate the likelihood of extensive cytoreduction and pre-operatively anticipate the input values for several intra-operative variables. We are not surprised why OT was the single most important contributor to the threshold risk for potential intervention. In theory, the duration of surgery can vary depending on the extent of disease spread, the surgical effort, the experience of the surgical team in addition to the patient’s overall health. Pre-operative prediction of the duration of the procedure is one of the critical elements of theatre scheduling, and one needs to know what surgical effort needs exerting at the beginning of surgery; this can then facilitate scheduling or anticipate needs for resources. Maximizing operating room utilization relies on accurate estimates of surgical control time. As the number of EOC diagnoses is rapidly increasing, effective management of theatre time is required. Anecdotal evidence suggests that surgeons grossly underestimate predicted surgical time [19]. Herein, we provided clear evidence of several surgical sub-procedures, indirectly linked to procedural time, which are associated with higher risk for intra-operative intervention. Nevertheless, extensive surgical approach is associated with increased blood loss and potentially higher peri-operative complications [20]. As studies continue to share knowledge about the optimal duration of surgical procedures [21], another study is currently underway to assign optimal procedural times from knowledge retrieved from operative notes using natural language processing (data not shown). This niche may have potential implications: to improve information delivery to surgeons so that careful planning is considered in reducing complications; secondly, it will improve patient counseling or discussion about extent of surgery or alternative therapeutic options. To date, most relevant studies have devised models to identify those patients who are at lowest risk and may need less pre-operative BT preparation [22,23]. This information can be used to develop EOC-specific institutional guidelines to predict crossmatch and transfusion needs, and promote the sensible use of blood products and cost savings. Owing to the high prevalence of BT in this population, this information can be of added value for accurate surgical planning and pre-operative optimization of potentially modifiable factors. We acknowledge that initial intra-operative anaesthetic interventions do not always involve transfusions of blood products. Goal-directed haemodynamic algorithms are routinely used to optimize stroke volume and provide better systemic stability and reduced need for fresh- frozen plasma [24]. Efforts to promote sensible use of intra-operative BT also include acute normovolemic haemodilution without increasing peri-operative complications [25]. Other strategies for “bloodless” surgery have been recently described [26,27]. There is insufficient evidence to recommend the routine use of tranexamic acid for reducing blood loss in women undergoing cytoreductive surgery for advanced EOC, as only limited data are available from a single, low quality RCT [28]. Optimizing intra-operative fluid management may improve short-term patient outcomes [29]. Notably, the intra-operative BT requirements are significantly lower in prehabilitation groups of EOC candidates to cytoreductive surgery [28]. The impact of BT on EOC survival has been uncertain until the ancillary analysis of the EORTC 55971 phase III trial, which concluded that BT does not negatively impact progression-free survival or overall survival. Nevertheless, it was associated with increased peri-operative morbidity without improvements in quality of life [30]. The case for overall survival was presented in our study albeit only 10% of patients in the >10%EBV group (rule of thumb) received BT peri-operatively. Intra-operative bleeding is the most relevant finding to surgical complications. Surgical time may not be related to complications, which are significantly related to delayed chemotherapy [31]. The strengths of this study included a large, population-based database inclusive of patient- and cancer-specific, but also surgical features. We cannot infer causality in variables that predicted the primary outcome. The use of an ML-based algorithm reliably predicted the risk for potential intervention. We used explainability AI to unveil the “black box” of ML, which is critical to overcome any resilience for clinical implementation. In gynaecological oncology, translatability of models is important when it comes to endpoint selections, due to the significant variability in model development [32]. Should another threshold be used, the results would have been different. Indeed, we tested the 20%EBV cut off, but the results were poor due to group imbalance. Despite the rigorous 5-fold internal validation, this single-center study design cannot provide real-world risk estimation for experienced centers. The top predictor could only be extracted from the operative details, which limits pre-operative counseling. Prospective evaluation may allow for model integration into EHR.

5. Conclusion

Our study identified the sole risk factor of weighted importance for intra-operative bleeding in EOC patients undergoing cytoreductive surgery that would require BT or use of surgical adjuncts. Provided that patients are not anaemic and have an intact anticoagulation system, the occurrence of intra-operative blood loss of at least 10%EBV was mainly associated with operative time followed by pre-treatment CA125, and a poor surgical outcome. The ROC curves indicated that by integrating these features into a single ML-based prediction system, the XGBoost reliably predicted intra-operative blood loss and anticipated requirements.

Such information will not only prevent serious post-operative mortality and morbidity but also avoid unnecessary waste of precious blood resources.