Development and Validation of a Logistic Regression Model to Predict Post-Operative Mortality in Emergency Cardiac Surgeries: A Comprehensive Analysis of Pre-Operative Factors and Model Performance

1. Introduction

1.1. Background

Cardiac surgeries, especially in emergency situations, carry a significant risk of post-operative mortality. Accurate prediction of these outcomes is essential for improving patient care and resource allocation. Traditional risk assessment tools, such as the ASA classification system, often fall short in predictive accuracy due to their limited scope and reliance on subjective parameters. Recent advancements in machine learning have introduced models that can analyze large datasets and identify complex interactions among variables, providing more accurate predictions of surgical outcomes (Fan et al., 2021).

1.2. Objective

The primary objective of this study is to develop a logistic regression model to predict post-operative in-hospital mortality rates for patients undergoing emergency cardiac surgeries. By integrating a comprehensive set of pre-operative factors, the study aims to enhance predictive accuracy compared to traditional risk assessment tools like the ASA classification. Logistic regression was chosen for its interpretability and effectiveness in handling binary outcomes, while also exploring its performance relative to more complex machine learning models.

1.3. Significance

Identifying significant pre-operative factors that can predict patient outcomes is crucial in cardiac surgery. Early and accurate prediction of post-operative mortality can inform clinical decision-making, guide pre-operative preparations, and improve patient stratification. This can lead to better resource allocation, timely interventions, and improved survival rates for high-risk patients. Previous studies, such as Fernandes et al. (2020) and Fritz et al. (2019), have shown the potential of machine learning models in providing superior predictive performance. This study builds on these findings by focusing on the pre-operative phase, aiming to develop a reliable and interpretable logistic regression model for clinical settings. Successful implementation of this model could revolutionize pre-operative risk assessments, enabling more precise and personalized patient care, improving outcomes, and optimizing resource use in healthcare systems.

2. Methods

2.1. Data Collection

The dataset for this study was derived from patients undergoing emergency cardiac surgeries at a tertiary hospital between 2008 and 2017. Initially, 5,443 patients met the inclusion criteria, which required complete pre-operative records and crucial variables. Patients with incomplete records or missing data were excluded, resulting in a final sample size of 4,855 patients. Patient data were anonymized to maintain confidentiality and comply with ethical standards.

2.2. Variables

The analysis included demographic, anthropometric, and clinical factors. Demographic factors were age and sex. Anthropometric measurements included weight and height. Clinical factors comprised the ASA Classification and emergency status (emop). Pre-existing conditions were assessed using the Elixhauser Comorbidity Index. Preoperative laboratory values included preoperative prothrombin time/international normalized ratio (preop_ptinr), creatinine levels (preop_creatinine), potassium levels (preop_potassium), and hemoglobin levels (preop_hb). These variables were selected for their clinical relevance and established predictive value (Fan et al., 2021). The Elixhauser Comorbidity Index, calculated using standard ICD-10 codes, was chosen for its comprehensive assessment of 30 comorbidities, enhancing the model’s predictive accuracy. The ICD-10 codes used in this study were specifically chosen for their relevance to cardiac procedures. These codes encompass a comprehensive range of interventions and diagnostic measures pertinent to cardiac surgeries. By including these specific ICD-10 codes, the analysis ensures that the model accurately captures the complexity and variety of cardiac-related conditions and procedures, thus enhancing the predictive power and clinical relevance of the model. This comprehensive approach aligns with established medical coding practices and facilitates a robust assessment of patient comorbidities and surgical risk factors.

2.3. Statistical Analysis

The primary analytical approach involved developing a logistic regression model to predict post-operative mortality. Pre-operative variables were used as predictors. The Elixhauser Comorbidity Index, calculated using standard ICD-10 codes, was included for its comprehensive assessment of 30 comorbidities. Various statistical tests ensured the robustness of the findings: the Wald Test determined the significance of individual predictors, the Likelihood Ratio Test compared the goodness-of-fit of different models, and the Hosmer-Lemeshow Test assessed the model’s calibration. Statistical significance was assessed using P values. Logistic regression was chosen for its interpretability and proven effectiveness in binary outcome prediction (Fernandes et al., 2020).

2.4. Tools and Techniques

Statistical analysis and model development were conducted using Python (version 3.8). The scikit-learn library in Python was used for implementing machine learning algorithms and calculating performance metrics. The statsmodels library in Python was used for developing the logistic regression model, and the SHapley Additive exPlanations (SHAP) library in Python was employed to interpret feature contributions. These tools were selected for their robust analytical capabilities and acceptance in the scientific community (Kilic et al., 2019).

3. Results

3.1. Feature Importance

The logistic regression model identified several key pre-operative factors as significant predictors of post-operative mortality in emergency cardiac surgeries. These factors included emergency operation status (emop), ASA classification (asa), preoperative prothrombin time/international normalized ratio (preop_ptinr), Elixhauser Comorbidity Index, preoperative creatinine levels (preop_creatinine), preoperative potassium levels (preop_potassium), and preoperative hemoglobin levels (preop_hb). The ’emop’ variable was the most influential, highlighting the impact of emergency surgeries on patient outcomes. The importance of these features was quantified by their absolute coefficient values in the logistic regression model (Figure 1).

3.2. Model Performance

The logistic regression model demonstrated strong performance in predicting post-operative mortality, with an AUROC of 0.939. The model also achieved an AUPRC of 0.350, showing its capability to maintain high precision across varying recall thresholds. In terms of classification metrics, the logistic regression model exhibited an accuracy of 0.990, a precision of 1.000, a recall of 0.125, and an F1 score of 0.222. These metrics reflect the model’s robustness in minimizing false positives, though the recall rate indicates a need for improvement in identifying true positive cases. In contrast, the ASA classification system, a traditional risk assessment tool, performed significantly worse, with an AUROC of 0.524 and an AUPRC of 0.010. These results underscore the superior predictive accuracy of the logistic regression model over the ASA classification.

3.3. ROC Curve

The ROC curve for the logistic regression model underscores its predictive performance (Figure 2). The curve shows a clear distinction from the diagonal line, which represents a random classifier, and closely approaches the top left corner, indicative of high sensitivity and specificity. The AUROC value of 0.939 further confirms the model’s effectiveness.

3.4. Confusion Matrix

The confusion matrix (Figure 3) provides a detailed breakdown of the logistic regression model’s predictions against actual outcomes. Out of 1457 test cases, the model correctly predicted 1441 true negatives and 2 true positives, with no false positives and 14 false negatives. The high true negative rate underscores the model’s accuracy in identifying non-mortal cases. However, the relatively low number of true positives and the presence of false negatives highlight areas for improvement in the model’s sensitivity. Enhancing the model’s ability to capture true positive cases could improve its utility in clinical decision-making, ensuring that high-risk patients receive appropriate care.

3.5. Precision-Recall Curve

The precision-recall curve (Figure 4) illustrates the trade-off between precision and recall for the logistic regression model. The curve shows that the model maintains relatively high precision even as recall increases, with an AUPRC of 0.350. This performance is significantly better compared to the ASA classification’s AUPRC of 0.010, highlighting the logistic regression model’s superior ability to identify true positives while minimizing false positives. In clinical practice, this balance is crucial for ensuring that high-risk patients are accurately identified and treated, while avoiding the over-treatment of low-risk patients. The precision-recall curve reinforces the model’s potential as a valuable tool for improving patient outcomes in emergency cardiac surgeries.

4. Discussion

4.1. Interpretation of Results

The results indicate that the logistic regression model significantly outperformed the traditional ASA classification system in predicting post-operative mortality in emergency cardiac surgeries. The logistic regression model’s AUROC of 0.939 and AUPRC of 0.350 demonstrate its strong discriminative power. Key features such as emergency operation status, ASA classification, preoperative prothrombin time/international normalized ratio, Elixhauser Comorbidity Index, preoperative creatinine levels, preoperative potassium levels, and preoperative hemoglobin levels were significant predictors. These findings align with previous research, such as Fan et al. (2021), which highlighted the superiority of machine learning models in mortality prediction. The enhanced performance of the logistic regression model underscores its potential utility in clinical settings, providing clinicians with a more reliable tool for risk assessment.

The comparison with the ASA classification system underscores the limitations of traditional risk assessment tools. The ASA classification’s lower AUROC of 0.524 and AUPRC of 0.010 reflect its reduced effectiveness. This disparity emphasizes the need for predictive models that can integrate a broader range of variables and handle complex interactions. The superior performance of the logistic regression model, as demonstrated by Fernandes et al. (2020), suggests that incorporating diverse preoperative factors can enhance predictive accuracy, crucial for clinical practice, enabling more precise risk stratification and better-informed surgical decisions.

4.2. Strengths and Limitations

This study has several strengths, including the use of a comprehensive dataset from a tertiary hospital, providing a rich source of preoperative variables. The logistic regression model’s strong performance metrics, such as high AUROC and AUPRC, indicate the robustness of the analytical approach. The inclusion of the Elixhauser Comorbidity Index significantly enhanced the model’s predictive accuracy. These strengths contribute to the study’s reliability and relevance, offering valuable insights that can improve clinical outcomes in emergency cardiac surgeries.

However, the study has limitations. The retrospective nature of the data collection may introduce biases related to missing or incomplete records. Additionally, the study is based on data from a single institution, which may limit the generalizability of the findings. Future studies could benefit from multicenter datasets to validate the model’s applicability across different clinical environments. Prospective data collection methods could help mitigate biases associated with retrospective studies. While the logistic regression model demonstrated strong performance, there is room for improvement in capturing true positive cases, as indicated by the model’s recall rate. Enhancing data completeness and incorporating additional predictive factors could further refine the model’s accuracy.

4.3. Future Work

Future research should focus on expanding the dataset to include multiple institutions, enhancing the generalizability of the findings. Additionally, exploring more complex machine learning models, such as convolutional neural networks (CNNs) and recurrent neural networks (RNNs), could improve predictive accuracy. Studies by Fritz et al. (2019) and Yu et al. (2021) have shown the potential of deep learning models in predicting post-operative outcomes, suggesting that such approaches could be beneficial in emergency cardiac surgeries. These models can capture complex, non-linear relationships among variables and may offer superior performance.

Incorporating real-time intraoperative data into predictive models could provide dynamic risk assessments, enabling clinicians to make more informed decisions during surgeries. Techniques such as real-time data streaming and integration with electronic health records (EHR) systems could be employed. Studies by Kilic et al. (2019) and Tseng et al. (2020) have highlighted the importance of intraoperative data in improving predictive accuracy. Finally, further investigation into the specific contributions of individual preoperative and intraoperative factors could help refine and optimize predictive models. Methods such as feature selection algorithms and sensitivity analysis could identify the most impactful predictors, ensuring the models are both accurate and clinically useful.

5. Conclusion

5.1. Summary

This study developed and validated a logistic regression model to predict post-operative in-hospital mortality rates for patients undergoing emergency cardiac surgeries. Using a dataset from a tertiary hospital, the model incorporated a comprehensive set of pre-operative factors, including emergency operation status, ASA classification, and various preoperative laboratory values. The logistic regression model demonstrated superior performance with an AUROC of 0.939 and an AUPRC of 0.350, significantly outperforming the traditional ASA classification system, which had an AUROC of 0.524 and an AUPRC of 0.010. These findings underscore the importance of integrating diverse pre-operative factors to enhance predictive accuracy.

5.2. Implications

The findings of this study have important clinical implications. The superior predictive performance of the logistic regression model suggests that it can be a valuable tool in clinical settings, helping healthcare providers make more informed decisions about patient management and resource allocation. By incorporating a comprehensive set of pre-operative factors, the model provides a more accurate risk assessment, enabling timely and targeted interventions for high-risk patients. This approach can potentially improve patient outcomes, reduce post-operative complications, and enhance overall healthcare efficiency. Implementing such predictive models in routine clinical practice could transform pre-operative evaluations, leading to better-prepared surgical teams and optimized patient care pathways. However, potential barriers to implementation, such as the need for comprehensive data collection and integration into existing clinical workflows, must be addressed. Training healthcare providers on the use of these models and ensuring the availability of necessary technological infrastructure will be crucial for successful adoption. Doctors can use our calculator to understand the chance of a patient dying within 30 days of a cardiac procedure.