Identification of Perioperative Risk Factors for Early Sacral Nerve Stimulator Explantation: A Single Center Retrospective Cohort Study

1. Introduction

Urinary incontinence is a prevalent condition [1,2] with a complex pathophysiology involving the central and peripheral nervous systems [3]. Sacral nerve stimulation (SNS) is a treatment option used to alleviate urinary incontinence, chronic pelvic pain and constipation in patients unresponsive to more conservative therapies [4]. SNS is generally considered safe and effective when used correctly in carefully selected patients with various conditions including urgency incontinence [5], idiopathic urinary incontinence [6], and neurogenic lower urinary tract dysfunction [7]. However, device failure and explant continue to be important problems [8,9]. Long-term response to SNS remains unacceptably low with a failure rate of up to 50% and an explantation rate of up to 16% [8,9]. With a total implant cost estimated to be over US$10,000 [10], early explantation of SNS poses a significant economic burden to patients and the healthcare system. Key reasons for removal include poor SNS efficacy, pain, infection and facilitation of magnetic resonance imaging scans [9].

Unfortunately, identification of factors associated with SNS explant remains challenging [11]. Previous work seeking to identify risk factors for explantation has identified age less than 55 as predictive of explanation [12]. However, efforts to identify other demographic, medical, social, or behavioral factors predictive of explantation have been unsuccessful [12,13]. To address this challenge, we sought to apply machine learning in the form of a multivariate logistic regression to identify demographic, medical, social, or behavioral factors predictive of early sacral nerve stimulator explantation —defined as explantation within two years of device placement.

2. Materials and Methods

Subjects: We used the Medical Informatics Operating room Vitals and Events Repository (MOVER) dataset [14] that contained de-identified electronic health records (EHR) from 58,799 unique patients undergoing surgery at the University of California Irvine Medical Center between January 2018 and July 2023.

Eligibility: We selected adult (≥18 years of age) patients who had a neuraxial procedure between January 2018 and July 2022 (N = 128 unique entries). After removing 16 patients with peripheral nerve stimulator and 58 patients with spinal cord stimulators, the overall cohort consisted of 54 unique patients with SNS. Figure 1 illustrates the sampling process.

Outcome: The primary outcome of our study was early SNS explantation. Early explantation was defined as having surgery for removal of SNS due to any reason in the first two years post-implantation.

Covariates: All study variables were created from data elements in the MOVER dataset. The International Classification of Diseases (ICD)-9-CM, procedure codes/claims (ICD-9, -10, and CPT), demographic data, medications, vital signs, American Society of Anesthesiologist (ASA) score, laboratory results, anesthesia type, and postoperative events including hospitalizations, intensive care unit (ICU) admissions, and demographics were used to create study variables. Detailed definitions of all study variables are available in Table S1.

Statistical Analysis: The data analysis plan was written before accessing the data. Numerical variables were reported as means ± standard deviation (SD) or medians with interquartile range (IQR). Categorical variables were reported as proportions or percentages. As an initial step, we compared patients with early explantation to those without early explantation using Fisher’s exact test for nominal variables and the Mann-Whitney U test for ordinal or continuous variables. Statistical significance was set at p < 0.05. Following preliminary comparisons between the groups, the data was imported into Anaconda Version 2.3.1. (Anaconda Software Distribution. Austin, TX). The following add-ons were used for analysis: sklearn23 [15], MatLab (The MathWorks, Inc., Natick, MA), pandas24 [16], scikit-learn-extra [15], and seaborn [17]. The target variable (explantation or no explantation) was defined, the features were scaled, and Synthetic Minority Oversampling Technique (SMOTE) was applied to balance the classes. The data was split into training (80%) and testing (20%) sets using random state to create deterministic train-test sets and the multivariate logistic regression model was trained. The model was then used to make predictions, and performance was evaluated using a train-test approach with reporting of precision, recall, f1-score, and the Area Under the Receiver Operating Characteristic Curve (AUC-ROC). As a next step, recursive feature elimination (RFE) was imported from sklearn.feature_selection and cross_val_score was imported from sklearn.model_selection. The algorithm iteratively evaluated different combinations of variables to identify the optimal set based on the AUC-ROC with cross-validation (Figure 2).

The optimal number of features was identified, and logistic regression was repeated as above with optimal number of features. Again, performance was measured using precision, recall, f1-score, and AUC-ROC. Once performance was assessed, the model was instructed to calculate odds ratios and 95% confidence intervals. The odds ratio was considered statistically significant (p < 0.05) if the entire 95% confidence was greater than 1 (increased risk) or less than 1 (decreased risk).

3. Results

3.1. Cohort Demographics

Baseline cohort characteristics are shown in Table 1. The overall cohort was 87.0% female with an average age of 63.7 (+/- 15.9) years. Urinary dysfunction was the most common indication, comprising 98.1% of patients. The average score ASA was 3.0 (IQR: 2-3) and the cases were predominantly done under monitored anesthesia care (72.2%). The average length of stay following implantation was 0.1 (+/- 0.5) days, with no statistically significant difference in length of stay in patients with explanation compared to those without explantation. The most common medical comorbidities were musculoskeletal pain (27.8%), sleep disorders (24.1%) arthritis (20.4%), and hypertension (16.7%). Hyperlipidemia was more frequently seen in patients without explantation (31.4%) compared to those with explantation (7.9%). No other statistically significant differences were seen between the cohorts.

3.2. Multivariate Logistic Regression Model

Given the limited associations identified by basic statistical analysis, we hypothesized an effective predictive model would need to be able to account for the interactions between multiple variables. To this end, we applied supervised machine learning in the form of a multivariate logistic regression model. The initial logistic regression model displayed good performance with an average precision of 0.81, an average recall of 0.69, and average f1-score of 0.65, and an average AUC-ROC of 0.86. Feature optimization identified 9 features: sex, ASA score, length of stay, peripheral neuropathy, low back pain, atrial fibrillation, chronic kidney disease, malignancy, and alcohol use as most predictive. Repeating the logistic regression analysis with these 9 features, the model displayed excellent performance with an average precision of 0.86, an average recall of 0.81, an average f1-score of 0.81, and an average AUC-ROC of 0.93.

Amongst the assessed variables, seven were predictive of increased risk of explantation. They were medical history of malignancy (OR: 3.88; CI: 3.84 – 3.93), female sex (OR: 3.72; CI: 3.68 – 3.76), ASA score (OR: 2.39; CI: 2.36 – 2.42), alcohol use (OR: 2.08; CI: 2.06 – 2.08), length of stay (OR: 1.92; CI: 1.90 – 1.94), peripheral neuropathy (OR: 1.86: CI: 1.83 – 1.88), and low back pain (OR: 1.82; CI: 1.79 – 1.84). Atrial fibrillation (OR: 0.38; CI: 0.37 – 0.38) and chronic kidney disease (OR: 0.47; CI: 0.47 – 0.48) were included in the model, however both conferred decreased risk of explantation (Figure 3; Table S2).

4. Discussion

SNS is an effective therapeutic option for multiple types of urinary incontinence [5-7]. However, targeting therapy to the correct patients remains challenging, with an explantation rate of 5-16% [8,12], resulting in significant expense to patients and the healthcare system [10]. Here, we applied supervised machine learning in the form of multivariate logistic regression to a large publicly available database seeking to identify factors predictive of SNS explantation. Utilizing multivariate logistic regression combined with feature optimization, we were able to create a model using 9 key patient factors which was able to predict explantation with an average AUC-ROC of 0.93. In doing so, we identified six factors predictive of increased risk of SNS explantation (malignancy, female sex, ASA score, alcohol use, length of stay during implant, peripheral neuropathy, and low back pain).

We are not the first to attempt to address this challenge. A single-center retrospective cohort study used a logistic regression approach seeking to identify patients factors predictive of SNS explantation. While they did identify age under 55 years as predictive of explantation or revision, they were unable to identify further associations with other demographic factors or medical comorbidities [12]. A second retrospective study sought to identify risk factors for battery explantation in a Medicare database. While they were able to identify interstitial cystitis as predictive of explantation, again, no other associations between demographic factors or medical comorbidities were identified [18]. Another related study sought to identify factors predictive of SNS infection, which may necessitate explantation or revision. The authors identified postoperative hematoma and a deep pocket (>3cm) as predictive of infection within 180 days following implantation [19].

In comparison to previous attempts, our approach achieved significantly higher performance and identified six novel patient factors predictive of explantation. We propose the reason for this efficacy lies in the utilization of factor optimization. Given the paucity of literature on risk factors for SNS explantation [12,18], we started our analysis using a broad battery of demographic, pre-operative, and perioperative factors. With all factors included, the logistic regression model demonstrated average performance (AUC of 0.86) with poor precision (average of 0.69). We hypothesized this was due to many factors offering limited predictive value. To optimize the model systematically and objectively, we performed factor optimization using an approach removing the least predictive factor until an optimal AUC was achieved. This resulted in improved overall performance (AUC: 0.93) and improved precision (average precision: 0.86), while simultaneously narrowing the number of assessed factors to nine total factors, with six factors conveying increased risk of explantation.

Strengths of our study include the robust number of demographics, medical history, perioperative, and behavioral factors assessed, the use of supervised machine learning, the use of factor optimization, and the robust follow up period. There were multiple limitations to our study, many of which are inherent to retrospective databases research. We were unable to perform detailed chart reviews to verify the accuracy of the database information. This is a single center study with a relatively small number of patients included (n = 54). Thus, results may not generalize to other centers. In addition, like with any observational study, we also cannot exclude the possibility that residual confounding could have biased our results. Future study will aim to prospectively assess the model performance in predicting explantation of SNS, with the aim to create a survey to identify patients at high risk of explantation prior to the procedure.