preprints.org > computer science and mathematics > artificial intelligence and machine learning > doi: 10.20944/preprints202308.1478.v2

Preprint Review Version 2 Preserved in Portico This version is not peer-reviewed

Version 1 : Received: 21 August 2023 / Approved: 21 August 2023 / Online: 21 August 2023 (13:12:21 CEST)
Version 2 : Received: 24 September 2023 / Approved: 25 September 2023 / Online: 26 September 2023 (05:17:49 CEST)
Version 3 : Received: 7 November 2023 / Approved: 8 November 2023 / Online: 8 November 2023 (10:22:33 CET)
Version 4 : Received: 16 November 2023 / Approved: 17 November 2023 / Online: 17 November 2023 (14:15:58 CET)

In this paper, a variety of machine learning techniques, including Artificial Neural Networks, Decision Trees, Support Vector Machines, Random Forests, Logistic Regression, and three gradient boosting techniques (XGBoost, LightGBM, and CatBoost), were employed to predict customer churn in the telecommunications industry using a publicly available dataset. To address the issue of imbalanced data, various data sampling techniques, such as SMOTE, the combination of SMOTE with Tomek Links, and the combination of SMOTE with Edited Nearest Neighbors, were implemented. Additionally, hyperparameter tuning was utilized to optimize the performance of the machine learning models. The models were evaluated and compared using commonly used metrics, including Precision, Recall, F1-Score, and the Receiver Operating Characteristic Area Under Curve (ROC AUC). The results revealed that the performance of the models was enhanced by the application of hyperparameter tuning and the combined data sampling methods on the training data.Overall, after applying SMOTE, XGBoost achieved impressive ROC AUC scores of 90% and an F1-Score of 92%. When SMOTE was combined with Tomek Links, LightGBM performed exceptionally well, achieving a ROC AUC of 91%. XGBoost continued to outperform with an F1-Score of 91% and an ROC AUC of 89%. SMOTE with ENN led XGBoost to outperform other techniques with an F1-Score of 88% and an ROC AUC of 89%. However, LightGBM exhibited a performance decline of 4% in F1-Score and 1% in ROC AUC compared to exclusive SMOTE usage. Lastly, after Optuna Hyperparameter Tuning, CatBoost excelled, achieving an impressive F1-Score of 93% and an ROC AUC of 91%.

machine learning; churn prediction; imbalanced data; combined data sampling techniques; hyperparameter optimization

Computer Science and Mathematics, Artificial Intelligence and Machine Learning

* All users must log in before leaving a comment