Machine Learning for Credit Risk Prediction: A Systematic Literature Review

1. Introduction

The digitization of processes and artificial intelligence (AI) is already part of our daily life and have been developing in all areas with which we interact, especially in the period of the COVID-19 pandemic [1,2]. This trend continued with the promotion of online loans and internet sales [3], Shih et al. [4] underlines that in 2019 consumer loans in the United States of America (USA) reached the order of 4 trillion US dollars, with an expectation of 4% annual growth; by 2022 it will increase to 7.9% reaching 4.8 trillion [5] and crowdfunding over the Internet will reach 300 billion dollars by 2025 [6]. The World Bank (WB) emphasizes that the banking sector is important because it improves the well-being of a country’s population and is essential for the growth of the economy [7]. In this crisis, to continue fulfilling their role and to maintain their validity, financial institutions need to reinvent themselves by generating new products and services, creating shareholder value, and improving the customer experience [8]. The Superintendency of Banking, Insurance, and AFP (SBS - Peru) maintains in its stability report of May 2022 that, in a scenario with great uncertainty caused by the "... international crisis generated by the COVID-19 pandemic and in amid the armed conflict between Russia and Ukraine, which, beyond the humanitarian crisis it has generated, is having a global economic impact." [9].

In this competitive environment, financial institutions seek to differentiate themselves, generate shareholder value, improve the customer experience, and promote financial inclusion. In this sense, they face the challenge of developing hyper-personalized products and services and distinguishing themselves from their competitors [8]. Microfinance has the challenge of adopting data-based innovation or data driver innovation (DDI), applying BIG DATA for product design, risk management, customer management, seeking efficiency, and increasing sales [10]. Information technology - cloud services, Internet of Things (IoT), BIG DATA, AI, and mobile telephony - is considered the fourth industrial revolution [11]. According to Del [8], 51% of consumers expect that banks already generate suggestions and forecast needs, even anticipating contact with customers.

For DDI and ML in financial institutions, there are still many use cases to be solved [12,13]. Hariri et al. [14], argues that there are gaps in the evaluation and uncertainty in the results when processing large volumes of information, using ML algorithms for real-time applications [10]. On the other hand, Hani et al. [10] argues that using ML techniques, through association, collaborative filtering, recommended and personalized content, systems can provide personalized assistance to users to address their individual preferences . Likewise, BIG DATA tools and technology could help increase competitiveness in changing markets, considering their potential to create value for companies and their environment [12,15].

Using machine learning in business intelligence to reduce the uncertainty of payment behavior, also called credit risk, is a necessity in the microfinance industry since it allows the analysis of large volumes of information generated [16], especially in the post-pandemic context COVID-19 and technological development [17]. The challenges are determining which configurations of attributes and algorithms are best suited and identifying limitations in the application. Consequently, we propose the research topics shown in Table 1.

2. Materials and methods

We use the Preferred Reporting Items for Systematic Review and Meta-Analysis (PRISMA) method; consequently, a search string defines our interest topic. The most relevant words identified to build our search string We apply in the recognized databases: IEEE Xplore, Scopus, and Web Of Science (WOS), considering that the studies belong to journal and conference articles published from 2019 to May 2023 and that are related to the area of computer science (view Figure 1). We identified 275 studies, 77 removes due to being duplicates, 131 delete applying the eligibility criteria: full access to the document, ranking in Scimagojr, irrelevant title or abstract, and 15 delete for not having relevance after the complete analysis of the document.

2.1. Aim

Identify models, features, and limitations that could predict credit risk using machine learning in the microfinance industry.

2.1.1. Research Questions

For this study, we propose the following research questions:

(RQ1) What are the algorithms, methods, and models used to predict credit risk?.

(RQ2) Which are the metrics to evaluate the performance of algorithms, methods, or models?.

(RQ3) What are these models’ accuracy, precision, F1 measure, and AUC?.

(RQ4) What datasets are using in the prediction of credit risk?.

(RQ5) What variables or features are using to prediction credit risk?.

(RQ6) What are the main problems or limitations of predicting credit risk?.

2.2. Research Strategy

2.2.1. Research string

TITLE-ABS-KEY ( ( "credit" OR "loan" ) AND ( "machine learning" ) AND ( "model" OR "algorithm" OR "method" ) AND ( "financial" ) AND ( "credit risk" ) ) AND ( LIMIT-TO ( PUBYEAR , 2023 ) OR LIMIT-TO ( PUBYEAR , 2022 ) OR LIMIT-TO ( PUBYEAR , 2021 ) OR LIMIT-TO ( PUBYEAR , 2020 ) OR LIMIT-TO ( PUBYEAR , 2019 ) ) AND ( LIMIT-TO ( DOCTYPE , "cp" ) OR LIMIT-TO ( DOCTYPE , "ar" ) ) AND ( LIMIT-TO ( SUBJAREA , "COMP"))

Figure 1. PRISMA Research Strategy.

Figure 1. PRISMA Research Strategy.

2.3. Selection Criteria

In this systematic review, we have considered the inclusion and exclusion criteria presented in Table 2. We also included the number of studies and their percentage of participation in the total articles identified with our research string.

2.4. Data extraction from study

To determine the importance of studies, we assess that they include: relevant conclusions or results; attributes or features of data sets; description of the proposed model or algorithm; the metrics with which the models evaluate; preprocessing techniques; identified problems or limitations; and future studies. We also consider that when the article includes more than one set of data or experiments, we choose one of them to present the metrics considering the most relevant in terms of ACC, AUC, or another metric in case the indicated ones not use. We also according to our criteria and in the cases in which it applies, we chose the UCI German Dataset dataset, considering that it is unbalanced [18], it is a frequent use, and it will allow us to compare results between investigations that include it.

2.5. Risk of bias

The main limitation when carrying out a systematic review is bias, considering that the decisions of the researchers influence the application of the method framed in the experiences and previous knowledge, for example, the choice of the topic, the choice of electronic resources, the proposal of the research questions, the methodology for data collection, and their evaluation. We have tried to meticulously follow the PRISMA method, taking into account the application of the most appropriate criteria and procedures in the construction of this document.

3. Current Research

The demand for online credits generates considerable information, which, when used through BIG DATA [19], to design new products, machine learning models, and credit risk assessment methods [20]. Consequently, in scenarios of increased demand, credit risks also increase considerably, in a non-linear manner, considering the level of risk, rate, and terms of credit [21]. In the same sense, there is an expectation of an increase in fraud in the following years [22]. Another problem to consider is the consistency of the information that records in the different stages of the process, such as sales data [23], cultural variables, environmental [24], macroeconomics [25], capacities innovation management and development, exchange rate evolution, Gross Domestic Product (GDP) growth trend [25,26], economic activity, experience [27].

The relevant problems are addressed by various research papers, using different ML approaches for the respective interpretations and decision-making [28]. However, there are also difficulties with implemented models [29], which generally follow the black box model; that is, predict, for example, the good and bad payers [30,31,32]. These models have presented problems, especially in difficult times, such as "the financial crisis of 2008" [1,33], since financial institutions focus on loans that generate the most income, being, therefore, the higher risk due to payment defaults [21,24]. It was also observed by Zou et al. [22] that automatic evaluation models, based on credit data, could confuse good paying customers for bad ones, and apply penalties on possible benefits [34]. The low explainability of advanced non-linear ML models is slowing down their implementation [35], the challenge in the development of Explainable artificial intelligence (XAI) whose objective is to provide predictive models with inherent transparency, especially in complex cases [36,37], in that sense Ariza-Garzón et al. [38] proposed the use of Shapley Additive exPlanations (SHAP), to expose the parameter dependency , and Dumitrescu et al. [39] proposes a model interpretable payment risk forecast from penalized logistic tree to enhance the performance of Logistic Regression (LR). Ma and Lv [40], indicated that the MLIA model and variable coefficient logistic regression more intuitively reveal the contribution of a variable to the predicted outcome. The "non-payment" problem is important because it could generate significant losses for financial entities [41,42]. Here the challenge of machine learning is to consider the existing multicollinearity in the input data [43,44], with high correlation of variables and some not useful for classification [2], imbalance in the data actual used, with probable overfit [22], which could generate biases in machine learning [45,46], chaotic reputation management, malicious or criminal deception [47]. In other words, the challenge is to determine the effective, relevant features [48], for example, for the training of neural networks (NN) –which is end-to-end by interactions–, additional constraints with desirable features known a priori in order to reduce the number of interactions, and prioritize smoothness, factors of explanatory interest, in the domain, control and generative modeling of features [49]. Other authors recommend the use of genetic algorithms to guide training, with data sequences that have the best result Tripathi et al. [50], the use of hive intelligence is also highlighted for this purpose [51,52,53]; noise removal using fuzzy logic (FL) and contribution to the identification of main attributes [30,54,55]. Rishehchi Fayyaz et al. [56] and Wang and Yang [19], discuss the interaction of borrowers within their supply chain. The use of images, interviews, text and social information, and interaction with mobile technology would give the credit risk assessment a multidimensional and multi-focus characteristic [17,57,58]; the inclusion of integrated accounting information with statistical variables of profitability, liquidity, asset quality, management indices, capital adequacy, efficiency, SCORECARD scorecard and the maximization of the internal rate of return (IRR), risks of the industry, GDP [1,25].

While Haro et al. [59] maintains that the most relevant characteristics are: gender, educational level, mortgage loan, microfinance, and debt balance, days past due . For de Castro Vieira et al. [60], the most relevant variable is days in arrears, especially those greater than 90 days, to determine non-payment behavior, and considers that discriminatory variables such as gender, age, and marital status should not be considered , [59], consequently the challenge of validating the quality of the features [61,62] arises. For this part, Wang and Yang [19], maintains that the number of influential variables for risk assessment has increased, and the linear and non-linear time series relationships have increased their complexity.

Alam et al. [63] proposes the following methods to address the imbalance problem: Random Undersampling (RUS), Near Miss (NMISS), Cluster Centroid (CC), Random Oversampling (ROS); Adaptive Synthetic (ADASYN), Synthetic Minority Oversampling Technique (SMOTE), Borderline-Smot (B-SMOT), and Smotetomek (SMOTE-T), other authors propose the use of CS-classifiers [64], and KFold [31,60]. However, other authors propose considering imbalance and missing data as characteristics to take into account in the [65,66] evaluation. As an initial part of the model evaluations in the experiments, the importance of optimizing the hyperparameters highlight using, for example, GA [17], K-Fold CV [39], random search (RS) [22], grid search (GS) [55,63], and other methods [65].

4. Results

4.1. Answer to RQ1: What are the algorithms, methods, and models used to predict credit risk?

In their research to forecast credit risk, the authors use ML models: 72.76% non-assembled (N-Ass) and 27.24% assembled (Ass), which are shown in Table 3. For a better presentation, in the Figure 2 and Figure 3, we group them into the following families: Boosted Category, the models related to the Boosted algorithm; Collective Intelligence, models related to collective or swarm intelligence, Fuzzy Logic, to models related to Fuzzy Logic; NN / DL, to models related to neural networks or Deep Learning (DL); Traditional, to the models cataloged but not related to previous models; and Other Models, to the models not cataloged. Of the total models used, 50.83% correspond to the Traditional model family, 27.24% to the Boosted Category, and 11.96% to NN / DL.

Analyzing the groups separately corresponding to Ass and N-Ass, and as shown in Figure 2 and Figure 3, the Category Boosted family was used at 21% and 43.9% for N-Ass and Ass, respectively; NN/DL has use of 12.8% and 9.8%; Traditional has a use rate of 61.6% and 22%; Other Models have a use at 4.6% and 3.7%; Fuzzy Logic has used at 12.2% only on Ass; and Collective Intelligence, in the order of 8.5% only in Ass.

Figure 2. Non Assembled Models.

Figure 2. Non Assembled Models.

Figure 3. Assembled Models.

Figure 3. Assembled Models.

4.2. Answer to RQ2: Which are the metrics to evaluate the performance of algorithms, methods, or models?

We collected the metrics used by the researchers in their articles, taking into account that for the cases in which they evaluate more than one data set and more than one model is applied, a pair is searching, taking into account the best ACC and AUC values, or another in the cases where these not use, as shown in the Figure 4. From this simplification, the authors propose 48% assembled models and 52% non-assembled models. Among the assembled models: Boosted Category has 21%; Collective Intelligence has 8%; NN/DL has 8%; Traditional has 8%; and Fuzzy Logic has 4%. On non-assembled models: Boost Category has 25%; Traditional has 21%; and NN/DL has 6%. In these models, the metrics used by the authors are: AUC with 16%, ACC with 14%, F1 measure with 11%, others with 11%, Precision with 10%, Recall with 9%, True Positive Rate (TPR) with 7%, True Negative Rate (TNR) with 6%, Geometric Mean (GMEAN) with 4%, Kolmogorov-Smirnov (KS) with 3%, Brier Score (BS) with 3%, GINNI Score (GINNI) with 2%, Root Mean Squared Error (RMSE), KAPPA coefficient (KAPPA), and Mean Absolute Error (MAE) whose group participation is 2%, the detail is shown in the Table 4.

4.3. Answer to RQ3: What are these models’ accuracy, precision, F1 measure, and AUC?

We list the values of the five metrics most used by the authors in their research: AUC, ACC, F1 measure, Precision, and Recall, as shown in Table 4. We have taken the values in each case according to the tuple defined in the previous question. The following figures show the metrics the authors obtained in their research Table 5.

Considering that there are experiments in which the same dataset to use, we can compare their prediction capabilities as in the case of the UCI German Dataset in which the XGB + DT model proposed by Alam et al. [63], has an ACC of 84, against the LR models proposed by Bao et al. [30], and Random forest (RF) + C4.5 proposed by García et al. [18]. Other less-used datasets on which we can compare the metrics are Tsinghua - Pekin U RESSET database and the Kaggle Prosper dataset, as shown in Table 5. It is essential to indicate that the Lending Club dataset is possible; however, it is necessary to consider different date ranges for the investigations [37,38].

4.4. Answer to RQ4: What datasets are using in the prediction of credit risk?

The dataset used by the authors can be divided into 53.85% public and 46.15% private, seeking mainly in the first case to validate new models, to be able to compare them with experiments, we show the data in the Figure 5—private datasets to used to extract knowledge by revalidating existing models in real scenarios. In the public group, the most used is the UCI German dataset with a use of 15.38%, possibly due to its characteristics [18]; the second most used is the Lending Club platform, which is a P2P loan platform with 15.38%. Some authors in their experiments validate both public and private data, Ass and N-Ass models, and determine their behavior in different scenarios.

4.5. Answer to RQ5: What variables or features are using to prediction credit risk?

The authors propose many variables using different methods to identify the variables with the best predictive capacity, genetic algorithms, fuzzy logic, hive intelligence, statistical methods, and functions to determine the correlation are applied. To simplify the analysis, we have grouped the proposed variables into Demographic, which has a 54.09% share; Operation with 29.18%; Payment behavior with 7.62%; External factors with 6.69%; Unstructured data with 1.30%, and Transaction with 1.12%, as shown in the Table 6, in the Table 7 we show details.

4.6. Answer to RQ6: What are the main problems or limitations of predicting credit risk?

In the reviewed articles, the authors state limitations or problems that they have faced during the experiments, although in each case there are nuances, we have grouped them into: Representativeness of reality with 32%, refers to the fact that many of the existing variables do not reflects the true nature of the information; Unbalanced data with 28%, refers to the fact that, according to some authors, the use of highly unbalanced data significantly reduces the performance of the models; Inconsistency in information recording with 17%, reference is made to the fact that the existing records have been entered with errors, bias, noise that generates the need to apply cleaning techniques, with the risk of losing certain information; Lack of ability to explain the proposed results with 13%, refers to the explainability limitation that the most robust ML models have; Availability of information and centralized processing with 6%, refers to the need to process information centrally, which can generate additional losses, noise and delays; and Adaptability in processing structured and unstructured information with 4%, refers to the need to process structured and unstructured data within the operation process, we show the results in the Table 8.

5. Additional findings

During the development of this RSL, we identified preprocessing techniques that the authors refer to during their experiments

5.1. Dataset balancing techniques

The most used algorithms to mitigate the problem of imbalance in the data sets are SMOTE at 28.24%, followed by KFold with 20%, RUS at 11.76%, and ROS in a similar way at 11.76%. We show the detail in Table 9. Also use: ADASYN, some variants of SMOTE like B-SMOTE, SMOTE-T, adapted classification algorithms like KN-SMOTE, Under-Bagging, techniques to identify missing values NMISS, CC, CS-Classifiers, and RESAMPLE.

5.2. Techniques for determination of hyperparameters

The techniques most used by the authors for estimating the hyperparameters are KFold at 58.33% and Grid Search at 22.22%. We show the detail in Table 10.

6. Discussion

The most widely used ML models to assess credit risk correspond to the traditional family, possibly due to their easy implementation, and many developments have already for them. On the other hand, those with the best prediction results correspond to the Boosted Category, both in the Ass and in the N-Ass groups. This trend evidences in the Table 11, where this family obtains 24 evaluations out of 52 and is constantly growing in recent years. Another trend identified is that better results are obtained with the N-Ass models [68], for example, Alam et al. [63] and Pandey et al. [27], which obtained AUC 91.00 with the AdaBoost + RF model and 91.20 with XGBoost + Decision Tree (DT), respectively, as shown in Table 5 and Figure 4, these results could be explained by gradient-based optimization features, parallelism, high volume throughput, and missing values. The most used metrics correspond to AUC, ACC, F1 measure, Recall, and Precision; the authors propose more specialized metrics according to the situation to are evaluating, AUC and ACC with 16.11% and 14.22%, mainly due to the ability to measure the capacity of different types of ML models, in the first case it does not vary before the transformation by normalization, it allows analyzing unbalanced data or with a high cost of classification; and in the second case, it works better with balanced data, and of easy explanation.

Likewise, of the experiments carried out, the most significant number of databases used are 53.85% public data sets, while 46.15% correspond to private ones; the first serves to evaluate the predictive capacity of the new models proposed by the authors, comparing the results with previous experiences and the second to generate new knowledge through its application in the real world. In the experiments, the authors identify as the main problem in the data sets for the design of ML models the misrepresentation of reality due to possible bias, inconsistency, or error when recording the information, the second problem corresponds to the imbalance of the data, which can impair the excellent performance of the models, to face this problem the SMOTE algorithm is mainly is using, and for the optimization of the hyperparameters, the Kfold CV and Grid Search Method techniques are using. However, some authors propose intelligence of hive [53] and genetic algorithms [28,52] to guide optimization. Finally, the most used variables that best represent the intention and ability to pay, which in turn originate credit risk, correspond to the demographic, transaction, and payment behavior features; these encompass the main characteristics expected to predict it, see Table 7. Although the corresponding external features and unstructured data must consider, considering the former, the influence in the hyperconnected world, the growing development of DDI, and the processing capacity of BIG DATA.

7. Conclusion

In this article, we systematically review the literature following the PRISMA method on credit risk prediction using ML. We identified 52 articles that met the inclusion and exclusion rules, relevance to the study, and full access to the document. By answering the research questions, we identified the following:

• Boosted Category is the family of ML models that are being researched the most in Ass and N-Ass, due to the best results it is showing, although the trend is its use in Ass.
• The five most used metrics are AUC, ACC, Recall, F1 measure, and Precision, although, in practice, the problem must consider choosing the most appropriate metrics.
• Public data sets are more used; of this group, the most used are UCI German Dataset and Landing Club Dataset. Its main use is to validate the behavior against other models under the same conditions. Private Datasets generate knowledge from the application to a specific situation.
• For the evaluation through ML of credits, demographic and operational variables are mainly used since they are oriented to identify patterns to predict behaviors. However, external variables and those related to unstructured data should consider the hyper-connectivity and development of DDI and BIG DATA processing.
• The main problems are the representativeness of reality, the imbalance of data for the training, and the inconsistency in recording information; both cases arise due to biases, errors, or problems in recording the information.
• The most widely used method to solve the imbalance problem is SMOTE to optimize the performance of ML models, while the methods to determine the hyperparameters are KFold-CV and Grid Search to guide their optimization.

8. Future Research

The credit risk prediction contribution corresponds to the stage that originates the loan; In this sense, we propose extending the application of ML to accurate credit data sets from specialized companies, including these models in other processes, such as credit collection and customer retention, considering the regulatory impositions that governments are implementing as an initiative to mitigate possible losses in the industry. This research can continue to focus on BIG DATA analysis, especially on unstructured data such as images, text, writing, sentiment, and hive intelligence, to assess adaptability to changing scenarios. Finally, and in the same sense, including variables that represent the state of the environment could contribute to reducing uncertainty in this sector.