Interpretable Multi-Model Framework for Early Warning of SME Loan Delinquency

1. Introduction

The rapid expansion of credit to small and medium enterprises (SMEs) has created a pressing need for accurate and interpretable tools to monitor portfolio quality and prevent systemic risk. As regulators tighten capital requirements and stress--testing frameworks, lenders must anticipate shifts in delinquency behaviour rather than simply react to losses. The literature on credit risk forecasting spans statistical models, probabilistic methods and modern machine learning techniques. Classical statistical methods such as exponential smoothing and autoregressive integrated moving average (ARIMA) models provide baseline forecasts but offer limited interpretability (De Livera et al. 2011; Hyndman and Khandakar 2008; Hyndman et al. 2002). Bayesian vector autoregressions and state--space approaches improve multivariate capability yet are computationally intensive for large portfolios (Carriero et al. 2019; Litterman 1986).

In the credit risk domain, the traditional workhorses have been logistic regression and survival analysis, which deliver well--calibrated probabilities of default but cannot easily capture non--linear interactions among borrower characteristics (Addo et al. 2018). Recent years have witnessed a proliferation of machine--learning models—including random forests, gradient boosting and deep neural networks—to enhance prediction accuracy for loan--level default and loss--given--default estimation (Addo et al. 2018; Ben Ameur et al. 2023). At the portfolio level, dynamic conditional correlation and GARCH--type volatility models highlight the importance of co--movements and tail risk in banking portfolios (Atahau et al. 2022; Wallbaum et al. 2021; Zahid et al. 2022; Likitratcharoen and Suwannamalik 2024). However, early--warning systems for SME delinquency remain under--researched, particularly in emerging markets such as Kazakhstan, and existing studies often favour black--box neural architectures that hinder regulatory acceptance (Taylor and Letham 2018).

This paper proposes a multi--stage analytical framework that balances interpretability with predictive performance for forecasting business loan delinquency. The contribution is threefold. First, we employ correlation screening and variance--inflation factor (VIF) analysis to select the most informative portfolio variables from a rich administrative dataset. Second, an ordinary least squares (OLS) regression quantifies the direction and strength of each driver, thereby offering an interpretable explanation of delinquency dynamics. Third, we layer a probabilistic Naïve Bayes classifier to anticipate the sign of monthly changes and a seasonal ARIMA (SARIMA) model to produce point and interval forecasts. The combination of explanatory and predictive models delivers actionable insights for risk management. Unlike purely neural approaches that emphasise predictive accuracy at the expense of explainability (Taylor and Letham 2018), our framework yields statistically significant coefficients and forecast intervals while still achieving competitive error rates. The remainder of the paper is organised as follows. Section 2 presents the empirical results; Section 3 discusses the findings in light of prior research; Section 4 describes the data and methods; Section 5 concludes.

2. Results

This section may be divided by subheadings. It should provide a concise and precise description of the experimental results, their interpretation, as well as the experimental conclusions that can be drawn.

2.1. Feature Selection and Linear Regression

The initial dataset contained 81 numeric attributes describing portfolio exposures, delinquency buckets, restructur-ing amounts, write--offs and customer counts. Monthly observations from January 2021 through June 2025 yield-ed 54 time points. We used Pearson correlation to identify variables highly correlated with the response variable (total delinquency). Figure 1 visualises the correlation matrix for the four most informative variables: delinquency over 15 days (Overdue 15+ days), restructurings, the number of active clients, and the target itself. Absolute corre-lations above 0.8 were considered for modelling. A stepwise VIF procedure removed multicollinear predictors, leaving three regressors.

Table 1 reports the OLS regression coefficients, which quantify how a one--unit change in each explanatory variable affects the expected delinquency (all else equal). Delinquency over 15 days exhibits the largest and most statistical-ly significant coefficient (β ≈ 1.336, p < 10⁻²²), implying that a one--thousand--unit increase in the 15+ bucket leads to roughly 1.34 thousand additional overdue loans. The coefficient for restructured loans is positive but not significant at conventional levels (p ≈ 0.18). The number of clients has a very small effect and is also statistically insignificant. The model explains approximately 98.5% of the variance (adjusted R² = 0.984), indicating that most delinquency dynamics are captured by the 15+ bucket.

Figure 2 illustrates the magnitude and sign of the regression coefficients. The bar chart underscores the dominance of the overdue 15+ days bucket and the relatively minor contributions of restructurings and the number of clients.

2.2. Probabilistic Classification

To evaluate whether delinquency is likely to increase or decrease in the following month, we constructed a binary target indicating whether the monthly change in delinquency is positive. A Gaussian Naïve Bayes classifier was trained on the three selected predictors. The model achieved an out--of--sample accuracy of roughly 55% on the last eleven observations. Although modest, the model captures the imbalanced nature of the series (rising delinquency dominated). The confusion matrix in Table 2 and Figure 3 indicates that the classifier correctly identifies 75% of decreases but only 43% of increases.

For managerial purposes it is useful to examine predicted probabilities and their evolution over time. Figure 3, Figure 4, Figure 5 and Figure 6 illustrate the probabilistic diagnostics of the Naïve Bayes classifier, including ROC and Precision–Recall performance, posterior distributions of $P\left(increase\right)$ and the resulting confusion matrix. These reveal limited separability between monthly increase and decrease states, consistent with the model’s low ROC AUC (0.371). Nevertheless, posterior probabilities provide an interpretable early-warning signal of likely directional change.

Naïve Bayes confusion matrix

Figure 4. ROC Curve — Naïve Bayes (Binary Direction of Delinquency). The AUC equals 0.371 on the chronological test set.

Figure 4. ROC Curve — Naïve Bayes (Binary Direction of Delinquency). The AUC equals 0.371 on the chronological test set.

Figure 5. Precision–Recall Curve — Naïve Bayes. The curve reflects class imbalance in monthly direction changes.

Figure 5. Precision–Recall Curve — Naïve Bayes. The curve reflects class imbalance in monthly direction changes.

Figure 6. Posterior Probability Distributions $P\left(\mathrm{increase}\right)$— Naïve Bayes. Overlapping densities indicate weak separation with the current feature set.

Figure 6. Posterior Probability Distributions $P\left(\mathrm{increase}\right)$— Naïve Bayes. Overlapping densities indicate weak separation with the current feature set.

Table 3 presents the posterior probabilities of delinquency increase for April, May and June 2025. All three months have very low values (< 6%), suggesting that the classifier expects delinquency to decline relative to March. Indeed, the observed values fall in April (Figure 7), although May shows a minor uptick not captured by the classifier. This demonstrates that even with moderate accuracy, probabilistic outputs can support qualitative decision-making in risk management.

2.3. Time--Series Forecasting

To generate point forecasts of future delinquency, we fit a seasonal ARIMA model with a first difference and two moving average terms (order (0,1,2)) and a seasonal (0,1,1) component with annual periodicity. The model was es-timated using data up to September 2024 and validated on October 2024 through February 2025. The hold--out mean absolute percentage error (MAPE) was 7.6%, while the coefficient of determination on the test set was 0.49. These metrics are competitive with more complex neural architectures reported in the literature (Caetano et al. 2023; Tsiotas et al. 2025) and satisfy the 8–9% error band recommended for business risk forecasting (Petropoulos et al. 2022). Figure 4 compares the actual series with forecasts through June 2025 and three--month projections thereafter. The model captures the upward trend and seasonal fluctuations but slightly underestimates the May 2025 peak. Table 4 lists the forecasts for the future months. The projected delinquency for June 2025 is around 26.8 thousand, consistent with the observed value.

Forecast evaluation.

Figure 8 and Figure 9 present the forecast dynamics of monthly delinquency.

The model accurately reproduces the medium-term trend, with prediction intervals remaining consistent with observed volatility.

Absolute forecast errors remain within operational tolerance across months, confirming the model’s stability and calibration on recent data.

3. Discussion

The linear regression results emphasise that the overdue 15+ days bucket (Overdue 15+ days) is the primary driver of overall delinquency. This finding aligns with empirical evidence that persistent arrears are a strong precursor to default (Addo et al. 2018). By contrast, restructuring volumes and the number of clients do not exert significant marginal effects once severe arrears are controlled for. Our VIF analysis indicates that other variables, such as the outstanding portfolio (outstanding debt) and repayments, are highly collinear with Overdue 15+ days and thus provide little incremental information.

The Naïve Bayes classifier offers a probabilistic early--warning signal. Although its accuracy is modest, the model successfully identifies most decreasing periods, which are of particular interest for provisioning and capital release. The classifier’s under--prediction of increases suggests that additional explanatory variables—perhaps macroeco-nomic indicators or borrower credit scores—could improve discrimination. More sophisticated probabilistic models, such as Bayesian additive regression trees (Burns et al. 2023) or Transformer--based quantile forecasters (Caetano et al. 2023), may enhance performance but at the cost of interpretability.

Our extended diagnostics (Figure 4, Figure 5 and Figure 6) confirm that the under-prediction of increases is primarily due to the limited separability in posterior probabilities and to the class imbalance. The ROC AUC of 0.371 and overlapping densities in Figure 3c demonstrate that the Gaussian Naïve Bayes model, while simple, offers probabilistic interpretability rather than predictive strength. In practical risk monitoring, such a model can complement regression and time-series layers by translating structural indicators (e.g., overdue 15+, restructurings, or macro shocks) into monthly probabilities of deterioration, enabling early qualitative alerts even when numerical accuracy is moderate.

The SARIMA forecasts yield error metrics comparable to those of neural architectures reported in the literature. For example, transformer models for probabilistic time series forecasting report MAPE values between 5% and 10% on real--world datasets (Caetano et al. 2023), while our model achieves 7.6%. The moderate R² reflects the inherent volatility of delinquency data but is sufficient for tactical planning. Future work could incorporate exogenous vari-ables in the SARIMA framework or utilise hybrid models such as NeuralProphet or NBEATSx (Olivares et al. 2022), which combine autoregressive components with deep neural networks.

4. Materials and Methods

4.1. Data description

The study uses a proprietary dataset of business loan portfolios provided by a regional bank in Kazakhstan. The spreadsheet contains monthly observations from January 2021 to June 2025, yielding 54 time points. Columns in-clude outstanding debt (ОД пoрт), total delinquency (Overdue), delinquency over 15 days (Overdue 15+days), non--performing loans (NPLs), restructurings (Рестр--ия), write--offs, risk zones, monthly issuances and repayments, and the number of active clients. All monetary figures are expressed in thousands of Kazakhstani tenge. The target variable is the total delinquency, which we aim to explain and forecast.

4.2. Feature Selection

Numeric features were first screened using Pearson correlation with the target. Predictors with an absolute correla-tion above 0.8 were retained. To mitigate multicollinearity, we employed a stepwise variance inflation factor (VIF) procedure: at each iteration the variable with the highest VIF above 10 was removed until at least two predictors remained. This process selected Overdue,15+ days, Рестр--ия and the number of clients. OLS regression was then estimated on these variables using the statsmodels library; significance was assessed using t-tests.

The linear regression model used to explain the delinquency time series can be written as

$$y_t={\beta }_0+{\beta }_1{x}_{1,t}+{\beta }_2{x}_{2,t}+{\beta }_3{x}_{3,t}+{\epsilon }_t,$$

where $y_t$ denotes the total delinquency in month $t$, $x_{1,t}$, $x_{2,t}$ and $x_{3,t}$ correspond to the selected predictors (15--day delinquency, restructuring volume and number of clients, respectively), ${\beta }_0$ is the intercept, ${\beta }_i$ are regression coefficients and ${\epsilon }_t$ is a zero--mean error term. The ordinary least squares estimator chooses $\widehat{\mathsf{\beta }}$ to minimise the sum of squared residuals.

4.3. Probabilistic Classification

We defined a binary class indicating whether the monthly change in delinquency was positive (increase) or non--positive (decrease). A Gaussian Naïve Bayes classifier from scikit--learn was trained on 80 % of the observations (chronologically) and evaluated on the remaining 20 %. Model performance was measured using accuracy, precision, recall and the confusion matrix. Predicted class probabilities were extracted for April–June 2025.

Under the Naïve Bayes assumption that the predictors are conditionally independent, the posterior probability of an increase is given by

$$P\left(y_t=1\mid {\mathbf{x}}_t\right)={\displaystyle \frac{\prod _{i=1}^3\phi \left(x_{i,t};{\mu }_{i,1},{\sigma }_{i,1}^2\right)}{\sum _{c\in \{0,1\}}\prod _{i=1}^3\phi \left(x_{i,t};{\mu }_{i,c},{\sigma }_{i,c}^2\right)}},$$

where ${\mathbf{x}}_t={\left(x_{1,t},x_{2,t},x_{3,t}\right)}^{\mathrm{\top }}$ denotes the predictor vector at time $t$, $\phi \left(\cdot ;\mu ,{\sigma }^2\right)$ is the Gaussian density with mean $\mu$ and variance ${\sigma }^2$, and ${\mu }_{i,c}$ and ${\sigma }_{i,c}^2$ are the class--specific mean and variance parameters estimated from the training data. The predicted class corresponds to the larger of $P\left(y_t=1\mid {\mathbf{x}}_t\right)$ and $P\left(y_t=0\mid {\mathbf{x}}_t\right)$.

4.4. Time--Series Forecasting

Due to limitations on external libraries, we used a seasonal ARIMA model rather than NeuralProphet. The SARIMA(0,1,2)×(0,1,1)ₛ pattern was selected by minimising the Akaike information criterion on the training set. The model was estimated using the statsmodels implementation. Forecast accuracy was evaluated on a hold--out sample covering October 2024–February 2025 using mean absolute percentage error (MAPE) and the coefficient of determination (R²). Multi--step forecasts were generated up to June 2025.

Formally, the seasonal ARIMA model of order $\left(0,1,2\right)\times {\left(0,1,1\right)}_s$ can be expressed as

$$\left(1-B\right)\left(1-B^s\right)y_t=\left(1+{\theta }_{1}B+{\theta }_2{B}^2\right)\hspace{0.17em}\left(1+{\Theta }_1{B}^s\right)\hspace{0.17em}{\epsilon }_t,$$

where $B$ is the backshift operator ($B{y}_t=y_{t-1}$), ${\theta }_1$ and ${\theta }_2$ are non--seasonal moving--average parameters, ${\Theta }_1$ is the seasonal moving--average parameter, $s$ denotes the seasonal period (12 months) and ${\epsilon }_t$ is white noise. Differencing the series once at the regular and seasonal lag accounts for trend and annual seasonality.

4.5. Software and Reproducibility

All analyses were performed in Python 3.11 using the pandas, statsmodels, scikit--learn and matplotlib libraries. The code and data used to generate the figures and tables are available in the accompanying repository. Where possible, generative artificial intelligence (AI) was not used beyond standard data wrangling; the authors take full responsibility for the results.

5. Conclusions

This paper proposes an interpretable, multi--model framework for forecasting business loan delinquency. By combining correlation filtering, VIF--based feature selection, linear regression, probabilistic classification and seasonal ARIMA forecasting, the framework balances explanatory insight with predictive accuracy. The 15+ days delinquency bucket emerges as the dominant driver of total arrears, underscoring the importance of early intervention for borrowers with persistent overdue balances. The Naïve Bayes model provides a crude early--warning signal, while the SARIMA forecasts achieve a MAPE of approximately 7.6 %. Future research may explore integrating macroeconomic indicators and borrower credit scores, experimenting with hybrid neural models and applying Bayesian methods to quantify parameter uncertainty.