Comparative Study of Mda, Logit and Probit Models in Bank Financial Unsoundness Prediction

1. Introduction

The 2008 financial crisis and its consequences have imposed significant costs on the economies of countries worldwide. Early crisis warning systems in the banking sector proved to be ineffective (Kerstein and Kozberg 2013). The recent failures of Silicon Valley Bank, Signature Bank, and Silvergate Bank have highlighted the necessity to improve tools for identifying troubled banks more promptly (Kabir and Winters 2023).

Factors such as low asset quality, high exposure to market risk, and inadequate internal monitoring have contributed to the vulnerability of banks in Kazakhstan during times of crisis. Predicting and monitoring financially unsound banks is crucial to minimizing the costs of bank failures. This necessitates the use of statistical methods to predict bank failures as early and accurately as possible to allow for timely intervention. Previous studies have primarily focused on developed countries, particularly the USA. There is a lack of research on bankruptcy prediction models in developing countries in general, and post-Soviet countries in particular. Additionally, statistical methods such as MDA, logit, and probit models have been successfully employed in prior studies to predict bankruptcy or bank distress (e.g., Meyer and Pifer 1970; Espahbodi 1991; Catanach and Perry 2001; Canbas, Cabuk, and Kilic 2005; Ioannidis, Pasiouras, and Zopounidis 2010; Betz et al. 2014; Mitchell 2015; and Kimmel, Thornton Jr., and Bennett 2016, Othman 2018; Ekinci and Sen 2023). These studies have confirmed that statistical models have high predictive accuracy in detecting and predicting financial unsoundness.

The purpose of the current study is to employ statistical models to predict bank unsoundness for a sample of Kazakhstani banks and to develop an integrated model to improve the predictability of bank financial unsoundness.

This research examines MDA, logit, and probit statistical models and integrates them to enhance the model's predictive power. For this study, a sample of Kazakhstani banks was taken. The structure of the banking sector was determined using a cluster-based methodology described in the article by Salina, Zhang, and Hassan (2020). Six unsound Kazakhstan banks and six matching sound banks were selected as a sample for this study. Sound banks were selected based on their size (total assets), specialization, and branch networks.

The signaling ability of the MDA, logit, and probit models in predicting the financial unsoundness of banks was assessed. These models identified major indicators of unsoundness in capital adequacy, management, earnings, and liquidity. The predictability of bank failure using these models was enhanced through the calibration of cutoff points by percentile and the combination of all three models into one integrated prediction model. All constructed models exhibited high predictive accuracy. The integrated model minimized Type I errors and demonstrated superior results.

Researchers consider terms such as “unsound,” “problem,” “distressed,” and “failed” as steps leading to bankruptcy. Bankruptcy is the worst-case scenario for an organization, and therefore, the majority of research studies examine this case. “Insolvency,” “failure,” and “default” are other terms with distinct definitions. Altman (1993) posits that all these terms can be combined under the concept of “distress” and share certain common features and signs of potential distress that appear long before bankruptcy. This study aims to develop a model for forecasting financial unsoundness as an earlier step towards distress, reflecting vulnerable and unsafe conditions in Kazakhstan’s banking sector.

The initial studies on analytical coefficients for predicting potential difficulties in the financial performance of companies were conducted in the United States in the early 1930s (Horrigan, 1968). Beaver (1966) and Altman (1968) were pioneers in employing financial ratios and advanced statistical techniques to predict bank bankruptcy or failure. Since then, this approach has gained considerable popularity.

The further development of statistical models for predicting the future status of banks occurred in the 1990s. These models focus on the use of early warning systems. The impetus for studying such models was the wave of bank defaults in the United States in the early 1990s. As a result, statistical models are now used by two regulatory bodies: the Federal Reserve and the Federal Deposit Insurance Corporation (FDIC). These models are known as SEER (System for Estimating Examination Ratings) and SCOR (Statistical CAMELS Off-site Rating).

Bankruptcy forecasting models can be classified into statistical models, artificially intelligent (AI) models, and other models (Aziz and Dar, 2004; Du Jardin, 2009; Citterio, 2024). In their reviews, Aziz and Dar (2004), Du Jardin (2009), and Citterio (2024) provide a critical analysis of the most frequently used bankruptcy forecasting models and compare them in terms of their predictive powers. They concluded that these models are not significantly different from each other in terms of their effect, and that historically, researchers primarily employed statistical models. Aziz and Dar (2004) noted that more than 30% of bankruptcy prediction studies use the MDA model, while another 21% apply the logit model. Together, these models constitute 77% of the statistical models group. Likewise, Bellovary et al. (2007) demonstrated that discriminant analysis, logit analysis, and probit analysis collectively accounted for 61% of their sample, which consisted of bankruptcy studies. Logit and probit analysis belong to the same family of binary choice statistical models, but the probit model is less commonly used. The major difference between these two models lies in their function distributions: it is logistic in the case of the logit model and normal in the case of the probit model. The logit model is more attractive because it is similar to the cumulative normal function but uses simpler calculations.

Recently, academics have been motivated to develop technology-oriented models, such as Artificially Intelligent models, which can be considered a sophisticated automated outgrowth of the statistical approach. Over time, the proportion of studies utilizing statistical methods has decreased due to the emergence of numerous new methods. However, statistical models continue to play a crucial role in predicting bank failures and exhibit a high level of predictive accuracy. Despite this, Aziz and Dar (2004) noted that the superiority of AI models becomes questionable when considering the predictive powers of individual models. In this context, MDA and logit models consistently provide superior predictive accuracies and report low average Type I and Type II error rates.

Khermkhan et al. (2015) compared the forecasting efficiency of the three most popular statistical models. They found that the logit and probit models are flexible and easy to understand and explain. MDA is also an appropriate tool but requires more complex techniques to identify several multivariate groups. Indeed, these classical bankruptcy prediction models have given rise to an extensive body of literature. The MDA, logit, and probit models formed the basis for the vast majority of studies on bank bankruptcy prediction, designing prediction rules and assessing the determinants of financial failure.

Finally, Kimmel (2016) concluded that researchers use statistical methods such as MDA, logit, and probit because they are proven and widely accepted, while newer, more complex models are still under development, with no clear consensus on which version or implementation is superior. Shi, Y., & Li, X. (2019) also identified that the one of the two most frequently used and studied models in bankruptcy prediction area is Logistic Regression. However, most prior studies have been conducted in developed countries, resulting in a lack of evidence about the effectiveness of these models in predicting bank financial unsoundness in developing countries. Therefore, the current study examines the ability of three statistical models to predict bank financial unsoundness in Kazakhstan.

To achieve the purpose of this research, gain insights into the current debate, and identify prominent trends in literature, we have compiled key papers published between 1970 and 2023.

As can be seen from Table 1, these studies cover a publication period from 1970 to 2023. Starting from the 1970s, statistical models like multiple discriminant analysis (MDA), logit analysis, and probit analysis emerged as pioneering techniques for predicting financial distress and were subsequently widely employed in developing bankruptcy prediction models. Generally, statistical methods have been extensively utilized in the literature because of their high interpretability and relatively low computational complexity. Despite dedicated efforts over more than five decades, academics still tend to disagree on which particular models are more reliable, useful, and have higher prediction accuracy for predicting bank unsoundness.

Most of the studies are conducted in developed countries, particularly the USA: Meyer and Pifer (1970), Sinkey (1975), Martin (1977), Bovenzi (1983), West (1985), Espahbodi (1991), Thomson (1991), Bell (1997), Estrella, Park, and Peristiani (2000), Catanach and Perry (2001), Mitchell (2015), Affes and Hentati-Caffel (2016), and Kimmel, Thornton Jr., and Bennett (2016). There are also studies in Europe, such as those by Betz et al. (2014), Filippopoulou, Galariotis, and Spyrou (2020), and Boos et al. (2023). However, there are far fewer studies on developing countries. Examples include Rahman et al. (2004), Canbas, Cabuk, and Kilic (2005), Ioannidis, Pasiouras, and Zopounidis (2010), and Othman (2018). Additionally, Kuznetsov (2003) and Barajas, Krakovich, and López-Iturriaga (2023) used the logit model to predict the default of Russian banks, representing a case of a transition country.

Observed studies utilized either a single prediction model (45%) or two or more models (55%). They employed statistical models such as the MDA model (35%), the logit model (80%), the probit model (20%), and proposed integrated models (20%) to improve the classification accuracy of individual prediction models.

A study by Ioannidis, Pasiouras, and Zopounidis (2010) investigated a sample from several countries and used six quantitative techniques to classify banks into three groups: very strong and strong banks; adequate banks; and banks with weaknesses or serious problems. They compared models developed with only financial variables to models that incorporated regulatory, institutional, and macroeconomic variables. Models using only financial variables exhibited weak prediction accuracy, while the inclusion of country-level variables substantially improved accuracy. The highest accuracy was achieved by models using multi-criteria decision aid and artificial neural networks. Additionally, they developed stacked models that combine the predictions of individual models at a higher level. Although the stacked models outperformed the corresponding individual models.

While Ioannidis, Pasiouras, and Zopounidis (2010) developed a stacked model at a cross-country level, studies by Canbas, Cabuk, and Kilic (2005) and Othman (2018) proposed integrated models using samples from Turkish and Malaysian banks, respectively. Canbas, Cabuk, and Kilic (2005) employed four well-known statistical techniques. Principal component analysis was used to explore the basic financial characteristics of the banks. Based on these characteristics, discriminant, logit, and probit models were developed. The Integrated Early Warning System (IEWS) was effectively employed in bank supervision and could help to avoid bank restructuring costs.

Bagheri et al. (2023) concluded that combining the results of multiple predictive models improves value-at-risk estimation. Citterio (2024), in his review, concluded that ensemble methods, which construct a set of learners and integrate their outputs, achieve reduced variance, improved accuracy, and enhanced model stability, surpassing the performance of individual classifiers. Additionally, current research aims to develop an integrated model, following the approaches of Canbas, Cabuk, and Kilic (2005), Ioannidis, Pasiouras, and Zopounidis (2010), Mitchell (2015), and Othman (2018), to improve the predictability of bank financial unsoundness.

2. Results

2.1. Results: MDA Model

2.1.1. Analysis of the Independent Variables

The purpose of variables analysis is to identify the variables that can efficiently distinguish sound banks from unsound banks. Mean values, standard deviations, Wilk’s Lambda, T, F, and Mann-Whitney U-test results for the fifteen variables are calculated and presented in Table 2.

The Wilk’s lambda is used in discriminant analysis and involves the stepwise inclusion of predictors in the regression equation. It uses the criteria for inclusion of a predictor in the regression equation and the criteria to exclude a predictor from the regression equation.

A two-sample F-test for variances is used to check if the variances of two groups are the same or different, where the null hypothesis H0 is σ1=σ2. Based on the F-test result, an appropriate T-test is then chosen to compare the means. If the p-value from the F-test is smaller than 0.05, H0 is rejected, and the T-test assuming unequal variances is used. If the p-value from the F-test is higher than 0.05, H0 cannot be rejected, and the T-test assuming equal variances is used.

The Mann-Whitney U-test is a non-parametric test used to determine whether two population means are equal. Unlike the t-test and the F-test, it does not require a specific distribution of the dependent variable and is robust against outliers and heavy-tailed distributions.

As seen from Table 5.3, according to the F-tests, seven variables (R1, R2, R3, R5, R12, R13, and R15) were defined as indicators that do not have discriminating power for sound and unsound banks. The T-test selected four insignificant variables: R5, R9, R11, and R15. The Mann-Whitney U-test recognized six variables as significant, with nine variables (R1, R2, R3, R4, R5, R7, R10, R11, and R15) defined as insignificant. Thus, the current research follows the results of the F-test, T-test, and Mann-Whitney U-test given in Table 5.3. The results are ambiguous, which is why all 15 variables were taken into account for the construction of the MDA, logit, and probit models to allow the statistical methods to choose the required variables.

2.1.2. Determination of Discriminant Function Coefficients

Next, discriminant function coefficients are calculated and analyzed. The values of this function should discriminate between the two groups as clearly as possible. A measure of the success of this discrimination is the correlation coefficient between the calculated values of the discriminant function and the group membership indicator. Table 3 shows the canonical correlation coefficient for this study.

In the Function column of Table 3 the value “1” indicates that one discriminant function was obtained in the course of the discriminant analysis. If the dependent variable had not two but three levels, two discriminant functions would be composed.

The high Eigenvalue (0.958) indicates that the obtained model has a high potential for discrimination. Additionally, the high index of canonical correlation (0.700) suggests a close relationship with the variables that define this index.

Table 4 of the Wilks' Lambda lists the indicators that determine the significance of the model obtained because of discriminant analysis.

Wilks’ Lambda is a standard statistic used to denote the statistical significance of discriminating power in the current model. Its value varies from 1.0 (no discrimination) to 0.0 (complete discrimination). Wilks’ Lambda at 0.511 indicates a sufficient level of discrimination

The higher the value of Chi-square, the stronger the discriminant function distinguishes between groups, and the more effectively it fulfills its intended use. The chi-square measure of group overlap indicates that the distributions of the individual vectors of the two groups overlap substantially. Given the high degree of group overlap, the classification results are "better" than might be expected. In this case, the chi-square value is 37.976. Its consistency is demonstrated by statistical significance (Sig.), which in this case is 0.000, and noticeably lower than 0.05.

Table 5 of Standardized Canonical Discriminant Function Coefficients and Table 6 of Structure Matrix allow for the assessment of the correlation of individual independent variables used in the discriminant function with the standardized coefficients. Table 5 summarizes the standardized coefficients, and Table 8 summarizes the correlation coefficients.

Using the standardized coefficients, the relative contribution of each independent variable in the discrimination of the two study groups can be directly compared.

For example, R13 has a stronger effect on the probability of financial unsoundness than R6.

Table 6. Structure Matrix

	Function
R13	0.647
R12a	0.571
R6a	-0.380
R4a	0.335
R14	-0.297
R10a	-0.282
R7a	-0.280
R3a	0.255
R8	0.252
R1a	0.226
R2a	0.214
R11a	-0.136
R15a	0.054
R5a	-0.050
R9a	-0.042

• Pooled within-groups correlations between discriminating variables and standardized canonical discriminant functions
• Variables ordered by absolute size of correlation within function.
• a. This variable is not used in the analysis.

Table 6. Structure Matrix

	Function
R13	0.647
R12a	0.571
R6a	-0.380
R4a	0.335
R14	-0.297
R10a	-0.282
R7a	-0.280
R3a	0.255
R8	0.252
R1a	0.226
R2a	0.214
R11a	-0.136
R15a	0.054
R5a	-0.050
R9a	-0.042

• Pooled within-groups correlations between discriminating variables and standardized canonical discriminant functions
• Variables ordered by absolute size of correlation within function.
• a. This variable is not used in the analysis.

Further, the discriminant function coefficients are calculated, and the discriminant equation is derived based on them. These coefficients are included in Table 7.

As a result, given the constant, the discriminant function equation has the following form:

Z=-1.925 + 37.865×R8 + 36.726×R13 - 1.686×R14

(1)

Now, based on this equation, the probability that a bank will lose its financial soundness can be calculated.

Table 10 of the Functions at Group Centroids lists the mean values of the discriminant function for each of the analysed groups of the dependent variable.

Table 8. Functions at Group Centroids

Status	Function
Sound	0.963
Unsound	-0.963

Unstandardized canonical discriminant functions evaluated at group means

Table 8. Functions at Group Centroids

Status	Function
Sound	0.963
Unsound	-0.963

Unstandardized canonical discriminant functions evaluated at group means

Figure 3 shows the point of discrimination between the two groups of financially sound and unsound banks.

Figure 1. Plot of Bank Centroids with Financially Sound and Unsound Banks

Figure 1. Plot of Bank Centroids with Financially Sound and Unsound Banks

The point for discrimination in the estimated model is 0: if Z is higher than 0, the bank is financially sound; if it is lower, the bank is financially unsound.

2.1.3. Quality Assessment of the Model

The quality of the obtained MDA model was estimated by using an out-sample test with 2013 and 2014 data.

Appendix C shows the assessment of the quality of the model for predicting the financial unsoundness of banks using the constructed MDA model during the out-sample period. Appendix C lists the assigned status Z values calculated by the formula and the predicted status of banks in 2013 and 2014.

As can be seen from Appendix C, the MDA model has predicted the status of financially sound banks for one observation previously defined as financially unsound, such as Kazkommerts bank in 2014. Additionally, it predicted the status of financially unsound banks for three financially sound observations: Halyk Bank of Kazakhstan, SB Sberbank, and Bank Centercredit in 2013.

The classification results are summarized in Table 9 of Classification Results, where the last two rows provide information on the accuracy of predictions.

Based on Table 9 the classification of MDA model errors has been compiled and is presented in Table 10.

Table 10. Classification of MDA Model Errors 2013 - 2014

Type of Error	Number correct	% correct	% error	Total observations
Type I	11	91.7	8.3	12
Type II	9	75.0	25.0	12
Total	20	83.3	16.7	24

83.3% of original grouped cases correctly classified.

Table 10. Classification of MDA Model Errors 2013 - 2014

Type of Error	Number correct	% correct	% error	Total observations
Type I	11	91.7	8.3	12
Type II	9	75.0	25.0	12
Total	20	83.3	16.7	24

83.3% of original grouped cases correctly classified.

The results of the Multiple Discriminant Analysis in Table 10 show that Type I errors in the out-sample period were 8.3% and Type II errors were 25.0%. The overall accuracy of predictions is 83.3%. The results of the assessment of classification correctness range from 50% to 100%, so the result of 83.3% can be considered more than satisfactory.

2.2. Results: Logit Model

The logistic regression or logit model is a statistical model that can be used to predict the probability of an event by fitting the data to a logistic curve. Using the binary logistic regression the dependence of dichotomous variables on the independent variables that have any kind of scale can be elucidated.

In the case of dichotomous variables, the question is whether a certain event can occur or not. Binary logistic regression, in such a case, calculates the probability of an event occurring based on the values of the independent variables.

The main advantage of using the logit model is that there are no problems with the interpretation of the resulting indicator (p), which can have values ranging from 0 to 1 and determines the nominal value of the probability of a bank's failure.

In discriminant models, the probability of bankruptcy is not determined by a nominal value. Additionally, in discriminant models, there commonly exist so-called “zones of uncertainty,” from which it is impossible to draw an unequivocal conclusion about the probability of bankruptcy based on the calculated indicator.

In the logit models such zones do not exist because, if the assessed probability (p) is greater than 0.5, it is predicted that the event will occur and, if it is less than or equal to 0.5, it is predicted that the event will not occur.

The variables used for building the logit model are the same as those used in the discriminant analysis.

2.2.1. Determination of Logit Model Coefficients

The methods used are Inclusion: Likelihood Ratios (LR) and Exclusion: Likelihood Ratios, which are stepwise. The joint criteria for the coefficients of the model are summarized in Table 13 of the Omnibus Tests of Model Coefficients.

The methods used are the Inclusion: Likelihood Ratios (LR) and the Exclusion: Likelihood Ratios and are stepwise. The joint criteria for the coefficients of the model are summarised in Table 11 of the Omnibus Tests of Model Coefficients.

Chi-Square, step, block, or models are the criteria for the statistical significance of the effects on the dependent variable of all predictors in a specified model, block, or step. In step 1, all three criteria of Chi-square are equal for models and steps because, at step 1, they are identical for block and model as the model contains only one block. Large values of the Chi-square criterion indicate that all included variables have a significant effect on the dependent variable.

The parameters to assess the likelihood of the model accuracy are summarised in Table 12.

The value of the -2 Log Likelihood describes the model and shows how well it matches the original data. Cox and Snell's R square and Nagelkerke R square are approximations of the R value, indicating the proportion of the impact of all predictors of the model on the variance of the dependent variable.

In this study, Nagelkerke R square is 0.723, meaning that the behaviour of the dependent variable is explained at a level of 72.3% by the predictors included in the model.

Table 13 shows the effects of the inclusion of variables in the equation at each step of its compilation. The line Constant for each step corresponds to the constant a of the regression equation.

Table 13. Omnibus Tests of Model Coefficients

		B	S.E.	Wald	df	Sig.	Exp(B)
Step	R5	-0.516	0.296	3.033	1	0.082	0.597
1	R8	-245.762	101.767	5.832	1	0.016	0.000
	R13	-119.94	41.915	8.188	1	0.004	0.000
	R14	3.804	1.366	7.758	1	0.005	44.898
	Constant	9.794	3.476	7.938	1	0.005	17931.929

a. Variable(s) entered on step 1: R5, R8, R13, R14

Table 13. Omnibus Tests of Model Coefficients

		B	S.E.	Wald	df	Sig.	Exp(B)
Step	R5	-0.516	0.296	3.033	1	0.082	0.597
1	R8	-245.762	101.767	5.832	1	0.016	0.000
	R13	-119.94	41.915	8.188	1	0.004	0.000
	R14	3.804	1.366	7.758	1	0.005	44.898
	Constant	9.794	3.476	7.938	1	0.005	17931.929

a. Variable(s) entered on step 1: R5, R8, R13, R14

Wald chi-square tests the null hypothesis that the B coefficient or constant equals 0. If the p-value from the column Sig. is less than 0.05, the hypothesis is rejected, and the B coefficient or constant is not 0. Exp(B) is an odds ratio and is the exponentiation of the B coefficient. P values for all ratios and the constant are less than 0.05. Based on the Wald chi-square test and its p-value (Sig.), all coefficients of the logit model are statistically significant. Thus, equation Z will be:

Z_lfs = -0.516×R5-245.762×R8-119.94×R13+3.804×R14+9.794

(2)

2.2.2. Assessment of Logit Model Quality

An out-sample test with 2013 and 2014 data was applied to assess the quality of the constructed logit model. The calculated probability values and the prediction of distribution into groups are listed in Appendix D.

As can be seen from Appendix D, the logit model has predicted the status of financially sound banks for one case previously defined as financially unsound, such as TemirBank in 2014. It also predicted the status of financially unsound banks for two financially sound observations: Halyk Bank of Kazakhstan and Bank Centercredit in 2013.

The comparison of predicted values for the dependent variable based on the logit model and the assigned status is shown in Classification Table 14.

As the data in the last column of the Table 15 show, the results of prediction proved to be correct for 87.5% of objects. It is more convenient to interpret the results in the form of the following indicators in Table 15.

Table 15 shows that Type I errors are 8.3% and Type II errors are 16.7%. A total of 87.5% of cases are classified correctly. The predictive ability of the model is high.

This section may be divided by subheadings. It should provide a concise and precise description of the experimental results, their interpretation, as well as the experimental conclusions that can be drawn.

2.3. Results: Probit Model

In probit analysis, the probability of banks falling into one of two groups is presented as a function of the normal distribution:

$$P_{pa}={\int }_{-\infty }^{Z_{pa}}{\displaystyle \frac{1}{\sqrt{2\pi }}}{e}^{{\displaystyle \frac{-z}{2}}dz}$$

(3)

2.3.1. Determination of Probit Model Coefficients

The statistics provided in Table 16 generated in Eviews will help to assess the quality of the model; coefficients for the calculation of Zpa are also provided there.

Table 16 includes the statistics, which can be used to assess the significance of the probit model. All coefficients of the probit model are statistically significant, as seen from the z-statistics.

Z_pa = 3.247906 -75.34322 x R8 - 65.96576 × R13 + 2.474162 × R14

(4)

When determining the predicted status, the probit model calculates the probability for each object and, based on this probability, assigns one of the two values of the dichotomous variable to a bank. If the probability is less than 0.5, the bank is assessed as financially sound (the value of the variable "status" is set to 0); otherwise, the bank is considered financially unsound (the value of the variable "status" is set to 1).

2.3.2. Assessment of Probit Model Quality

The values of Zpa, the probability, and the status of banks calculated for the out-sample period 2013-2014 based on the constructed probit model are listed in Appendix E.

As seen from Appendix E, the probit model has predicted the status of financially sound banks for one case previously defined as financially unsound, such as Kazkommerts bank in 2014. It also predicted the status of financially unsound banks for three financially sound observations: Halyk Bank of Kazakhstan, SB Sberbank, and Bank Centercredit in 2013.

The quality of the assessed probit model and the correctness of classification for 2013–2014 are summarized. Based on the predicted status and the percentage of correct observations, the classification of errors in the out-sample period is compiled in Table 17.

Thus, the probit model has 8.3% of Type I errors and 25.0% of Type II errors in the 2013–2014 period. A total of 83.3% of the observations are classified correctly. This indicates a high predictive ability of the probit model during the out-sample period.

In summary, both the MDA and probit models exhibited high predictive accuracy of 83.3% during the out-of-sample period. The logit model outperformed the others, achieving a predictive accuracy of 87.5%. Furthermore, Khashei, Etemadi, and Bakhtiarvand (2024) demonstrated that the newly developed discrete direction-based logistic regression, a widely used statistical classifier for bankruptcy forecasting, provides an efficient and robust approach for achieving superior predictive performance.

The high predictive accuracy of these models is satisfactory. Nevertheless, the cut-off points were adjusted to improve the models’ performance, and the results of the three models were combined and reported in Table 18.

The new cut-off points improved the quality of all models. The predictive ability of the MDA model increased from 83.3% to 87.5%, Type II errors decreased from 25.0% to 16.7%, and Type I errors remained unchanged. The predictive accuracy of the logit model improved from 87.5% to 91.67%, with Type II errors remaining at 16.7% and Type I errors decreasing from 8.3% to 0%. Most significantly, the predictive ability of the probit model improved from 83.3% to 95.83%, with Type II errors decreasing from 25.0% to 0% and Type I errors remaining at 8.3%.

2.4. Results: Integrated Prediction Model of Bank Unsoundness

Research devoted to enhancing the prediction models has increased, and some studies integrating and comparing two or more models have appeared. Lee (1990) was one of the first researchers who integrated two models in decision support systems and noted that integration can synergistically benefit both. He affirmed that integration implies the unification of problem specifications and solution procedures, which encompass both integrating methodologies. Jo and Han (1996) suggested the use of an integrated model that used a combination of discriminant analysis, neural networks, and a case-based forecasting system. They used bankruptcy prediction to validate the effectiveness of the integrated model, and the prediction ability of the integrated model was superior to the three independent prediction techniques. They concluded that prediction error was reduced when the prediction results of various methods were combined.

Also, Lam and Moy in 2002 presented a method which combines several discriminant methods to predict the classification of new observations. They drew conclusions that as, no single-discriminant method outperforms other discriminant methods under all circumstances, decision-makers may solve a classification problem using several discriminant methods and examine their performance for classification purposes in the training sample.

The current study tries to improve the predictability of the three empirical models (MDA, logit, and probit) obtained above. They can be systematically combined to construct an integrated prediction model of bank financial unsoundness as a reliable decision tool in bank supervision and examination. This model could help increase the probability of correct forecasting.

Figure 2 shows the structure of the integrated prediction model of bank financial unsoundness and its data flow. Basically, the predicted values from the three models are employed as the input variable of the integrated model. The processes of the integrated model are presented in the view of the data flow. The model consists of four types of data: (i) four variables, (ii) computed coefficients of MDA, logit, and probit models, (iii) predicted values of the discriminant score by the MDA model and probabilities by the logit and probit models, and (iv) prediction output (sound/unsound bank).

When assessing a new bank according to the integrated model, all the system data will remain unchanged, except for the four financial ratios of the analyzed bank. These ratios are the basis for the MDA, logit, and probit models. Hence, the input to the system consists of four ratios, which are used in calculating the discriminant score and the logit and probit probabilities of bank unsoundness. The system provides early warning signals for each of the discriminant, logit, and probit models. These three empirical models together increase prediction accuracy regarding the future problems of the bank. The integrated model assigns an unsound status to a bank if even one of the three models predicts the bank as unsound.

For example, the estimated discriminant score for Kazkommerts bank on 1st January 2014 is 0.096, which is lower than the cut-off score. The estimated logit and probit unsoundness probabilities for this bank are 69.0% and 44.3%, respectively. According to the MDA and probit models, this bank is sound; according to the logit model, it is unsound. Thus, the logit model gives true information about this bank, which was actually unsound in 2014. The integrated model also assigns the status of unsound to this bank. Therefore, the integrated model provides more cautious and conservative forecasts that should reduce Type I errors. The ability to detect bank unsoundness will reduce the cost of monitoring and provide valuable information to the supervisor to prevent bank failure.

The integrated model determined the predicted statuses for each case from 1st January 2013 to 1st January 2014, as presented in Appendix F.

As can be seen from Appendix F, the integrated model has predicted the status of financially unsound for three financially sound observations: Halyk Bank of Kazakhstan, SB Sberbank, and Bank Centercredit in 2013.

The classification results of the integrated model on out-sample data for predictive accuracy, Type I errors, and Type II errors are shown in Table 19.

Table 19 shows that the predictive accuracy of the integrated model in the out-sample period is high. Type I errors are absent, and Type II errors are 25.0%. A total of 87.5% of cases are classified correctly. The integrated model did not outperform the MDA model in overall predictive accuracy. However, it reduced the rate of Type I errors compared to the MDA, logit, and probit models. As is known, Type I errors are more costly than Type II errors.

3. Discussion

We illustrate the usefulness of the integrated model to predict bank unsoundness by comparing its predictive accuracy. The comparative analysis starts by examining the accuracy of the models in predicting bank unsoundness that occurred during the sample period. This study focused on three statistical models in an attempt to evaluate their effectiveness with respect to each other, in addition to an integrated model based on all three. Four criteria will be used to assess the performance of these models, namely:

• percentage of Type I errors;
• percentage of Type II errors; and
• predictive ability of the model.

This section analyzes the predictive ability of the three empirical models and the integrated model. Prior studies conclude that integrated models produce higher prediction accuracy than individual models (Jo and Han, 1996). Results from this study are in line with these findings (Table 20).

All employed models demonstrated high overall predictive accuracy in 2013–2014. The MDA and probit models had an accuracy of 83.3%, while the logit and integrated models achieved 87.5%. The integrated model showed the lowest rate of Type I errors at 0.0%. The logit model had the lowest rate of Type II errors at 16.7%. The MDA, probit, and logit models' Type I errors were 8.3%, and the Type II errors were 25.0% for the MDA and probit models, and 16.7% for the logit model. The integrated model had the expected lowest rate of Type I errors at 0.0% and Type II errors at 25.0%. All models were effective in predicting the unsoundness status of banks, but the integrated model slightly outperformed the MDA, logit, and probit models in reducing Type I errors.

The loss from Type I errors is significantly larger than that of Type II errors because a Type I error occurs when a bank predicted to be financially sound defaults, while a Type II error implies that a bank predicted to be financially unsound survives. Sahajwala and Van den Berg (2000) confirm that Type I error is potentially more serious than Type II error because a weak bank that may escape supervision entails a higher risk. Supervisory authorities aim at minimizing the Type I error rate and calibrating models to carry a low Type I error. Therefore, the integrated model demonstrated superior results, reducing Type I errors to 0.0%.

Jo and Han (1996), Lam and Moy (2002), Canbas, Cabuk, and Kilic (2005), and Othman (2013) affirmed that combining several models provides superior information about future prospects. The results of this study show that an integrated model decreases Type I error.

Indeed, two approaches—discriminant analysis and choice models—are often compared in terms of predictive power in prior studies. Lennox (1999) and Lin (2009) noted the superiority of the logit model, while Altman et al. (1994) and Jagtiani (2003) did not find a significant difference in the predictive power of the two approaches. The results of the current study are consistent with both findings because the predicted values of these models are very close, but the predictive power of the logit model is slightly higher.

This study concludes that an integrated model can serve as an effective and cost-efficient supervisory tool, capable of identifying unsound banks over extended periods without the need for modification. This implies that the signal indicators employed by such models must remain stable over time. Furthermore, the study finds that all the examined models efficiently detect signs of bank unsoundness within a five-year horizon and can therefore be considered reliable predictive techniques. Although Bro de Comères (2025) argues that a period of four quarters preceding actual distress events is sufficiently long to enable policymakers to take preventive action once a distress signal has been issued, while at the same time limiting the likelihood of reverse causality bias.

The power of these empirical models lies in the indicators used by them. Focusing on the ability of financial ratios to highlight banks vulnerable to financial distress, the four variables reflecting capital adequacy, management, operating efficiency, and liquidity were chosen. In the models using the MDA, logit, and probit analysis, the significant coefficients calculated were the R13 interest rate spread and the R14 working capital to total assets ratio. The MDA and logit analysis consider one more ratio, R8 salary to assets. Additionally, the logit model considers the R5 debt to equity ratio.

The R13 interest rate spread is calculated as the difference between the average interest rate paid to depositors and the average interest rate earned from borrowers. This indicator reveals bank operating efficiency and allows a superior understanding of the sources of bank profitability, and hence the degree of vulnerability of its profitable sources. A negative or very low value indicates an ineffective interest rate policy or a loss, but a high value could also be a negative sign because high rates are often earned on assets that are excessively risky.

The R14 net working capital to total assets ratio is used to measure liquidity. It is an indicator from the modified Altman four-factor model for non-manufacturing companies. It is calculated as the ratio of net working capital to total assets. Working capital is the difference between current assets and current liabilities. Altman (1968) considered this indicator as the most valuable of the three liquidity ratios.

Rahman et al. (2004) proved that capital adequacy, loan management, and operating efficiency are three common performance dimensions able to identify problem banks. This result comes as no surprise, as banks with a high R5 debt to equity ratio are more fragile during crises. Estrella, Park, and Peristiani (2000) suggest using simple coefficients such as leverage to predict bank failure, as it is a very informative indicator. Also, sometimes less frequently mentioned indicators have a higher ability to discriminate depending on the particular situation and may change over time (Ohlson, 1980). In this case, they include the R8 salary to assets ratio.

These four early indicators could provide supervisory bodies with a head start in identifying the root causes of changes in a bank’s financial soundness and could potentially enhance the effectiveness of off-site monitoring. Understanding the root causes of a bank’s unsoundness is likely to improve the overall effectiveness of bank monitoring and supervision.

4. Materials and Methods

This study utilizes statistical models to predict bank financial unsoundness, specifically using MDA, logit, and probit analyses. It employs a set of indicators selected and analyzed in Salina, A.P., Zhang, X., and Hassan, O.A.G. (2020). It seeks to answer the following research question:

Can the predictability of bank financial unsoundness be improved by using statistical models such as MDA, Logit and Probit?

To achieve this goal, the predictability of these models is investigated, and then an integrated model using MDA, logit, and probit analyses is developed. The logic behind this approach is that the integration of several simple models into a more complex one is easier than building a single complex model (Ferreira and Figueiredo, 2012).

Figure 1. Process and Design of Integrated Prediction Model of Bank Unsoundness

Figure 1. Process and Design of Integrated Prediction Model of Bank Unsoundness

The process starts with data collection. Our sample consists of 12 banks: six unsound and six sound banks. The status of unsound banks was assigned by cluster analysis on January 1, 2014, as described in Salina, Zhang, and Hassan (2020). The six sound banks were matched from a group of financially sound banks based on asset size, specialization, and branch network. The entire sample comprises 12 banks, representing 81.3% of the total assets in the banking sector (Table 2).

Table 2. Selected Sample of Banks and Asset Share, 1st January, 2014

Unsound Bank	Share in Assets of Banking Sector, %	Ranking	Sound Bank	Share in Assets of Banking Sector, %
Kazkommertsbank	16.2	1	Halyk Bank of Kazakhstan	15.8
BTA Bank	9.8	2	Bank Centercredit	6.9
ATF Bank	5.8	3	SB Sberbank	6.7
Alliance Bank	3.6	4	Tsesnabank	6.0
Temirbank	2.0	5	Kaspi Bank	5.5
Total	39			42.3
Total of two groups				81.3

Table 2. Selected Sample of Banks and Asset Share, 1st January, 2014

Unsound Bank	Share in Assets of Banking Sector, %	Ranking	Sound Bank	Share in Assets of Banking Sector, %
Kazkommertsbank	16.2	1	Halyk Bank of Kazakhstan	15.8
BTA Bank	9.8	2	Bank Centercredit	6.9
ATF Bank	5.8	3	SB Sberbank	6.7
Alliance Bank	3.6	4	Tsesnabank	6.0
Temirbank	2.0	5	Kaspi Bank	5.5
Total	39			42.3
Total of two groups				81.3

Variables used in this study were carefully chosen and analyzed in the descriptive statistics in Salina, Zhang, and Hassan (2020) (Table 3).

The next step involves selecting indicators based on a review of relevant prior studies. In this study, accounting-based variables were chosen as indicators due to their proven high predictive ability. Citterio (2024) concluded that, although the predictive accuracy of macroeconomic information has been examined in several studies, there is only limited evidence regarding the influence of non-financial performance on a bank’s probability of default.

Following the CAMELS acronym, indicators were selected to reflect the main characteristics of capital adequacy, asset quality, management, earnings, and liquidity (Table 3). These 15 variables are widely used in studies devoted to bank financial soundness, distress, failure, and bankruptcy, and they reflect the nature of the banking sector.

Table 2. Description of Selected Financial Ratios

	Code	Ratio	Measurement
Capital Adequacy	R1	Capital adequacy ratio (CAR)	Equity / Total Assets
	R2	Regulatory capital to risk-weighted assets	Regulatory Capital / Risk-Weighted Assets
	R3	Regulatory Tier 1 capital to risk-weighted assets	Tier 1 Regulatory Capital / Risk Weighted Assets
	R4	Equity to debt ratio	Book Value Equity / Book Value of Total Liabilities
	R5	Debt to equity ratio (financial leverage)	Total Liabilities / Total Equity
Asset Quality	R6	Nonperforming loans to total gross loans ratio	Value of NPLs / Total Value of the Loan Portfolio
	R7	Nonperforming loans net of provisions to capital ratio	(NPLs - the Value of Specific Loan Provisions) / Total Regulatory Capital
Management	R8	Salary to assets ratio	Gross Salary Accrued / Total Assets
Earnings	R9	Return on assets (ROA)	Earnings after Tax / Total Assets
	R10	Return on equity (ROE)	(Gross Income - Gross Expenses) / Average Value of Capital
	R11	EBIT to total assets ratio	Earnings Before Interest and Tax / Total Assets
	R12	Net interest margin	(Interest Income - Interest Expenses) / Earning Assets
	R13	Interest rate spread	Lending Rate – Deposit Rate
Liquidity	R14	Working capital to total assets ratio	(Current Assets – Current Liabilities) / Total Assets
	R15	Current ratio	Average Current Assets / Average Demand Deposit Liabilities

Table 2. Description of Selected Financial Ratios

	Code	Ratio	Measurement
Capital Adequacy	R1	Capital adequacy ratio (CAR)	Equity / Total Assets
	R2	Regulatory capital to risk-weighted assets	Regulatory Capital / Risk-Weighted Assets
	R3	Regulatory Tier 1 capital to risk-weighted assets	Tier 1 Regulatory Capital / Risk Weighted Assets
	R4	Equity to debt ratio	Book Value Equity / Book Value of Total Liabilities
	R5	Debt to equity ratio (financial leverage)	Total Liabilities / Total Equity
Asset Quality	R6	Nonperforming loans to total gross loans ratio	Value of NPLs / Total Value of the Loan Portfolio
	R7	Nonperforming loans net of provisions to capital ratio	(NPLs - the Value of Specific Loan Provisions) / Total Regulatory Capital
Management	R8	Salary to assets ratio	Gross Salary Accrued / Total Assets
Earnings	R9	Return on assets (ROA)	Earnings after Tax / Total Assets
	R10	Return on equity (ROE)	(Gross Income - Gross Expenses) / Average Value of Capital
	R11	EBIT to total assets ratio	Earnings Before Interest and Tax / Total Assets
	R12	Net interest margin	(Interest Income - Interest Expenses) / Earning Assets
	R13	Interest rate spread	Lending Rate – Deposit Rate
Liquidity	R14	Working capital to total assets ratio	(Current Assets – Current Liabilities) / Total Assets
	R15	Current ratio	Average Current Assets / Average Demand Deposit Liabilities

As seen from Table 3, these 15 variables are widely used in studies devoted to bank financial soundness, distress, failure, and bankruptcy, reflecting the nature of the banking sector. For instance, Guillen (2025), in assessing the main factors that led to the collapse of Silicon Valley Bank and First Republic, found that capital adequacy, management quality, earning ability, and liquidity position ratio can predict bank failure up to five years before the official bankruptcy. Guillen’s study highlights the importance for policymakers to closely monitor these financial indicators to prevent potential bank runs and financial crises. They are also part of the IMF’s Financial Soundness Indicators (FSI) (R1, R2, R3, R6, R7, R9, R10, and R15) and the prudential norms of Kazakhstan banks (R1, R2, R3, R5, R6, R7, R10, R12, R13, and R15).

This set of 15 selected indicators was used in Salina, Zhang, and Hassan (2020) to identify the structure of the Kazakhstan banking sector by degree of financial soundness. In Salina, Zhang, Tong, and Hassan (2023), indicators R4, R9, R11, and R14 were used to test the ability of Altman’s models to predict bank financial unsoundness. This research employs the set of 15 financial ratios to construct prediction models of bank financial unsoundness using MDA, logit, and probit techniques.

Thus, the MDA, logit, and probit models are employed on a sample of 12 Kazakhstan banks annually from January 1, 2008, to January 1, 2014. Since sound and unsound groups of banks were defined on January 1, 2014, this date serves as the benchmark. This sample was divided into in-sample and out-sample groups. The first group was used for the models’ design, while the second group was used to check the models’ ability to predict financial unsoundness. The in-sample period consists of observations from January 1, 2008, to January 1, 2012, while the out-sample period includes January 1, 2013, and January 1, 2014. A set of fifteen financial ratios used in Salina, Zhang, and Hassan (2020) were computed annually for the period from January 1, 2008, to January 1, 2014 (Appendix A, Appendix B).

MDA analysis, based on certain features (independent variables), assigns the object to one of two (or a few) pre-set groups. This setting of the problem, especially in the case of two predefined groups, closely resembles the problem statement for the logistic regression method. The core of discriminant analysis is the construction of the so-called discriminant function.

D = β₁Х₁ + β₂Х₂ + ... + B_nХ_n + α,

(2)

where

D – the discriminant value;

X1 and Xn — the values of variables relevant to the cases under consideration;

β1 – βn — the coefficients to be assessed using the discriminant analysis; and

α – the constant.

The purpose is to determine such coefficients that would enable partitioning into groups with maximum accuracy based on the discriminant function values.

The logit model is used to predict the probability of an event by fitting the data to a logistic curve. Using binary logistic regression, the researcher can explore the dependence of dichotomous variables on independent variables that have any scale.

Commonly, a dichotomous variable refers to an event that may or may not occur; binary logistic regression, in such a case, estimates the probability of the event occurrence based on the values of independent variables.

The probability of event occurrence for some cases shall be calculated by the formula.

$$p={\displaystyle \frac{1}{1+e^{-z}}},$$

(2)

where

z= β₁Х₁ + β₂Х₂ + ... + B_nХ_n + α,

(2)

where

X1 and Xn — the values of variables relevant to the cases under consideration

β1 – βn — the factors to evaluate and

α – error term (the probability of Type I error occurs).

The advantage of using logit models is that there are no problems with the interpretation of the resulting indicator (R), which takes on values only in the range from 0 to 1 and determines the nominal value of the probability of entity insolvency. In logit models, such zones are absent because if the estimated probability (R) is more than 0.5, it is predicted that the event will occur, and if it is less than or equal to 0.5, the event will not occur.

For the calculation of factors in the model, methods of Inclusion: Likelihood Ratios (LR) and Exclusion: Likelihood Ratios are used, which are stepwise. Quality assessment of the model is made by calculating multiple indicators. The Log Likelihood value describes the model and shows how well it matches the original data. Cox and Snell's R square and Nagelkerke’s R square are approximations of the R-square values, indicating the share of influence of all predictors of the model on the variance of the dependent variable.

The probit model is a statistical non-linear model used in various areas, providing a method for analyzing the dependence of qualitative variables on a set of factors based on normal distribution. In econometrics, probit models are employed in models of binary or multiple choice between different alternatives to model default rates of companies.

The term "probit" is derived from the English “probability unit” and was introduced by Chester Ittner Bliss [1899—1979]. The probit model allows for the estimation of the probability that the analyzed (dependent) variable takes the value 1 at pre-set values of factors (an estimation of the share of "units" at a given value of factors). In the probit model, the probit function is modelled as a linear combination of factors, including a constant.

In probit analysis, the probability of banks falling into one of the two groups is presented as a function of the normal distribution.

$$P_{pa}={\int }_{-\infty }^{Z_{pa}}{\displaystyle \frac{1}{\sqrt{2\pi }}}{e}^{{\displaystyle \frac{-z}{2}}dz}$$

(2)

where Zpa equation takes the following form:

Z_pa = β₁Х₁ + β₂Х₂ + ... + B_nХ_n + α,

(2)

where

X1 and Xn — the values of variables relevant to the cases under consideration

β1 – βn — the factors to evaluate and

α – error term.

Logit and probit models are very similar as both are models of binary choice. The difference between the models lies in the distribution of the error term. If the error term has a standard normal distribution, the probit model should be used. If the error term has a logistic distribution, the logit model should be used.

To increase the predictability of MDA, logit, and probit models, an approach by Begley et al. (1996) and Wu et al. (2010) was used. The obtained discriminant score and probabilities from logit and probit analysis were ranked from lowest to highest. It was assumed that the superior cut-off point lies between the 25th and 75th percentiles. The cut-off points were selected as the percentile at which the sum of Type I and Type II classification errors was minimized and the predictive ability was highest.

Finally, as a concluding step, a comparison between the outcomes of all empirical models employed in this study is developed for the MDA, logit, probit, and integrated models. The comparative analysis focused on four features: the percentage of Type I errors, the percentage of Type II errors, the predictive ability of the model, and the prediction accuracy annually. Type I error represents the misclassification of an unsound bank as sound. Conversely, Type II error is a statistical term identified with misclassification by a model when the system wrongly classifies a sound bank as unsound (Sahajwala and Van Den Bergh, 2000).

5. Conclusions

This study analyzed the ability of three statistical models in predicting the financial soundness of banks, namely the MDA, logit, and probit models. In addition, it developed an integrated model based on these three models. Firstly, the explanatory power of the independent variables and the correlation between them were assessed. Next, the MDA, logit, and probit models were constructed and integrated to find the most reliable model by exploring their predictive ability. Finally, a comparative analysis of the predictive ability of the empirical models was carried out.

The empirical results of this study are listed as follows:

1. In the out-sample period, the MDA model predicted the status of financial soundness for one observation previously defined as a financially unsound bank, such as Kazkommerts bank in 2014, and the status of financial unsoundness for three financially sound observations: Halyk Bank of Kazakhstan, SB Sberbank, and Bank Centercredit in 2013.

For the Multiple Discriminant Analysis model, in the out-sample period in 2013–2014, Type I errors in the model were 8.3% and Type II errors were 25.0%. The overall accuracy of predictions is 83.3%.

2. The logit model predicted the status of financial soundness for one case previously defined as a financially unsound bank, such as TemirBank in 2014, and the status of financial unsoundness for two financially sound observations: Halyk Bank of Kazakhstan and Bank Centercredit in 2013.

Type I errors are 8.3% and Type II errors are 16.7% in the out-sample period in 2013–2014. A total of 87.5% of cases are classified correctly. The predictive ability of the model is high.

4. The probit model predicted the status of financial soundness for one case previously defined as a financially unsound bank, such as Kazkommerts bank in 2014, and the status of financial unsoundness for three financially sound observations: Halyk Bank of Kazakhstan, SB Sberbank, and Bank Centercredit in 2013.

The probit model has Type I errors at 8.3% and Type II errors at 25.0%. In general, 83.3% of observations have been classified correctly in 2013–2014.

5. The integrated model predicted all unsound banks correctly, but it assigned the status of financial unsoundness to three financially sound observations: Halyk Bank of Kazakhstan, SB Sberbank, and Bank Centercredit in 2013.

In 2013–2014, the integrated model had no Type I errors and its Type II errors were at 25.0%. A total of 87.5% of cases were classified correctly.

6. All constructed models demonstrated high predictive ability. The logit and integrated models had superior overall predictive ability to forecast bank financial unsoundness compared to the MDA and probit models. The predictive ability of the integrated model was equal to the logit model, but it proved its superiority in reducing Type I errors. This research has confirmed the conclusions of Jo and Han (1996), Canbas, Cabuk, and Kilic (2005), and Othman (2013) that when the prediction results of various methods were combined, the prediction accuracy improved.