An Integrated Structural Equation Modeling and Machine Learning Framework for Measurement Scale Evaluation – Application to Voluntary Turnover Intentions

1. Introduction

Contemporary research in the field of work psychology and organisational management increasingly emphasises the significant role of accurate and reliable measurement of psychometric variables, such as voluntary turnover intentions. Measurement scales of this kind play a crucial role not only in modeling the mechanisms of organisational behaviour but also in predicting personnel phenomena that directly impact the functioning of enterprises [1,2,3]. Due to the substantial costs associated with employee turnover, developing tools that allow for its early detection and the explanation of predictive factors remains a problem of high applied value [4]. Despite the availability of various measurement scales, many of them are tested without simultaneously considering the quality of structural model fit and their predictive effectiveness, which limits their usefulness in practical applications.

While numerous studies have utilised either structural equation modeling (SEM) or machine learning (ML) methods to assess psychometric instruments, these approaches are typically applied in isolation, which limits their capacity to address theoretical model fit and predictive accuracy simultaneously. Traditional SEM procedures often emphasise model fit indices such as RMSEA or CFI but do not evaluate how individual items contribute to out-of-sample prediction performance [5,6]. Conversely, ML models are optimised for classification or regression accuracy but lack theoretical grounding in latent construct measurement [7]. This methodological separation creates a significant gap: current psychometric validation frameworks fail to integrate construct validity with predictive utility in a unified approach. Recent studies have highlighted the potential of combining SEM and ML, but no standardised or replicable methodology has yet emerged for doing so in scale refinement [8,9]. Addressing this gap, the present study proposes an integrative SEM-ML framework for psychometric scale evaluation that accounts for theoretical validity and predictive effectiveness.

This methodological gap defines the aim of the present article, which is to develop an integrated method for evaluating psychometric scales that combines theoretical validation with an assessment of predictive effectiveness. The approach proposed in this article integrates structural equation modeling (SEM) with machine learning (ML), allowing for simultaneous analysis of the scale's fit to the theoretical concept and its utility in case classification. To achieve the stated goal, the covariance-based SEM method was employed (with maximum likelihood as the parameter estimation method), alongside the following machine learning algorithms: naive Bayes, linear and nonlinear support vector machines, decision trees, k-nearest neighbours, and logistic regression.

Such integration places the study at the core of applied mathematics, as it merges optimisation techniques, parameter estimation, and algorithmic learning to solve real-world empirical problems in an organisational context [5,6,10]. Thus, the article contributes to the growing interest in using applied mathematics tools in analysing social and psychometric data, offering a novel approach to constructing and testing research instruments.

The Literature Review section presents the essence of the issues related to using measurement scales for voluntary turnover intentions and explains the core and mathematical formalisation of structural equation modeling (SEM) and machine learning. The Methodology section outlines the procedure of the proposed method for evaluating measurement scales, integrating structural equation modeling with machine learning. The Results chapter presents the implementation of the method using the example of evaluating a measurement scale for employee voluntary turnover intentions.

2. Literature Review

2.1. Measurement Scales for Employee Voluntary Turnover Intentions

Turnover intention refers to the likelihood or propensity of an employee to exit their current organisational affiliation voluntarily [11]. This construct is typically operationalised through temporal measurement frameworks within empirical research, capturing the individual's deliberative process regarding organisational departure [12]. Prior studies have demonstrated a significant positive association between turnover intentions and actual voluntary turnover behaviour, underscoring the predictive validity of the construct [13].

Voluntary turnover intention is one of organisational behaviour research's most frequently analysed variables. The literature indicates that turnover intentions are a reliable predictor of actual employee departures [14]. A key issue in this area is the selection of appropriate measurement tools—namely, scales for assessing turnover intentions and related psychological and organisational variables. One of the most commonly used instruments is the three-item scale developed by Mobley and colleagues [11], which includes questions about thoughts of leaving, intentions to search for a new job, and the likelihood of leaving in the near future—this scale has demonstrated good validity and reliability [15].

Subsequent research has introduced extended and multidimensional scales for measuring voluntary turnover intentions, for example:

• Maertz and Campion [16] distinguish eight dimensions of turnover (e.g., avoidance, calculative),
• Tett and Meyer [17] propose separating the measurement of intentions from the emotional reasons for leaving,
• Lee et al. [18] develop a "push-pull" scale assessed using 5-point Likert scales.
• Bothma and Roodt [19] confirm the factorial validity as well as the reliability of the TIS-6 scale.
• Ike et al. [20] proposed and evaluated twenty-five items with five-factor scale of turnover intention.

In these proposed scales, turnover intentions are strongly associated with factors such as job satisfaction [21], organisational commitment [22], and stress and burnout [23]. Schaufeli and colleagues [20] point out that indicators such as voluntary turnover intention are conceptualised as latent changes in SEM models or aggregated into composite scales.

However, an increasing number of contemporary studies are linking psychometric scale development with the construction of machine learning models. In measurement scales used as datasets for machine learning, various variables—most commonly rated on a 5-point Likert scale—are collected. Predictive analyses of this type frequently employ algorithms such as logistic regression, support vector machines, and decision trees [24].

2.2. Structural Equation Modeling

Structural Equation Modeling (SEM) is an advanced statistical method for analysing relationships between observed and latent variables. SEM combines features of factor analysis and regression modeling, allowing for the testing of complex theoretical models through the use of matrix equations [5]. An SEM model consists of two main components:

• The measurement model, which describes the relationship between latent variables and observed variables according to formula (1)

$$\mathrm{x}={\mathsf{\Lambda }}_x\xi +\delta y={\mathsf{\Lambda }}_y\eta +ϵ$$

(1)

Where:

 $\mathrm{x},y$ – vectors of observed variables,

 $\xi$ and $\eta$ – exogenous and endogenous latent variables,

 ${\mathsf{\Lambda }}_x$, ${\mathsf{\Lambda }}_y$– factor loading matrices,

 $\delta , ϵ$ – measurement errors.

2.

The structural model, which describes the relationships between latent variables (formula 2)

$$\eta =B\eta +\mathsf{\Gamma }\xi +\varsigma$$

(2)

Where:

$B$ – matrix of regression coefficients among endogenous variables,

$\mathsf{\Gamma }$ – matrix of regression coefficients from exogenous to endogenous variables,

$\varsigma$ – vector of structural errors.

The most commonly used method for parameter estimation in SEM is the Maximum Likelihood (ML) method, which involves minimising function (3).

$${\mathrm{F}}_{ML}=\ln \left|\Sigma \left(\theta \right)\right|-\ln \left|S\right|+\mathrm{tr}\left({\mathrm{S}\mathsf{\Sigma }}^{-1}\right)-p$$

(3)

Where:

$\mathsf{\Sigma }\left(\mathsf{\theta }\right)$ – model-implied covariance matrix,

$\mathrm{S}$ – observed covariance matrix,

$p$ – number of observed variables.

Alternative estimation methods include Generalised Least Squares (GLS), Unweighted Least Squares (ULS), and Bayesian SEM [10]. The fit of an SEM model to the data is assessed using multiple indices, such as those presented in Table 1 [25]; [26].

The main advantages of SEM include the ability to model latent variables while accounting for measurement error, testing complex theoretical hypotheses, and assessing both direct and indirect effects. The most commonly cited limitations of the method are its high sample size requirements (recommended N > 200), sensitivity to deviations from data normality, and the possibility of fitting a model with low theoretical validity [27].

2.3. Machine Learning

Machine learning (ML) offers a range of algorithms for classification and regression that allow for modeling relationships in data without the need to specify their functional form strictly. The present article employs several key machine learning algorithms, including: naive Bayes, linear and nonlinear support vector machines, decision trees, k-nearest neighbours, and logistic regression.

The first algorithm analysed is the naive Bayes classifier. This model is based on Bayes' theorem and the assumption of conditional independence of features [28] (formula 4):

$$P\left(C_k|x\right)={\displaystyle \frac{P\left(C_k\right){{\displaystyle \prod }}_{i=1}^{n}P\left(x_i|C_k\right)}{P\left(x\right)}}$$

(4)

Where:

Gdzie:

$P\left(C_k|x\right)$– probability of belonging to class $C_k$,

$P\left(C_k\right)$ – prior probability of class,

$P\left(x_i|C_k\right)$ – conditional probability of feature $x_i$ given class $C_k$.

In the Gaussian classifier, a normal distribution of features is assumed (formula 5):

$$P\left(x_i{C}_k\right)={\displaystyle \frac{1}{\sqrt{2\pi {\sigma }_k^2}}}\exp \left(-{\displaystyle \frac{{\left(x_i-{\mu }_k\right)}^2}{2{\sigma }_k^2}}\right)$$

(5)

The next algorithms addressed in the study are linear and nonlinear support vector machines (SVM). In the linear SVM model, for a dataset ${\left\{\left(x_i,y_i\right)\right\}}_{i=1}^n$, where and $x_i\in {\mathbb{R}}^d$ and $y_i\in \{-1,1\}$, the objective is to determine a decision function of the form $f\left(x\right)=w^{T}x+b$ that separates the classes while simultaneously solving the optimisation problem defined by the objective function (6) [29]:

$$\underset{w,b}{\min }\left({\displaystyle \frac{1}{2}}{w}^2+C{\displaystyle \sum }_{i=1}^n{\xi }_i\right)$$

(6)

under the assumption that the following margin constraints are satisfied:

$y_i\left(w^T{x}_i+b\right)\ge 1-{\xi }_i$${\xi }_i\ge 0; i$.

In the mathematical context, nonlinear SVM addresses the classification problem in its dual form by maximising the objective function (7):

$$\underset{\alpha }{\max }{\displaystyle \sum }_{i=1}^n{\alpha }_i-{\displaystyle \frac{1}{2}}{\displaystyle \sum }_{i=1}^n{\displaystyle \sum }_{j=1}^n{\alpha }_i{\alpha }_j{y}_i{y}_{j}K\left(x_i,x_j\right)$$

(7)

subject to the constraints: $0\le {\alpha }_i\le C$; ${\displaystyle \sum }_{i=1}^n{\alpha }_i{y}_i=0.$

where $K\left(x_i,x_j\right)$ is a kernel function, e.g., RBF. Once the coefficients ${\alpha }_i$ are determined, the classification of a new observation x is based on function (8):

$$f\left(x\right)={\displaystyle \sum }_{i=1}^n{a}_i{y}_{i}K\left(x_i,x\right)+b$$

(8)

In contrast to the linear variant, which operates directly on the original features, nonlinear SVM uses a kernel function to transform the data space, allowing it to handle more complex patterns more effectively [30].

Another algorithm applied in this study was decision trees. Decision trees are constructed based on data splits that maximise information gain [31]. For the entropy function (9):

$$H\left(S\right)=-{\displaystyle \sum }_{i=1}^c{p}_i{\log }_2{p}_i$$

(9)

Information gain from splitting by an attribute (10):

$$IG\left(S,A\right)=H\left(S\right)-{\displaystyle \sum }_{v\in Values\left(A\right)}{\displaystyle \frac{\left|S_v\right|}{\left|S\right|}}H\left(S_v\right)$$

(10)

Where:

 $p_i$ – frequency of class $i$,

 $S_v$ – subset of data with value $v$ of attribute $A$.

The article also applied the k-nearest neighbors (k-NN) method. In this algorithm, for a given point, the closest training points are found (11) (e.g., using the Euclidean metric) [32]:

$$d\left(x,x_i\right)=x-{x_i}_2=\sqrt{{\displaystyle \sum }_{j=1}^d{\left(x_j-x_{ij}\right)}^2}$$

(11)

The decision is made through majority voting of the classes (12):

$$\widehat{y}=\arg \underset{y}{\max }{\displaystyle \sum }_{i\in N_k\left(x\right)}1\left(y_i=y\right)$$

(12)

where 1($·$) is an indicator function that takes the value 1 if the condition is met and 0 otherwise. The final algorithm applied in the article is logistic regression. This algorithm models the probability of belonging to class 1 using the function [33] (13):

$$P\left(y=1|x\right)=\sigma \left(w^{T}x+b\right)={\displaystyle \frac{1}{1+e^{-\left(w^{T}x+b\right)}}}$$

(13)

To fit the model to the data, the log-likelihood function is maximised, expressed as (14):

$$\underset{w,b}{\max }{\displaystyle \sum }_{y=1}^n\left[y_1\log P\left(y_i|x_i\right)+\left(1-y_i\right)\log \left(1-P\left(y_i|x_i\right)\right)\right]$$

(14)

Optimisation is performed, for example, using the gradient descent method.

3. Materials and Methods

The proposed method for evaluating measurement scales using structural equation modeling and machine learning can be presented as a five-step procedure.

Step 1. Development of a dataset based on the prepared measurement scale

After the measurement scale is developed, a questionnaire study is conducted on a selected research sample. The respondents' answers are collected into a dataset. Considering the formal and substantive requirements of SEM methodology, the research sample should not be smaller than 200 participants.

Step 2. Construction of a structural model in which the latent variable is the selected psychometric construct

In this step, an SEM model is developed consisting of two components:

• Measurement model – this model tests whether all the scale's factors can be reduced to a single component (the examined psychometric construct).
• Structural model – this model tests the regression relationship between the analysed factors and the label. In this case, the label is the dependent variable, and its predictors are the factors from the psychometric scale.

At this research stage, it is necessary to determine the key SEM model fit indices, especially χ², RMSEA, CFI, and TLI. In the proposed method, it is standardly assumed that an acceptable model fit corresponds to an RMSEA value not exceeding 0.08. If the RMSEA value exceeds 0.08, it indicates that the psychometric scale is not suitable for measuring the selected psychometric construct. Although the developed method is primarily intended to enhance the performance of well-constructed psychometric scales, improving the SEM model to achieve the desired fit level is still possible even when the RMSEA slightly exceeds 0.08.

Step 3. Selection of the best machine learning algorithm for predicting the selected psychometric construct

In this step of the method, a machine learning process is conducted on the dataset using the following algorithms: naive Bayes, linear and nonlinear support vector machines, decision trees, k-nearest neighbours, and logistic regression. To avoid the issue of "lucky sampling," each algorithm is evaluated using cross-validation and repeated random splits of the data into training and test sets. For each algorithm, the average value of the prediction quality metric accuracy is calculated across all learning processes, along with the standard deviation of this metric. The algorithm with the highest average accuracy is then selected for further analysis.

Step 4. Simulation of the impact of removing factors on the SEM model and the effectiveness of the machine learning model

In this step, SEM model fit simulations are conducted by iteratively removing items from the scale. If the scale consists of n items, n SEM simulations are performed. The difference between the initial RMSEA (with no items removed) and the RMSEA after item removal is computed for each simulation.

Analogous simulations are carried out for the best-performing machine learning model (as selected in Step 3). That is, items are sequentially removed from the machine learning model, and the average accuracy metric is calculated after each removal. Differences between accuracy before and after elimination are also determined.

In the structural equation modeling literature, it is common practice to eliminate items solely based on improvements in fit indices (e.g., RMSEA, CFI, or TLI) [34]; [35]. Although such a procedure may lead to a less complex model and formally better fit, lowering the RMSEA alone does not guarantee the maintenance—or improvement—of the scale's predictive capacity. In practice, removing even a single item may reduce the measurement tool's validity in terms of classification or forecasting, thereby limiting its practical utility.

Therefore, the proposed method balances the SEM fit criterion with an assessment of each variable's contribution to the effectiveness of the machine learning model. Machine learning enables the quantification of the impact of removing a particular item on prediction quality (measured, for example, by accuracy), allowing for the selection of variables whose elimination does not deteriorate—and in the best case even improves—both SEM fit and classification quality. As a result, the scale achieves an optimal compromise: it retains theoretical construct coherence (good fit indices) while preserving the tool's real predictive power.

Step 5. Refinement of the Psychometric Scale Based on SEM-ML Simulations

Following the simulations conducted in Step 4, variables are identified whose removal improves one of the two components—SEM model fit or machine learning prediction quality—without simultaneously worsening the other. According to the proposed method, such variables should be excluded from the psychometric scale. This results in at least a non-deteriorated SEM model fit and no decrease in the predictive quality of the selected psychometric construct, with the additional benefit of a shortened scale.

In the most favourable scenario, beyond reducing the number of items in the measurement scale (which is a significant benefit in itself), both the SEM model fit and the predictive accuracy of the psychometric construct using machine learning are improved.

4. Results

Step 1. Development of a dataset based on the prepared measurement scale

Table 2 presents a custom-developed measurement scale regarding the occurrence of employee voluntary turnover intentions (after the whitening process of grey numbers).

Additionally, for machine learning purposes in particular, the survey questionnaire included a question asking whether the respondent demonstrates an intention to leave their job voluntarily. The survey was conducted between August 1 and September 30, 2024. The sample included in the present study comprised 854 individuals.

Step 2. Construction of a structural model in which the latent variable is the occurrence of employee voluntary turnover intention

The developed SEM model consisted of two components:

• Measurement model – the latent factor is voluntary turnover intention, onto which all 27 items are loaded. This model tests whether all 27 items can be reduced to a single component (turnover intention),
• Structural model – this model tests the regression relationship between the 27 items and the label, which is the occurrence of turnover intention. The label is the dependent variable in this model, and all 27 items are predictors.

The key parameters of the measurement and structural models are presented in Table 3.

In the measurement model, all loadings are statistically significant (p < 0.001) and generally high (> 0.8), which confirms that each indicator effectively reflects the latent construct. In the structural model, we examine the influence of this construct on the label (turnover intention). The negative coefficient (Estimate = –0.332, p < 0.001) indicates that a higher level of the latent construct is associated with a lower probability of turnover intention. Both variances are significant, suggesting meaningful variability in both the construct and the intention to leave. The SEM model fit indices are presented in Table 4.

The overall model fit can be considered good despite the statistically significant Chi² test (p < .001)—a typical result for large samples. The key RMSEA index of 0.073 falls below the 0.08 threshold, indicating an acceptable approximation error. The CFI = 0.878 and TLI = 0.868, though slightly below the conventional 0.90 cutoff, still suggest satisfactory model fit. Additionally, GFI = 0.856, AGFI = 0.844, and NFI = 0.856 confirm that the model structure adequately reflects the data. The AIC and BIC values can be used for comparison with alternative models, but in themselves, they raise no concerns.

Step 3. Selection of the Best Machine Learning Algorithm for Predicting the Occurrence of Voluntary Employee Turnover

Following the methodology outlined in the previous section, a training process was carried out using the following algorithms: naive Bayes, linear and nonlinear support vector machines, decision trees, k-nearest neighbors, and logistic regression. Cross-validation was used in the analysis. Table 5 presents the training process results for all algorithms, along with the standard deviations of the accuracy metric.

Based on the obtained results, it can be concluded that the analysed models perform well in predicting the occurrence of voluntary employee turnover intentions. Each of the analysed models demonstrates over 80% accuracy. For further research, the nonlinear support vector machine algorithm was the most effective of the analysed algorithms.

Step 4. Simulations of the impact of factor removal on the SEM model and on the effectiveness of the machine learning model

At the beginning of this step, simulations of the SEM model were conducted by successively excluding individual items from the scale. The results of the SEM model fit, measured by changes in the RMSEA index following successive reductions, are presented in Table 6.

In the next step, the impact of removing successive variables on the accuracy metric of the best-performing model – the nonlinear support vector machine – was verified. The results of these simulations are presented in Table 7.

Step 5. Improvement of the psychometric scale based on the conducted SEM-ML simulations

In the next step, those factors were identified whose potential removal neither worsens the fit of the SEM model (i.e., leads to a decrease in the RMSEA index or maintains it at the same level) nor reduces the predictive performance of the machine learning model (measured by the average accuracy value in the cross-validation method).

It was found that out of the 27 analysed items in the scale measuring turnover intention, three indicators meet the exclusion criteria. These factors are presented in Table 8.

The table identifies three items (X₉, X₄, X₁₈) whose exclusion from the "voluntary turnover intentions" scale does not deteriorate either the measurement validity assessed by SEM (ΔRMSEA ≥ 0) or the predictive power of the best ML model (Δaccuracy ≤ 0). The removal of X₉ and X₁₈ even leads to a slight reduction in RMSEA without any change in accuracy, while the removal of X₄ results in the most considerable reduction in RMSEA (–0.00506) with a negligible impact on classification performance (–0.00001), making these items natural candidates for elimination. As a result, a shorter and more structurally compact scale is obtained, while still maintaining satisfactory SEM fit indices and high predictive power.

In the final step, a SEM model was constructed, and the accuracy metric was calculated for the measurement scale after removing factors X₉, X₄, and X₁₈. Table 9 presents the fit indices of the new, simplified SEM model for voluntary employee turnover intention.

The results presented in the table for the simplified SEM model (after removing X₉, X₄, and X₁₈) show a clear improvement in all key fit indices compared to the initial model. RMSEA decreased from 0.073 to 0.065, indicating a significant enhancement in model quality. At the same time, CFI increased from 0.878 to 0.911, and TLI from 0.868 to 0.903 – both now exceed the commonly accepted threshold of 0.90, signaling a strong representation of the theoretical structure. GFI (0.856 → 0.890), AGFI (0.844 → 0.880), and NFI (0.856 → 0.890) also improved by more than 0.03 points, confirming the overall better quality of the model. Lower values of the information criteria AIC (from 107.4 to 97.0) and BIC (from 373.4 to 334.5) indicate that a more economical model was obtained with fewer parameters, offering a better balance between parsimony and accuracy.

The new machine learning model (nonlinear support vector machine), without variables X₉, X₄, and X₁₈, achieved an average accuracy metric (calculated via cross-validation) of 0.8630 with a standard deviation of 0.017565.

The predictive performance of the selected ML classifier (nonlinear SVM) for the shortened scale also appears promising – the mean accuracy increased from 0.862 to 0.863, and the standard deviation remained at a similar level. Although the accuracy gain is modest, it demonstrates that eliminating the three variables improved SEM fit without any loss, and even with a slight enhancement of the ML model's predictive power. These results confirm that the applied method for item selection achieves its intended trade-off: it yields a more concise and theoretically coherent measurement tool while maintaining (and even slightly improving) its practical utility in classifying turnover intentions.

5. Conclusions

This article makes a significant contribution to the field of applied mathematics by extending the methodology of psychometric scale evaluation through an integrated approach that combines covariance-based SEM with machine learning algorithms. By proposing a general algorithm that simultaneously assesses the impact of individual indicators on model fit measures (RMSEA, CFI, TLI) and their role in classification performance (accuracy), the article sheds new light on the trade-offs between construct validity and the practical utility of measurement tools. Unlike traditional studies where SEM optimisation and predictive validation are treated separately, the proposed procedure integrates SEM parameter estimation (via maximum likelihood) with variable selection processes in the context of classifiers such as nonlinear SVMs, logistic regression, and decision trees. This approach enriches the theoretical foundations of structural equation modeling with an algorithmic learning perspective and demonstrates how optimisation tools and simultaneous data analysis can be used to construct more concise and effective psychometric scale structures.

From a practical standpoint, the method provides researchers and HR professionals with a useful tool for optimising the length and validity of applied scales. It enables the identification of items whose exclusion leads to maintained or improved structural fit without degrading the predictive capacity of ML models, ultimately resulting in a shorter and more easily implementable questionnaire. In the context of human capital management, this allows for faster and more precise diagnosis of employee turnover intentions under limited research resources and reduces respondent burden. The case study on voluntary turnover intentions, conducted with a sample of over 850 individuals, demonstrates that removing three indicators from a 27-item scale is possible without any significant negative impact on construct validation or predictive accuracy. As a result, the tool becomes more economical and adaptable, while organisations benefit from a scale better suited for the rapid identification of turnover risk.

Despite its clear advantages, the developed method has certain limitations. First, the application of covariance-based SEM assumes compliance with sample size and distribution normality requirements—conditions not always met in field research. Second, the ML analysis was limited to selected classifiers and the accuracy metric; alternative performance measures were not considered, which may affect optimal variable selection. Additionally, the procedure is based on cross-sectional data and cross-validation, which does not eliminate the potential for overfitting to a specific sample. Finally, the study pertains to one specific turnover intention scale—generalising the results to other psychometric tools or organisational cultures requires further verification.

The methodological development should progress in several directions. First, it would be valuable to explore the adaptation of the procedure in the context of PLS-SEM or Bayesian SEM, allowing application in complex models and smaller samples. Second, expanding the range of ML algorithms and evaluation metrics (including multiclass scenarios or continuous data) would enable a more comprehensive assessment of the predictive utility of scales. Moreover, longitudinal analysis using panel data could reveal how stable the selection recommendations are over time and what factors influence the variability of turnover intention intensity. Finally, applying the method to areas beyond human resource management—such as social psychology or consumer research—would allow verification of the universality and scalability of the proposed approach. Such a broadening of research horizons would contribute to the fuller integration of applied mathematics and machine learning techniques in the process of creating and validating measurement tools.