Optimizing Parkinson's Disease Prediction: A Comparative Analysis of Data Aggregation Methods Using Multiple Voice Recordings via an Automated Artificial Intelligence Pipeline

1. Introduction

In the dynamic field of machine learning and artificial intelligence, comprehensive research and accumulated expertise have consistently demonstrated a fundamental principle: there is no universal, one-size-fits-all approach that can optimally solve every computational challenge or problem across diverse scenarios[1,2,3]. When dealing with diverse structures of data, it is essential to customize methodologies based on the data’s intrinsic characteristics to provide reliable outcomes. This work concentrates on a particular form of imbalanced grouped data, where each group encompasses multiple individual samples. This data structure is common in fields such as public health and medicine, where individual patients would have multiple records and diagnoses need to be made based on these records. In these fields, imbalanced data is also common and must be addressed in analyses[4].

Multiple Instance Learning (MIL) is a specialized approach designed to handle grouped data structures. This method is particularly useful when dealing with data organized into “bags,” where each bag contains multiple instances. A practical example of this is in medical research: when studying a specific disease, individual patients can be considered as bags, with their various medical readings or tests serving as instances within those bags. The distinguishing feature of MIL is its ability to work with partially labeled data. In these scenarios, labels are assigned to the bags (e.g., patient diagnoses), but the individual instances within each bag (e.g., specific test results) often lack their own labels. This characteristic makes traditional supervised learning algorithms unsuitable for direct application. The primary goal of MIL is to bridge this gap by developing techniques to transform these collections of unlabeled instances within labeled bags into single, labeled instances. This transformation allows for more effective analysis and prediction in scenarios where instance-level labels are unavailable or impractical to obtain[5,6]. In addressing multiple instance learning challenges, researchers typically employ two main strategies: the bag-based approach and the instance-based approach. The bag-based method focuses on consolidating data at the bag level. This approach involves representing each bag (a collection of instances) as a single vector. By doing so, it becomes possible to apply conventional supervised learning models to these bag-level representations. One effective technique within this approach is the use of neural network embedding to extract features for each bag. This consolidation allows for a more streamlined analysis, as it transforms the complex, multi-instance structure into a format that’s compatible with standard machine learning algorithms. Essentially, it simplifies the problem by treating each bag as a single, cohesive unit rather than a collection of separate instances. The bag-based approach offers a practical solution to the challenges posed by multiple instance learning, enabling the application of well-established supervised learning techniques to datasets that would otherwise be difficult to analyze using traditional methods[7,8,9,10].While the bag-based approach offers simplicity and ease of implementation, it comes with a significant drawback: the potential loss of valuable information from individual instances within each bag. This limitation is particularly concerning in medical research and practice. In healthcare contexts, the detailed data from each instance (such as individual test results or specific symptoms) often represents what is known as intraindividual variability. This term refers to the variations observed within a single individual over time or across different measurements. Medical professionals and researchers widely recognize the critical importance of intraindividual variability. This granular level of detail can provide crucial insights into a patient’s condition, treatment response, or disease progression that might be obscured when data is aggregated at the bag level. Therefore, while the bag-based approach simplifies analysis, it risks overlooking subtle but potentially vital patterns or indicators that exist at the instance level, which could be essential for accurate diagnosis, personalized treatment, or in-depth medical research[11].

The instance-based approach offers an alternative strategy to handle multiple instance learning challenges. This method focuses on labeling individual instances within each bag, allowing for the application of supervised learning models at the instance level. Once these models predict labels for each instance, the collective results are used to determine the overall label for the bag. One innovative technique within this approach involves clustering instances into several groups, effectively creating distinct classes. This clustering strategy provides a systematic way to assign labels to all instances, enabling a more granular analysis of the data. By preserving the detailed information of each instance, this approach can potentially capture nuanced patterns that might be lost in bag-level aggregation, though it may also introduce additional complexity in terms of computation and result interpretation[12]. The instance-based approach preserves the unique details of each instance but labelling instances requires careful design to capture the nature of instances[5]. Nevertheless, both approaches have distinct advantages and disadvantages, and the selection primarily depends on the characteristics of the data and the specific objectives of the project.

To evaluate the effectiveness of bag-based and instance-based approaches in handling multiple instances, we developed two distinct data aggregation strategies tailored to a specific dataset. Our comprehensive AI-driven analysis employs a custom-designed pipeline optimized for supervised learning, with a focus on classification tasks. This sophisticated workflow comprises three core components: data preprocessing, model fitting, and evaluation, with each component adaptable to either data aggregation strategy. We rigorously tested twelve AI algorithms, assessing their performance across seven diverse evaluation metrics. To ensure the robustness and reliability of our findings, we incorporated several advanced techniques, including data augmentation to expand our dataset, cross-validation for thorough model testing, and grid search for optimal hyperparameter tuning. This methodical approach allows for a thorough comparison of the two strategies, providing valuable insights into their relative strengths and weaknesses in multiple instance learning scenarios.

2. Parkinson Disease and Dataset Description

Parkinson’s disease (PD) is a chronic, progressive neurological disorder that primarily affects motor function[13]. It is marked by the progressive deterioration of dopamine-producing neurons in the substantia nigra, a region near the brain’s base. The loss of neurons hinders the brain’s capacity to regulate bodily movements, leading to a trifecta of hallmark motor symptoms: tremors, muscle rigidity, and bradykinesia[14,15,16,17,18]. In addition to movement abnormalities, PD frequently entails cognitive deterioration and several non-motor symptoms, including depression, sleep disturbances, and anosmia. The disorder profoundly affects patients’ quality of life and presents an escalating global health concern, with an estimated 500,000 to 1 million individuals impacted in the United States alone and a consistently increasing global occurrence[19,20,21,22,23,24,25]. Numerous research studies have consistently linked the persistent accumulation of cytotoxic intracellular and extracellular protein aggregates to various neurodegenerative disorders (NDs), including PD[26,27,28,29]. This established connection has catalyzed extensive exploration into potential pharmacological interventions, with a significant focus on developing and identifying novel non-steroidal anti-inflammatory drugs (NSAIDs) that show promise in inhibiting protein aggregation associated with NDs. These innovative compounds include Aspirin, various quinoline analogs, TDZD analogs such as PNR886 and PNR962, combretastatin-A4 analog PNR502, among others [26,27,28,29]. These compounds are being designed to target not only PD, but also a broader spectrum of NDs. The pursuit of such multi-faceted therapeutic approaches reflects the growing understanding of shared pathological mechanisms across different NDs and the potential for more comprehensive treatment strategies that could address multiple aspects of these complex disorders simultaneously. However, despite these advancements, no intervention has been able to cure PD to date, reinforcing the critical need for continued research in this field. Timely diagnosis and comprehensive care are vital for preserving patients’ independence and quality of life as the PD advances[30,31,32,33,34,35,36].

The dataset used in this research was obtained from the UCI Machine Learning Repository, specifically the Parkinson’s Disease Classification dataset (https://archive.ics.uci.edu/dataset/470/parkinson+s+disease+classification)[37]. The objective of this study is to develop AI models capable of predicting Parkinson’s disease (PD) using voice recordings as input. We utilized a comprehensive dataset consisting of 756 voice samples collected from 252 individuals, encompassing 188 PD patients (107 males and 81 females, aged 33-87) and 64 healthy controls (23 males and 41 females, aged 41-82). Each participant provided three sustained phonations of the vowel /a/, recorded at a sampling rate of 44.1 kHz. A unique identifier was assigned to link samples from the same individual. The dataset includes binary labels (1 for PD, 0 for healthy) and an extensive set of 754 features for each voice recording. These features, derived from advanced speech signal processing algorithms, comprise Time Frequency Features, Mel Frequency Cepstral Coefficients (MFCCs), Wavelet Transform based Features, Vocal Fold Features, and TWQT (Tunable Q-factor Wavelet Transform) features. This rich array of acoustic characteristics provides a robust foundation for analyzing vocal patterns associated with PD, enabling the development of sophisticated AI models for disease prediction[37]. Extensive research has revealed that vocal biomarkers, derived from patients’ speech patterns and acoustic characteristics, offer a promising avenue for the identification and diagnosis of Parkinson’s disease (PD). This method stands out for its reliability and non-invasive nature. Even in the early stages of the disease, when speech abnormalities are often subtle, advanced vocal analysis techniques can detect these changes. This capability not only suggests a powerful tool for early detection but also presents an opportunity for ongoing monitoring of disease progression. By leveraging sophisticated analysis of speech, healthcare professionals may be able to identify PD earlier and track its development more effectively, potentially leading to improved patient outcomes through timely intervention and personalized treatment strategies[38,39].

PD frequently manifests in speech abnormalities, including reduced volume (hypophonia), slowed speech rate, imprecise articulation, and voice tremor [40,41,42]. These vocal changes stem from the progressive degeneration of neural pathways governing motor control in the brain. Leveraging advanced artificial intelligence algorithms and speech processing technologies to analyze these vocal biomarkers offers a promising avenue for early PD detection and diagnosis [43]. This non-invasive method may facilitate earlier diagnosis and intervention strategies, thereby enabling more timely and targeted treatment with the potential to halt the progression of Parkinson’s disease and enhance the quality of life for individuals with the condition[44]. The application of AI in vocal analysis has shown remarkable potential, with some studies demonstrating the ability of artificial neural networks (ANNs) to detect PD based on voice samples of vowels, with test accuracy rates as high as 86.47%[45].By utilizing these technologies, clinicians could potentially intervene at the pre-clinical stage of PD, a critical period when neuroprotective therapies might be most effective in preserving neurological function.

In this research, we leveraged the meticulously pre-processed features from the original dataset to construct a bespoke multi-algorithm AI pipeline specifically designed for PD classification. Our approach builds upon this solid foundation, emphasizing the implementation and refinement of various AI algorithms. The primary objective of our study is to enhance the accuracy of PD detection while eliminating the need for additional signal processing steps. By focusing on algorithm application and optimization within our custom pipeline, we aim to streamline the classification process and potentially improve its effectiveness, contributing to the advancement of non-invasive diagnostic tools for PD.

3. Methods (Workflow Description)

This study was implemented using Python, with the scikit-learn package[46] serving as the primary tool for data preprocessing and AI tasks.

3.1. Data Preprocessing

The data preprocessing stage involved two key steps: first, splitting the data into training and test sets, and second, conducting data augmentation to address issue of class imbalance.

3.1.1. Train-Test Split

To assess the generalizability of our AI pipeline, we implemented a strategic 70:30 split of the dataset into training and test sets [47]. This division resulted in a training set of 528 voice recordings from 176 subjects and a test set of 228 recordings from 76 subjects. Crucially, we employed stratified sampling based on the diagnosis status to maintain a consistent distribution of PD cases (approximately 75%) and healthy controls across both sets. This stratification is essential for ensuring reliable performance evaluation, especially given the imbalanced nature of our dataset. By preventing overrepresentation of the majority class in either set, this approach enhances the robustness of our model assessment and helps guarantee that our AI pipeline’s performance is accurately evaluated across diverse data subsets[48].

3.1.2. Bag Over-sampling (BOS)

The dataset exhibits a mild class imbalance, with Parkinson’s disease (PD) cases outnumbering healthy controls in a ratio of approximately 3:1. Although this level of imbalance may not significantly impact the classification performance of our models, it remains an important consideration, particularly when contemplating real-world applications. For instance, in population-based screening scenarios, the class distribution is likely to be reversed, with healthy individuals substantially outnumbering those with PD. Addressing this imbalance is crucial for ensuring our models can generalize effectively to diverse real-world situations, maintain fairness in predictions, and potentially optimize performance across various population distributions[49].

Data augmentation was implemented on the training set to enable the AI models to place more emphasis on the minority class during the training phase. Specifically, we use Bag Over-sampling (BOS) [50], which is a generalization of Synthetic Minority Oversampling Technique (SMOTE) [51] in MIL. Since this is an interpolation method in Euclidean space, all features of the voice recordings are first standardized to mitigate the influence of different feature scales.

In the BOS algorithm, a new bag is synthesized based on two existing bags. Since there is no information showing that the three recordings from each subject have any inherent order or weight, we assumed that the recordings are equivalent and should be treated with equal importance for each subject [37]. Considering two existing bags (i.e., subject) ${\mathit{B}}_{\mathit{i}}$ and ${\mathit{B}}_{\mathit{j}}$, an instance (i.e., voice recording) is randomly sampled from each of them, say ${\mathit{x}}_{\mathit{p}}^{\mathit{i}}$ and ${\mathit{x}}_{\mathit{q}}^{\mathit{j}}$. Then an instance in the new bag ${\mathit{B}}_{\mathit{i}\mathit{j}}$ can be generated as:

$$\begin{array}{c}{\mathit{x}}_{\mathit{p}\mathit{q}}^{\mathit{i}\mathit{j}}={\mathit{x}}_{\mathit{p}}^{\mathit{i}}+{\delta }_{pq}\times \left({\mathit{x}}_{\mathit{q}}^{\mathit{j}}-{\mathit{x}}_{\mathit{p}}^{\mathit{i}}\right),\end{array}$$

(1)

where ${\delta }_{pq}\in \left[0,1\right]$ is a random number. This process assigns equal sampling probabilities to the three instances in a bag, ensuring equivalent treatment. This step is repeated three times and then a new bag ${\mathit{B}}_{\mathit{i}\mathit{j}}$ containing three instances is synthesized.

To select two bags for synthesis, the $K$-nearest neighbor method is used. For a randomly selected bag, the$K$-nearest neighbors ($K=2$ in our case) is identified, and new bags are then synthesized based on this bag and each of its neighbors respectively. Since this method requires the distance between bags, we defined it as the distance between their centroids, which are calculated as the mean vector of the three instances in a bag. The oversampling process is repeated for the minority class until the ratio of the two classes reaches 1:1, thereby completing the data augmentation process.

3.2. Handling Multiple Instances

In this PD dataset, each subject provided three voice recordings, resulting in multiple instances that need to be addressed through MIL[52]. Therefore, we designed two customized data aggregation strategies, one following the instance-based approach and another following the bag-based approach, to handle these multiple instances.

3.2.1. Post-aggregation strategy

Our exploratory data analysis (EDA) revealed a high degree of similarity among the three voice recordings from each subject. When clustered, these recordings typically grouped together, suggesting they would likely receive the same label. This observation led us to adopt a straightforward approach: assigning the subject’s overall label to all three of their voice recordings. This strategy eliminates the need for a separate clustering process and provides practical meaning to the labels. Consequently, we labeled all recordings from Parkinson’s disease (PD) patients as diseased (1) and those from healthy individuals as non-diseased (0). This approach allows AI algorithms to directly predict whether a voice recording indicates the presence of disease, simplifying the classification task while maintaining clinical relevance.

Once predictions for the voice recordings are generated, they must be aggregated to produce the final predictions for each subject, a process known as post-aggregation. We proposed three aggregation methods: mean, min, and max. These methods calculate the mean, minimum, and maximum of the predicted probabilities of being diseased from the three voice recordings to represent the overall likelihood of each subject having Parkinson’s disease (PD). This aggregation process ensures that we capture a comprehensive assessment of the subject’s condition based on their voice recordings. These methods can be represented as follows:

$$\begin{array}{c}{P}_i^{mean}={\displaystyle \frac{p_{i,1}+p_{i,2}+p_{i,3}}{3}},\end{array}$$

(2)

$$\begin{array}{c}{P}_i^{min}=\min \left\{p_{i,1},p_{i,2},p_{i,3}\right\},\end{array}$$

(3)

$$\begin{array}{c}{P}_i^{max}=\max \left\{p_{i,1},p_{i,2},p_{i,3}\right\},\end{array}$$

(4)

where $p_{i,j}$ is the predicted probability of being diseased for the $j$th voice recording of the $i$th subject, and $P_i^{mean}$, $P_i^{min}$, $P_i^{max}$ are the three kinds of overall probabilities of having PD for the $i$th subject derived from the three post-aggregation methods. During this process, we avoid assigning any different weights to the three voice recordings from each subject to ensure the equivalent treatments. The overall probability is then compared to a threshold of 0.5 to determine whether the subject is classified as having PD or not.

The selection of specific aggregation methods—mean, min, and max—was based on varying priorities. The post-min aggregation method requires all three voice recordings to be classified as diseased in order to predict that the subject has Parkinson’s disease (PD). This approach is ideal for situations where it is crucial to avoid misdiagnosing healthy individuals as diseased. Conversely, the post-max aggregation method predicts a subject as having PD if any one of their three voice recordings is classified as diseased, making it suitable for scenarios where it is vital not to overlook individuals with the disease. The post-mean aggregation method, on the other hand, does not favor either class and operates similarly to a voting system, providing a balanced assessment. This strategy allows us to leverage all available data points, enabling the model to learn from the nuances of each recording. However, it is important to note that assuming each voice recording has a definitive label indicating whether it is diseased may not be entirely accurate, which could limit the model’s effectiveness and necessitates validation in practical applications.

3.2.2. Pre-aggregation strategy

In the pre-aggregation strategy, we employed the bag-based idea by aggregating the features of the three voice recordings from each subject into a single vector to represent the subject. Since the recordings are assumed equally important, we simply aggregate the features of the three voice recordings by taking their mean value. In other words, the centroid of the three instances is used to represent the entire bag. This produces a composite feature vector for each patient, subsequently utilized to train AI models on subject level. This strategy facilitates direct modeling of the subjects, but at the cost of omitting certain details from the individual recordings. By simply mean-aggregating the three recordings into one, we may lose some information of variations across recordings that could be useful for diagnosis.

3.3. Artificial Intelligence (AI) Algorithms

In the modeling stage, we applied a range of AI classification algorithms to predict Parkinson’s disease (PD). The inclusion of multiple machine learning algorithms in the pipeline serves to assess whether the aggregation methods yield consistent results across different algorithms, thereby evaluating their generalizability. This approach allows us to determine the robustness of our findings and ensures that the predictive performance is not overly dependent on a single algorithm. The algorithms employed include Logistic Regression[53], Decision Tree [54], Random Forest [55], Gradient Boosting [56], XGBoost [57], LightGBM [58], K-Nearest Neighbors (KNN)[59], Support Vector Machine (SVM) [60], Naive Bayes [61], AdaBoost[62] , and Multi-layer Perceptron (MLP)[63].

In addition, we included a stacking classifier, which consists of two base estimators (KNN and SVM) and a final estimator (Logistic Regression). The stacking model leverages the predictive power of base models, with the final model learning to optimally combine the predictions of these base estimators [64,65].

3.3.1. Hyperparameter Tuning

We employed cross-validation and grid search for hyperparameter tuning to minimize the risk of overfitting while enhancing the generalizability and stability of our AI models[66,67]. For each AI algorithm, we delineate a spectrum of hyperparameter values and use 5-fold cross-validation to identify the best hyperparameter combinations, where models are evaluated based on accuracy.

In particular, we implemented group 5-fold cross-validation when applying the post-aggregation strategy, ensuring that all voice recordings from the same subject are kept within the same fold to prevent data leakage[68]. The final model is then trained on the entire training set using the optimal hyperparameters identified in this process.

3.3.2. Model Evaluation

Once the final models have been developed through hyperparameter tuning, we evaluate the performance of each AI model using a range of evaluation metrics. This comprehensive assessment is designed to capture various aspects of classification performance, providing a well-rounded understanding of how effectively each model predicts outcomes. These metrics include Accuracy, Precision, Sensitivity, F1 Score, Area Under the Receiver Operating Characteristic Curve (AUC), and Matthews Correlation Coefficient (MCC) [69,70].By standard conventions:

Accuracy measures the overall correctness of the model. Mathematically,

$$\begin{array}{c}Accuracy={\displaystyle \frac{TP+TN}{TP+FP+TN+FN}},\end{array}$$

(5)

where TP refers to True Positives, TN denotes True Negatives, FP represents False Positives, and FN refers to False Negatives.

Precision measures the proportion of positive identifications that are accurate:

$$\begin{array}{c}Precision={\displaystyle \frac{TP}{TP+FP}}\end{array}$$

(6)

asure the proportions of actual positives and negatives that are accurately identified, respectively:

$$\begin{array}{c}Sensitivity={\displaystyle \frac{TP}{TP+FN}},\end{array}$$

(7)

$$\begin{array}{c}Specificity={\displaystyle \frac{TN}{TN+FP}}.\end{array}$$

(8)

The F1 Score is the harmonic mean of Precision and Sensitivity, providing a balance between the two metrics:

$$\begin{array}{c}F1Score=2\times {\displaystyle \frac{\mathrm{P}\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{i}\mathrm{s}\mathrm{i}\mathrm{o}\mathrm{n}\times \mathrm{S}\mathrm{e}\mathrm{n}\mathrm{s}\mathrm{i}\mathrm{t}\mathrm{i}\mathrm{v}\mathrm{i}\mathrm{t}\mathrm{y}}{\mathrm{P}\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{i}\mathrm{s}\mathrm{i}\mathrm{o}\mathrm{n}+\mathrm{S}\mathrm{e}\mathrm{n}\mathrm{s}\mathrm{i}\mathrm{t}\mathrm{i}\mathrm{v}\mathrm{i}\mathrm{t}\mathrm{y}}}.\end{array}$$

(9)

The Area Under the Receiver Operating Characteristic Curve (AUC) quantifies a model’s capacity to differentiate between classes, with a higher AUC reflecting superior performance. Finally, the Matthews Correlation Coefficient (MCC) provides a balanced measure even in the presence of class imbalance:

$$\begin{array}{c}MCC={\displaystyle \frac{TP\times TN-FP\times FN}{\sqrt{\left(TP+FP\right)\left(TP+FN\right)\left(TN+FP\right)\left(TN+FN\right)}}}.\end{array}$$

(10)

Each AI model was trained on the training set and evaluated on the test set by these performance metrics. Subsequently, comparisons between models were made to identify the best-performing AI model for PD classification.

3.4. Workflow

To clearly illustrate our customized multi-algorithm AI pipeline for predictive modeling, we have included a detailed flowchart that outlines the methods employed in this study. The entire process is summarized in Figure 1.

4. Results

We present the evaluation results of the models trained using the four aggregation methods—post-aggregation (mean/min/max) and pre aggregation via our customized multi-algorithm AI pipeline. The models were evaluated on the test set using the seven-performance metrics: Accuracy, Precision, Sensitivity, Specificity, F1 Score, AUC, and MCC. The results for each aggregation method are summarized in Table 1, Table 2, Table 3 and Table 4.

Figure 2 presents a visual comparison of the seven metrics for the best-performing models—those with the highest accuracy—across the four aggregation methods.

Figure 3 offers a visual comparison of the averages of seven metrics for the twelve machine learning algorithms across the four aggregation methods.

To further investigate the effect of aggregation methods on classification performance, we performed a comprehensive comparison of each AI algorithm across the four aggregation methods within our customized multi-algorithm AI pipeline. Figure 4 displays the performance metrics for each algorithm across these four aggregation strategies.

5. Discussion

5.1. Findings

Previous research that employed same dataset and similar AI algorithms have reported accuracies between 0.66 and 0.90[4,71,72,73,74,75,76,77,78,79,80,81,82,83]. Notably, in the post-mean aggregation method, the XGBoost algorithm achieved peak performance with an accuracy of 0.921, precision of 0.918, sensitivity of 0.982, specificity of 0.737, F1 score of 0.949, AUC of 0.896, and MCC of 0.783.

Our results demonstrate that the post-mean aggregation method is the overall best aggregation approach, as evidenced by the mean accuracy, F1 score, AUC, and MCC presented in Figure 3. The other three methods display varying performance across these metrics without demonstrating significant superiority. Figure 2 provides an additional comparison of the top-performing machine learning algorithms across different aggregation methods.

Figure 4 ranks the machine learning models based on their mean accuracy across the four aggregation methods, revealing that higher-performing models tend to be more complex, such as MLP and boosting algorithms, with post-mean aggregation achieving the best results. Conversely, for the lower-performing models, post-max aggregation demonstrates superior performance.

The metrics indicate that post-min aggregation generally results in higher specificity, but lower sensitivity compared to post-mean aggregation. For instance, the top-performing LightGBM algorithm achieved an accuracy of 0.908, with a sensitivity of 0.947 and a specificity of 0.789. This distinction is evident when the metrics are analyzed in detail. To explain this phenomenon, it is important to examine how the aggregation method affects the balance between true positives (TP), true negatives (TN), false negatives (FN), and false positives (FP). The post-min aggregation method employs a conservative threshold by focusing on the lowest prediction scores. This approach emphasizes the accurate classification of negative samples (i.e., TN) while minimizing false positives (FP), resulting in higher specificity. However, this method also has the drawback of increasing false negatives (FN) and decreasing true positives (TP), which ultimately leads to reduced sensitivity.

In contrast, post-max aggregation prioritizes the maximum prediction scores, leading to the opposite effect: it emphasizes the identification of positive cases, thereby enhancing sensitivity (1.0 for XGBoost). However, this approach also decreases specificity (0.526 for XGBoost) because it is more prone to misclassifying negative samples as positives, resulting in an increase in false positives (FP). This may explain why post-max aggregation is more effective for lower-performing models. By applying straightforward formula-based reasoning, we can infer that in an imbalanced dataset with more positive cases than negative ones, sensitivity has a greater influence on accuracy than specificity. For models like MLP or boosting algorithms, the sensitivity achieved through post-mean aggregation is already close to 1, and post-max aggregation does not result in a significant increase in sensitivity. However, for models that underperformed, the enhancement in sensitivity provided by post-max aggregation becomes crucial for improving overall accuracy.

These differences underscore the importance of selecting between post-min and post-max aggregation based on specific requirements. In applications where it is essential to prevent misclassifying negatives as positives—such as situations where false positives could lead to serious consequences (e.g., unnecessary medical interventions) —post-min aggregation would be the preferable choice. Conversely, when the goal is to maximize the identification of positive cases, such as in screening for rare but critical conditions, post-max aggregation may be the more suitable option[84,85].

Another important consideration is the inherent class imbalance in the dataset, with positive samples outnumbering negative ones by a ratio of 3:1. This type of imbalance leads machine learning models to prioritize positive samples, resulting in higher sensitivity but lower specificity[84]. Although the data augmentation performed by BOS mitigates the imbalance to some extent, this trend persists. In this context, even post-min aggregation shows high sensitivity and low specificity, though it maintains the highest specificity among the four aggregation methods. Post-max aggregation exacerbates this trend, increasing mean sensitivity by only 0.164 while decreasing mean specificity by 0.355 compared to post-min aggregation. This illustrates the potential limitations of post-max aggregation, particularly in scenarios where specificity is critical[86]. This is noteworthy given that our study utilized a relatively large dataset for PD voice analysis[87,88,89].

Although many studies utilizing the same dataset and comparable AI algorithms have reported accuracies ranging from 0.66 to 0.90, [4,71,72,73,74,75,76,77,78,79,80,81,82,83], our pipeline demonstrated far promising performance, achieving a maximum accuracy of 92.1%. This achievement is particularly significant considering the simplicity and ease of implementation of our pipeline.

5.2. Future Directions

The current design of the pre-aggregation method in our research can be improved, necessitating further exploration. To preserve intraindividual variability, more advanced techniques are required to extract maximum information from each voice recording during the pre-aggregation phase. One potential strategy is to concatenate various descriptive statistics, such as mean and variance, from the features of multiple voice recordings to create subject-level features. This approach captures more information beyond the mean and has the potential to enhance classification performance. Additionally, neural networks could be effective if a specialized encoder is developed to extract features from multiple voice recordings [90,91,92,93,94,95,96].

While our pipeline shows promising results, certain limitations might be addressed in future studies to enhance its generalizability. The data imbalance identified in this study stems from the case-control design, which maintains a 3:1 ratio of cases to controls. However, in practical applications, such class imbalances may not always be present. For instance, in population-based research, the number of individuals with a disease can be significantly lower than that of healthy individuals. This potential reversal in class distribution requires modifications to the pipeline for effective adaptation. Possible adjustments include implementing additional resampling techniques or recalibrating classification thresholds to strike a balance between sensitivity and specificity. Furthermore, developing aggregation strategies that maintain robustness across varying class distributions by automatically adjusting for class imbalance could improve the pipeline’s applicability in real-world scenarios.

6. Conclusion

This study highlights the critical importance of appropriate data aggregation strategies and methods in multiple instance learning, particularly in the context of class imbalance. We developed a customized multi-algorithm AI pipeline for non-invasive Parkinson’s disease screening using voice recordings. The results revealed that the post-mean aggregation method achieved the best overall performance, with an accuracy of up to 0.921 and an MCC of up to 0.783, underscoring the effectiveness of our AI pipeline. The observed variations in performance metrics among the different aggregation methods, particularly in sensitivity and specificity, may provide valuable insights for selecting aggregation methods in imbalanced grouped data. These findings have significant practical implications for the application of various aggregation methods in broader medical diagnostic tasks.