Rapid Salmonella Serovar Classification Using AI-Enabled Hyperspectral Microscopy with Enhanced Data Preprocessing and Multimodal Fusion

1. Introduction

Salmonella is one of the most significant foodborne pathogens, with serovars that exhibit distinct host reservoirs, pathogenicity, and epidemiological profiles, making accurate serovar-level discrimination essential [1,2]. The current standard for detection and identification of Salmonella serovars in food relies on a multi-step culture-based method outlined in regulatory guidelines such as the Food and Drug Administration (FDA) Bacteriological Analytical Manual (BAM). This method involves an initial 24-h non-selective pre-enrichment, followed by a 24-h selective enrichment, and subsequent isolation on selective agar media for an additional 24–48 h. Confirmatory serotyping via slide agglutination using O and H antigen-specific antisera (the Kauffmann-White scheme) further extends the identification process by 2–3 days to the result [3,4,5]. Efforts to reduce the overall processing times have led to the development of rapid molecular detection methods, which partially minimize the need for selective enrichment [6,7,8]. However, these methods still require initial enrichment step, may detect non-viable cells, and continue to necessitate pure isolates for definitive serovar identification [5]. Additionally, robust data analysis often involves specialized domain expertise and skilled personnel. Therefore, there remains an unmet need for a rapid one-step method capable of simultaneously detecting and identifying viable Salmonella serovars without extensive enrichment, thereby complementing and enhancing rapid screening approaches.

To address these unmet needs, recent advancements in optical imaging and artificial intelligence (AI) present promising solutions. Optical imaging techniques that leverage intrinsic bacterial characteristics, such as scattering patterns, autofluorescence, and refractive index, allow for label-free analysis, minimizing extensive sample preparation and reducing dependence on selective enrichment process [9,10,11]. Similarly, classical microscopy-based approaches have been enhanced by image processing algorithms that differentiate bacterial species based on observable parameters such as shape, size, and refractive index [12]. More recently, incorporating deep convolutional neural networks (CNNs) further accelerated the analysis of high-resolution image data by leveraging convolutional filters to extract fine-grained image-based features and hierarchical relationships, thus enabling simultaneous detection and identification of viable bacteria [13,14]. The integration of AI models underscores the value of spatial relationships within image data, highlighting hidden yet informative patterns that can enhance rapid classification of pathogens.

Hyperspectral microscopy provides an advanced optical imaging by capturing high-dimensional data across a broader electromagnetic spectrum, significantly enhancing pathogen classification capabilities. This non-invasive technique utilizes intrinsic bacterial light-scattering fingerprints, offering detailed insights into biochemical, metabolic, and structural properties beyond basic cellular morphology [15]. Historically, hyperspectral microscopy applications primarily focused on spectral data for pathogen classification, using image-based data mainly for identifying cellular regions of interest (ROIs) [15,16,17]. These studies employed various data preprocessing methods, followed by comparative evaluations of traditional machine learning classifiers, such as random forest (RF), support vector machine (SVM), and k-nearest neighbors (k-NN), to determine optimal algorithm. Early studies demonstrated successful differentiation of closely related Salmonella serovars [18,19], yet their emphasis on model optimization, rather than end-to-end automation, necessitated manual preprocessing steps for data quality and interpretation. More recent advancements in AI-enabled hyperspectral microscopy have effectively integrated spatial features through CNNs, enabling automated feature extraction and hierarchical modeling. These improvements have enhanced classification accuracy at the species level [20] and facilitated discrimination of physiological states such as viability [21] or stress responses [22]. Furthermore, some of these studies have shown that multimodal fusion of spectral and spatial data (e.g., morphology descriptors) improves classification performance. Given these promising advancements in other pathogens, applying multimodal fusion specifically to Salmonella serovar classification represents a critical next step toward achieving rapid, accurate, and generalizable serovar-level identification.

Thus, this study introduces a rapid AI-enabled hyperspectral microscopy method for the simultaneous detection and identification of Salmonella serovars with enhanced generalizability. The objectives of this study are to i) develop a rapid serovar classification approach requiring minimal sample preparation without enrichment, ii) compare manual feature selection and data-driven feature extraction approaches to optimize spectral data preprocessing, and iii) evaluate multimodal fusion of spectral and image-based features to enhance classification accuracy and reduce overfitting. As illustrated in Figure 1, our AI-enabled hyperspectral microscopy method addresses these objectives by directly analyzing fresh overnight cultures of five Salmonella serovars (Kentucky, Infantis, Enteritidis, 4,[5],12:i:-, and Johannesburg), without prolonged enrichment and extensive reagents. To leverage the rich information present in hyperspectral data cubes, the analysis employed two parallel branches: a spectral branch and an image-based branch. Within the spectral branch, single-cell spectra derived from segmented cells were preprocessed using manual feature selection or data-driven feature extraction approaches, followed by traditional machine learning algorithms to determine the optimal combination of preprocessing method and classification algorithm. In the image branch, microscopy images (i.e., RGB composites representing the full microscopy field) were analyzed using a CNN architecture. Outputs from both branches were then integrated using prediction-level multimodal fusion. Overall, this study not only accelerates serovar-level detection and identification but also establishes the groundwork for automated end-to-end pathogen classification, significantly enhancing food safety monitoring and enabling timely interventions.

2. Materials and Methods

The Materials and Methods should be described with sufficient details to allow others to replicate and build on the published results. Please note that the publication of your manuscript implicates that you must make all materials, data, computer code, and protocols associated with the publication available to readers. Please disclose at the submission stage any

2.1. Data Acquisition

2.1.1. Bacterial Strains and Sample Preparation

In this study, a total of five Salmonella serovars (Enteritidis, Infantis, Kentucky, Johannesburg, and 4,[5],12:i:-) were selected due to their critical roles in food safety and public health and obtained from the Food Safety Laboratory at Cornell University. The Salmonella stock cultures were maintained at 4°C on LB Lennox agar plates (Thermo Fisher, Waltham, MA, USA). Fresh cultures for data collection were inoculated into 9 mL of tryptic soy broth (Difco, BD, Sparks, MD, USA) supplemented with 0.6% (w/v) yeast extract (Difco, BD) and incubated at 37°C for 16 h. Cells in the logarithmic phase were harvested by centrifugation at 4,450 rpm for 15 min at 4°C (SYTO 9, SLA-1500, Hampton, NH, USA). The final pellet was suspended in sterile de-ionized (DI) water for a final concentration of 105 CFU/mL for hyperspectral microscopy image acquisition. Microscope slides of bacterial cells were prepared following a modified version of a previously published method [22]. For each serovar, 2 µL of bacterial suspension was deposited onto a sterile glass microscope slide and air dried for 15 min at 23°C with an average relative humidity of 43% in a fume hood. After 15 min, 1 µL of sterile DI water was added to affix a plastic coverslip, securing the cells in place, as depicted in Figure 2.

2.1.2. Hyperspectral Microscopy and Data Consistency Check

Hyperspectral microscopy data collection was conducted for all five serovars following our previously published method [22]. A total of 500 hyperspectral data cubes (100 for each serovar) were acquired using an Olympus BX43 upright optical microscope (Evident Scientific, Waltham, MA, USA) equipped with CytoViva patented enhanced darkfield illumination optics, a tungsten halogen lamp (CytoViva, Auburn, AL, USA), and a HinaLea 4250 hyperspectral camera (HinaLea, Emeryville, CA, USA). The spatial resolution of the imaging system was 1936 × 1216 pixels, covering a spectral range of 399–1,000 nm with 303 spectral wavebands with 2-nm bandwidth. Data collection was performed using a 100× oil immersion objective lens (Olympus UPLFLN 100×), with an exposure time of 200 ms and a gain setting of 16 dB that were obtained by preliminary experiments for optimizing high quality of hyperspectral data cubes.

For each data collection session, three subcultures per serovar were incubated across two independent sessions to yield a total of six biological replicates. Each subculture was allocated 3 microscope slides for data collection, serving as technical replicates. For each technical replicate, 6 distinct fields of view (FOV) were imaged per microscope slide, with careful selection to prevent overlapping cells. This approach yielded a total of 500 hyperspectral data cubes (100 per serovar) for further analysis. Raw hyperspectral data were analyzed with the ENVI software (version 6.0, NV5 Geospatial) to ensure consistency between the dataset within each serovar.

2.2. Data Preprocessing and Spectral Feature Engineering

After confirming data consistency, raw hyperspectral data were processed following the workflow illustrated in Figure 1. The dataset was initially partitioned into training (70%) and testing sets (30%) (fixed random state = 42 for replication) to prevent data leakage during preprocessing and ensure reproducibility. Spectral information from bacterial cells were obtained following previously published methods. Briefly, automated segmentation of single-cell regions of interest (ROIs) was performed using attention-gated residual U-Net (ARG2U-Net) [23] to derive mean single-cell spectra. The resulting spectra were then transformed using standard normal variate (SNV) to remove baseline effects caused by imaging conditions [24].

These mean spectra served as inputs in the spectral branch for subsequent comparisons between two feature engineering approaches: i) manual feature selection and ii) data-driven feature extraction. For manual feature selection, three characteristic wavebands corresponding to the most intense regions near the red, green, and blue ranges across all serovars were manually selected, following the methods in our previous study [22]. For data-driven feature extraction, principal component analysis (PCA) was applied to the training set to reduce dimensionality by capturing the most informative linear combinations across all wavebands, as opposed to manually selecting only three wavebands. PCA was implemented using the singular value decomposition function from the Python library NumPy. Before PCA, the training data were mean-centered, and the same mean value was used for test data transformation. The number of principal components (PCs) was chosen to retain 99% of the original variance, ensuring minimal information loss and data redundancy. The PCA transformation was fitted exclusively on the training set and subsequently applied to the test set.

2.3. Machine Learning Models for Spectral Branch

2.3.1. Model Architectures

Four supervised machine learning algorithms were implemented in the spectral branch using Python scikit-learn library to determine the best performing model for Salmonella serovar classification. These included three traditional machine learning models widely used in food safety research, k-NN, RF, and SVM [16,25,26,27,28,29,30,31], as well as a shallow multilayer perceptron (MLP) model. k-NN uses a distance-based metric to classify data points based on their nearest neighbors. RF employs an ensemble approach, combining multiple decision trees to improve classification accuracy and reduce overfitting. SVM classifies samples by identifying an optimal hyperplane to maximize the margin between different classes in feature space. The shallow MLP model was designed to include one hidden layer containing 100 neurons with a rectified linear unit (ReLU) activation function to introduce non-linearity, followed by a linear output layer.

2.3.2. Model Training

All machine learning models were trained using the PyTorch library for Python [32]. Hyperparameter tuning for each model was performed via grid search and 10-fold cross-validation. For k-NN, hyperparameter tuning optimized the number of neighbors (k), leaf size, and distance metric (p), with a search space including k values from 1 to 4, leaf sizes from 1 to 50, and distance metrics of 1 and 2. For RF, hyperparameters optimized included tree depth, minimum samples per split, minimum samples per leaf, and the number of features considered for splitting. The search space included maximum depths of 10, 25, or unlimited, minimum samples per split (5, 7), and leaf sizes (3, 5), with the number of estimators fixed at 100. SVM was optimized with a search space that included a fixed regularization parameter (C = 1), a radial basis function kernel, and polynomial kernel degrees of 3 and 4. Model selection was based on classification accuracy. MLP was trained for 200 epochs using the Adam optimizer (initial learning rate = 0.001) and cross-entropy loss.

2.4. Multimodal Spectral-Spatial Fusion

2.4.1. CNN for Image Branch

In the image branch, microscopy images (i.e., RGB composites representing the full microscopy field) were analyzed using a CNN to capture spatial context without relying on predefined morphological descriptors. The microscopy images were created by combining data points at wavelengths of 620 nm (red), 550 nm (green), and 450 nm (blue). Building on our previously published method [22], a modified EfficientNetV2 architecture was trained using the PyTorch Lightning deep learning framework [33]. Various combinations of standard augmentation strategies from the Albumentations library [34] were applied to enhance model robustness. The pretrained EfficientNetV2 variant (`efficientnetv2_rw_s`) was fine-tuned on the training dataset with a total batch size of 16, using a cross-entropy loss function. Optimization was performed using the AdamW optimizer with an initial learning rate of 0.0001 and a weight decay of 0.001, while a step-based learning rate scheduler reduced the learning rate by a factor of 0.3 every 10 epochs. To mitigate overfitting, early stopping was implemented based on validation loss monitored throughout each epoch.

2.4.2. Fusion of Spectral and Image-Based Prediction Outputs

The multimodal fusion model concatenated prediction-level outputs from the best-performing spectral machine learning model and the CNN image classification model, as shown in Figure 1. Note that the output dimensions from the spectral and image branches differ, as EfficientNetV2 typically outputs a prediction vector matching the number of classes (5 serovars in this case), whereas the spectral branch outputs a vector of length 32. Thus, we modified the EfficientNetV2 model to output a length-32 feature vector that can be concatenated with the output of the spectral branch. The resulting spectral-spatial fused vector had a length of 64. This vector was then fed into the final classifier head, a feed-forward neural network consisting of one hidden layer with 64 neurons, followed by an output layer with 5 neurons corresponding to the number of Salmonella serovar classes. The fusion model was trained for 50 epochs using an AdamW optimizer, with different learning rates for the image (0.0001) and spectral (0.0001) branches and the fusion classifier head (0.001) to balance learning. The batch size, weight decay, and the remaining training procedures and hyperparameters were the same as in Section 2.4.1

2.5. Model Evaluation and Performance Metrics

Each model’s performance was assessed using classification accuracy, precision, and recall, ensuring a comprehensive evaluation. The formulas used for evaluation were as follows:

$$\mathrm{A}\mathrm{c}\mathrm{c}\mathrm{u}\mathrm{r}\mathrm{a}\mathrm{c}\mathrm{y}={\displaystyle \frac{\mathrm{T}\mathrm{P}+\mathrm{T}\mathrm{N}}{\mathrm{T}\mathrm{P}+\mathrm{T}\mathrm{N}+\mathrm{F}\mathrm{P}+\mathrm{F}\mathrm{N}}}\times 100$$

(1)

$$\mathrm{P}\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{i}\mathrm{s}\mathrm{i}\mathrm{o}\mathrm{n}={\displaystyle \frac{\mathrm{T}\mathrm{P}}{\mathrm{T}\mathrm{P}+\mathrm{F}\mathrm{P}}}\times 100$$

(2)

$$\mathrm{R}\mathrm{e}\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{l}={\displaystyle \frac{\mathrm{T}\mathrm{P}}{\mathrm{T}\mathrm{P}+\mathrm{F}\mathrm{N}}}\times 100$$

(3)

where TP, TN, FP, FN are true positive, true negative, false positive, and false negative, respectively.

3. Results

This section may be divided by subheadings. It should provide a concise and precise description of the experimental results, their interpretation, as well as the experimental conclusions that can be drawn.

3.1. Comparision of Hyperspectral Data of Salmonella serovars

The results in Figure 3 illustrate image-based and spectral data derived from the raw hyperspectral data cubes. Figure 3A shows example microscopy images of individual Salmonella cells, cropped from the full microscopy field to illustrate the representative morphology for each serovar. Spatially, bacterial cells from all serovars appear visually similar to the human eye. However, Figure 3B reveals clear spectral distinction by the mean single-cell spectra obtained from segmented bacterial cells. Each serovar shows characteristic variations in spectral intensity and peak wavelengths. Salmonella Kentucky exhibited a maximum intensity of 848 at 570 nm, Salmonella Johannesburg had a maximum intensity of 1,095 at 564 nm, Salmonella Enteritidis had a maximum intensity of 812 at 562 nm, Salmonella Infantis had a maximum intensity of 1191 at 555 nm, and Salmonella 4,[5],12:i:- had a maximum intensity of 1159 at 553 nm.

3.2. Selection of the Optimal Classification Model within the Spectral Branch

3.2.1. Influences of Manual Feature Selection and Data-Driven Feature Extraction

Global manual feature selection was performed on the mean single-cell spectra to identify characteristic wavebands useful for Salmonella serovar classification. By analyzing these spectra, three characteristic spectral wavebands at 499, 555, and 628 nm were identified. These wavebands exhibited significantly higher intensity levels compared to those typically captured by wavebands represented by standard red, green, blue channels. This observation highlights the potential importance of these specific wavebands for differentiating among Salmonella serovars. Additionally, data-driven feature extraction using PCA was performed on the single-cell spectral dataset. It was found that retaining 18 principal components (PCs) was sufficient to explain 99% of the total data variance. By selecting these 18 PCs, dimensionality was effectively reduced without loss of critical information needed for accurate classification.

3.2.2. Performance Comparison of Machine Learning Models Using Spectral Features

To identify the best-preforming model within the spectral branch, the classification performance of machine learning algorithms was compared using both manually selected spectral wavebands and PCA-derived spectral features. Table 1 summarizes the classification accuracies obtained from each approach. When employing the three manually selected wavebands, classification accuracies on the test sets (n=150) for k-NN, SVM, RF, and MLP were 60.1%, 54.1%, 59.5%, and 62.2%, respectively, with corresponding precision values of 60.1%, 56.0%, 58.8%, and 62.1%, and recall values of 60.1%, 54.1%, 59.5%, and 62.2%. In contrast, the use of 18 PCA-derived spectral features, which captured 99% of the total data variance, substantially improved the performance of all models. Specifically, test-set classification accuracies increased to 73.7%, 75.0%, 77.7%, and 81.1%, for k-NN, SVM, RF, and MLP, respectively, along with improved precision values of 74.2%, 76.3%, 78.6%, and 81.1%, and recall values of 73.7%, 75.0%, 77.7%, and 81.1%, respectively.

3.3. Multimodal Classification by Fusion of Spectral and Image-Based Features

The multimodal fusion model demonstrated superior classification performance compared to the standalone spectral and image-based models (Table 2). Specifically, the multimodal approach achieved a test-set classification accuracy of 82.4%, precision of 82.4%, and recall of 82.4%, surpassing the standalone image classification model (i.e., EfficientNetV2) accuracy, precision, and recall of 66.4% obtained from microscopy images, as well as the best-performing standalone spectral model (i.e., PCA-MLP) accuracy of 81.1%. By integrating spectral and image-based features, the multimodal fusion reduced classification ambiguity, especially among serovars with overlapping spatial characteristics or subtle biochemical differences. Additionally, multimodal fusion substantially reduced overfitting observed in standalone models, narrowing the performance gap between training and test datasets (e.g., accuracy gap reduced from spectral-only model: 99.4% train vs. 81.1% test, to multimodal model: 93.6% train vs. 82.4% test). This demonstrates improved generalization and highlights the advantage of combining complementary spectral and image-based features.

Serovar-specific classification performance is further illustrated in Figure 4. The multimodal model performed particularly well for the serovar 4,[5],12:i:-, achieving the highest classification accuracy of 97%. The lowest accuracy was observed for Infantis (73%), which was commonly misclassified as Johannesburg. Enteritidis (81%) and Johannesburg (82%) showed strong classification results with minimal confusion. Kentucky achieved moderate accuracy (79%) but showed occasional misclassification with Enteritidis. These serovar-specific patterns emphasize the model's strength as well as highlight areas requiring further refinement.

4. Discussion

4.1. Hyperspectral Microscopy Captures Intrinsic Differences Among Salmonella serovars

Hyperspectral microscopy in this study enabled a comprehensive spectral and image-based characterization of Salmonella serovars by simultaneously capturing spectral signatures and pixel-level spatial information at the single-cell level. The mean single-cell spectral profiles exhibited distinct intensity patterns for each serovar (Figure 3), despite relatively small variability compared to the spectral differences reported across bacterial species or viability states in previous studies [21,22]. This limited inter-serovar variation reflects phenotypic similarities yet reveals subtle biochemical differences that can support classification. These observations align with previous studies using hyperspectral microscopy with different light sources, confirming that the captured features reflect intrinsic characteristics of Salmonella [19,35]. Eady and Park (2016b) reported three critical spectral peaks between 400–800 nm using a tungsten halogen lamp [35], which is consistent with our data (Figure 3), alongside three additional peaks at 446 nm, 555 nm, and 628 nm. These spectral differences can be attributed primarily to variations in surface components among serovars, including lipopolysaccharides, outer membrane proteins, and other envelope-associated structures [2,36]. Such molecular differences influence the way each serovar interacts with incident light, leading to distinct spectral signatures [37,38]. Additionally, the source of each serovar isolate, whether environmental, clinical, or food-related, may contribute to variations in surface chemistry and structure, thereby affecting spectral properties [39,40,41]. Follow-on studies using higher-resolution or molecularly specific techniques, such as transmission electron microscopy or Raman spectroscopy, could help validate and localize the structural differences underlying the observed spectral variations. Furthermore, expanding the spectral range to include shortwave infrared (SWIR) could potentially improve serovar discrimination by capturing biochemical constituents.

4.2. Data-Driven Feature Extraction Improves Spectral Data Representation

Our results demonstrate that data preprocessing plays a crucial role in optimizing hyperspectral microscopy data for accurate Salmonella serovar classification (Table 1). The high dimensionality of hyperspectral data introduces challenges such as noise, redundancy, and irrelevant variability, all of which can negatively impact AI model performance [30,42,43]. In our previous study, manual feature selection based on domain knowledge achieved high classification accuracy for E. coli under different physiological states [22]. However, applying this approach to serovar-level classification of Salmonella resulted in reduced performance (Table 1). This decline may be attributed to the heuristic selection of only three wavebands, which may have excluded subtle yet informative spectral cues critical for distinguishing among serovars. While manual feature selection offers advantages in computational efficiency and data storage, it introduces subjective bias and risks omitting spectral regions that, though individually weak, contribute collectively to classification performance. This can result in the loss of spectral richness necessary for fine-scale discrimination.

In contrast, data-driven feature extraction using PCA substantially improved model performance (Table 1). PCA transforms hyperspectral data by projecting them onto orthogonal components, yielding a reduced set of uncorrelated features that retain the most informative variance. This transformation not only reduces dimensionality but also enables downstream models to focus on latent spectral patterns that may not be apparent through manual selection based on individual wavebands [44]. The superior performance of PCA-preprocessed data highlights the effectiveness of data-driven feature extraction in preserving subtle and distributed spectral information critical for serovar-level discrimination. Supporting this observation, a previous study using multivariate data analysis of Salmonella serovars reported classification accuracies above 90% when PCA was applied to mean single-cell spectra [18]. While that study relied on traditional statistical modeling, our results extend these findings by demonstrating the value of PCA as a preprocessing step within AI-enabled classification frameworks. These results underscore the effectiveness of unsupervised data-driven approaches in capturing the spectral complexity required for high-resolution microbial classification.

These findings are consistent with other strategies designed to manage high dimensionality of both micro- and macroscale hyperspectral data. For instance, one study reported improved classification performance and reduced computation time when using PCA-guided waveband selection [45]. Moreover, Tanskin et al. (2017) proposed a data processing approach that combined dimensionality reduction with hierarchical function decomposition, which outperformed conventional feature selection methods [46]. These examples reinforce a broader conclusion observed across the literature that analytical decisions during preprocessing directly impact model outcomes. Improper preprocessing can introduce artifacts or remove biologically meaningful information [47], and variations in steps like normalization, noise reduction, and baseline correction significantly influence downstream classification performance [48]. Therefore, selecting appropriate preprocessing methods is essential to ensure that results reflect true biological variation rather than artifacts of data handling.

4.3. Multimodal Fusion of Spectral and Image-Based Features Enhances Classification and Mitigates Overfitting

The enhanced performance of the multimodal fusion model highlights the value of integrating multiple feature types derived from hyperspectral microscopy (Table 2). Combining spectral and image-based modalities is particularly advantageous for classifying microorganisms with low interclass variability [49]. Spectral data, which capture how samples scatter light across a wide range of wavebands, encode information related to biochemical compositions [18,19]. These signatures are especially useful for differentiating Salmonella serovars that may exhibit similar morphology but differ in molecular characteristics. As such, spectral features alone can provide strong biochemical fingerprints for classification tasks [50,51]. However, image-based features add complementary value to spectral signatures. As a type of spatial information, these features capture structural patterns directly observable within hyperspectral data cubes at the single-cell level. In our study, spectral and image-based feature vectors were combined at the prediction level, enabling a more comprehensive representation of each Salmonella serovar. This multimodal integration improved classification accuracy from 57.3% (image-based only) and 81.1% (spectral only) to 82.4%, while also mitigating overfitting, as indicated by the reduced gap between training and test performance metrics (Table 2). However, classification performance varied by serovar (Figure 4), with Infantis exhibiting the lowest accuracy. This may reflect underlying biological variability, as Salmonella Infantis is known to be more adaptive to environmental stressors, potentially contributing to its phenotypic heterogeneity [52]. Such variability highlights the importance of integrating complementary data modalities to improve classification robustness.

This finding aligns with a broader body of work demonstrating that integrating spectral and spatial information can improve classification performance in hyperspectral imaging applications. A conceptually related approach combined spatial features derived from segmented cells with spectral features to classify Bacillus megaterium and Bacillus cereus, achieving classification accuracies above 95% using traditional machine learning algorithms [53]. Although this method did not employ prediction-level fusion, it demonstrates that integrating spatial and spectral features can significantly enhance classification performance. Additionally, several studies have explored CNN architectures to better exploit the rich spectral and spatial information embedded in hyperspectral imaging data [20,24,54]. One study employed feature fusion strategies to combine spectral and spatial representations extracted from hyperspectral data cubes, while another implemented prediction-level fusion by integrating morphological descriptors, single-channel images, and spectral features within a unified classification framework [20,24]. In macroscale applications, 3D CNNs have also been developed to jointly learn spatial–spectral patterns by treating hyperspectral data cubes as volumetric inputs [54]. These findings underscore the effectiveness of multimodal fusion for robust pathogen classification and support future studies aimed at enhancing generalization across diverse microbial strains and environmental contexts.

5. Conclusions

This study demonstrates a rapid Salmonella serovar classification approach that reduces the need for selective enrichment by integrating hyperspectral microscopy with AI-enabled analysis. Hyperspectral data cubes were processed to extract two complementary data types: single-cell spectral profiles obtained from segmented cells and microscopy images constructed as RGB composites of the full microscopy field. By capturing both spectral and image-based spatial features, this approach enables comprehensive characterization of serovar-level bacterial differences, leveraging both biochemical signatures and pixel-level spatial patterns. The results illustrate the critical role of data preprocessing, showing that data-driven feature extraction via PCA outperforms manual feature selection by effectively preserving informative spectral variance while reducing data dimensionality. Furthermore, the image branch employed a CNN to learn spatial features directly from images, without relying on explicit morphological descriptors. The multimodal fusion of spectral and image-based features not only enhanced classification performance but also mitigated overfitting, ensuring robust and generalizable serovar classification. Overall, these results highlight the potential of AI-enabled hyperspectral microscopy as a rapid, culture-independent solution for pathogen identification and food safety monitoring.