Deep Learning-Based Recognition of Traditional Saudi Arabian Foods: A New Dataset and Comparative Evaluation of CNN and Transformer Models

1. Introduction

Food is an essential part of human existenc, which is considered part of human identity, where some food dishes are culturally inherited from generations and considered a valuable part of the identity for many humans and countries [1]. Culinary traditions reflect the history, values, and diversity of nations, serving as a lens through which we can understand the unique tapestry of human experience [2]. The flavors, ingredients, and methods of preparing traditional dishes are passed down from generation to generation, creating a deep connection between individuals and their heritage [3]. Saudi Arabia is known for its rich culinary landscape, which speaks of a deep history for the region’s cultural identity.

Artificial Intelligence (AI) has demonstrated significant advancements in the field of food computing, yielding promising results in areas such as food recognition, food detection, calorie estimation, and robotic manipulation of food items [4,5]. In particular, recent breakthroughs have enabled AI systems to accurately identify the type and state of food based solely on visual cues, leveraging sophisticated deep learning approaches [6,7]. Additionally, the emergence of foundation models has further enhanced the capability to segment complex food images, thereby improving the precision of food-related tasks [8,9].

Despite these technological achievements, a notable limitation persists regarding the diversity and inclusivity of available datasets. Most existing food image datasets primarily focus on a limited selection of global cuisines, leaving others underrepresented or entirely absent. For instance, public datasets for Saudi cuisine are nonexistent, which poses challenges for developing AI models that are generalizable across diverse culinary traditions. The inclusion of food images from a broader range of cuisines is essential to enhance the robustness and applicability of AI models in food computing, ensuring that these technologies can effectively serve and generalize globally.

To address the current lack of diversity and representation in food image datasets—especially the absence of resources dedicated to Saudi cuisine, this study introduces a comprehensive approach that both preserves and highlights Saudi culinary heritage within food computing. Specifically, we present a deep learning-based food recognition framework designed to accurately identify traditional Saudi dishes. Central to our work is the creation of the first publicly available, labeled dataset featuring over 3,000 high-quality images of Saudi traditional foods. By making this dataset accessible to the research community, we aim to facilitate further innovation and inclusivity in food computing technologies. Furthermore, we rigorously trained and evaluated a range of deep learning models on this dataset, employing both fine-tuning of existing architectures and training from random initialization. This dual approach enables a thorough assessment of model performance and adaptability when exposed to previously underrepresented culinary categories, ultimately contributing to the development of more robust and generalizable AI systems in the food recognition domain. Our contribution can be summarized as follows:

• Curation of the first dataset for Saudi Arabian Traditional food images, which we release publicly to the computer vision research community.
• Assessment of CNN and ViT on the classification of Saudi Arabian Traditional food images.

2. Background

The field of food recognition has emerged as a critical area of research, bridging technological innovation and cultural understanding. With the dynamic evolution of global culinary practices, there is a growing demand for sophisticated food recognition systems that can accurately identify and interpret a wide range of cuisines. These advancements are essential for deepening our understanding of diverse food cultures and enriching the overall culinary experience [10].

In recent years, research in this field has included numerous studies that have utilized machine learning and deep learning techniques to develop models capable of analyzing food images [7,8,11,12]. Among these models, Convolutional Neural Networks (CNNs) have been the most commonly used [13], accurately identifying and classifying food dishes [14,15], and estimating food waste [16]. CNNs consist of several convolutional layers, non-linear activation layers, and pooling layers, enabling them to capture complex patterns in images. CNNs are used in various applications such as image classification [17], object detection [18], and image segmentation [19]. Despite their effectiveness, CNNs require a large annotated image dataset to learn discriminative features [20]

Additionally, the Vision Transformer (ViT) model has been explored, leveraging transformer architecture to enhance food classification performance. ViT was introduced in a research paper in 2021, demonstrating excellent performance in image classification tasks, surpassing traditional convolutional neural networks in certain cases. ViT processes images by dividing them into smaller patches and treating them as sequences, allowing it to capture global relationships within the image more effectively. However, ViT requires larger amounts of data for training compared to CNNs [21,22].

In their research, Ciocca et al. [4] introduced a dedicated dataset and conducted experiments focused on food recognition involving various dishes arranged on trays in different configurations. The dataset comprises images and manually annotated masks for 1,027 trays, encompassing a total of 3,616 food instances across 73 distinct food categories. Notably, each food category is represented by multiple instances, enhancing the dataset’s diversity. The authors evaluated the performance of convolutional neural networks (CNNs) for the task of food recognition using this dataset, achieving an accuracy of 79%.

Several studies have explored food recognition and calorie estimation using portable devices, such as smartphones. These approaches leverage the widespread availability and convenience of mobile technology to enable users to capture images of their meals and receive immediate feedback on nutritional content [23,24,25,26]. By integrating advanced image analysis algorithms with user-friendly interfaces, these systems provide practical solutions for individuals seeking to monitor their dietary intake on the go. In addition to calorie estimation, other research efforts have concentrated on the health-related applications of food recognition. These include tools designed to assist with dietary management, support individuals with specific nutritional requirements, and promote healthy eating habits through personalized recommendations [27,28]. The convergence of mobile computing and AI-driven food recognition technologies is thus shaping a new era of accessible, health-focused solutions that empower users to make informed choices about their nutrition in everyday life.

In Table 1, we provided a summary of previous work for food category recognition using various datasets. Despite significant advances in food recognition, existing studies primarily focus on widely represented cuisines, while traditional Saudi cuisine remains largely absent from publicly available datasets and benchmarking efforts. This limitation restricts the development of food recognition systems that can effectively represent the cultural diversity of global cuisines.

To address this gap, we introduce the first comprehensive dataset of traditional Saudi foods, comprising over 3,000 manually verified images across 13 regional dishes. We further conduct a comparative evaluation of CNN- and transformer-based architectures under different training strategies, providing a new benchmark for Saudi food recognition. By combining cultural heritage preservation with modern AI techniques, this work contributes both a valuable dataset and new insights into deep learning-based food classification.

3. Dataset

3.1. Data Collection

The dataset compiled for this research is rooted in the official culinary heritage of the Kingdom of Saudi Arabia (KSA), as recognized by the Culinary Arts Commission (CAC) 1. Thirteen traditional dishes classified by CAC were selected to represent the unique gastronomic identity of the thirteen regions of the Kingdom of Saudi Arabia. These categories were chosen to ensure that the dataset captures the cultural and culinary diversity across the kingdom, reflecting the distinct flavors, ingredients, and preparation styles associated with each region. To build a robust and diverse image dataset, we systematically collected images of each of the selected dishes, resulting in a comprehensive dataset comprising 3,000 high-quality images. The image acquisition process was carried out in three sequential rounds, employing a combination of automated and manual methods. Initially, an automated Python script was utilized to scrape images from publicly available sources on the web, ensuring broad coverage and efficient collection. To further enrich the dataset and capture authentic representations, additional images were manually sourced from various social networking platforms, where users share images of traditional Saudi Arabian cuisine.

Quality assurance was a critical component of the dataset construction process. Every image underwent two rigorous rounds of verification to confirm both the accuracy of its content and its correct classification within the designated food category. This dual-check system was implemented to eliminate mislabeling and ensure that each photograph authentically depicted the intended dish. As a result, the final dataset offers reliable, category-specific visual information suitable for robust model training and evaluation. Furthermore, some images were eliminated from the dataset due to poor lighting, blurriness, duplications, and irrelevant images. After cleaning and quality check, the total number of images in our dataset is 3,000. The dataset was split into training with a total of 2100 images, validation with a total of 300 images, and a test set with a total of 600 images. The split ratio is 70:10:20 for train, validation, and test sets, respectively.

Figure 1 presents a detailed summary of the distribution of images across the thirteen food categories, highlighting the balance and representation within the dataset. Additionally, Figure 2 shows some images from each category, illustrating the variety and visual characteristics of traditional Saudi dishes included in the dataset. These categories not only serve as a cultural identity of their respective regions but also provide valuable resources for advancing research in food recognition and cultural informatics.

3.2. Data Augmentation

To enhance the model’s ability to generalize and perform effectively on novel and unseen food images, we applied a range of data augmentation strategies to the training set. These augmentations were carefully chosen to introduce controlled variability into the dataset, thereby reducing overfitting and improving robustness. Specifically, the images were randomly rotated 40 degrees to simulate variations in camera angle, and width and height shifts of up to 20% were applied to account for positional differences during image capture. In addition, a shearing transformation of 20% was included to mimic distortions, while zooming operations of up to 20% were used to replicate differences in scale. Horizontal flipping was employed to reflect natural variations in object orientation, and brightness adjustments within the range of 0.8 to 1.2 were applied to capture changes in lighting conditions. This augmentation step was applied to the training set only using ImageDataGenerator class from the Keras software package. This augmentation contributes to creating a more diverse training dataset that better represents real-world scenarios and strengthens the model’s capacity to make accurate predictions on previously unseen samples.

4. Methodology

For the experimental phase of this study, all model training and evaluation were conducted on the Google Colab platform, which provides a flexible and scalable environment well-suited for deep learning research. Leveraging the computational capabilities of the NVIDIA T4 GPU available within Google Colab, we were able to significantly accelerate both the training and fine-tuning processes, ensuring efficient experimentation across multiple architectures.

Three state-of-the-art deep learning models were selected for comparative analysis: MobileNet, ResNet, and the Vision Transformer (ViT). MobileNet and ResNet, both well-established convolutional neural network (CNN) architectures, were incorporated due to their proven effectiveness in image classification tasks and their ability to balance accuracy with computational efficiency. The Vision Transformer was included to assess the performance of transformer-based models on food image recognition, given its recent success in visual tasks. Each model underwent two distinct training regimes: (1) training from random initialization (“from scratch”) and (2) fine-tuning of pre-trained weights on the ImageNet dataset. Training and fine-tuning were performed for 50 epochs to ensure thorough convergence and allow for robust evaluation of each model’s capacity to generalize to the Saudi Arabian cuisine dataset. The Adam optimizer was utilized throughout, with a fixed learning rate of 0.001, selected for its adaptive learning capabilities and effectiveness in deep learning applications.

In addition to established architectures, we developed a custom CNN tailored for the specific characteristics of our dataset. This network was deliberately designed to be lightweight, facilitating rapid experimentation and serving as a baseline for comparison. The architecture consists of two convolutional layers, followed by a pooling layer, and a flattening layer, followed by two fully connected layers. This structure was chosen to enhance feature extraction while maintaining computational efficiency, making it suitable for the relatively scale of the dataset. All models were trained using identical pre-processing and augmentation pipelines to ensure consistency across experiments. Performance metrics—including accuracy, precision, recall, and F1-score were systematically recorded for each setup, facilitating a comprehensive assessment of both off-the-shelf and customized models in the context of traditional Saudi food recognition.

Figure 3 illustrates the workflow, starting with data collection, followed by annotation, AI model training, and concluding with evaluation on a separate test set.

5. Deep Learning Architectures

In this section, we describe each of the models we used in this paper. First, MobileNet is a model designed for mobile devices, focusing on achieving high efficiency while maintaining good accuracy [40]. It relies on depthwise separable convolutions to reduce the number of computations, making it ideal for applications on smartphones and IoT devices. MobileNet is considered the best option due to its speed and resource efficiency. Second, ResNet, or Residual Networks, uses residual connections that allow bypassing some layers, making it easier to train deep networks and reducing performance degradation issues [41]. ResNet has achieved great success in competitions like the ImageNet challenge, making it one of the most popular models in deep learning. Third, The Vision Transformer uses the transformer blocks by dividing the images into blocks [42]. This facilitates accurate recognition and classification. The model has achieved notable success in image processing competitions, making it an effective tool in the field of food classification. It benefits from a rich dataset that includes diverse images of food, enhancing its accuracy in recognizing different ingredients and flavors. Fourth, our designed CNN relies on two convolutional layers to extract features, followed by a max-pooling layer and two fully connected layers. The goal of keeping this CNN very small is to evaluate the performance of a small-weight CNN on our dataset, since the end goal is to deploy the CNN on end devices for real-time inference. The convolutional operation excels at recognizing various patterns in images, such as edges and shapes, which makes it ideal for applications in image classification and object detection. An illustration of our customized CNN is shown in Figure 4.

6. Experiment

In our study, we leveraged two main strategies for model training. The first strategy involved utilizing pre-trained models, where we fine-tuned established architectures with weights initialized from large-scale datasets such as ImageNet. By applying these pre-trained models—specifically ResNet and MobileNet—to our own tasks, we were able to reduce the overall training time and achieve significantly improved performance. This fine-tuning approach allows the models to benefit from prior knowledge gained on extensive and diverse datasets, making them more effective even when working with a smaller amount of domain-specific data.

Alongside using pre-trained models, we also explored training various architectures entirely from scratch. In this method, the model weights are initialized randomly, and all learning is conducted using only our own dataset. While training from randomly initialized weights offers the flexibility to tailor the models specifically to our unique data, it also comes with increased computational demands and the requirement for a much larger training set to achieve competitive performance. This dual approach enabled us to compare the benefits and trade-offs of fine-tuning versus building models from the ground up.

7. Results

In this section, we present the outcomes of our experiments, providing a comprehensive overview of the test metrics for each model evaluated. Our analysis covers four primary types of models: MobileNet, ResNet50, our customized CNN, and ViT architecture.

MobileNet, known for its lightweight architecture and efficiency, is particularly suited for applications where computational resources are limited, such as on mobile devices. We trained two variants of MobileNet: one leveraging pre-trained weights from ImageNet and another trained entirely from scratch. The model initialized with ImageNet weights achieved on the test set an accuracy of 92%, precision of 0.92, recall of 0.92, and F1-score of 0.92. In contrast, the MobileNet model trained from scratch achieved a lower accuracy of 66%, with precision, recall, and F1-score of 0.69, 0.66, and 0.65, respectively. These results highlight the benefit of using pre-trained weights. For the ResNet50 model, a widely adopted deep learning architecture featuring skip connections to mitigate the vanishing gradient problem, we also explored two training approaches: training from random weights and fine-tuning from ImageNet pre-trained weights. The randomly initialized ResNet50 model achieved an accuracy of 49%, precision of 0.54, recall of 0.49, and F1-score of 0.48. Conversely, the ResNet50 model initialized with ImageNet weights demonstrated a significant improvement, achieving an accuracy of 73%, precision of 0.84, recall of 0.85, and F1-score of 0.84. These results further underscoring the effectiveness of fine-tuning of pre-trained deep learning architectures.

The customized CNN model achieved an accuracy of 85%, a precision of 0.80, a recall of 0.82, and an F1-score of 0.81. These findings demonstrate that even with a compact architecture, strong performance can be attained on the test set, making this approach highly suitable for deployment on mobile devices.

The Vision Transformer (ViT) architecture, a cutting-edge architecture recognized for its strong performance in image recognition tasks that demand high accuracy, such as classifying a diverse range of food dishes. This model achieved a test accuracy of 94%, precision of 0.94, recall of 0.93, and F1-score of 0.93. These results highlight its balanced and reliable performance in correctly classifying food images. These results underscore the effectiveness of the Vision Transformer approach in complex image classification scenarios, particularly for applications requiring nuanced recognition capabilities.

A summary of the results can be found in Table 2, with graphical representations shown in Figure 5 and Figure 6.

To offer a more in-depth understanding of classification per food category, we provided the confusion matrices for the best two models: MobileNet fine-tuned model as shown in Figure 7, and ViT as shown in Figure 8. Based on the classification results of both models, visually comparable categories such as Madini Rice and Haneeth show misclassification across all models, most likely as a result of common texture and color traits as well as the inclusion of rice in both dishes. On the other hand, categories with unique visual characteristics, like Kleeja, reached 100% in the ViT model. These results show ViT’s superior performance in classifying Saudi food dishes.

8. Discussion

This research marks significant progress in leveraging deep learning techniques to identify and classify traditional Saudi cuisine. The primary objective of the study is to introduce a distinctive dataset, which includes carefully curated images paired with the corresponding names of various Saudi dishes. By assembling and annotating this collection, the study not only aims to improve the accuracy of food recognition models but also contributes valuable resources for future research in both computer vision and culinary heritage preservation

A computational comparison of training time, inference time, and models’ trainable weights is provided in Table 3. Even though ViT has the best accuracy, MobileNet fine-tuned has a much faster training and inference time than ViT, which makes MobileNet more appropriate for mobile and edge deployment. These findings emphasize the trade-off between computational cost and accuracy when choosing a model for edge devices during deployment.

As illustrated in Figure 9, examples from our traditional Saudi dishes featured in the dataset are prominently displayed: Kubaibah Hail, Salig, Kleeja, and Ragish, along with the prediction distribution. These dishes have specifically highlighted the performance of the MobileNet model for Saudi traditional food recognition.

Distinguishing between certain traditional Saudi dishes can present a challenge, especially when there are similarities in their main ingredients or appearance. For example, Saliq often gets confused with dishes like Sayadiyah, Madani rice, and Hassawi rice due to their shared use of rice and similar presentation. As demonstrated in Figure 10, the Grad-CAM (Gradient-weighted Class Activation Mapping) visualization technique was employed to highlight the specific regions of each image that the deep learning model focused on during classification. This approach provides deeper insight into how the model interprets and differentiates between dishes. We provided the Grad-CAM in Figure 10 for MobileNet fine-tuned pretrained model, since it is a small weighted model with a low cost of training and testing suitable for edge devices.

The Grad-CAM analysis revealed that the model was able to identify and concentrate on the primary components within each dish, such as rice for Saliq and unique elements for Ragish, effectively capturing the distinguishing ingredients that contribute to the cultural identity of Saudi cuisine. By focusing on these signature features, the model not only improved its classification accuracy but also enhanced interpretability, offering users more detailed information about the ingredients and preparation methods of each dish. This capability is particularly valuable for users interested in learning about the culinary traditions of Saudi Arabia, as it supports both food recognition and cultural heritage preservation.

The experimental results demonstrate that transfer learning remains a highly effective strategy for food recognition tasks when domain-specific datasets are relatively small. The substantial performance improvement observed for MobileNet and ResNet50 after fine-tuning highlights the value of knowledge transferred from large-scale datasets such as ImageNet. In contrast, models trained from random initialization generally achieved lower performance, indicating that feature representations learned from generic visual datasets provide a strong foundation for recognizing culturally specific food categories. Interestingly, the Vision Transformer achieved the highest classification accuracy, suggesting that its ability to capture long-range spatial relationships is particularly beneficial for distinguishing traditional dishes that share similar ingredients but differ in texture, presentation, or subtle visual patterns.

Another important finding is the strong performance of the proposed lightweight CNN despite its simple architecture. Although it contains significantly fewer parameters than the state-of-the-art models, it achieved competitive accuracy while requiring substantially lower computational resources. This characteristic makes the model an attractive candidate for deployment in real-world applications such as mobile dietary assessment systems, smart restaurant services, and tourism-related applications where computational efficiency is essential. Collectively, these findings demonstrate that both high-capacity transformer models and lightweight CNN architectures have complementary advantages, offering practical deployment options depending on the target application’s computational constraints and accuracy requirements.

Although the proposed models achieved promising classification performance, this study has several limitations. First, each deep learning model was trained and evaluated using a single experimental run. While identical training settings were applied to ensure a fair comparison, multiple independent runs with different random initializations would provide a more comprehensive assessment of model stability and reduce the influence of stochastic training behavior. Consequently, the reported performance metrics should be interpreted as representative results rather than definitive estimates of each model’s robustness.

Future work will focus on conducting repeated experiments with multiple random seeds and applying statistical significance tests to rigorously compare model performance. Such analyses will enable the quantification of performance variability, confidence intervals, and the statistical significance of observed differences between CNN- and transformer-based architectures. These extensions will provide stronger evidence regarding the robustness and reliability of deep learning models for Saudi traditional food image classification while establishing a more rigorous benchmark for future research.

Furthermore, future work will involve broadening the dataset by incorporating additional traditional Saudi dishes, ensuring an even more comprehensive representation of the nation’s culinary heritage. This study directly supports Saudi Vision 2030 by promoting the tourism sector, showcasing the diversity of Saudi culinary traditions, and strengthening national identity on a global scale. By connecting people through food and technology, the project contributes to broader goals of cultural preservation and international engagement.

9. Conclusions

In summary, this study introduces a novel dataset dedicated to traditional Saudi cuisine, carefully curated from thirteen main regions within the Kingdom of Saudi Arabia. By assembling a diverse and representative collection of images and their corresponding dish names, we aimed to support both culinary heritage preservation and technological innovation. We rigorously evaluated the dataset using both fine-tuned state-of-the-art deep learning models and randomly initialized models. The findings demonstrate that these models can effectively and accurately recognize a wide range of Saudi traditional dishes, showcasing the potential for deep learning applications in food identification. Our approach not only improves classification performance but also provides valuable insights into ingredient recognition and dish differentiation, as discussed with the aid of visualization techniques like Grad-CAM. This work lays the groundwork for future research in computer vision and cultural studies and contributes a valuable resource for both technological advancement and the celebration of Saudi culinary identity.