Toward Smart Agriculture: A Novel Multimodal Deep Learning Framework for Olive Disease Stage Classification and Severity Estimation

Olive is a major agricultural crop extensively cultivated throughout the Mediterranean region. However, olive trees are vulnerable to several diseases that can negatively affect productivity and yield. One of the most widespread foliar diseases is olive leaf peacock spot, caused by the fungus Cycloconium oleaginum. Early detection of this disease is essential for preventing leaf drop, limiting disease spread, maintaining tree health, and reducing treatment costs before the infection reaches an advanced stage. In this study, a multimodal hybrid deep learning framework is developed to detect peacock spot disease in olive leaves and assess disease severity based on visual and numerical features. The proposed framework integrates olive leaf images with soil conditions, environmental conditions, and vegetation and stress indices to provide a more comprehensive disease analysis than image-only approaches. A ResNet50-based convolutional neural network is used to extract visual features from leaf images, while a multilayer perceptron processes the numerical sensor-based and index-based data. These features are then fused within a unified learning framework to classify disease stages and estimate leaf damage severity, including lesion coverage and yellowing percentage. The performance of the proposed model was evaluated using standard performance metrics suitable for both classification and regression tasks. For classification, the model was evaluated on 494 testing samples and achieved an overall accuracy of 97.77 %, with a macro F1-score of 0.9809 and a weighted F1-score of 0.9776. In addition, the model achieved low regression errors, with mean absolute errors of 1.16 % for lesion coverage and 1.42 % for yellowing estimation. These results demonstrate the effectiveness of the proposed multimodal framework for accurate peacock spot detection and severity assessment, supporting its potential use in smart agricultural monitoring and disease management.