A Hybrid CNN–MLLM Architecture for Image-Based Nutrition Estimation and Advisory Insulin Decision Support in Type 1 Diabetes

Accurate estimation of meal composition from food images can support safer and more reliable insulin bolus decision-making for individuals with Type 1 diabetes. Existing food recognition and nutrition estimation systems are often designed for general dietary logging and do not directly integrate food analysis with personalized insulin therapy parameters. This study presents an image-based nutrition estimation and insulin decision-support module developed within the AI-assisted Diabetes Care (AIDCARE) platform. The proposed system uses a convolutional neural network (CNN) to classify food items from a single meal image and retrieves reference nutritional values from a food composition database. A separate multimodal large language model (MLLM)-based estimation component is then used to estimate portion size, allowing carbohydrate and nutrient values to be scaled according to the observed serving. A curated food image dataset containing 40 food categories was used to evaluate three CNN architectures: ResNet50, Inception V3, and EfficientNet-B0. EfficientNet-B0 achieved the best classification performance, with 94.91% validation accuracy, 95.55% precision, 94.87% recall, and 94.90% F1-score. The portion-estimation component achieved an MAE of 12.27 g and an RMSE of 15.11 g. The estimated carbohydrate value is combined with user-specific clinical parameters, including the insulin-to-carbohydrate ratio and insulin sensitivity factor, to generate advisory bolus guidance. To support safety, the system requires user confirmation or correction of the recognized food category and estimated portion before insulin guidance is displayed. The proposed system is intended for advisory decision support only and is not designed to replace clinical judgment or autonomous insulin delivery systems.