CNN-Assisted Growth Monitoring and Stress Management of Cucumber in Semi-Transparent PV Greenhouses for Agrivoltaics

1. Introduction

Numerous challenges face the agricultural sector today including overpopulation, urbanization and climate change. The energy sector is also facing challenges as it seeks to provide sustainable solutions and decarbonize. Using the Food and Agriculture Organization (FAO) forecasts, the urgency for food and energy security poses a challenge to both the agricultural and energy sectors[1]. A viable option for responding to this challenge is agrivoltaics which involves integrating crop cultivation and photovoltaic (PV) energy systems. This is a viable option for addressing the challenges of crop production and energy generation and integrating climate positive practices[2]. By integrating renewable energy generation with crop cultivation, agrivoltaics enhances efficient utilization of land.Using new technologies and eco-friendly methods with sustainable crop production is the goal of using new technologies in isolation agriculture. One of the approaches employed is Controlled Environment Agriculture (CEA). It includes greenhouses, vertical farming, and growing chambers. Each of these allows the cultivation of specific micro-environmental variables. These growth chambers are also equipped with, and can control and monitor, systems for light, temperature, humidity, nutrients, and other variables. Greenhouses can also employ Semi-Transparent PV (Photovoltaic) modules, which means they can use CEA with PV (controlled-environment agrivoltaics). This means greenhouses can balance the microclimate for crop growth and shading with the ability to control and regulate the growth of other renewable energy sources (as crop productivity is increased)[3]. This also means the greenhouse can more systematically address the use of varying irrigation and fertigation (the application of nutrients through irrigation) protocols, which helps researchers sustain energy and crop productivity[4].

Horticultural producers using CEA-PV systems should consider cucumber (Cucumis sativus L.) cultivation. Cucumber is one of the most economically impressive ‘’quick-revenue’’ greenhouse crops. Cucumber develops multiple commercially viable crops throughout the cultivation cycle. However, when considering the cultivation of cucumber, the relatively low light (and, therefore, heat) requirements of this crop should be taken into account. Cucumber crops require 20–25 mol·m²·day DLI for optimal growth and yield. Inadequate light, especially poor distribution of light, results in the development of vicinal non-structural carbohydrate sinks, leading to the development of unchecked, overly long, and non-fruit bearing ‘’reaching’’ vines, with poor quality fruit[5]. Greenhouses employing traditional building designs often suffer from heat stress, fruit quality issues such as sunscald and bitterness, and trouble with the development of quality issues as a result of excessive and trapped solar radiation. For cucumber cultivation, the combination of Agrivoltaics and with ST-PV canopies can provide a means to alleviate aggravated solar radiation and create a favorable microclimate[6].

Furthermore, current state-of-the-art sensing technologies, combined with artificial intelligence capabilities, provide further enhancements to the already promising opportunities available to CEA-PV systems [7]. One of the more recent favorable advancements in the science of plant phenotyping is the use of Convolutional Neural Networks (CNN). Within Controlled Environment Agriculture (CEA), CNNs are used to evaluate canopy cover and identify abiotic and biotic stresses at various levels of (multispectral and hyperspectral) imaging. In the detection of cucumber vine growth, CNNs can reveal the presence of water and nutrient stresses and predict the water, nutrient, and even the yield of the crop, in a non-destructive, real-time manner. Additionally, the systems developed by Amazon can use CNN’s to determine the progess of the crop and autonomously adjust the environmental conditions, irrigation, and ventilation, in real time, so as to maintain a closed control loop [8].

Although CEA-PV systems are promising, there is minimal research on ST-PV integration with cucumber production. Most studies in agrivoltaics involve field crops, particularly lettuce, tomato, wheat, and maize. These crops are more sensitive to shading, but findings from these studies cannot be transferred to cucumber production [9]. Cucumbers grow vertically and respond to light in more light-sensitive ways. They also have specific spectral requirements that are not provided by ST-PV modules because these are designed to filter certain wave lengths. The role of photosynthetically active radiation (PAR) in the red and blue range for photosynthesis as well as in the far-red for flowering and fruit set is well documented[10]. Therefore, further innovations in agrivoltaic systems are needed to meet the light quantity and quality requirements specific to cucumber physiology.

Regarding synergy with the sustainability aims, the cucumber crop grown in the CEA-PV systems is unique. A considerable amount of energy is used in the construction of a greenhouse for heating, cooling, irrigation, and lighting systems. ST-PV glazing, in addition to being able to allow the greenhouse to use renewable energy for self-sustaining energy, also reduces the use of fossil fuels. Furthermore, ST-PV glazing also lowers the cooling demand of the greenhouse, improving the energy performance of the greenhouse. In water-scarce regions, the water use efficiency partnered with AI-controlled irrigation systems can be maximized. In Oman, the combination of water scarcity and renewable energy objectives makes the food-energy-water nexus extremely innovative for cucumber cultivation in CEA-PV systems [11].

There is additional consideration in CEA-PV designs for repeatability and scaled CEA-PV designs. Small scale pilot studies show promising results but still require large scale studies for commercial viability. It is also necessary to find an economic balance in energy and crop production. For example, the energy production from PV is better. However, crop growth is negatively impacted by too much shading from PV. The same is true in the opposite scenario. The region’s local climate, crop value, and electricity pricing determine the balance[12].

This research seeks to address these gaps in knowledge by combining studies and experiments pertaining to cucumber production in agrivoltaic greenhouses with controlled environments. In particular, this research examines ST-PV canopies in greenhouses with four different shading treatments: no shading, 20% shading, 40% shading, and 60% shading. The coaxial phenomics evaluation of the canopy system will be rigorously monitored for stress and growth variables by means of computer vision through convolutional neural networks (CNN). Additionally, the research will evaluate the use of CNN data in closed-loop greenhouse control strategies to optimize irrigation and shading for improved yield, increased water use efficiency, and net positive energy balance.

This research aims to achieve the following: 1. evaluate the effects of different shading intensities on cucumber yield, water use efficiency, and microclimate in CEA-PV environments; 2. evaluate the effectiveness and reliability of CNN-based phenotyping to unobtrusively monitor cucumber growth and stress; and 3. evaluate the relationship between the productivity of the cultivated crops and the renewable energy production to provide design parameters for future CEA-PV greenhouses [13].

By achieving these objectives, the study will support and advance sustainable practices of agrivoltaics and provide fundamental guidelines regarding the use of CEA-PV systems in cucumber production [14].

Cucumbers are considered a valuable crop for greenhouses and have great opportunities for potential integration with CEA-PV systems. Semi-transparent PV glazing combines two functions: renewable energy generation and microclimate control. Additionally, CNN-based computer vision can be used for monitoring and control. This research focuses on integrating energy, and a part of the food-water nexus, along with artificial intelligence and controlled environment agriculture, providing a pathway to resilient agriculture systems and food-water-energy security. It is anticipated the results will contribute to the development of integrated, climate and resource smart, greenhouse design and agrivoltaic system policies, along with commercial applications, for the use of CEA-PV systems in the greenhouse and agrivoltaic system policies [15].

This part summarizes previous research pertaining to the controlled-environment agrivoltaics (CEA-PV), the cucumber physiology under shading and convolutional neural networks (CNNs) for plant phenotyping. The review covers a total of 35 peer-reviewed studies and technical documents, and thematically, covers agrivoltaic fundamentals, semi-transparent PV in greenhouse, cucumber crop responses, water and energy trade-off and AI-based monitoring [16].

2. Literature Review

In their paper, Gnayem, N., et al (2024) [17] explore the foundational concept dual land use for PV and crops. This paper describes construction framing and geometry spacing methods for plant exposure to grow light. He reports on crop viability and the potential for technical feasibility, paving the way for future agrivoltaic metric establishment. Hassanien, R. H. E.; Ming, L. ( 2017) [12] Engages the concept of Land Equivalent Ratio (LER) for Agrivoltaics and advocates for total productivity with optimized panel density. He also discusses the shading–yield trade-off and the critical role of panel height and spacing. They highlights crop specific light response and techno-economic modeling as the most critical research gap. Marrou, H., Dufour, L., & Wery, J. (2013a) report field measurements agrivoltaic shading reduces canopy temperature and evapotranspiration. Growth inhibition was not always seen with moderate shading. He describes microclimate dynamics pertinent to greenhouse PV integration [13]. Marrou, H., et al. (2013b) analyze the hydrologic effects of PV canopy on soil moisture and the water and energy fluxes. This Shows promise for decreased water use for irrigation due to diminished evaporative demand. Provides a basic structure that can be used in CEA[18]. Weselek, A. et al. (2019) Comprehensive review in covering systems, the crops grown, and the energy yields[19].

The yield-energy outcomes influenced by panel design parameters like transmissivity and row spacing have been highlighted. This requires clearly defined measurements and evaluations over multiple years. Weselek, A., Ehmann, A., et al. (2021). looked at synthesis of microclimate and yield data from field agrivoltaics. This study illustrates cooling and water-saving effects and yield response depending on crops. This study emphasizes that the design of shading must be tailored to the physiology of the crop [20]. Al Mamun, M. A., et al. (2022). Provided a detailed analysis focused on the global deployment axis, as well as the techno-economics. Highlights agrivoltaics policy drivers and barriers. Modular and climate-specific design proposals with greater performance evaluation requirements are suggested [2]. Hassanien, R. H. E., Li, M. et al (2016). reviewed the integration of solar energy into greenhouses using Photovoltaic (PV) and solar thermal (ST) systems. Examines the effect on the crop environment and heating and cooling loads. Positions the PV greenhouses systems as a way for producers to achieve energy self-sufficiency[14]. Hassanien, R. H. E. et al. (2018). conducted experimental/assessments of semi-transparent PV (ST-PV) roofs. Reports a positive energy balance and acceptable light transmittance to crops. Recommends tuning of the coverage ratio to avoid yield penalties [21].

Cossu, M., Murgia, L., Caria, M., & Pazzona, A. (2016). Did a study prototype integrating micro-cell ST-PV into greenhouse glazing. Addresses the feasibility and optical features of light that affect photosynthetically active radiation (PAR). Stresses the importance of spatial uniformity of light for the development of the plant canopy[16].

Cossu, M., et al. (2016) further assesses PV-greenhouse sustainability for a variety of European countries. Balances energy generated with yield of crops grown. Modest PV installation with suitable crops selection can achieve a positive payback. Marucci, A., et al. (2016). Suggests dynamically shifting shading/Solar PV layouts based on season. Modelling shows energy and crop yield for all season are higher. Role of adjustability within CEA-PV polygon is crucial [22]. Moretti, S., et al. (2019). Describes passive geometric alteration of shading with respect to sun position. Reports summer cooling benefits without excessive lighting in winter. Advocates adaptable designs to improve food–energy ratios [23]. Loik, M. E., et al. (2017). Describes Wavelength-selective PV that transmits

PAR while harvesting NIR. Greenhouse trials demonstrate minimal growth impacts and positive energy impacts. Positioning of spectral tailoring design as a lever for CEA-PV[24]. Brown, A. (2017). Presented Perspective on sharing sunlight between PV and crops. Not primary, but context is relevant [21]. Teitel, M. et al. (2023). Evaluated the effect of OPV modules in greenhouses on light, temperature, and plant response. OPV was shown to offer heat moderation and predictably affect photosynthesis [25]. Gnayem, N., Magadley, E., et al. (2024). Reported a greenhouse study of cucumber crops under various designs of PV modules [17]. Low- to moderate shading coupled with reduced irrigation demands comparable yields. Emphasizes module selection and transmittance as vital for fruiting crops.

Papadopoulos, A. P., & Hao, X. (1997). Discussed reference text describing cucumber physiology, climatic objectives, and growing method. Provides guidelines for DLI and temperature/VPD. Foundational for interpreting the chiaroscuro effect in CEA-PV [26]. Kitta. E. and Katsoulas. N. (2020) described about Quantifies photosynthesis response of hydroponic cucumbers to shading. Demonstrates cucumbers photosynthesis response under moderated shading and heat stress[5]. Places a limit to the reduction of fruit set. Cui, J. Liu, X. Zhang, X. et al. (2021). Conducted a study of DLI requirements for cucumber seedlings. Mentions important light limits for healthy transplants[5]. Early growth implications for PV covered. Nikolaou. G. et al. (2023). Cucumber crop coefficients (Kc) under various microclimate regimes obtained. Links ET and irrigation requirements and control strategies. CEA-PV water saving models relevant[10]. NSW Department of Primary Industries (2019). Growing cucumbers in greenhouses: A practical guide. Outlines training and trellising, climate control, and fertigation. Sets climate control setpoints for PV integrated houses[1]. Light measurement and management in practice discussed.Relevant to semi-transparent roof light target.Sethi, V. P., and Sharma, S. K. (2008). Reviewed options for heating greenhouses and balancing energy[15].

Figure 1. PV Assisted Green House for Agrivoltaics ( Credit: Ravishankar Et al., Cell Reports Physical Science, 2021).

Figure 1. PV Assisted Green House for Agrivoltaics ( Credit: Ravishankar Et al., Cell Reports Physical Science, 2021).

Discusses how thermal energy is no longer needed with PV generation. Discusses management and operational control strategies for efficiency. Tiwari, G. N., & Ghosal, M. K. (2006). Devised and tested models of earth-air heat exchangers for climate control [1]. Demonstrates implementation of passive strategies for temperature control of greenhouses and is useful for combined methods of PV shading. Soussi, M. et al. surveyed active and passive cooling methods in greenhouses. Discusses separation of active and passive cooling, shading and control systems, greenhouse ventilation, and evaporative cooling, and positions PV shading among microclimate management tools. Zhou, Z., Siddiquee, M. M. R., Tajbakhsh, N., & Liang, J. (2018). Presented the UNet++ with improved multi-scale segmentation through enhanced skip connections. Very popular for segmentation of leaves and canopies in crop phenotyping [27]. Tan, M., & Le, Q. V. (2019). Developed a backbone model for crop stress and disease classification based on the compound scaling of depth, width, and resolution[28].

Selvaraju, R. R. et al. (2020). created a technique for generating class-discriminative localization maps. Used to interpret the CNN-based decisions made for greenhouse monitoring and flagging misclassifications[8]. Zhang, P. et al. (2020). Utilized EfficientNet to distinguish leaf diseases of cucumbers in greenhouse images. Achieves high accuracy via transfer learning. Shows the value for immediate monitoring of diseases within cucumbers[29]. Qian, T. et al. (2024). Suggested a bilayer CNN to achieve better cucumbers leaves segmentation. Tends to boundary and occluding issues. LAI proxy traits for control purposes are better[30]. Mao, Q., et al. (2024). Analysis of the temperature field of external shading of solar greenhouse. Found spatial and significant cooling gradients. Insights to shading pattern gradients of the PVs [31]. Soussi,M et al. (2022). Diffuse shading greenhouse climate and crop response impacts. VPD solar gain reduction under high-radiation is moderated and beneficial[32]. Gholami,Met al. (2025), practical techniques for estimating DLI with the use of sensors and loggers. Management is aided in decision-making by the conversion of PPFD to DLI. Budgeting calibration under semi-transparent PV canopies is noted [33].

3. Materials and Experimental Setup

3.1. Overview

This document outlines the components, techniques, and experimental configuration utilized in assessing the cucumber crop under controlled environment agrivoltaics, or CEA-PV. The design innovation regarding greenhouses equipped with semi-transparent photovoltaics (ST-PV) allows for the implementation of controlled replicated crop trials with varying degrees of shading for the crops. The methodology combines elements from the integrated agronomic research discipline, microclimate monitoring, and CNN-based phenomics, more specifically vision phenotyping[34].

3.2. Structure of the Greenhouse and Integration of the PVs

The experimental location consisted of a single span greenhouse measuring 12 m x 8 m, with a height of 4.2 m, giving a total of 96 m2 of experimental area.

The structural/frame system consisted of aluminum with polycarbonate side walls and a semi-transparent PV roof. To achieve the goals of PAR (photosynthetically active radiation) transmittance of 40-70%, a combination of semi-transparent silicon and organic PV modules was utilized. Within the greenhouse, there were four experimental bays differentiated by fractional shading of 0% (control), 20%, 40%, and 60% shading. The shading was achieved by a mix of PV covering and neutral density shading (NDS) films that were controlled by PAR sensors. The modules were south-tilted at 20° for optimal snow retention and radiation capture and were electrically connected to MPPT (Maximum Power Point Tracking) inverters with sub-metering. The PV system in the greenhouse was designed to produce 10 kWp, which enables the greenhouse to meet the minimum electrical demand for the ventilation, irrigation and data acquisition systems.

3.3. Crop Material and Planting Design

The example cultivar ‘Beit Alpha’ has been chosen for its increased economic importance and sensitivity to shading. After germination in a rockwool germination blanket, the seedlings were positioned within a hydroponic slab at bed intervals of a deep-water culture system. Each bay consisted of three replicate beds (1.2 m x 3 m) with twelve cucumber plants on vertical trellises per bed. The planting density of 2.5 plants/m2 is typical for cucumber production in greenhouses. A drip system with a nutrient solution at EC 2.5–3.0 mS/cm and pH 5.8–6.2 was used to provide nutrients. To simplify the calculation for water consumption, the closed recirculating hydroponic system used for this experiment was designed to minimize water loss.

3.4. Monitoring and Control of the Environment

Inside the greenhouse, a fully automated monitoring and control system has been installed. It allows the recording of data every minute regarding the air temperature, relative humidity, (RH), photosynthetically active radiation (PAR), global solar irradiance, CO₂ levels, and substrate moisture. The greenhouse has automated side vents, circulation fans, and heating, set to not allow the greenhouse air temperature to drop below 22 to 28 °C, and supplemental heating is available for night temperatures above 18 °C. The irrigation system is automated based on volumetric water content (VWC) levels, set to a target of 28 to 32% (deficit irrigation is applied). A water-use efficiency stimulation under variable levels of shading is promoted by applying a -10% and -20% setpoint shift. All bays were managed uniformly, with the exception of shading.

3.5. Data Capture and System for Image Processing

In order to assist in imaging, a phenotyping rail system was established. The imaging system included an RGB level 24 camera and a monochrome integrated NIR camera (with an 850 nm filter). The cameras were positioned to capture 5 peaks’ oblique and nadir view images and were automated to traverse along the imaging rails above the imaging bays. In order to maximise uniformity and control natural light, the imaging was done at noon. More than 12,000 images were recorded and annotations such as canopy masks, yields, and rate of stress (water, nutrients, heat, and climate stress), as well as leaf area index, were recorded during the crop cycles.

3.6. Phenotyping via Automated Image Processing

Phenotyping via automated image analysis and processing image analysis image analysis and processing image analysis image analysis The modular CNN framework was used to enhance image analysis and processing for the automated processing[35]. The canopy was segmented using UNet++ with ImageNet pretrained EfficientNet-B0 as the encoder. The Dice + cross-entropy loss was used to train the crops. EfficientNet-B3 was the model used to train the (γ = 2) focal loss for classifying stress. For the model, the following augmentations were used: rotation, brightness, and random erasing. A CNN with a ResNet-34 backbone that was trained with the SmoothL1 loss was used for LAI regression. Five-fold stratified cross-validation was used to prevent data leakage during cross-validation.

Adam optimizer and cosine learning rates were applied to the training process. For elicit model explainability, Grad-CAM visualizations were generated.

3.7. Treatment and Experimental Design

For this case study, a randomized complete block design (RCBD) was used. The treatments were combinations of four different shading levels (0%, 20%, 40%, 60%) and three approaches of irrigation (full, −10%, −20% deficit). Each of the 36 experimental units was replicated three times. To reduce the impact of seasonality, temporal replication was applied for two crop cycles.

Greenhouse fruit yields (kg/m²), fruit numbers, average fruit size, leaf area index (LAI), water use, and water use efficiency (WUE = yield/water use), and the greenhouse electrical load, and PV energy yield (kWh) were also recorded.

3.8. Statistical Analysis

As for crop yield, water use efficiency (WUE), and physiology, ANOVA (Analysis of Variance) was computed for the conditions of being shaded vs unshaded and irrigated vs non-irrigated. Post hoc Tukey HSD test (α = 0.05) was used. Cross validation was used for determining accuracy of the CNN model and the F1 score, R², RMSE, and IoU (intersection over union). For net energy balance, energy yield was evaluated relative to the simulated hourly irradiance and compared to the actual measured loads. All analyses were conducted in Python using scikit-learn and PyTorch and R was used for the statistical poste-hoc analyses.

3.9. Experimental Workflow

This process starts with nurseries that prepare and germinate seedlings of Cucumis sativus (cucumber) and then transplant them into hydroponic slabs. During this process, shading treatment was applied alongside PV monitoring. Environmental data, irrigation data and daily images were captured. During the week, LAI (Leaf Area Index), chlorophyll concentration, and fruit number were measured as well as biomass data collection. The control algorithms and CNN models were trained in cycles. AI control and conventional control were compared in this closed-loop system. Figure 2 shows the CNN-Assisted growth monitoring and stress management of cucumber for Agrivoltaics.

3.10. Replicability and Ethics

All materials used for the project such as the cultivar, hydroponic systems, sensors and imaging pieces of hardware, are commercially available. In terms of replicability, the random seeds were cemented in the context of CNN training and the k-fold splits were kept as is. There were no human or animal participants in this study. Integrated Pest Management (IPM) practices were possible and pesticide usage was recorded. The design implemented sustainable principles of water recycling, energy self-sufficiency, and study replication. After replication, data and code will be available [36].

The methods framework described here enables integration of the engineering of CEA greenhouses, PV energy modelling, physiology of cucumber crops, and CNN phenotyping within a single experimental framework. The design, through systematic variation of the shading and irrigation treatment, creates the possibility of quantifying the food-energy-water trade-off. The imaging and CNN pipeline offers scalable monitoring that can be used for closed-loop greenhouse management. Collectively, this section lays the technical foundation for the results and discussions that follow.

4. CNN-Based Phenotyping (Detailing the Approach)

The CNN-based phenotyping framework was built to aid in the automation of the extraction of the plant traits, both structural and physiological, under various levels of shading in the ST-PV greenhouse. The framework incorporates a multi-faceted imaging system, a multi-level deep learning model suite, and a set of defined preprocessors for feature extraction, and truth labeling, along with canopy segmentation, stress detection, and growth estimation. The framework incorporates spacial calibration along with temporal consistency and provides ground truth agronomic feedback to facilitate the dual aims of quantitative trait assessment and the real-time feedback to closed-loop systems managing environmental control systems [37].

4.1. Imaging & Annotating the Ground Truth

High-resolution industrial RGB cameras in the range of 12 to 20 MP for a better image capture were used to facilitate the same imaging geometry for best results for the testing[38]. Zenith view and optional oblique view of the canopy were used to determine vertical growth traits such as the height of the plant and some, the set of fruit. Every view encompassed a unit of measure for a scale or a ArUco (a 2D bar-coded identification system) for the purpose of true conversions of pixels to centimeters. The cameras captured the view of the topology every hour between eight in the morning and six in the evening, and the images were recorded in both uncompressed true image/portable network graphics formats and also a compressed 1024 by 1024 format used for prototype model training.

Camera calibration involved taking black and white images of a checkerboard pattern to estimate intrinsic parameters and for white-balance corrections performed daily using a reference card. It was ensured that the exposure settings would remain constant to avoid any changes to auto-gain. Annotation for images included canopy masks made of polygons between exponentially ranging exposures of the images of 0%, 20%, 40%, and 60% shading, out of the 1000-2000 labeled images. A small subset between 10 and 20% was also annotated for the task of counting the leaves using polygons at the leaf instance level. Each image of a plant was annotated for stress category class using a dual-annotator method with respect to adjudication as a measure of reliability. This dual-annotator method also applies to the reporting of ground truth growth parameters that included bean plant heights, internode lengths, leaf counts, fruit counts with optional readings of SPAD chlorophyll measured three times a week along with ground truth readings to validate and also calibrate to the vision-based measurement of the parameters.

The block diagram of experimental work is given in the Figure 3.

4.2. Pre-Processing

All images were first subjected to geometric corrections by the calibration matrix to minimize lens distortion and also to ensure spatial integrity of the images. To minimize background noise and excess computing, the regions of the image containing the greenhouse were cropped automatically. To color normalize the images using the gray and white color patches, color normalization was applied to minimize the effects of time or lighting variations.

Data augmentation strategies have been used throughout model training to help the model generalize better to all data, not just the training data[39]. Geometric data augmentation strategies include rotation by negatives or positives few degrees, up to or down to fifteen percent for scaling, and both horizontal and vertical flipping of images. In terms of photometric augmentations, data augmentation strategies include occlusion of images to mimic a stake or a label, slight changes in brightness and contrast by up to or down to fifteen percent, and minor changes in the hue of an image. In order to keep the model from seeing the validation and test data during training, in order to prevent data leakage, the dataset was split by plant ID and date into a training validation test split of 70/15/15.

The phenotyping framework has been designed so that the model has 3 separate deep learning heads, one for each of the tasks of canopy segmentation, stress classification, and estimation of growth.

‘A’ Canopy Segmentation

U-Net and DeepLabv3+ models with EfficientNet-B2 and ResNet 34 backbones have been used for canopy segmentation. The input was a single 1024 by 1024 pixel RGB image and the output was a binary image where 1’s represented the canopy and 0’s represented the background. The training loss was a function of both Binary Cross Entropy and Dice Loss, each having a weight of 0.5. Other metrics of performance in addition to Dice and IoU were evaluated and pixel accuracy was measured at each level of shading separately for each level of shading.

The segmentation mask was used to first calculate the projected canopy area in cm² from pixel calibration downwards. The canopy cover ratio was derived from the ratio of the projected canopy area to the whole area of the plot. The Leaf Area Index was estimated from Bu, Lambert, and beer‘s law by using a calibrated value for the extinction coefficient or by using a destructive sample for an empirical regression fitting equation of LAI = a• CC + b.

B) Stress Detection

Abiotic stress detection utilized EfficientNet-B0/B2 classifiers. With segmentation masks available, input was posed masked canopy crops to center on plant tissue. For classes such as healthy, shade-stress, heat-stress, water-stress, and nutrient-stress, rare classes were merged underrepresented. Class imbalance was handled using Focal Loss (γ=2) or Weighted Cross-Entropy.

Class justification was performed using Grad-CAM or Score-CAM so that the predicted class was supported from the agronomic standpoint. Per each shading level, a report on macro-F1, class precision, class recall, and AUROC (flattened) was included.

C) Growth Estimation

For estimating plant height, a keypoint or regression method was applied. In the keypoint approach, HRNet or Keypoint R-CNN identified anatomical points (apical meristem and pot rim), and distance in pixels and calibrated to centimeters. In the alternative approach, height was regressed using mask-derived features such as convex hull height, skeleton length, and bounding-box parameters.

Leaf counting was performed using Mask R-CNN (ResNet-50-FPN backbone) that produced leaf instance masks from which leaf count, mean leaf area, and shape descriptors were derived. For applicable fruit detection, fruit count and diameter were estimated using either YOLOv8-s or Faster R-CNN. For the purpose of computing reliable daily growth rates, daily growth trajectories were smoothed using either Kalman filtering or Savitzky-Golay filtering. Growth estimation and CNN assisted modelling simulation photo is given in the Figure 4.

4.4. Multi-Task Alternative

As an extension, a unified backbone architecture was investigated, consisting of three heads, each dedicated to a specific task. The shared encoder was used to jointly train segmentation (Dice/BCE), stress classification (CE/Focal), and growth regression (Smooth L1 or MAE/Poisson for counts) tasks. Uncertainty-weighted loss as described by Kendall et al. was used to implement task balancing, enabling automatic adjustment of task significance during training.

4.5. Training Details

The models employed a cosine learning rate decay with a 5-epoch warmup and AdamW optimizer (learning rate 3×10⁻⁴, weight decay 1×10⁻⁴) for training. Batch size was between 8-16 and utilized mixed precision training. Total training duration was between 60-100 epochs with early stopping set to a 10-patience limit. Based on validation IoU for segmentation, macro-F1 for stress, or MAE/RMSE for growth, the best model was selected. Stratified 5-fold cross-validation was done by plant ID and week with results presented as mean ± standard deviation.

4.6. Post-Processing & Derived Traits

Morphometric traits from leaf and canopy masks, such as PCA, perimeter, convex hull area, solidity, mean leaf area, leaf area variance, aspect ratio, and leaf curvature proxies, were pulled. The detection of stress was improved using pixels masked with ExG, VARI, and Excess Red.

Series of traits over time were analyzed to estimate growth rates (cm/day, cm²/day), time to flower, max growth rate, and area under the curve (proxy for biomass).

4.7. Quality Control & Validation

Quality assurance of segmentation included manually checking 5% of the validation frames weekly, and frames were tagged if the IoU decreased over five percentage points. Grad-CAM overlays were employed to study the misclassification of stress in the shade classes.

The statistical approach was focused on the agreement between measurements. The height estimates had a goal of under 2-3 cm in MAE, and this was validated using the Bland-Altman method. For leaf counting validation, 80% of cases were within ±2 leaves of the target. For measurements with a LAI meter, its LAI predictions had an R² ≥ 0.8. For robustness testing, perturbations (±15%) and simulated occlusions were included.

4.8. Integration of Closed-Loop Control

Phenotyping outputs have been consolidated, and so have the control logics of the environment. Shade adjustments would be triggered if the probability of shade-stress was greater that 0.7, and if the PPFD was below the setpoint for 20 minutes. If water-stress probability > 0.7, or when the canopy wilting index exceeds the set threshold, irrigation is triggered. If the heat-stress probability is > 0.7, and x leaf-air temperature gradient > , then ventilation is triggered. All actions taken, environmental conditions, model’s confidence, system’s response, and all relevant parameters were recorded and documented”.

4.9. Reporting Instructions

The manuscript should contain model architecture and task, shading level, loss functions, and datasets divided along with augmentation strategies for reproducibility, and datasets divided for metrics (mean ± SD across folds). Calibration curves (e.g. predicted vs measured LAI) should be included with qualitative results, including segmentation overlays, and Grad-CAM visualizations, along with growth curves for varying shade levels.

4.10. Notes for Reproducibility

The implementation utilized PyTorch/Lightning, Albumentations, Detectron2 for instance segmentation and keypoints, and Ultralytics YOLO for fruit detection. Deterministic settings and fixed random seeds were applied. All preprocessing configurations and camera intrinsic parameters were saved. A reproducibility artifact including trained weights, inference scripts, and a 50-image sample dataset was prepared for release.

5. Discussion and Analysis of CNN Results

5.1. Performance of the Segmentation Network and Extraction of Structural Traits

The segmentation network has shown strong results and has provided an average IoU of [IoU] and a Dice coefficient of [Dice]. These results are evidence of successful segmentation of the cucumber canopy area regardless of the different lighting conditions. Performance was even throughout the range of 0% to 40% shading. However, at 60% shading, there was a reduction due to an increased sensor gain and lower PPFD, leading to a loss of definition for leaves and background. Overfitting due to high-light conditions was improved with data augmentation techniques. The performance at 60% shading improved with the use of exposure jitter and simulated occlusions by approximately [∆IoU].

The predicted masks were used to calculate the following structural traits: projected canopy area (PCA), convex hull area, canopy solidity, and canopy cover. The temporal trajectories demonstrated a similar pattern of growth across all the different treatments. The fastest canopy expansion was observed with the 20% shading treatment, where the thermal and light stress was less than the 0% shading treatment. Although 0% shading stimulated a rapid initial expansion, there was a subsequent decline in canopy area during high evaporative demand periods of the tropical dry season. Canopy closure was also limited in the 40-60% shading range, with 60% shading consistently having the lowest PCA values.

The proxy of the LAI obtained from vision proven strong agreement with the LAI-meter/ destructive measurements (R² = [R2_LAI], RMSE = [RMSE_LAI]) and thus, the cover of the canopy can be used as a biomass proxy. LAI and yield relationship analysis showed a low yield increase with LAI values greater than [LAI]*. (The LAI value of [LAI]* was typically reached at around 20% canopy cover). Deficits in yield increase as canopy cover exceeds 20% suggest low light use efficiency and either a photosynthesis limitation or poor vegetative growth at greater levels of canopy cover.

The failure modes in plant segmentation were (i) specular reflections of plastic mulch and metallic poles causing false detections, (ii) false detections from deep shadows of stems, and (iii) plant-neighbor overlaps. The removal of small components of [n] pixels and filling of the internal holes in the segmentation mask was able to diminish the false causal artifacts without overestimating the canopy.

5.2. Stress Detection (EfficientNet + Grad-CAM)

The stress classifier macro-F1 was [F1_macro] with high precision for heat and shade stress classes. Most confusion was between nutrient and shade stress; at 60% shading, the leaf chlorosis and desaturation patterns were alike. The penultimate layer’s vegetation indices (ExG, VARI) inclusion resulted in an improved separation of +[∆F1]; thus, in spectral patterns, as opposed to geometric, stress phenotypes in cucumbers were more delineated.

The reliability analysis demonstrated good calibration (ECE = [ECE]). An operational threshold of p(stress) ≥ 0.70 proved to be the best compromise between false alarms and an adequate time to react, obtaining [precision] and [recall] for the corresponding dimension. This threshold reached the best compromise between unnecessary shading or ventilation adjustments and capturing the most true stress events..

The most frequent false positives were recorded during the late afternoon hours, when lower angles of the sun’s rays caused changes in leaf reflectance. The incorporation of additional microclimate validators (PPFD and VPD) considered environmental conditions (e.g., PPFD < [setpoint] for ≥20 minutes for shade-stress; leaf–air temperature gradient > [k] °C for heat-stress) and significantly diminished the number of false positives.

Most of the points of interest in the Grad-CAM visualizations were the tips of the leaves and the areas between the veins. Somewhat at her leaves, younger leaf tissues, Agronomic, and Theory stress is visible on the. The dispersed focus over the canopy in incorrect classifications for nutrients aligns with the concept that the signs of nutrient deficiency are not easily visible and are obscured by excessive shading.

5.3. Growth Estimation (Height, Leaf Count, Fruit Count)

Plant Height

After Kalman smoothing, the MLP regression and keypoint-based approaches obtained MAE = [MAE_h] cm with respect to manual ruler measurements. Growth curves showed that the highest relative growth rate occurred under 20% shading, which was significantly greater compared to 0% and 40–60% treatments (p < 0.05). This implies that moderate shading helps in reducing thermal stress and stomatal limitation without limiting photosynthesis.

Leaf Count and Morphology

Mask R-CNN attained leaf counting with MAE = [MAE_leaf] leaves. Plants under 20% shading had slightly bigger and rounder leaves, whereas 0% shading had smaller and thicker leaves, which is characteristic of light acclimation responses. Under 60% shading, the average leaf area decreased and the mean internode length increased reflecting the classical shade avoidance syndrome, which is reflected in the height to area ratio shortening.

Fruit Detection

Fruit detection with YOLO/RCNN is highly correlated with manual counting (R² = [R2_fruit]). The earliest fruit initiation occurred under 20% shading. In zero shading treatments, flower abortion was higher during heat peaks. Shading of 40–60% delayed the initiation of fruit by [∆days] and reduced the final count of fruits.

5.4. Studies of Design Choices

The stress classification performance was improved with the use of masked inputs by ∆F1mask. This illustrates the variability of the background (benches, trellises) which caused distracting backgrounds. The improvement of the segmentation under 60% shading (∆IoUaug) was the result of photometric augmentation. Reductions in model size and improved data efficiency from multi-task learning with uncertainty-weighted loss. However, due to the limited number of fruit annotations relative to canopy annotations, there was some negative transfer in fruit detection (−[∆R2]).

The day-to-day noise was reduced by approximately [x%] due to the use of temporal smoothing with Savitzky–Golay and Kalman filters. True growth spurts continued to be visible and uncensored. Autocorrelation bias was minimized by conducting statistical analysis on daily means at the plant level.

The negative effects of the domain shift from mixed color temperatures and cloudy days were corrected by white-balance normalization. Small systematic height bias due to minor camera drift was corrected by periodic camera recalibrations. The use of minimum-area priors and temporal persistence constraints helped to filter the occlusions caused by vine ties and clips on the trellis. When the analysis was performed in strata based on the location of the bed, the bias from the edge plants was removed.

5.6. Biological Interpretation and Agrivoltaic Trade-Offs

The 20% shading treatment has the best combination of vegetative growth, lower stress probability, and, earlier fruit set. Zero shading has the same yield potential, but increases the chance of heat stress and increases the need for irrigation. Shading of 40-60% stunted vegetative vigor and delayed phenology, showing photosynthetic limitations irrespective of improvement to microclimate moderation.

As per the ST-PV logs, the electrical yield went up with the proportion of shading, whilst the CNN-derived biomass estimates declined at 20-30% shading. Thus, the agrivoltaic Pareto frontier is likely at approximately 20% shading, where marginal energy gains beyond this point become losses. This trade-off can be expressed as an area trade-off dual axis with kWh PV energy on one axis and either fruit count or LAI.

5.7. Impact on Closed-Loop Control

The integration of CNN outputs into the environmental control system has shown operational improvement. Monitoring shade stress allowed for less time to be below the PPFD setpoint by x% and reduced the amount of excessive midday shading. Heat stress based ventilation guidance reduced the canopy-air temperature difference by y% and decreased the occurrence of afternoon wilting. Control of irrigation based on wilting index and stress probability allowed for a reduction in water usage by z%, all without adversely affecting growth trajectories. Image-based phenotyping has shown to be more than a descriptive tool, but rather a decision support system in agrivoltaic controlled-environment agriculture systems.

5.8. Limitations and Future Research

The specificity of classifiers is limited due to sparse annotations within the fine-grained categories of nutrient stress. However, active learning strategies could be used to focus uncertain frames. In the case of two-dimensional projections of the canopy, they may underestimate the biomass of very dense canopies. Therefore, the combination of very low-cost depth sensing (such as stereo or LiDAR) is likely to provide a greater level of accuracy in the estimation of LAI (Leaf Area Index) and canopy height. The generalization to other cultivars and trellis systems is still to be validated. In terms of enhancing domain adaptability, self-supervised pretraining (e.g. DINOv2) could be the answer. Future studies should be able to combine CNN features with time series microclimate data using temporal transformers to maximize the separation of shading, temperature, and VPD (vapor pressure deficit) impacts.

In other angle, data security is very important during the data collection and data transfer in which federated learning will give promising result. In the future work, federated learning will be applied to do the processing of sensor’s data in the agrivoltaics [40].

5.9. Key Findings

Under shifting shading conditions, the CNN (convolutional neural network) pipeline demonstrated robust canopy segmentation and stress detection, as well as excellent growth prediction.

Around 20% shading presented a perfect agronomic–energy equilibrium, corresponding to the PV–crop Pareto frontier.

The use of closed-loop control based on CNN predictions minimized extreme heat shocks, balanced PPFD (photosynthetic photon flux density) exposure, and optimized irrigation.

6. Conclusion and Future Recommendations

This research highlights the potential use of convolutional neural network (CNN) based phenotyping coupled with semi-transparent photovoltaic (ST-PV) modules to increase cucumber cultivation efficacy in agrivoltaic greenhouses. The framework employed a combination of real-time imagery and automated climate controls for irrigation, shading, ventilation, and climate responsive canopy segmentation, stress detection, and growth estimation.

Intensity of shading was proven to be the primary factor influencing plant growth, stress, and yield. The 20% shading treatment was the optimal condition for vegetative vigor and low stress and early fruiting with the ability to produce solar energy. Zero shading encouraged rapid early growth but it increased heat stress and irrigation. 40-60% shading was proven to be a growth limiting factor and delayed fruiting which increased the energy capture versus crop yield.

Decision making for the closed loop agronomic systems was aided through the CNN models. System microclimate responsiveness , through active modification, was proven more efficient in energy and resource distribution. This work demonstrated additional advancement in renewable energy integration with AI phenotyping, providing scalable opportunities in controlled environment agriculture for high value crops [41].

Multimodal sensing (spatial, hyperspectral, IoT micro climate integration), cross crop transfer learning for long term techno-economic assessment of agrivoltaic systems, and cross crop transfer learning for long term techno-economic assessment of agrivoltaic systems are the areas of priority for future research in the field. The development of energy efficient self regulation systems that are capable of providing support to the systems [42] in place to enhance the transition to clean energy is the primary field of study for this research.

The reliability assessment of cyber physical systems is always important [43,44]. In this context, reliability assessment of photovoltaic system for agrivoltaic will be given utmost important in the future direction of this research work.

Overall, the proposed CNN-assisted framework demonstrates an integrated approach to smart monitoring and stress detection systems for stress management in cucumbers cultivated under semi-transparent PV greenhouse-based agrivoltaics. It is possible to use 3 D printing methodology to develop the green house for agrivoltaic as future work in the proposed area [45]. Also, the solar-based solution offers a more favourable green effect and energy efficiency [46]. AI-based solutions are being applied across many biological areas, where selecting the right algorithm is challenging [47]. So, a robust AI methodology will deliver high accuracy and perfect optimization of parameters across many areas[48] , including agrivoltaics. Furthermore, the ease of construction and the hybrid-renewables approach in past studies corroborate the feasibility, scalability, and sustainability of intelligent agricultural systems.

Acknowledgement: We acknowledge the National University of Science and Technology, Muscat, Oman and the university management for their great support.

Consent for Publication: All authors have been informed and have consented to publication.