Data Analytics in Agriculture: Enhancing Decision-Making for Crop Yield Optimization and Sustainable Practices

1. Introduction

Smart farming, also known as smart agriculture, represents a transformative approach to traditional farming practices. The main feature in this approach is the adoption of advanced technologies and data-driven operations to optimize and enhance sustainability in crop production. Smart farming key features include Artificial Intelligence (AI), Automation, and Internet of Things (IoT). AI algorithms analyze data collected from various resources to enable better decision-making for farmers. Farming processes such as planting, irrigation, and harvesting can be automated using proper devices. IoT devices facilitate the collection of real-time data from sensors, drones, and other connected devices on a farm [1].

Smart farming emerges as a response to critical global challenges such as food security, climate change, rising costs, and regulatory requirements. By using technology and data-driven decisions, smart farming aims to tackle these issues while improving productivity and resource efficiency. The urgency is underlined by the overwhelming task of producing 85% more food by 2050 to feed an estimated global population of 8 billion people [2]. However, this endeavor is not easy taking into consideration that natural resources are increasingly scarce, environmental degradation persists, and impacts of climate change are obvious. In this context, traditional farming practices, particularly on small family-owned farms, can no longer meet these demands without the adoption of innovative technologies [3].

Data-driven agriculture plays a pivotal role in achieving sustainable farming practices. To be precise, data collected from sensors and analyzed using AI can provide valuable insights. Farmers can make decisions about planting, fertilizing, and soil management, leading to higher yields and improved sustainability. In addition to that, data-driven approaches enable precision agriculture techniques, such as variable-rate application of inputs which leads to resource optimization. By optimizing resource usage, farmers reduce wastage and minimize environmental impact [4]. As a result, data-driven insights help farmers understand the interplay between natural systems. This knowledge is crucial for achieving both sustainability and food security [5].

While data-driven agriculture provides potential solutions, it faces some challenges. For example, to develop a comprehensive decision-making system, legal complexities related to shared data protection should be guaranteed. Moreover, technical barriers must be addressed to streamline the integration of data across various platforms [6]. Ensuring high-quality data and compatibility between different systems remains a challenge. Standardizing data formats and protocols is essential for effective smart farming [1]. Building data science capacity among farmers and agricultural producers is also crucial. Training programs and educational initiatives are required to empower stakeholders in utilizing data and collaborating effectively to improve their farming practices [7].

This paper explores the outcomes of collaborative smart agriculture, achieved through technology and community partnerships. The main objective is to develop machine-learning models that enhance crop yields while promoting sustainable farming practices within the context of smart farming. More precisely, this project focuses on the collaboration between small family farms located in the southern region of Zagreb, Croatia, and academic institutions. Through this partnership, we aim to demonstrate the pragmatic benefits of smart farming based on IoT.

These family-owned farms have been dedicated to producing organic food using traditional methods. However, they face several challenges, including low production quantities, irregular supply chain operations, and a noticeable lack of entrepreneurial skills. Moreover, the impact of climate change aggravates their struggles, leading to increased drought effects and water scarcity, further impeding productivity.

Our research contributes to the advancement of smart farming through a multi-phase objectives approach. The first phase is the collaboration with a producer of organic vegetables and fruits to install sensors in the fields to collect data. The sensors collect data on various environmental parameters, including moisture, humidity, pH levels, light intensity, and nutrient content. By leveraging data-driven insights, boosting crop yield sustainability is expected. Moreover, a conceptual framework will be designed to train and educate farmers to improve their practices efficiency. Moreover, the findings from the first phase will be used for an educational prototype that focuses on enhancing entrepreneurial and digital skills among them. Ultimately, this research aims to empower farmers. They are expected to become proficient in administering data collection and monitoring. By implementing IoT systems, farmers can increase their yields while maintaining sustainability practices.

Capacity building for small farms includes topics such as AI, IoT, GIS, robotics, drones, data analytics, finance, digital marketing, and “soft skills” to enable smart agricultural practices. In this research, we aim to include frameworks for enhancing collaboration among stakeholders and assessments of the economic and environmental impacts of collaborative approaches. Enhanced collaboration and data-driven decision-making foster cooperation and communication between all parties involved in the agricultural supply chain. This can lead to improved processes, shared resources, and enhanced problem-solving.

By integrating educational modules and best practices into the platform, all stakeholders can have access to valuable information that can drive more effective and sustainable farming practices. Through this research, farmers will be trained on the use of IoT systems, including sensors for data collection and monitoring, to improve their yields and sustainability. By fostering collaboration among various stakeholders in the agricultural supply chain, from farmers to distributors, this paper seeks to illustrate the potential of smart farming to transform traditional agricultural practices, making them more efficient, cost-effective, and sustainable.

This research effort is a collaborative work, drawing on the expertise of researchers from RIT-Dubai and RIT-Croatia. The interdisciplinary approach, combined with industrial partnerships, enhances the potential impact and scalability, along with its educational benefits.

The remainder of the paper is organized as follows. After the literature review in Section 2, we introduce the methodology adopted to achieve the main objective which is the framework for enhancing collaboration among stakeholders and conducting assessments of the economic and environmental impacts of these collaborative approaches in Section 3. As part of the framework, a database will be developed based on the data collected, which will be used to build a comprehensive platform for smart monitoring and education. In Section 4, the results of the first phase are discussed. Finally, the conclusion and future insights are presented in Section 5.

2. Literature Review

In this section, literature on the evolution of agricultural technology is presented. Additionally, sensor technology in agriculture, the impact of NPK, humidity, and temperature monitoring on crop yields is reviewed. Finally, the latest advances in IoT and data analytics for farm management are explored.

Precision agriculture or farming (PA), which emerged in the late 20th century, is a rapidly advancing field that aims to enhance agricultural productivity, sustainability, and efficiency through data-driven approaches. Sensor technology is the backbone of PA by enabling real time monitoring of farming parameters. Adamchuk et al. [8] addressed the role of the soil sensors in PA in measuring soil moisture, pH, temperature, and nutrient levels. According to the authors, the sensors provide crucial data that enable farmers to make decisions about irrigations, fertilization, and other management practices thus enhancing crop yields and sustainability.

Shafi et al. [9] discussed and proposed transforming the agriculture sector into PA through IoT-based automation for crop monitoring. The proposed solution includes a wireless sensor network (WSN) for real-time monitoring of crop health status and a remote sensing platform for obtaining multispectral imagery to classify healthy and unhealthy crops. Their results demonstrated the enhancement of agricultural practices through automation and data-driven decision-making through increased production with reduced human effort.

Singh et al., [10] highlighted the importance of utilizing remote sensing for efficient operation of irrigation to improve water use efficiency in agriculture. The use of unmanned aerial vehicles (UAVs) as remote sensing has become an essential tool in PA. UAVs provide aerial images that can be used to detect issues early and apply precises intervention, thereby optimizing resources and improve crop management [11].

PA composes advanced technologies to monitor and optimize key environmental and soil parameters, significantly impacting crop yields. The most important parameters are nitrogen (N), phosphorus (P), potassium (K) levels, humidity, and temperature. N is essential for plant growth. It encourages vegetation development and leaf growth. While P plays a crucial role in energy level in plants. It impacts root development, flowering, and fruiting. K reflects the quality nutrient in plants. It enhances the resilience of plants to adapt to changes through controlling photosynthesis, protein synthesis, and nutrient uptake. A balanced NPK values have a great impact on healthy growth, resilience to stress, and crop yield of plants [12].

Recently, Weddell [13] discussed a complex precision management strategy with the focus on variable application of nitrogen (N) fertilizer, because N is a main limiting factor in cereal production. The authors managed to control the level of N using the PA technologies. Chen et al., [14] investigated the integration of soil-crop system management in China to enhance grain yield for rice, wheat, and maize while reducing environmental impacts. Through 153 field experiments, the integrated system increased average yields significantly without additional nitrogen fertilizer, showcasing lower nutrient use, loss, and greenhouse gas emissions compared to traditional practices. The study revealed the possibility of meeting food demands sustainably by adopting precision management system widely, ensuring food security and reducing the environmental footprint of agriculture.

Ragul et al., [15] proposed an automated irrigation system for efficient water management and intruder detection, incorporating sensors for pH, soil moisture, and humidity measurements. The authors discussed empowering farmers to remotely access field information, reducing manual labor and time while improving productivity and resource utilization. Communication with the farmer is facilitated through a GSM module, enabling real-time updates on field conditions via SMS.

Recently, Murali et al., [16] implemented an IoT-enabled soil nutrient system to assist farmers in making informed decisions throughout the farming process. Their model incorporates sensors, cloud computing, and machine learning tools to collect real-time data on soil conditions and analyze nutrient levels. The ML models could recommend suitable crops while minimizing fertilizer usage for enhanced productivity. According to the authors, the ML proposed model achieved high accuracy in nutrient classification and crop recommendation, providing farmers with a cost-effective and efficient solution for soil management and crop selection.

Another study found that the use of digital platforms and tools can help to improve communication and coordination among supply chain partners, leading to increased efficiency and reduced costs. Overall, the literature suggests that collaboration among various stakeholders in the farming supply chain can lead to many benefits, including increased efficiency and productivity, reduced costs, and improved product quality [17].

In the context of decision-making, an additional critical factor should be considered. While the possession of valuable information is expected to enhance the decision-making process, it is essential to evaluate the accuracy of the data. Furthermore, understanding the factors that influence the data is important. Therefore, it is insufficient to merely collect data; the data must be collected, cleaned, structured, and prepared for analysis. "The information that crops offer is turned into profitable decisions only when efficiently managed" [1].

This literature review provides a concise overview of the key findings related to the area of improving and collaboration within the farming supply chain. Smart farming supply chains can bring benefits for all stakeholders. However, it is important for all parties to have open and transparent communication, and for there to be a balance of power among stakeholders. Moreover, technology and digitalization can play a key role in facilitating collaboration.

3. Methodology

The core of this research hinges on advanced sensor technology designed to monitor critical environmental and soil parameters that influence crop yields. In addition to that, the researchers managed to increase awareness of smart farming practices among the region of the small family farm. This section explained the methodology that was followed to perform the research.

3.1. Sensors Selection and Implementations

To facilitate the farmer's interaction with the technology, a single piece of equipment integrating multiple sensors was selected. This approach precluded the need for the farmer to implement four distinct sensors. The sensor selected is IP68 ABS Soil NPK Sensor, High Precision Soil Nutrient Intelligent Fertilizer Detector Tester Meter NPK Sensor. Its specifications are: Measuring Range: 0-1999mg/kg - Working Humidity: 5 to 95% (relative humidity), no condensation - Measurement Accuracy: ±2%F. s - Resolution: 1mg/kg (mg/l) - Working Temperature: 5 to 45 ° C - Power Supply: 12V-24V DC. Figure 1 presents a detailed close-up view of the sensor.

We developed a sensor system utilizing the ESP32 microcontroller, incorporating an NPK 5-pin sensor for measuring soil parameters including potassium, phosphorus, humidity, temperature, and pH levels. The system was equipped with Wi-Fi connectivity for real-time data transmission and featured local data logging onto an SD card to ensure data backup in the event of Wi-Fi instability. Figure 2.

The sensor was strategically installed within the fields of the small family farm in the southern region of Zagreb, Croatia, Figure 3.

The methodology was tailored specifically for growing spring onions, ensuring the monitored parameters align with the optimal conditions for this crop. The following ideal conditions were considered [19]:

Temperature: 15 - 21 degrees Celsius, pH Level: 5.3 to 5.8, Moisture Level: Should be moist but not too wet (50-60%), Nitrogen Level: 10-50 mg/kg, Phosphorus Level: 5-25 mg/kg, Potassium Level: 10-40 mg/kg, Electrical Conductivity (EC): Low to moderate level of EC.

3.2. Data Collection Process

The data collection process was designed to capture comprehensive environmental and soil parameter data across the study site. The steps involved are as follows. Sensors continuously recorded data at set intervals (every 2 minutes) to capture enough variations and overall trends. The data were transmitted wirelessly to a central database using IoT communication protocols. Also, data logger was used as a backup plan to prevent data loss. The collected data were logged in a cloud-based storage system, allowing for remote access and ensuring data integrity. Regular calibration of sensors was conducted to maintain accuracy. Validation of sensor data was performed by comparing sensor readings with expected values. Table A1 in the Appendix exhibits samples from the data collected from the sensor.

3.3. Analytical Techniques Used

The analysis process involved a combination of statistical and predictive modeling techniques to derive meaningful insights from the collected data. Initial data analysis involved basic descriptive statistics analysis to identify any outliers or anomalies. Pearson correlation coefficients were computed to examine the relationships between different parameters (e.g., NPK levels, soil moisture, temperature, and humidity). This analysis helped in understanding how these factors interact and influence crop yields.

3.3.1. Principal Component Analysis (PCA) & Biplot

PCA was used in conjunction with biplot visualization to explore the multivariate structure of our dataset. PCA is a dimensionality reduction technique that identifies the underlying patterns and relationships within high-dimensional data by transforming it into a lower-dimensional space while preserving most of the original variance. The biplot representation extends PCA by simultaneously displaying both the observations (samples) and the variables (features) in a single plot, providing insight into their relationships and contributions to the principal components. In our analysis, each data point represents a sample, and its position in the biplot is determined by its scores on the principal components. Additionally, the direction and length of the variable vectors in the biplot indicate the strength and direction of their influence on the principal components, allowing for the interpretation of variable relationships and their contributions to the overall variance in the dataset. This combined approach of PCA biplot analysis enables a comprehensive visualization of the complex interrelationships among the variables and facilitates the identification of underlying patterns and trends within the data.

3.3.2. Analyzing Humidity Curve

For the automated detection of watering events based on humidity data, we leveraged the functionality provided by the latest version of the SciPy library in Python. Specifically, we employed the find peaks function, which offers robust peak detection capabilities. This method utilizes various parameters, such as peak prominence and peak width, to identify local maxima in the dataset corresponding to significant changes or peaks in humidity levels as shown in Figure 4. By carefully tuning these parameters to suit our dataset and experimental conditions, we were able to accurately detect peaks indicative of watering events. This automated approach not only facilitated the timely detection of watering occurrences but also eliminated the need for manual intervention, thereby streamlining the data analysis process and enhancing its efficiency.

3.4. Developing the Entrepreneurial Skills of Small Family Farm Owners

Alongside the technological methodology mentioned earlier, we also focused on enhancing the sustainability of the project by developing the entrepreneurial skills of small family farm owners. The owner of the small farm where we installed the sensors actively participated in the process related to greenhouse activities. We applied the Six Sigma concept, mainly the DMAIC methodology, to initiating discussions about inefficiencies in agricultural production. Through these discussions, the owner became aware of unnecessary visits to the greenhouse and excessive water and energy consumption. Fortunately, he trusted the process and was willing to collaborate. Together, we analyzed the most critical issues and identified water consumption as a priority area for improvement. To address this, we implemented sensors, and his collaboration in monitoring the process was invaluable. After applying the Six Sigma concept, other small family farm owners visited the greenhouse. During discussions about scaling the project, they concluded that collaboration could enhance their production and distribution.

4. Discussion and Results

Utilizing the 5-in-1 sensor and the data logger, the data on N, P, K, pH, and the temperature were collected over several days. Figure 5 below presents a snapshot of the data plotting. As it can be seen from the figure, the N, P, and K values decreased as the plant grew.

4.1. Potassium and Electrical Conductivity Association

Our analysis revealed consistent patterns between EC measurements and potassium levels within the soil. Given the known association between potassium ions and EC, this observation suggests the potential for approximating potassium levels based on electrical conductivity readings, or vice versa, Figure 6.

This finding offers an intriguing avenue for leveraging electrical conductivity as a proxy for potassium levels, thereby providing a cost-effective and efficient means of estimating potassium concentrations in soil. Additionally, this correlation underscores the interplay between these two parameters and highlights their utility in assessing soil fertility and nutrient dynamics. Further investigation into the precise relationship between electrical conductivity and potassium levels could yield valuable insights into soil health and management practices. The observation of decreasing trends in both electrical conductivity measurements and potassium levels over time suggests a progressive depletion of soil minerals and ions. This decline in essential nutrients has significant implications for plant health and yield. The diminishing availability of key minerals and ions in the soil could adversely affect plant growth, development, and overall productivity. As nutrient reserves become depleted, plants may experience nutrient deficiencies, leading to stunted growth, reduced vigor, and diminished yield potential. Moreover, the prolonged depletion of soil minerals and ions may exacerbate soil degradation, compromising its fertility and resilience. These findings underscore the importance of proactive soil management strategies, such as nutrient replenishment and conservation practices, to mitigate nutrient depletion and sustain optimal soil health for agricultural productivity. Further investigations into the dynamics of soil nutrient depletion and its impact on plant performance are warranted to inform targeted interventions and enhance agricultural sustainability.

4.2. PCA Biplot Reveals Dependencies in Data

The PCA biplot analysis unveiled distinct patterns among the various parameters under investigation. Notably, the pH value exhibited a unique and independent pattern, primarily associated with PC2. This separation from the other parameters suggests that pH variations are driven by different underlying factors compared to the remaining parameters. In contrast, the parameters including moisture, nitrogen, potassium, phosphorus, temperature, and electrical conductivity displayed similar variation patterns in the data, predominantly characterized by PC1, Figure 7.

Furthermore, these parameters clustered together and exhibited coherent directional trends, indicating shared variance and potential interdependencies. This collective behavior suggests that changes in one of these parameters may be indicative of concurrent changes in others, reflecting underlying relationships and interconnectedness within the soil ecosystem. Overall, the PCA biplot analysis provides valuable insights into the multivariate structure of the dataset, highlighting the distinctiveness of pH dynamics and the cohesive behavior of other key parameters, thereby enhancing our understanding of soil variability and dynamics.

5. Conclusions

This paper explored collaborative smart agriculture within the context of technology and community partnerships. The main objective was to develop data-driven models that enhance crop yields while promoting sustainable farming practices. By collaborating with small family farms in Zagreb, Croatia, and academic institutions, we demonstrated the pragmatic benefits of smart farming using IoT to overcome their challenges.

The findings from our analysis hold promising implications for agricultural practices aimed at optimizing plant yield and enhancing water use efficiency for sustainability. By leveraging the insights gained from the PCA biplot analysis, we can develop predictive models to forecast plant yield based on soil parameters and tailor watering intervals to maximize productivity while minimizing water usage. This approach offers a data-driven strategy for precision agriculture, enabling farmers to make informed decisions regarding irrigation scheduling and resource allocation, thereby promoting sustainable water management practices. Furthermore, our study lays the groundwork for future research endeavors aimed at expanding the scope and applicability of sensor-based monitoring systems in agricultural settings.

In addition to implementing technology, we succeeded in raising awareness about the benefits of digital transformation and smart farming for the development of entrepreneurial skills. As a result, other small family farm owners are now willing to collaborate and participate in our future research. The goal is to educate them on how to use technology for building a sustainable agricultural supply chain.

Scaling up this experiment to encompass multiple plant species and diverse environmental conditions will enable a comprehensive assessment of how sensor data correlates with variations in yield quality and quantity across different agricultural contexts. By exploring the intricate relationships between soil parameters, plant performance, and environmental factors, we can gain deeper insights into the underlying mechanisms driving crop productivity and resilience. Moreover, by examining the impact of sensor data on yield outcomes, we can identify opportunities for fine-tuning agricultural management practices to optimize yield quality and quantity while minimizing resource inputs. This iterative approach holds promise for enhancing agricultural sustainability by promoting efficient resource utilization, reducing environmental impacts, and enhancing overall crop resilience in the face of changing climatic conditions and evolving agricultural landscapes.

In conclusion, our study underscores the potential of sensor-based monitoring systems coupled with advanced data analytics techniques to revolutionize agricultural practices and contribute to the realization of sustainable food production systems. Through continued interdisciplinary research efforts and collaborative partnerships, the significant potential for impact and scalability was highlighted. By embracing smart farming, we can create a more resilient and productive agricultural supply chain—one that benefits both farmers and the environment.