ARTICLE | doi:10.20944/preprints202304.0926.v1
Subject: Environmental And Earth Sciences, Remote Sensing Keywords: artificial intelligence; image analysis; visual language model
Online: 25 April 2023 (10:47:52 CEST)
Recent advancements in Natural Language Processing (NLP), particularly in Large Language Models (LLMs), associated with deep learning-based computer vision techniques, have shown substantial potential for automating a variety of tasks. One notable model is Visual ChatGPT, which combines ChatGPT’s LLM capabilities with visual computation to enable effective image analysis. The model’s ability to process images based on textual inputs can revolutionize diverse fields. However, its application in the remote sensing domain remains unexplored. This is the first paper to examine the potential of Visual ChatGPT, a cutting-edge LLM founded on the GPT architecture, to tackle the aspects of image processing related to the remote sensing domain. Among its current capabilities, Visual ChatGPT can generate textual descriptions of images, perform canny edge and straight line detection, and conduct image segmentation. These offer valuable insights into image content and facilitate the interpretation and extraction of information. By exploring the applicability of these techniques within publicly available datasets of satellite images, we demonstrate the current model’s limitations in dealing with remote sensing images, highlighting its challenges and future prospects. Although still in early development, we believe that the combination of LLMs and visual models holds a significant potential to transform remote sensing image processing, creating accessible and practical application opportunities in the field.
ARTICLE | doi:10.20944/preprints202103.0220.v1
Subject: Environmental And Earth Sciences, Atmospheric Science And Meteorology Keywords: Convolutional Neural Network; Deep Learning; Environmental Monitoring
Online: 8 March 2021 (13:37:58 CET)
Accurately mapping individual tree species in densely forested environments is crucial to forest inventory. When considering only RGB images, this is a challenging task for many automatic photogrammetry processes. The main reason for that is the spectral similarity between species in RGB scenes, which can be a hindrance for most automatic methods. State-of-the-art deep learning methods could be capable of identifying tree species with an attractive cost, accuracy, and computational load in RGB images. This paper presents a deep learning-based approach to detect an important multi-use species of palm trees (Mauritia flexuosa; i.e., Buriti) on aerial RGB imagery. In South-America, this palm tree is essential for many indigenous and local communities because of its characteristics. The species is also a valuable indicator of water resources, which comes as a benefit for mapping its location. The method is based on a Convolutional Neural Network (CNN) to identify and geolocate singular tree species in a high-complexity forest environment, and considers the likelihood of every pixel in the image to be recognized as a possible tree by implementing a confidence map feature extraction. This study compares the performance of the proposed method against state-of-the-art object detection networks. For this, images from a dataset composed of 1,394 airborne scenes, where 5,334 palm-trees were manually labeled, were used. The results returned a mean absolute error (MAE) of 0.75 trees and an F1-measure of 86.9%. These results are better than both Faster R-CNN and RetinaNet considering equal experiment conditions. The proposed network provided fast solutions to detect the palm trees, with a delivered image detection of 0.073 seconds and a standard deviation of 0.002 using the GPU. In conclusion, the method presented is efficient to deal with a high-density forest scenario and can accurately map the location of single species like the M flexuosa palm tree and may be useful for future frameworks.
ARTICLE | doi:10.20944/preprints202102.0516.v1
Subject: Environmental And Earth Sciences, Remote Sensing Keywords: machine learning; insect-damage; spectral data; theoretical model
Online: 23 February 2021 (14:12:28 CET)
In cotton cultivars, an insect that causes irreversible damage is the Spodoptera frugiperda, known as the fall armyworm. Since the visual detection of plants is a burdensome task for human inspection, the spectral information related to plant damage, registered on a spectral scale, can be useful. These measurements, associated with machine learning techniques, produce useful information for a rapid and non-invasive inspection method development. To contribute to this gap fulfillment, this paper proposes a machine learning framework to model the spectral response of cotton plants under the attack of the fall armyworm. Additionally, a theoretical model is presented, built from the results of the machine learning analysis, to infer this damage with up-to-date orbital sensors. The data was composed of the reflectance measurements collected at a cotton field with control plants and plants submitted to Spodoptera frugiperda damage. Their spectral response was recorded with a hand-held spectroradiometer ranging from 350 to 2,500 nm, for eight consecutive days. Different machine learning models were evaluated and the overall best model was defined by accuracies comparisons on a testing-set. A ranking approach was adopted based on the model accuracy, returning the most contributive wavelengths for the classification. Sequentially, an unsupervised neural network (Self-Organizing Map - SOM) was implemented to reduce data-dimensionality and assist in the definition of important spectral regions. The regions were associated with the spectral bands of the two sensors (OLI and MSI) and a theoretical model using a band simulation process with the overall best machine learning model was proposed. The results indicated that the Random Forest (RF) algorithm is the most suitable to predict cotton-plants damaged by insects and that the last day of analysis (8th day) was better to separate it, with F-measure equals 0.912. The ranking approach combined with the SOM method indicated the spectral regions at the red to near-infrared (650 to 1,350 nm) and shortwave infrared (1,570 to 1,640 nm) as the most important regions to the analysis. The proposed theoretical model simulated with the OLI and MSI sensor-bands returned an F-Measure of 0.865 and 0.886, respectively. In conclusion, this framework can be used to map cotton-plants under insect-attack. The theoretical model presents high accuracy to infer the insect-damaged on cotton plants based on multispectral bands from other sensors, being a useful tool for future research that intends to evaluate it in other areas and at different field scales.
ARTICLE | doi:10.20944/preprints202102.0498.v1
Subject: Environmental And Earth Sciences, Atmospheric Science And Meteorology Keywords: proximal hyperspectral sensing; precision agriculture; random forest
Online: 22 February 2021 (17:20:41 CET)
A strategy to reduce qualitative and quantitative losses in crop-yields refers to early and accurate detection of insect-damage caused in plants. Remote sensing systems like hyperspectral proximal sensors are a promising strategy for managing crops. In this aspect, machine learning predictions associated with clustering techniques may be an interesting approach mainly because of its robustness to evaluate high dimensional data. In this paper, we model the spectral response of insect-herbivory-damage in maize plants and propose an approach based on machine learning and a clustering method to predict whether the plant is herbivore-attacked or not using leaf reflectance measurements. We differentiate insect-type damage based on the spectral response and indicate the most contributive wavelengths to perform it. For this, we used a maize experiment in semi-field conditions. The maize plants were submitted to three different treatments: control (health plants); plants submitted to Spodoptera frugiperda herbivory-damage, and; plants submitted to Dichelops melacanthus herbivory-damage. The leaf spectral response of all plants (controlled and submitted to herbivory) was measured with a FieldSpec 3.0 Spectroradiometer from 350 to 2500 nm for eight consecutive days. We evaluated the performance of different learners like random forest (RF), support vector machine (SVM), extreme gradient boost (XGB), neural networks (MLP), and measured the impact of a day-by-day analysis into the prediction. We proposed a novel framework with a ranking strategy, based on the accuracy returned by predictions, and a clusterization method based on a self-organizing map (SOM) to identify important regions in the reflectance measurement. Our results indicated that the RF-based framework algorithm is the overall best learner to deal with this type of data. After the 5th day of analysis, the accuracy of the algorithm improved substantially. It separated the three treatments into different groups with an F-measure equal to 0.967, 0.917, and 0.881, respectively. We also verified that the most contributive spectral regions are situated in the near-infrared domain. We conclude that the proposed approach with machine learning methods is adequate to monitor herbivory-damage of S. frugiperda and stink bugs like Dichelops melacanthus in maize, differentiating the types of insect-attack early on. We also demonstrate that the framework proposed for the analysis of the most contributive wavelengths is suitable to highlight spectral regions of interest.