We introduce a novel microscopic image dataset augmented with segmentation and detection labels specifically designed for microplastic analysis in sewage environments. Recognizing the increasing concern over microplastics — particles of synthetic polymers smaller than 5mm — and their detrimental effects on marine ecosystems and human health, our research focuses on enhancing detection and analytical methodologies through advanced computer vision and deep learning techniques. The dataset comprises high-resolution microscopic images of microplastics collected from sewage, meticulously labeled for both segmentation and detection tasks, aiming to facilitate accurate and efficient identification and quantification of microplastic pollution. In addition to dataset development, we present example deep learning models optimized for segmentation and detection of microplastics within complex sewage samples. The models demonstrate significant potential in automating the analysis of microplastic contamination, offering a scalable solution to environmental monitoring challenges. Furthermore, we ensure the accessibility and reproducibility of our research by making the dataset and model codes publicly available, accompanied by detailed documentation on GitHub and LabelBox. The dataset and example deep learning models are publicly available at the following GitHub link. https://anonymous.4open.science/r/Microplastics-in-Sewage-1BEF

Background: This study addresses the burgeoning impact of Large Language Models (LLMs) such as ChatGPT on the integrity of online examinations and seeks to demonstrate latent and advanced LLM reasoning capabilities. The research devises an iterative self-reflective strategy for invoking critical thinking and higher-order reasoning in LLMs based on multimodal exam questions using visuals and text. Methods: The proposed strategy was demonstrated and evaluated on real exam questions by subject experts and the performance of ChatGPT (GPT-4) with vision was estimated on an additional dataset of 600 text descriptions of multimodal exam questions, focusing on its response to complex, visual, and textual prompts. Results: The results indicate that latent multi-hop reasoning capabilities can be invoked using the proposed self-reflective strategy, which can effectively steer ChatGPT towards correct answers by integrating critical thinking from each modality into the final response. Meanwhile, ChatGPT demonstrated considerable proficiency in being able to answer multimodal exam questions across 12 subjects. Conclusions: This work challenges findings from recent studies which claim that existing LLMs are unable to reason on complex multimodal tasks, and therefore this study underscores the need for enhanced online exam security measures like proctoring software, that can mitigate the potential misuse of AI technologies in educational settings.

A surprising insight from the advent of genomic sequencing was that many genes are deeply conserved during evolution. With a particular focus on genes that interact with light in animals, I explore the metaphor of genetic toolkits, which can be operationalized as lists of genes involved in a trait of interest. A fascinating observation is that genes of a toolkit are often used again and again during convergent evolution, sometimes across vast phylogenetic distances. Such a pattern in the evolution of toolkits requires three different stages: origin, maintenance, and redeployment of the genes. The functional origins of toolkit genes might often be rooted in interactions with external environments. The origins of light interacting genes in particular may be tied to ancient responses to photo-oxidative stress, inspiring questions about the extent to which the evolution of other toolkits were also impacted by stress. Maintenance of genetic toolkits over long evolutionary timescales requires gene multifunctionality to prevent gene loss when a trait of interest is absent. Finally, the deployment of toolkit genes in convergently evolved traits like eyes sometimes involves the repeated use of similar, ancient genes but other times involves different genes specific to each convergent origin. How often a particular gene family is used time and again for the same function may depend on how many possible biological solutions are available. When few solutions exist and are maintained, evolution is constrained to use the same genes over and over. However, when many different solutions are possible, the innovative possibilities of evolution are often on display. Therefore, a focus on genetic toolkits highlights the combination of legacy plus innovation that drives the evolution of biological diversity.

Vertical Indoor Farming (VIF) with hydroponics offers a promising perspective for sustainable food production. Intelligent control of VIF system components plays a key role in reducing operating costs and increasing crop yields. Modern machine vision (MV) systems use Deep Learning (DL) in combination with camera systems for various tasks in agriculture, such as disease and nutrient deficiency detection, and flower and fruit identification and classification for pollination and harvesting. This study presents the applicability of MV technology with DL modelling to detect the growth stages of chilli plants using YOLOv8 networks. The influence of different bird’s eye and side view datasets and different YOLOv8 architectures was analysed. To generate the image data for training and testing the YOLO models, chilli plants were grown in a hydroponic environment and imaged throughout their life cycle using four camera systems. The growth stages were divided into growing, flowering and fruiting classes. All trained YOLOv8 models showed reliable identification of growth stages with high accuracy. The results indicate that models trained with data from both views show better generalisation. YOLO’s middle architecture achieved the best performance.

Microscopic vision plays an important role in automated micro-assembly. However, some uncertain factors in the assembly process, such as occlusion and stains can lead to the mistakes of feature extraction. Herein, to solve the problem, the deep learning techniques are introduced into the feature recognition tasks, focusing on the attention mechanism and visualizing CNNs for DL-based microscopic vision. The main contributions are summarized as follows: The CBAM attention mechanism is combined with the YOLOv5 algorithm to improve the accuracy and robustness of feature extraction. The micropart feature occlusion experiment results show that at 70% occlusion degree, YOLOV5-CBAM can reach 97.9% mAP@0.5, which is 4.6% higher than the original one. Visualization analysis of DL-based model is conducted using Grad-CAM to make the decision result more transparent and avoid potential visual detection risks during assembly. The heatmap matching degree between GT area and high-light area is increased by 27.81% on average, which further verify the effectiveness of attention mechanism in micropart feature localization. Additionally, micropart surface stain and droplet quality classification models based on ResNet50 are trained to replace the manual sorting. The visual results are consistent with human eye discernment and judgement, confirming the reliability of parts and droplets sorting.

All types of sports are potential application scenarios for automatic and real-time visual object and event detection. In rink hockey, the popular roller quad skate variant of hockey team sports, it is of great interest to automatically track player’s movements and positions, player’s sticks and, also, making other judgments, such as being able to locate the ball. In this work, we introduce a real-time pipeline composed by an object detection model, created specifically for rink hockey games, followed by a knowledge-based event detection module. Even in the presence of occlusions and quick motions, our deep learning object detection model effectively identifies and tracks, in real-time, important visual elements such as: ball; players; sticks; referees; crowd; goalkeeper; and goal. Using a curated dataset composed by a collection of videos of rink hockey, comprising 2525 annotated frames, we trained and evaluated the algorithm performance and compare it to state of the art object detection techniques. Our object detection model, based on YOLOv7, presents a global accuracy of 80%, and presents a good performance in terms of accuracy and speed, according to our results, making it a good choice for rink hockey applications. In our initial tests, the event detection module successfully detected one important event type in rink hockey games, the occurrence of penalties.

The state-of-the-art smart city has been calling for an economic but efficient energy management over large-scale network, especially for the electric power system. It is a critical issue to monitor, analyze and control electric loads of all users in system. In this paper, we employ the popular computer vision techniques of AI to design a non-intrusive load monitoring method for smart electric energy management. First of all, we utilize both signal transforms (including wavelet transform and discrete Fourier transform) and Gramian Angular Field (GAF) methods to map one-dimensional current signals onto two-dimensional color feature images. Second, we propose to recognize all electric loads from color feature images using a deep neural network with multi-scale feature extraction and attention mechanism. Third, we design our method as a cloud-based, non-intrusive monitoring of all users, thereby saving energy cost during electric power system control. Experimental results on both public and our private datasets have demonstrated our method achieves superior performances than its peers, and thus supports efficient energy management over large-scale Internet of Things.

The Robotics Vision Lab of Northwest Nazarene University has developed Orchard Robot (OrBot), which was designed for harvesting fruits. OrBot is composed of a machine vision system to locate fruits on the tree, a robotic manipulator to approach the target fruit, and a gripper to remove the target fruit. Field trials conducted at commercial orchards for apples and peaches during the harvesting season of 2021 yielded a harvesting success rate of about 85% and had an average harvesting cycle time of 12 seconds. Building upon this success, the goal of this study is to evaluate the performance of OrBot during nighttime harvesting. The idea is to have OrBot harvest at night, and then human pickers continue the harvesting operation during the day. This human and robot collaboration will leverage the labor shortage issue with a relatively slower robot working at night. The specific objectives are to determine the artificial lighting parameters suitable for nighttime harvesting and to evaluate the harvesting viability of OrBot during the night. LED lighting was selected as the source for artificial illumination with a color temperature of 5600 K and 10% intensity. This combination resulted with the lowest noise in the images. OrBot was tested in a commercial orchard using twenty Pink Lady apple trees. Results showed an increased success rate during the night, with OrBot gaining 94%. compared to 88% during the daytime operations.

This research introduces a pioneering solution to the challenges posed by gastrointestinal tract (GI) cancer in radiation therapy, focusing on the imperative task of precise organ segmentation for minimizing radiation-induced damage. GI imaging has historically used manual demarcation, which is laborious and uncomfortable for patients. We address this by introducing the ResECA-U-Net deep learning model, a novel combination of the UNet and ResNet34 architectures. Furthermore, we further augment its functionality by incorporating the Efficient Channel Attention (ECA-Net) methodology. By utilizing data from the UW-Madison Carbone Cancer Centre, we carefully investigate several image processing techniques designed to capture critical local characteristics. With its foundation in computer vision concepts, the ResECA-U-Net model is excellent at extracting fine details from GI images. Sophisticated metrics such as intersection over union (IoU) and the dice coefficient are used to evaluate performance. Our study's outcomes demonstrate the effectiveness of the suggested method, yielding an impressive 96.27% Dice coefficient and 91.48% IoU. These results highlight the significant contribution that our strategy has made to the advancement of cancer therapy. Beyond its scientific merits, this work has the potential to significantly enhance cancer patients' quality of life and provide better long-term outcomes. Our work is a significant step towards automating and optimizing the segmentation process, which can potentially change how GI cancer is treated completely.

Deep learning is widely used in image segmentation, effectively identifying crops and weeds and contributing to the reduction of herbicide use. Traditional methods of applying pesticides over large areas are inefficient in dealing with weeds as needed, leading to massive waste of pesticides, high-cost production, and serious environmental pollution. This affects crop yield and quality. Although weed recognition using conventional deep learning methods has evolved over time, there are still challenges in weed extraction, detection, and segmentation. Accurate recognition and detection of weeds are essential prerequisites for implementing variable spraying. This paper proposes a semantic segmentation method based on UNet++ for complex environments where accurate identification of plants and weeds is challenging, targeting weeds in sugar beets, peas, and rice. An attention module is integrated into the upsampling process of UNet++, and UNet++ is used as a backbone network to effectively integrate multi-scale information, efficiently suppressing external noise interference. The UNet++ model with an integrated attention mechanism module achieves higher IOU than the general UNet++ model used in medical image analysis. This method effectively detects crops and weeds in complex backgrounds, providing reference material for the accurate application of robotic herbicides.

Vision impairment may delay responses to hazards when driving. In a proof-of-concept driving simulator study, we evaluated a hazard warning device designed for vision impaired drivers. Three groups participated: 11 persons with central vision loss (CVL; median age 60 years), 12 with homonymous field loss (HFL; 52 years) and 11 with normal vision (NV; 60 years). CVL is characterised by impaired visual acuity and contrast sensitivity while HFL is a loss of vision in the same half of the visual field (blind side) in both eyes. Participants completed drives with and without the device which gave a directional vibro-tactile warning for an approaching pedestrian hazard when collision risk exceeded a pre-defined threshold. Warnings significantly (p< 0.001) reduced brake response times for CVL (∆= 0.54s) and HFL (∆= 0.30s), but not NV participants. For HFL participants, the reduction was greater (p = 0.02) for hazards on the blind (∆= 0.57s) than seeing side (∆= 0.10s). The warnings significantly reduced collision rates for CVL participants from 11 to 0% (all hazards) and for HFL participants from 30 to 1.7% (blind side hazards). Thus, hazard warnings may be beneficial in improving safety of vision impaired drivers; further research is warranted.

The Eurasian grapevine (\textit{Vitis vinifera L.}) is the most widely grown horticultural crop in the world and is important for the economy of many countries. In the wine production chain, grape varieties play an important role, as they directly influence the authenticity and classification of the product. Identifying the different grape varieties is therefore fundamental for quality control and control activities, as well as for regulating production. Currently, ampelography and molecular analysis are the main approaches to identifying grape varieties. However, both methods have limitations. Ampelography is subjective and prone to errors and is experiencing enormous difficulties as ampelographers are increasingly scarce. On the other hand, molecular analyses are very demanding in terms of cost and time. In this scenario, Deep Learning (DL) methods have emerged as a classification alternative to deal with the scarcity of ampelographers and avoid molecular analyses. In this study, the most recent and current methods for identifying grapevine varieties using DL classification-based approaches are presented through a systematic literature review. The steps of the standard DL-based classification pipeline were described for the 18 most relevant studies found in the literature, highlighting their pros and cons. Potential directions for improving this field of research were also presented.

The paper delineates the developed vision system of the UR5 cobot and the operating algorithm of the robotic quality control station. The hardware-software architecture of the developed robotic station was shown, consisting of a UR5 cobot equipped with a web camera and a stationary industrial camera with a lighting system. Image processing and analysis algorithms were described, the method of control and communication between the station components was discussed, and two operating scenarios were presented as a single robotic station and a robotic line. Based on the results obtained, the level of measurement noise, accuracy, and repeatability of the developed vision system were estimated.

The ever-evolving ways attacker continues to im prove their phishing techniques to bypass existing state-of-the-art phishing detection methods pose a mountain of challenges to researchers in both industry and academia research due to the inability of current approaches to detect complex phishing attack. Thus, current anti-phishing methods remain vulnerable to complex phishing because of the increasingly sophistication tactics adopted by attacker coupled with the rate at which new tactics are being developed to evade detection. In this research, we proposed an adaptable framework that combines Deep learning and Randon Forest to read images, synthesize speech from deep-fake videos, and natural language processing at various predictions layered to significantly increase the performance of machine learning models for phishing attack detection.

A human eye has about 120 million rod cells and 6 million cone cells. This huge number of light sensing cells inside a human eye will continuously produce a huge quantity of visual signals which flow into a human brain for daily processing. However, the real-time processing of these visual signals does not cause any fatigue to a human brain. This fact tells us the truth which is to say that human-like vision processes do not rely on complicated formulas to compute depth, displacement, and colors, etc. On the other hand, a human eye is like a PTZ camera. Here, PTZ stands for pan, tilt and zoom. We all know that in computer vision, each set of PTZ parameters (i.e., coefficients of pan, tilt and zoom) requires a dedicated calibration to determine a camera’s projection matrix. Since there is an infinite number of PTZ parameters which could be produced by a human eye, it is unlikely that a human brain stores an infinite number of calibration matrices for each human eye. Therefore, it is an interesting question for us to answer, which is to say whether simpler formulas of computing depth and displacement exist or not. Moreover, these formulas must be calibration friendly (i.e., easy process on the fly or on the go). In this paper, we disclose an important discovery of a new solution to 3D projection in a human-like binocular vision system. The purpose of doing 3D projection in binocular vision is to undertake forward and inverse transformations (or mappings) between coordinates in 2D digital images and coordinates in a 3D analogue scene. The formulas underlying the new solution are accurate, easily computable, easily tunable (i.e., to be calibrated on the fly or on the go) and could be easily implemented by a neural system (i.e., a network of neurons). Experimental results have validated the discovered formulas.

The rise in both the frequency of natural disasters and the ubiquity of artificial intelligence has led to novel applications of new technologies in improving disaster response processes, such as the labor-intensive assessment of disaster damages. Assessment of residential and commercial structure damages, a precursory step to government agencies being able to provide most of their financial assistance, has benefited from aerial and satellite imagery-based computer vision models; however, limitations of using such imagery include the ground structures being obscured by clouds or smoke, as well as the lack of resolution to distinguish individual structures from others. Using a different data source, we propose a damage severity classification model using ground-level imagery, focusing on residential structures damaged by wildfires. This classifier, a Vision Transformer (ViT) model trained on over 18,000 professionally labeled images of damaged homes from the 2020-2022 California wildfires, has achieved an accuracy score of over 95%. Further, we have open sourced the training dataset–the first of its kind and scale–as well as built a publicly available web application prototype, which we demoed to and received feedback from disaster response officials, both of which further contribute to the broader literature beyond the proposed model.

Robotic odor source localization (OSL) is a technology that enables mobile robots or autonomous vehicles to find an odor source in unknown environments. An effective navigation algorithm that guides the robot to approach the odor source is the key to successfully locating the odor source. While traditional OSL approaches primarily utilize an Olfaction-only strategy, guiding robots to find the odor source by tracing emitted odor plumes, our work introduces a fusion navigation algorithm that combines both vision and olfaction-based techniques. This hybrid approach addresses challenges such as turbulent airflow, which disrupts olfaction sensing, and physical obstacles inside the search area, which may impede vision detection. In this work, we propose a hierarchical control mechanism that dynamically shifts the robot’s search behavior among four strategies: crosswind maneuver, Obstacle-avoid Navigation, Vision-based Navigation, and Olfaction-based Navigation. Our methodology includes a custom-trained deep-learning model for visual target detection and a moth-inspired algorithm for Olfaction-based navigation. To assess the effectiveness of our approach, we implemented the proposed algorithm on a mobile robot in a search environment with obstacles. Experimental results demonstrate that our Vision and Olfaction Fusion algorithm significantly outperforms Vision-only and Olfaction-only methods, reducing average search time by 54% and 30%, respectively.

Efficient processing of end-of-life lithium-ion batteries of electric vehicles is important and a pressing challenge for a circular economy. Regardless of whether the processing strategy is recycling, repurposing or remanufacturing, the first processing step would usually involve disassembly. As battery disassembly is a dangerous task, efforts have been made to robotise it. In this paper, a robotic disassembly platform using four industrial robots is proposed to automate the non-destructive disassembly of a plug-in hybrid electric vehicle battery pack into modules. The work was conducted as a case study to demonstrate the concept of autonomous disassembly of an electric vehicle battery pack. A two-step object localisation method based on visual information is used to overcome positional uncertainties from different sources and is validated by experiments. Also, the unscrewing system is highlighted, and its functions, such as handling untightened fasteners, loosening jammed screws, and changing the nutrunner adapters with square drives, are detailed. Furthermore, the time required for each operation is compared with that taken by human operators. Finally, the limitations of the platform are reported and future research directions are suggested.

Centimeter-level localization and rotation angle estimation of flatbed trucks are major problems in unmanned forklift automated loading scenes. To solve it, this study proposes a method for high-precision positioning and rotation angle estimation of flatbed trucks based on BeiDou and vision. First, an unmanned forklift equipped with a ToF camera and double antenna mobile receiver for BeiDou positioning collects depth images and localization data near the front and rear endpoints of the flatbed. Then DDRNet-23-slim was used to segment the flatbed from the depth image and extract the straight lines at the edges of the flatbed using the Hough transform and compute the set of intersection points of the lines. A neighborhood feature vector was designed to locate the endpoint of a flatbed from a set of intersection points by feature screening. Finally, the relative coordinates of the endpoints were converted to a customized forklift navigation coordinate system by BeiDou positioning, and rotation angle estimation was performed by the front and rear endpoints. Experiments showed that the endpoint positioning error was less than 3 cm and the rotation angle estimation error was less than 0.1°. The precision of the method meets the demand for automated flatbed truck loading by unmanned forklifts.

In the age of social media, images come from unreliable sources that may change their content to fit a narrative. Automated detection of forged images is a complex task, especially considering recent technological advances in image manipulation software. However, in the existing literature, it is possible to identify two general approaches adopted by detection methods, active and passive. Active techniques preemptively act on an image by inserting structures before any manipulation is made that can verify its authenticity. In contrast, passive methods analyze the content of an image in search of traces of manipulation. In this way, this research proposes a novel approach to image manipulation detection, combining two passive methods through neural networks, creating a generalist approach capable of detecting with greater accuracy than the methods that compose it. Furthermore, this work used a combination of four datasets available in the literature for training and evaluation. After training, the merged approach obtained an accuracy of 89.59% in the set of validation images, significantly higher than the model trained with only unaltered images, which obtained 78.64%, and the two other models trained using images with a feature selection method applied to enhance inconsistencies that obtained 68.02% for Error-Level Analysis images and 50.70% for the method using Discrete Wavelet Transform. In addition to the performance improvement, the proposed approach’s accuracy variation was lower than the other models.

In recent times, using image-based deep learning techniques has been very useful for classifying fruits. However, mango is a particular challenge because its shelf life is directly related to the degree of maturity at the time of harvest, which is determined mainly through the visual assessment of the operators when sorting them, doing it incorrectly, coupled with the blows and compressions exposed during storage and transport, trigger the deterioration in their quality, hence the importance of this operation. In this sense, the purpose of this study focuses on designing a computational model based on Deep Learning that uses a convolutional neural network (CNN) parameterized to select mangoes according to their color tone based on their maturity stage. A dataset was established from 201 labeled images, then a CNN architecture was built, obtaining a model on which 3 criteria were evaluated: the number of convolution and grouping layers, the resolution of the input images and the number of training epochs required. As a result, an input image resolution of 32 x 32 pixels was achieved, the process included a convolution layer followed by pooling that fed another layer that processed the results and ended in an output layer with 2 neurons under the One Hot Encoding representation. The model required 20 epochs to perfect its training, achieving 96.04 % accuracy in mango image selection. This study supports the potential of CNNs in quality analysis through mango color selection, even in uncertainty scenarios.

The article presents the operation of the automatic pine sawn timber inspection system, which was developed at the Woodinspector company and offered commercially. The vision inspection system is used to detect various wood defects, including knots, blue stain, and mechanical damage caused by worms. Blue stain is a defect that is difficult to detect based on the color of the wood, because it can be easily confused with wood defects or dirt that do not impair its strength properties. In particular, the issues of detecting blue stain in wood, the use of artificial neural networks and improving the operation of the system in production conditions were discussed.

Amidst the burgeoning global concerns surrounding skin cancer, the quest for precise and efficient diagnostic methodologies has intensified. Conventional diagnostic approaches relying on subjective visual inspection by dermatologists often grapple with subjectivity and resource constraints. This predicament underscores the critical role of Artificial Intelligence (AI) in revolutionizing diagnostic paradigms by significantly enhancing accuracy and accessibility. This study embarks on an extensive exploration of skin cancer classification, specifically targeting the intricate task of 8-class skin cancer classification. Employing cutting-edge deep learning models on the ISIC2019 dataset, a comprehensive analysis was conducted utilizing a diverse array of pre-trained ImageNet architectures and Vision Transformer models. To counteract the inherent class imbalance in skin cancer datasets, a pioneering "Naturalize" augmentation technique was introduced, leading to the creation of indispensable tools—the Naturalized 2.4K ISIC2019 and groundbreaking Naturalized 7.2K ISIC2019 datasets—catalyzing advancements in classification accuracy. The pivotal role of AI in mitigating the risks of misdiagnosis and under-diagnosis in skin cancer becomes evident through this research. AI’s proficiency in analyzing vast datasets and discerning subtle patterns augments the diagnostic prowess of dermatologists significantly. Integration of AI-powered systems in skin cancer diagnosis holds immense promise in facilitating early detection and substantially improving patient outcomes. The meticulous evaluation employed quantitative measures such as confusion matrices, classification reports, and visual analyses using Score-CAM across diverse dataset variations. Astonishingly, the culmination of these endeavors culminated in an unprecedented achievement—100% average accuracy, precision, recall, and F1-Score—within the groundbreaking Naturalized 7.2K ISIC2019 dataset. This remarkable feat underscores the transformative potential of AI-driven methodologies, exemplifying their prowess in redefining the landscape of skin cancer diagnosis and management. In summary, this research represents a pivotal stride towards redefining dermatological diagnosis, showcasing the remarkable impact of AI-powered solutions in surmounting the challenges inherent in skin cancer diagnosis. The attainment of 100% across crucial metrics within the Naturalized 7.2K ISIC2019 dataset serves as a testament to the transformative capabilities of AI-driven approaches in reshaping the trajectory of skin cancer diagnosis and patient care.

Weeds are unwanted and invasive plants that proliferate and compete for resources such as space, water, nutrients, and sunlight, affecting the quality and productivity of the desired crops. Weed detection is crucial for the application of precision agriculture methods and for this purpose machine learning techniques can be used, specifically convolutional neural networks (CNN). This study focuses on the search for CNN architectures and technology used to detect and identify weeds in different crops; 61 articles applying CNN architectures and technology were analyzed in the last five years (2019-2023). The results show the used of different devices to acquire the image for training, such as digital cameras, smartphones, and drone cameras. Additionally, the YOLO family and algorithms are the most widely adopted architectures, followed by VGG, ResNet, Faster R-CNN, AlexNet, and MobileNet, respectively. This study provides an update on CNNs that will serve as a starting point for researchers wishing to implement these weed detection and identification techniques.

Reconstructing a 3D model of a scene from a set of multiple views poses a significant challenge in the field of computer vision. The advent of NeRF has marked a major breakthrough in this domain. However, there is a need to extend the capabilities of the NeRF model to enable editing tasks such as relighting and deformation, as well as to enhance its training efficiency. Neural reflectance fields introduce the concept of the reflectance equation into the NeRF framework to achieve relighting of the NeRF model. We leverage the TensoRF approach, which incorporates tensor decomposition and employs multiple tensors to store features, to expedite the training process of the NeRF model. Our novel method, called StensorR, combines the reflectance equation and tensor decomposition within the radiation field model framework. Differing from previous approaches, we employ a single tensor to store scene features and render the surface color of our scene using a simplified reflectance equation. This approach accelerates model convergence and enables relighting of the NeRF model. Experimental results demonstrate that our method achieves a 50% faster convergence rate compared to existing relighting radiation field models, while successfully enabling relighting and improving the quality of synthesized images from new viewpoints.

The intricate functionality of the vertebrate retina relies on the interplay between neurotransmitter activity and calcium dynamics, offering important insights into de-velopmental processes, physiological functioning, and disease progression. Neuro-transmitters orchestrate cellular processes to shape the behavior of the retina under diverse circumstances. Despite research to elucidate the roles of individual neuro-transmitters in the visual system, there remains a gap in our understanding of the holistic integration of their interplay with calcium dynamics in the broader context of neuronal development, health, and disease. To address this gap, the present review explores the mechanisms used by the neurotransmitters glutamate, GABA, glycine, dopamine, and acetylcholine and their interplay with calcium dynamics. This con-ceptual outline is intended to inform and guide future research, underpinning novel therapeutic avenues for retinal-associated disorders.

Leveraging the open-world understanding capacity of large-scale visual-language pre-trained models has become a hot-spot in point cloud classification. Recent approaches rely on transferable visual-language pre-trained models, classifying point clouds by projecting them into 2D images and evaluating consistency with textual prompts. These methods benefit from the robust open-world understanding capabilities of visual-language pre-trained models and require no additional training. However, they face several challenges summarized as prompt ambiguity, image domain gap, view weights confusion, and feature deviation. In response to these challenges, we propose PointBLIP, a zero-training point cloud classification network based on the recently introduced BLIP-2 visual-language model. PointBLIP is adept at processing similarities between multi-images and multi-prompts. We separately introduce a novel method for point cloud zero-shot and few-shot classification, which involves comparing multiple features to achieve effective classification. Simultaneously, we enhance the input data quality for both the image and text sides of PointBLIP. In point cloud zero-shot classification tasks, we outperform state-of-the-art methods on three benchmark datasets. For few-shot classification tasks, to the best of our knowledge, we present the first zero-training few-shot point cloud method, surpassing previous works under the same conditions and showcasing comparable performance to full-training methods.

Ultrasound (US) has become a widely used imaging modality in clinical practice, characterized by its rapidly evolving technology, advantages, and unique challenges such as low imaging quality and high variability. There is a critical need to develop advanced automatic US image analysis methods to enhance diagnostic accuracy and objectivity. Vision transformer, a recent innovation in machine learning, has demonstrated significant potential in various research fields, including general image analysis and computer vision, due to its capacity to process large datasets and learn complex patterns. Its suitability for automatic US image analysis tasks, such as classification, detection, and segmentation, has been recognized. This review provides an introduction to vision transformer and discusses its applications in specific US image analysis tasks, while also addressing the open challenges and potential future trends in its application in medical US image analysis. Vision transformer has shown promise in enhancing the accuracy and efficiency of ultrasound image analysis and is expected to play an increasingly important role in the diagnosis and treatment of medical conditions using ultrasound imaging as technology progresses.

This study advances the field of solar irradiance nowcasting by introducing a discrete-level classification approach, diverging from traditional continuous measurement methods. Grounded in a need for cost-effective and modular solutions, the research employs high-resolution computer vision coupled with a deep learning framework to predict Direct Normal Irradiance (DNI). By harnessing the capabilities of a Logitech C900 1080p HD camera and an NVIDIA Jetson module, the study achieves real-time data processing, pivotal for CSP systems' operational efficiency. The core of the methodology is the ResNet-50 convolutional neural network, refined via transfer learning on a bespoke dataset, culminating in a predictive accuracy of 85.78%. This discrete classification model contrasts with conventional, costly instruments like the MS-57, offering a novel and accessible alternative for DNI estimation. Such innovation not only demonstrates high predictive accuracy but also signifies a shift towards less resource-intensive and more adaptable solar energy forecasting tools, contributing a significant leap towards optimizing renewable energy systems.

Deep learning heavily relies on statistical correlations to drive artificial intelligence (AI) innovations, particularly in computer vision applications like autonomous driving and robotics. However, despite providing a solid foundation for deep learning, these statistical correlations can be vulnerable to unforeseen and uncontrolled factors. The lack of prior knowledge guidance can result in spurious correlations, introducing confounding factors and affecting the model's robustness. To address this challenge, recent research efforts have focused on integrating causal theory into deep learning methodologies. By modelling the inherent and unbiased causal structure, causal theory can potentially mitigate the impact of spurious correlations effectively. Hence, this paper explores the basics of causal methodologies in image classification.

Machine learning models facilitate the search for non-linear relationships when modeling hydrological processes, but they are equally effective for automation at the data preparation stage. The tasks whose automation was analyzed consisted in estimating changes in the roughness of a mountain streambed and the extent of floods from images. The Segment Anything Model (SAM) developed this year by Meta was used for this purpose. Images from many years from the Wielka Puszcza mountain stream located in the Polish Carpathians were used as the only input data. The model was not additionally trained for the described tasks. The model can be run in several modes, but the two most appropriate were used in this study. The first one is available in the form of a web application, while the second one is available in the form of a Jupyter notebook run in the Google Colab environment. Both methods do not require specialized knowledge and can be used by virtually any hydrologist. In the roughness estimation task, the average Intersection over Union (IoU) ranges from 0.55 for grass to 0.82 for shrubs/trees. Ultimately, it was possible to estimate the roughness coefficient of the mountain streambed between 0.027 and 0.059 based solely on image data. In the task of estimation of the flood extent, when selecting appropriate images, one can expect IoU at the level of at least 0.94, which seems to be an excellent result considering that SAM is a general-purpose segmentation model. It can therefore be concluded that SAM model can be a useful tool for a hydrologist.

This paper explores the application of machine learning techniques for automated object grasping. Current electronic prosthetics often require extensive training for users to gain fine motor control over the prosthetic fingers, hindering their usability and acceptance. To address this challenge, this paper proposes an automated method that leverages computer vision-based techniques and machine learning algorithms. In this study, three reinforcement learning algorithms, namely Soft Actor-Critic (SAC), Deep Q-Network (DQN), and Proximal Policy Optimization (PPO), are employed to train agents for automated grasping tasks. The results indicate that the SAC algorithm achieves the highest success rate of 99% among the three algorithms at just under 200,000 timesteps. This research also shows that object’s physical characteristics can affect the agent’s ability to learn an optimal policy. Moreover, the findings highlight the potential of the SAC algorithm in developing intelligent prosthetic hands with automatic object gripping capabilities.

Skin cancer is a global health concern with increasing prevalence, necessitating effective early detection and classification systems. Transfer learning has emerged as a powerful tool in this domain, enhancing diagnostic accuracy. Attention mechanisms, which selectively focus on pertinent image features, play a pivotal role in transfer learning. While previous studies have highlighted their effectiveness, their applicability across different models and datasets remains understudied. This empirical study explores the impact of various attention mechanisms on the performance of transfer learning models for skin cancer detection and classification. Using five transfer learning models (DenseNet121, InceptionV3, MobileNet, VGG16, and Xception) and six attention mechanisms (Channel Attention, Global Context Attention, Guided Attention, Nonlocal Attention, Positional Attention, and Spatial Attention), we conducted 105 experiments across three datasets. Traditional metrics (accuracy, precision, recall, and f1-score) were employed for empirical validation. The results reveal a nuanced relationship between attention mechanisms, transfer learning models, and datasets. Overall, attention mechanisms exhibit the potential to enhance skin cancer classification. Spatial and channel attention mechanisms consistently outperform others, offering simplicity and effectiveness. Model-specific selection of attention mechanisms is crucial, with a trade-off between model complexity and performance evident. This study provides insights into developing efficient skin cancer classification models utilizing attention mechanisms.

This paper introduces an innovative approach to lipreading, leveraging a web application designed to generate subtitles for videos where the speaker's mouth is visible and a comprehensive literature review that precedes the discussion, encompassing a thorough examination of various lipreading methods employed over the past decade. Our method employs a powerful deep learning model, featuring a 3D-convolution network and bidirectional LSTM, enabling accurate sentence-level predictions based solely on visual lip movements. With an impressive accuracy of 97%, our model is trained using pre-segmented lips regions, transformed into animated GIFs for effective pre-training. This work stands as a significant contribution to the evolving landscape of lipreading research, offering a practical and accurate solution for real-world applications.

Through the refinement of approaches to applying visual augmentation to virtual environments, the possibilities of utilising this process in robotics continues to be explored. However these endeavours have only yielded a collaborative relationship between the two technologies, where the augmented reality (AR) component is separate from the robot functionality. Regarding telepresence robots, research has shown that their perceived usefulness wanes in comparison to that of video conferencing tools as they do not offer a level of immersion and modified visual experience to justify their use. Therefore this paper contributes a novel virtually augmented robot (VAR) model that embeds AR functionality into telepresence, through the use of augmentation and facial recognition sub-models and algorithms. Experimental results show that using pulse width modulation (PWM), duty cycles, digital signals and accelerometer angles provides an approach for stabilising robot motion and the transference of human to robot head movement. The results also show the benefit of having a robot model with intrinsic AR functionality, as it offers a greater level of immersion and visual control of an environment.

In recent years, Synthetic Aperture Radar (SAR) has received attention in the Automatic Target Recognition (ATR) field due to its ability to acquire images both day and night, combined with the superior resolution offered by the latest sensors. However, despite advances, there remains a deficiency in dataset quality and quantity. This has necessitated the use of simulated data to overcome challenges associated with obtaining large quantities of high-quality images of targets and their subsequent labeling. In this study, we trained a Convolutional Neural Network (CNN) using a simulated dataset, subsequently employing it to classify real images from the MSTAR dataset. Traditional training methodologies often struggle with generalization when confronted with domain disparities between simulated and real data. To mitigate this, we exploited Margin Disparity Discrepancy (MDD), a domain adaptation technique not previously explored in SAR context. Findings from this study showcased a 13% increase in classification accuracy and improved generalization using MDD. Through Explainable AI (XAI) techniques such as t-SNE and other metrics like Kullback Leibler Divergence (KLD), we revealed a more distinct alignment of feature spaces between the two domains after domain adaptation, emphasizing MDD’s role in enhancing model accuracy and generalization.

This paper presents a methodology to detect the erosion and progradation effects in the Atrato river delta, located in the Gulf of Urabá in Colombia using SAR (Synthetic Aperture Radar) data. Other authors have studied this area by using optical images, encountering difficulties given the persistent presence of clouds in this region. SAR imagery is rescaled, despeckled, composed, classified, registered and processed to obtain a heatmap that shows the overall changes during eight years (2016–2023) in one single image. Since this technology uses microwave active signals, the presence of clouds and other environmental phenomena was not a problem, and all the images from the selected dates could be processed and analyzed. Also, detection of the dynamic processes was obtained for every period considered in this study with respect to the previous one in time, calculating areas and detecting the regions subject to the erosion and progradation processes. The multitemporal analysis performed found that the newly created mouth is the most active area for these processes, coinciding with other studies. A comparison of these findings with the Oceanic Niño Index (ONI) showed a relative delayed coupling to the erosion process and a coupling of progradation with dry and wet seasons.

Skin cancer is one of the widespread diseases that typically develop on the skin due to continuous exposure to sunlight. Although cancer can appear on any part of the human body, skin cancer reports account for over half of all cancer occurrences worldwide. There are substantial obstacles to the precise diagnosis and classification of skin lesions because of the morphological variety and indistinguishable characteristics across skin malignancies. Recently, Deep Learning models have been used in the field of image-based lesion diagnosis, and it has demonstrated diagnostic efficiency on par with that of dermatologists. To increase classification efficiency and accuracy for skin lesions, a cutting-edge multi-layer deep Convolutional Neural Network (CNN) termed SkinLesNet has been built in this study. The ResNetV50 and VGG16 models have been carefully compared to review the performance of the proposed model. The dataset used in this study, PAD-UFES-20, contains 1314 samples in total and includes three common forms of skin lesions. The proposed approach, SkinLesNet, significantly outperforms the well-known compared models in the given conditions

The twelve cranial nerves play a crucial role in nervous system, orchestrating a myriad of functions vital for our everyday life. These nerves are each specialized for particular tasks. Cranial Nerve I, known as the olfactory nerve, is responsible for our sense of smell, allowing us to perceive and distinguish various scents. Cranial Nerve II, or the optic nerve, is dedicated to vision, transmitting visual information from the eyes to the brain. Eye movements are governed by Cranial Nerves III, IV, and VI, ensuring our ability to track objects and focus. Cranial Nerve V controls facial sensation and jaw movement, while Cranial Nerve VII, the facial nerve, facilitates facial expressions and taste perception. Cranial Nerve VIII, or the vestibulocochlear nerve, plays a critical role in hearing and balance. Cranial Nerve IX, the glossopharyngeal nerve, affects throat sensation and taste perception. Cranial Nerve X, the vagus nerve, is a far-reaching nerve, influencing numerous internal organs such as the heart, lungs, and digestive system. Cranial Nerve XI, the accessory nerve, is responsible for neck muscle control, contributing to head movement. Finally, Cranial Nerve XII, the hypoglossal nerve, manages tongue movement, essential for speaking, swallowing, and breathing. Understanding these cranial nerves is fundamental in comprehending the intricate workings of our nervous system and the functions that sustain our daily lives.

The COVID-19 outbreak demonstrated the viability of various remote working solutions, telepresence robots (TPRs) being one of them. High-quality video transmission is one of the cornerstones of using such solutions, as most of the information about the environment is acquired through vision. This study aims to compare the camera capabilities of four models of popular telepresence robots using compact reduced LogMAR and Snellen optometry charts as well as text displayed on the projector screen. The symbols from the images are extracted using OCR (Optical Character Recognition) software and the results of the recognition are compared with the symbols on the charts. Double 3 TPR provides the best quality images of optometric charts, but the OCR results of measurements of the image on the projector do not show the clear advantage of one single model over the others. The results demonstrated by Temi 2 and Double 3 TPRs are generally better than the others, suggesting that these TPRs are more feasible to be used in teaching and learning scenarios.

In this paper, we redefine the boundaries of blood cell classification, expanding from 5 to 11 distinct blood cell types using the challenging 11-class PBC dataset. This shift enables a deeper understanding of blood cell diversity, overcoming previous limitations in medical image analysis. Our approach combines state-of-the-art deep learning techniques, including pre-trained ConvNets, ViTb16 models, and custom CNN architectures. We employ transfer learning, fine-tuning, and ensemble strategies, such as CBAM and Averaging ensembles, to achieve unprecedented accuracy and interpretability. Our fully fine-tuned EfficientNetV2 B0 model sets a new standard, with a macro-average precision, recall, and F1-score of 91%, 90%, and 90%, respectively, and an average accuracy of 93%. This breakthrough underscores the transformative potential of 11-class blood cell classification for more precise medical diagnoses. Moreover, our groundbreaking "Naturalize" augmentation technique produces remarkable results. The 2K-PBC dataset generated with "Naturalize" boasts a macro-average precision, recall, and F1-score of 97%, along with an average accuracy of 96% when leveraging the fully fine-tuned EfficientNetV2 B0 model. This innovation not only elevates classification performance but also addresses data scarcity and bias in medical deep learning. Our research marks a paradigm shift in blood cell classification, enabling more nuanced and insightful medical analyses. The "Naturalize" technique’s impact extends beyond blood cell classification, emphasizing the vital role of diverse and comprehensive datasets in advancing healthcare applications through deep learning.

Background: Retinal diseases such as neovascular age related macular degeneration (nAMD), diabetic macular edema (DME) or branch-/central retinal vein occlusion (B/CRVO) have significant implications for patients' social and psychological well-being. To gain a comprehensive understanding of patients' backgrounds and attitudes, we developed an exploratory questionnaire, the Basic Care and Patient Satisfaction Questionnaire (BPZ-9). Methods: The ALBATROS study aimed to assess the care situation of treatment-naïve patients diagnosed with either nAMD, DME or B/CRVO. The data collection took place at the beginning of treatment and after twelve months of anti-VEGF (vascular endothelial growth factor) therapy. The BPZ-9 instrument comprises nine questions to evaluate patients’ psychological and social situation and their satisfaction with treatment. Results: Data were collected from 1.478 nAMD (mean 78 years), 445 DME (67 years), 233 BRVO (70 years) and 144 CRVO (71 years) patients at 102 study centers throughout Germany. Approximately one third of the patients was living alone and six percent were in organized care. One in four patients had difficulties to walk and one in five needed accompany for treatment. Anxiety to lose vision was present in three out of four patients at the beginning and slightly decreased to two out of three over the 12-month treatment preriod. The distress of having a retinal disease was generally higher than the distress related to treatment. Most patients reported high treatment satisfaction (73%) and felt well informed (81%). Conclusion: Patients receiving anti-VEGF therapy commonly experience impaired mobility, dependency on others, distress and the fear of vision loss.

The overpopulation of feral pigeons in Hong Kong has significantly disrupted the urban ecosystem, highlighting the urgent need for effective strategies to control their population. In general, control measures should be implemented and re-evaluated periodically following accurate estimations of feral pigeon population in the concerned regions, which, however, is very difficult in urban envi-ronments due to the concealment and mobility of pigeons within complex building structures. With the advances in deep learning, computer vision can be a promising tool for pigeon monitoring and population estimation but has not been well investigated so far. Therefore, we propose an improved deep learning model based on Mask-RCNN (Swin-Mask R-CNN) for feral pigeon detection using computer vision techniques. Specifically, our model consists of a Swin transformer network (STN) as the backbone, a feature pyramid network (FPN) as the neck, and three decoupled detection heads. The STN is utilized to extract deep feature information of feral pigeons through local and cross-window attention mechanisms. The FPN is employed to fuse multi-scale features and enhance the multi-scale learning ability. Heads in the three branches are responsible for classification, pre-dicting best bounding boxes, and segmentation of feral pigeons, respectively. During the prediction phase, a Slicing Aided Hyper Inference (SAHI) tool is employed to zoom in on the feature infor-mation of small feral pigeon targets, and the segmentation head is frozen to expedite inference of large images. Experiments were conducted on feral pigeon dataset to evaluate model performance. The results reveal that our model is well-suited for detecting small targets in high-resolution images and achieves excellent recognition performance for feral pigeons with a mAP (mean average pre-cision) and an AP50 (average precision at 50% intersection over union) of 0.74 and 0.93, respectively. For small target feral pigeons, AP50 in small scale (AP50s) improved by 10% as compared to the Mask R-CNN (AP50s of 0.75), demonstrating its potential for dynamic pigeon detection and population estimation in the future.

Computer vision is a powerful technology that has enabled solutions in various fields by analyzing visual attributes in images. One field that has taken advantage of computer vision is agricultural automation, which promotes high-quality crop production. The nutritional status of a crop is a crucial factor in determining its productivity. This status is mediated by approximately 14 chemical elements acquired by the plant, and their determination plays a pivotal role in farm management. To address the timely identification of nutritional disorders, this study focuses on the classification of three levels of phosphorus deficiencies through individual leaf analysis. The methodological steps include: (1) generating a database with laboratory-grown maize plants that were induced to total phosphorus deficiency, medium deficiency, and total nutrition, using different capture devices; (2) processing the images with state-of-the-art transfer learning architectures (i.e. VGG16, ResNet50, GoogLeNet, DenseNet201, and MobileNetV2); and (3) evaluating the classification performance of the models using the created database. The results show that the VGG16 model achieves superior performance, with 98% classification accuracy. However, the other studied architectures also demonstrate competitive performance and are considered state-of-the-art automatic leaf deficiency detection tools. The proposed method can be a starting point to fine-tune machine vision-based solutions tailored for real-time monitoring of crop nutritional status.

Deep Learning (DL) has made significant advances in computer vision with the advent of Vision Transformers (ViT). Unlike Convolutional Neural Networks (CNNs), ViTs use self-attention to extract both local and global features from image data, and then use residual connections to feed these features directly into a fully networked multilayer perceptron head. In hospitals, hematologists prepare peripheral blood smears (PBSs) and read them under a medical microscope to detect abnormalities in blood counts such as leukemia. However, this task is time-consuming and prone to human error. This study investigates the transfer learning process of Google ViT and ImageNet CNNs to automate the reading of PBSs. The study used two online PBS datasets, PBC and BCCD, and transferred them into balanced datasets to investigate the influence of data amount and noise immunity on both neural networks. The PBC results show that Google ViT is an excellent DL neural solution for data scarcity. The BCCD results show that Google ViT is superior to ImageNet CNNs in dealing with unclean, noisy image data because it is able to extract both global and local features and use residual connections, despite the additional time and computational overhead.

To solve the problem of complex sensor installation and easy damage in contact displacement measurement, a non-contact measurement method of brace displacement in bridge swivel construction based on binocular vision is proposed in this study. Combined with monitoring data in a swivel construction interchange project, binocular vision system was compared and analyzed with the traditional displacement meter and total station. Results showed that force versus displacement variation plots were obtained by binocular visual measurement in the weighing test, the inflection point can be clearly distinguished, and the position of the inflection point was the same as that of the displacement meter; monitoring data of binocular vision measurement and total station were compared and analyzed in the bridge swivel process, the average error was no more than 0.18 mm, the maximum error was no more than 0.42 mm, and the standard deviation was no more than 0.12. The method proposed in this study can be used as an alternative to displacement meter and total station measurement methods in bridge swivel construction, the problem of complex sensor installation and easy damage is solved in brace displacement measurement, equipment and labor costs can be effectively saved.

Track fasteners play a pivotal role in infrastructure inspection for high-speed rail. Yet, images taken by drones often capture shadows cast by electrical towers flanking the high-speed rail tracks. These shadows can hinder the visibility of the track fasteners, thereby impacting detection efficiency and accuracy considerably. The present paper introduces an end-to-end shadow removal algorithm, rooted in generative adversarial network training. The comprehensive network framework is segmented into three sub-networks: pseudo-mask generation, shadow removal, and result refinement. We have integrated a Fourier convolutional residual module to bolster the feature extraction capability of the generator network. This integration ensures the network retains a global receptive field, even in its more superficial layers. By employing an overall weighted loss function, we enhance the quality of the images produced without shadows. Further, a perceptual loss function has been incorporated to retain the structural information of objects, setting the stage for subsequent defect detection. Our results highlight that Pse-ShadowNet adeptly eradicates fastener shadows while maintaining vital visual features, including object position, structure, texture, edges, and other key visual elements. Consequently, the reconstructed images are detailed and showcase superior image quality.

Cycle times at workstations in offsite construction factories fluctuate widely due to various influencing factors. Consequently, relying on average rates, such as length per unit of time, for estimating cycle times proves to be inaccurate, often leading to significant deviations between production schedules and actual operations. To address this issue, this study proposes an estimation system that leverages machine-learning-based prediction, statistical methods, 3D simulation, and computer vision to predict cycle times at the workstation level. Testing of the system on a semi-automated wood-wall framing workstation in a panelized construction factory shows that it reduces the mean absolute error and sum of errors by approximately 36% and 68%, respectively, compared to the fixed rate method. The results also highlight the efficacy of using computer vision data for training machine-learning models for cycle time estimation, the importance of identifying and understanding the factors influencing cycle times, and the impact of random delays on the accuracy of cycle time estimation systems.

Image classification in Remote Sensing and Geographic Information Systems (GIS) containing various land-cover classes is essential for efficient and sustainable land-use estimation, and other tasks like object detection, localization and segmentation. Deep Learning (DL) techniques have shown a tremendous potential in the GIS domain. While Convolutional Neural Networks (CNN) have dominated most of the image analysis domain, a new architecture called transformers have proved to be a unifying solution for several AI-based processing pipelines. Vision Transformers (ViT), a variant of transformers can have comparable and in some cases better accuracy than a CNN. However, they suffer from a significant drawback associated with an excessive use of training parameters. In this research we explore several modifications in the vision transformer architectures, especially MobileViT that can be optimized while boosting accuracy. To verify our proposed approach these new architectures are trained on four land-cover datasets AID, EuroSAT, UC-Merced, and WHU-RS19. Experiments reveal that combination of lightweight convolutional layers including ShuffleNet along with depthwise separable convolutions and average pooling can reduce the trainable parameters by 17.85% and yet achieve higher accuracy than the base MobileViT. It is also observed that utilizing a combination of convolution layers along with multi-headed self attention layers in MobileViT variants provide better performance in capturing local and global features unlike the standalone ViT architecture that utilizes almost 95% more parameters than the proposed MobileViT variant.

Self-driving vehicles demand efficient and reliable depth sensing technologies. Lidar, with its capacity for long-distance, high-precision measurement, is a crucial component in this pursuit. However, conventional mechanical scanning implementations suffer from reliability, cost, and frame rate limitations. Solid-state lidar solutions have emerged as a promising alternative, but the vast amount of photon data processed and stored using conventional direct time-of-flight (dToF) prevents long-distance sensing unless power-intensive partial histogram approaches are used. In this paper, we introduce a groundbreaking ‘guided’ dToF approach, harnessing external guidance from other onboard sensors to narrow down the depth search space for a power and data-efficient solution. This approach centers around a dToF sensor in-which the exposed time widow of independent pixels can be dynamically adjusted. We utilize a 64-by-32 macropixel dToF sensor and a pair of vision cameras to provide the guiding depth estimates. Our demonstrator captures a dynamic outdoor scene at 3 fps with distances up 75 m. Compared to a conventional full histogram approach, on-chip data is reduced by over 25 times, while the total laser cycles in each frame are reduced by at least 6 times compared to any partial histogram approach. The capability of guided dToF to mitigate multipath reflections is also demonstrated. For self-driving vehicles where a wealth of sensor data is already available, guided dToF opens new possibilities for efficient solid-state lidar.

Iron ore processing involves critical steps that affect the quality of the final product. Determining the diameter of the pellets is the necessary initial step for volume measurement and, ultimately, for porosity and bulk density measurement, crucial characteristics for optimizing the burning process in the blast furnace. Traditional measurement methods using mercury present issues related to operator safety and environmental preservation, while the practices described in ISO 4698 standard require lengthy preparation and execution time. In light of environmental needs, operator safety, and the time consumed in tests, implementing a new method based on digital image processing and convolutional neural networks for measuring the diameter of burnt iron ore pellets is proposed. The methodology involves capturing images of the pellets using a high-resolution camera and utilizing digital image processing and neural networks capable of performing pixel-by-pixel object segmentation in the images, providing precise information about the pellets to calculate their volume automatically. The results were compared with those obtained using traditional methods, evaluating their conformity with the ISO 4698 standard. In conclusion, this study offers a new approach to measuring the volume of burnt iron ore pellets, providing accurate and reliable results in compliance with safety and environmental preservation standards.

Medical image analysis is an important branch in the field of medicine, which mainly uses image processing and analysis techniques to interpret and diagnose medical image data. Medical image data helps doctors to effectively observe and diagnose patients' body structures, tissues and lesions. Medical image analysis has been an important research area in the medical field, and it is important for disease diagnosis, treatment planning, and condition monitoring. In recent years, the rapid development of deep learning and computer vision technologies has contributed greatly to the automation, multimodal data fusion, real-time application, and accuracy improvement of medical image analysis. In addition, the development of deep learning has given rise to some new research areas in medical image analysis, such as Generative Adversarial Networks (GANs) for synthetic medical images, self-supervised learning for unsupervised feature learning, and neural network interpretability. In this paper, we will introduce some optimisation methods for medical images which are effective in improving the accuracy, efficiency and reliability of medical image analysis.

Human Action Recognition is widely used in multiple fields to recognize activities and extract spatial and temporal information. This paper focuses on analyzing various methods as well as provides extensive knowledge of the foundational concepts of the HAR. A dataset is also very crucial for any research study so we discussed the popular dataset and its features in this paper.

This study proposes a method for image matching between infrared(IR)-RGB images using a deep learning network to estimate the deformation field. We propose a deformation field generator (DFG) that estimates the deformation field of the transformation matrix to match each pixel or IR image to the RGB image. DFG is a network that receives IR and RGB images as input; the output size is two channels and has the sample resolution as the input image. By warping the IR image through a grid-sampler that warps the image according to the value of the deformation field, we can obtain a warped IR image that matches the RGB image. Additionally, to check whether the warped IR image matched the RGB image, the masking images detecting the segmentation of objects were photographed in two images. Without directly comparing IR and RGB images, we proposed mask loss that warps the IR mask image through the deformation field and grid sampler and then compares the warped IR mask image with the RGB mask image. Mask loss solves the spatial similarity comparison problem with multi-modality images, such as IR and RGB images, by comparing the mask image with the same modality image as the mask image.

The relative position of the orchard robot to the rows of fruit trees is an important parameter for achieving autonomous navigations. The current methods for estimating the position parameters between rows of orchard robots obtain low parameter accuracy, and to address this problem, this paper proposes a machine vision-based method for detecting the relative position of orchard robots and fruit tree rows. Firstly, the fruit tree trunk is identified based on the improved YOLOv4 model; secondly, the camera coordinates of the tree trunk are calculated from the principle of binocular camera triangulation, and the ground projection coordinates of the tree trunk are obtained through coordinate conversion; finally, the midpoints of the projection coordinates of different sides are combined and the navigation path is obtained by linear fitting with the least squares method, and the position parameters of the orchard robot are obtained through calculation. The experimental results show that the average accuracy and average recall of the improved YOLOv4 model for fruit tree trunk detection are 97.05% and 95.42%, respectively, which are 5.92 and 7.91 percentage points higher than those of the original YOLOv4 model. The average errors of heading angle and lateral deviation estimates obtained based on the method in this paper are 0.57° and 0.02 m. The method can accurately calculate heading angle and lateral deviation values at different positions between rows, and can provide a reference for autonomous visual navigation of orchard robots.

People often make mistakes, so we try to automate every aspect of our lives. Sports is no exception. While just over a decade ago humans were analyzing games, today this is being done by artificial intelligence. Due to rapid development over the past decade, neural networks are now faster, more accurate, and in some areas even better than their human counterparts. In this paper, we present an algorithm that can detect player statistics during an NBA broadcast. It also helps users better understand the game and the use of augmented reality. The algorithm detects players on the court, tracks their movements, and assigns them to their respective teams. Using homography estimation, we transform the players’ positions from a three-dimensional space in the video to a two-dimensional space on the playing field plane. We define a new algorithm that predicts the players’ actions and their statistics. The results show that the proposed method can effectively identify the players, their respective teams, and their positions. It can also analyze their actions with high accuracy.

Some traditional robots are based on offline programming reciprocal motion, and with the continuous upgrading of vision technology, more and more tasks are replaced by machine vision. For the current problem of insufficient accuracy of robot target localization based on binocular vision, and an improved random sampling consistency algorithm is proposed to complete parallel robot target localization and grasping under the guidance For the current problem of insufficient accuracy of robot target localization based on binocular vision, an improved random sampling consistency algorithm is proposed to complete parallel robot target localization and grasping under the guidance of multi Firstly, the RANSAC algorithm is improved based on the SURF algorithm; then the parallax gradient method is applied to iterate the matched point pairs several times to further optimize the data; then the 3D reconstruction is completed by the program technique; finally the obtained data is finally the obtained data is input into the robot arm and the camera internal and external parameters are obtained by the calibration method so that the robot can accurately locate and The experiments show that the improved algorithm has advantages in recognition accuracy and grasping success rate under multi- The experiments show that the improved algorithm has advantages in recognition accuracy and grasping success rate under multi-vision system.

Understanding park events and their categorization offer pivotal insights into urban parks and their integral roles in cities. This study utilized images and event category data from the New York City Parks Events Listing database to train a Convolutional Neural Network (CNN) for image-based park event categorization. Different CNN models were tuned to complete this multi-label classification task, their performances compared. Preliminary results underscore the efficacy of deep learning in automating the event classification process, revealing the multifaceted activities within urban green spaces. The CNN showcased proficiency in discerning various event nuances, emphasizing the diverse recreational and cultural offerings of urban parks. Such categorization has potential applications in urban planning, aiding decision-making processes related to resource distribution, event coordination, and infrastructure enhancements tailored to specific park activities.

Quality control is a critical component in industrial manufacturing, directly influencing efficiency, product reliability, and ultimately, customer satisfaction. In the dynamic environment of industrial manufacturing, traditional methods of inspection may not adequately meet the evolving complexity, necessitating innovative approaches to bolster precision and productivity. In this study, we explore the application of mixed reality (MR) technology for real-time quality control in the assembly process. Our methodology involved the integration of smart glasses with a server-based image recognition system, designed to conduct real-time component analysis. The innovative aspect of our study lies in the harmonization of MR and computer vision algorithms, providing immediate visual feedback to inspectors and thereby improving the speed and accuracy of defect detection. YOLOv8 have been adopted in this study for detection object model. The project implementation occurred in a controlled environment to enable a comprehensive evaluation of the system functionality, the identification of possible problems and improvements in the system performance. The results indicated the viability of mixed reality as a powerful tool for enhancing traditional inspection processes. The fusion of MR and computer vision offers possibilities for future advancements in industrial quality control, paving the way for more efficient and reliable manufacturing ecosystems.

In autonomous driving systems, high-speed and real-time image processing, along with object recognition, are crucial technologies. This paper builds upon the research achievements in in-dustrial item sorting systems and proposes an object recognition and sorting system for auton-omous driving. In industrial sorting lines, goods sorting robots often need to work at high speeds to efficiently sort large volumes of items. This poses a challenge to the robot's real-time vision and sorting capabilities, making it both practical and economically viable to implement a real-time and low-cost sorting system in a real-world industrial sorting line. Existing sorting systems have limitations such as high cost, high computing resource consumption, and high power consump-tion. These issues lead to the fact that existing sorting systems are typically used only in large industrial plants. In this paper, we design a high-speed, low-cost, low-resource-consumption FPGA (Field-Programmable Gate Array) based item sorting system that achieves similar perfor-mance to current mainstream sorting systems at a lower cost and consumption than existing sorting systems. The recognition part employs a morphological recognition method, which segments the image using a frame difference algorithm and then extracts the color and shape features of the items. To handle sorting, a six-degree-of-freedom robotic arm is introduced in the sorting segment. The improved cubic B-spline interpolation algorithm is employed to plan the motion trajectory and consequently control the robotic arm to execute the corresponding actions. Through a series of experiments, this system achieves an average recognition delay of 25.26ms, ensures smooth operation of the gripping motion trajectory, minimizes resource consumption, and reduces implementation costs.

Identification of plants is a challenging task which aims to identify the family, genus, and species level according to morphological features. Automated deep learning-based computer vision algorithms are widely used for identifying plants and can help users to narrow down the possibilities. However, numerous morphological similarities between and within species make the classification difficult. In this paper, we tested a custom convolution neural network (CNN) and vision transformer (ViT) based models using the PyTorch framework to classify plants. We used a large dataset of 88K and 16K images for classifying plants at genus and species levels respectively. Our results show that for classifying plants at the genus level, ViT models perform better compared to CNN-based models ResNet50 and ResNet-RS-420, and other state-of-the-art CNN-based models suggested in previous studies on a similar dataset. The ViT model achieved top accuracy of 83.3% for classifying plants at the genus level. ViT models also perform better for classifying plants at the species level compared to CNN-based models ResNet50 and ResNet-RS-420, with a top accuracy of 92.5%. We show that the correct set of augmentation techniques plays an important role in classification success.

The traditional understanding of brain function has predominantly focused on chemical and electrical processes. However, new research in fruit fly (Drosophila) binocular vision reveals ultrafast photomechanical photoreceptor movements significantly enhance information processing, thereby impacting a fly's perception of its environment and behaviour. The coding advantages resulting from these mechanical processes suggest that similar physical motion-based coding strategies may affect neural communication ubiquitously. The theory of neural morphodynamics proposes that rapid biomechanical movements and microstructural changes at the level of neurons and synapses enhance the speed and efficiency of sensory information processing, intrinsic thoughts, and actions by regulating neural information in a phasic manner. We propose that morphodynamic information processing evolved to drive predictive coding, synchronising cognitive processes across neural networks to match the behavioural demands at hand effectively.

Obstructive Sleep Apnea (OSA) surgery is now a viable solution in selected patients and the “remodelling” palatopharyngeal surgery is the most common one. Recently it becomes less in-vasive with the introduction of Barbed Sutures (BS). An optimization of surgical technique is represented by Barbed pharyngoplasty (BP), that requires surgical precision and needs efficient and precise oropharyngeal visualization. Consequently, the lighting system is of pivotal im-portance in BP. The aim of this work is to describe the first experience on the use of a new light-ing system, called KlaroTM in BP for OSA. We evaluated the Klaro™ system in 15 consecutives BP for OSA in comparison with conventional headlamp illumination. The visualization of pala-topharyngeal muscle in the bottom of the tonsillar fossa, entry and exit needle, such as needle tip were statistically better with KlaroTM than headlamp illumination both for surgeon and resi-dent (p<0.05). No significant differences for the visualization of the posterior pharyngeal wall and uvula were reported. The KlaroTM lighting system allows a satisfied illumination of oral cavity and oropharynx in the majority of cases. We encourage to use of KlaroTM not only in BP for OSA, but also in all oral and pharyngeal surgeries, including tonsillectomy and oncological surgery.

Due to the influence of background conditions, lighting conditions, occlusion issues and the image resolution, how to extract robust person features is one of the difficulties in ReID research. Vision in Transformers (ViT) has achieved significant results in the field of computer vision. However, the existing problems still limit its application in ReID due to slow extraction of person features and difficulty in utilizing local features of people. To solve the mentioned problems, we utilize Pyramid Vision Transformer (PVT) as the backbone of feature extraction and propose a PVT-based ReID method in conjunction with other studies. Firstly, some improvements suitable for ReID are used on the PVT backbone, and we establish a basic model by using powerful methods verified on CNN-based ReID. Secondly, in an effort to further promote the robustness of the person features extracted by the PVT backbone, two new modules are designed. (1) The local feature clustering (LFC) is recommend to enhance the robustness of person features by calculating the distance between local features and global feature to select the most discrete local features and clustering them. (2) The side information embeddings (SIE) are used to encode non-visual information and send it into the network for training to reduce its impact on person features. Finally, the experiments show that PVTReID has achieved excellent results in ReID datasets and are 20% faster on average than CNN-based ReID methods.

Studying and understanding the behavior of people and vehicles on public roads can be of utmost importance for supporting the activities of many institutional stakeholders. It may allow automated supervision of the ongoing situation in a given place, with warnings or alarms raised in case of anomalies. It may be used to plan their interventions on road and town organization. It may provide them with advanced support to decision-making. The number of involved entities and places to manage makes it infeasible to manually handle all the traffic-related tasks. Moreover, the complexity of the tasks to be carried out requires the adoption of advanced approaches. Many AI solutions are nowadays mature to support these requirements. In some cases, the motivations and objectives of traffic management require the AI outcomes to be understandable, interpretable and explainable. In this paper, we propose TrAnSIT (TRaffic ANalysis Supervision and Interpretation Tool), an AI-based framework that combines several modules, each aimed at tackling a specific traffic-related task, so as to cover a wide landscape of traffic-related issues, from overall urban or suburban traffic management to surveying specific road segments that fall under the scope of one camera. Most of these modules are based on AI techniques that support a human-level understanding of the outcomes.

Regular inspections during construction work verify that the work completed is consistent with the plans and specifications and ensure that it is within the planned time and budget. This requires frequent physical site observations to independently measure and verify the completion percentage of the construction progress performed over periods of time. The current computer vision-based (CV) techniques for the measurement of as-built elements, predominantly use 3D laser scanning or 3D Photogrammetry modelling to determine the geometrical properties of as-built elements on construction sites. Both techniques require data acquisition from several positions and angles to generate sufficient information about the element’s coordinates making the deployment of these techniques on dynamic construction project sites a challenging task. In this paper, we propose a pipeline for automating the measurement of as-built components using artificial intelligence (AI) and computer vision (CV) techniques. The pipeline requires a single image obtained with a stereo-camera system to measure the size of selected objects or as-built components. We demonstrate our approach by measuring the size of concrete walls and columns. The novelty of this work is attributed to the fully automated CV-based method for measuring any given element using a single image only. The proposed solution is suitable for use in measuring the sizes of as-built components of built assets. It has the potential to be further developed and integrated with BIM models for use on construction projects for progress monitoring.

Artificial intelligence (AI) and 3D printing (3DP) play critical roles in what is known as the 4th Industrial Revolution by developing data- and machine-intelligence-based integrated production technologies. This shift has led to increasingly complex design requirements and fabrication methods, posing several challenges for implementing design, fabrication process, and architectural quality standardization. This paper systematically reviews AI techniques applied in diverse stages of 3DP production in architecture. The research goals are to (1) offer a comprehensive critical analysis of the body of literature; (2) identify and categorize approaches to integrating AI in the production of 3D-printed structures; (3) identify and discuss challenges and opportunities of AI integration in architectural production of 3DP structures; and (4) identify gap and provide recommendations for future research. The findings indicate that topics related to AI applications are current and trending, receiving increasing attention from researchers in general and particularly those working in machine and deep learning, artificial neural networks, and pattern recognition. However, the potential of the application of AI in architectural production using additive manufacturing still needs to be explored. Considering this, the paper emphasizes the necessity of redefining traditional field boundaries and opening new opportunities in the sector.

ORB (Oriented FAST and Rotated Brief) features are the most commonly used features in visual SLAM and visual odometry that have high computational speed, and rotation scale invariance. However, due to the homogenization of ORB features, the corner properties of specific features are poor in some artificial environments, and it can lead to poor matching performance when the environmental texture is not rich. The underlying reason is that the pixel grayscale changes in the less-richly textured regions on the image are not obvious, leading to a certain degree of mis-matching in the matching process. At the same time, when the camera motion speed is low, there is much overlap between adjacent frames. This results in minimal or almost no changes in the projection of feature points, and the system is highly sensitive to errors, In this case, even small errors can cause significant fluctuations in the calculation results. In response to the above issues, an improved feature point method for stereo vision mileage calculation is proposed to solve the problem of system stability deterioration when cameras move at low speeds in artificial envi-ronments. Firstly, weight calculation based on different texture regions is used to solve the problem of low corner properties of feature points caused by environmental texture differences. Then, keyframes are used for motion model estimation to improve the system's stability under low-speed motion. The experimental results of the dataset showed that the key frame rate was optimal between 10% and 12%, and the positioning accuracy was higher than these open-source systems compared with three common open-source VO systems.

In Mexico, the incorporation of deaf people into education has been lacking since only 14% of the deaf population in the age group between 3 and 29 years access education with the support of a hearing aid. Additionally, those who have been incorporated frequently face inappropriate educational strategies which poorly develop the use of Mexican Sign Language (MSL) and therefore academical success and opportunities for insertion in the workplace are difficult. This research explores a novel mexican sign language lexicon video dataset containing the dynamical gestures most frequently used by MSL. Each gesture consists of a set of different versions of videos under uncontrolled conditions. MX-ITESO-100 data set is composed of a lexicon of 100 gestures and 5,000 videos from three participants with different grammatical elements. Additionally, the data set is evaluated in a two-step neural network model with an accuracy greater than 99%. and thus serves as a benchmark for future training of machine learning models in computer vision systems. Finally, this research provides an inclusive environment within society and organizations in particular for people with hearing impairment.

Image analysis is widely applied in plant science for phenotyping and monitoring botanic and agricultural species. Although a lot of software is available, tools integrating image analysis and statistical assessment of seedling growth in large groups of plants are limited or absent, and do not cover the needs of the researchers. In this study, we developed Morley, a free, open-source graphical user interface written in Python. Morley automates the following workflow: (1) group-wise analysis of a few thousand seedlings from multiple images; (2) recognition of seeds, shoots and roots in seedling images; (3) calculation of shoot and root lengths and surface areas, (4) evaluation of statistically significant differences between plant groups, (5) calculation of germination rates, (6) visualization and interpretation. Morley is designed for laboratory studies of biotic effects on seedling growth, when molecular mechanisms underlying morphometric changes are analyzed. Performance was tested using cultivars of T. aestivum, P. sativum on seedlings of up to 1 week old. Accuracy of the measured morphometric parameters was comparable with the ones obtained using ImageJ and manual measurements. Dose-dependent laboratory tests for germination affected by new bioactive compounds and fertilizers, assuming extraction of seedlings from a substrate and/or dissection are among the suggested applications.

There is a need for cost-effective and non-invasive methods of monitoring feeding behavior in livestock operations, considering the significant impact of feed costs on economic efficiency and assisting in detecting health issues of group-fed animals. This paper proposes using deep learning-based computer vision techniques to detect pen-fed beef cattle feeding behavior using Mask Region-based Convolutional Neural Network (RCNN). A deep learning model was pre-trained on the Common Objects in Context (COCO) dataset to generate cattle instance segmentation. Manually defined feed bunk polygons are compared with these segmentation masks to derive feeding time for each bunk. A full day’s worth of video data and the corresponding physical sensor data are collected for the experiment. By benchmarking the computer vision detected data with physical ground truth over random time segments from morning to evening (thus various lighting conditions), the optimal thresholds for Mask RCNN are determined to be 0.7 for bounding boxes and 0.1 for masks. Using these parameters. The reports suggest that the computer vision system achieved a precision of 87.2% and a recall of 89.1%, signifying precise detection of feeding events. Our study, to the best of our knowledge, was one of the first investigations of instance segmentation on feeding time sense, which combines deep learning methods with traditional computer vision logistics, reporting on feeding time data collection and processing, camera testing and adjustment, and performance evaluation. Future research directions include computer vision applied in feed grading and animal re-identification for individual production analysis.

To handle the problem of solving the results of aircraft’s visual navigation with scene matching, this paper take the measure of the spherical EPnP positioning posture solving by measuring the central angle threshold value and approaches for constructing a measuring model. The detailed steps are as follows: firstly, this approach needs to construct a positioning coordinate model for the earth surface, makes sure the expression for the 3-dimensional coordinate of the earth surface and solves the positioning of constructing data model with EPnP positioning posture solving algo-rithm. Secondly, by contrasting and analyzing the positioning posture value of approximate plane coordinates, the critical value is acquired, which can be recognized as plane calculation. Lastly, this method should construct a theoretical model of measurement for the visual height and central angle with the decided central angle threshold value. The simulation experiment shows that the average positioning precision of taking the spherical coordinates as input is 16.42 percent way higher than taking the plane coordinates as input. When the central angle is less than 0.05 degrees and the surface district is less than 5585 square meters, the positioning precision of the plane co-ordinates is pretty much equal to the spherical coordinates. At this moment, the sphere can be seen as flat. The conclusion of this essay can theoretically guide the further study of positioning posture solving of the scene matching, which is also of vital significance for theory research and engineering application.

Space syntax can be potentially applied for the evaluation of quality of user flows particularly in solving problems of sustainable projects in spaces intended for users to move around. This study aims to analyze the concepts of pedestrian flows (open and closed) by space syntax based bibli-ographical approach on a global scale, demonstrating the capability of improvements in Sus-tainable Development Goals (SDGs) as applied to the architecture of sustainable flows. Scopus theoretical reference bases were used, which are directly related to the theme of space syntax in open and closed spaces. Frequency analyzes were carried out, applying the content analysis, to identify words with a degree of similarity, related to “space syntax: flow in urban environ-ments” and “space syntax in closed built systems”, in relations to the SDGs. The results show that pedestrian flows identified in the literature aid to understand the global production on space syntax in open and closed spaces, directed to user flows in the built environment, where many environments of flows in the built space analyzed become unsustainable because they do not present full flow efficiency. In our study on space syntax, the following central terms were identified: pedestrian movements (open urban systems) and space (closed built systems), which allows a better understanding of the flows, highlighting the importance of the urbanist architect on the functionality of user flows in sustainable architectural projects on a global scale. The most frequent categories for open environments were the terms space and form and for closed envi-ronments were space and flow.

Mice are one of the frequently used animal models in science research whose behavioral characteristics can provide much valuable information in biology, neuroscience, and pharmacology. Nowadays, artificial intelligence is widely used in mice behavior analysis. Integrated AI systems such as ChatGPT and VisualGPT are already available, and we discuss the feasibility of MiceGPT to help researchers identify and classify mouse behavior more easily. We review the applications of mice behavior analysis, analyze the tasks of deep learning on these applications based on an AI pyramid, and finally summarize the AI approaches to solve these tasks. Based on these summaries, we propose three MiceGPT architectures to demonstrate the theoretical feasibility of MiceGPT.

Indoor localization (IL) is a significant topic of study with several practical applications. The area of IL has evolved greatly in recent years due to the introduction of numerous technologies such as WiFi, Bluetooth, cameras, and other sensors. Despite the growing interest in this field, there are numerous challenges and drawbacks that must be addressed to develop more accurate and sustainable systems for IL and its real-life applications. This review study gives an in-depth look into IL, covering the most promising artificial intelligence-based and hybrid strategies that have shown excellent potential in overcoming some of the limitations of classic methods. In addition, the paper investigates the significance of high-quality datasets and evaluation metrics in the design and assessment of IL algorithms. Furthermore, this overview study emphasizes the crucial role that machine learning techniques, such as deep learning and transfer learning, play in the advancement of IL. A focus on the importance of IL and the various technologies, methods, and techniques that are being used to improve it. Finally, The survey highlights the need for continued research and development to create more accurate and scalable techniques that can be applied across a range of industries, such as evacuation-egress routes, hazard-crime detection, smart occupancy-driven energy reduction and asset tracking and management.

The use of visual signals in horticulture has attracted significant attention and encompassed a wide range of data types such as 2D images, videos, hyperspectral images, and 3D point clouds. These visual signals have proven to be valuable in developing cutting-edge computer vision systems for various applications in horticulture, enabling plant growth monitoring, pest and disease detection, quality and yield estimation, and automated harvesting. However, unlike other sectors, developing deep learning computer vision systems for horticulture encounters unique challenges due to the limited availability of high-quality training and evaluation datasets necessary for deep learning models. This paper investigates the current status of vision systems and available data in order to identify the high-quality data requirements specific to horticultural applications. We analyse the impact of the quality of visual signals on the information content and features that can be extracted from these signals. To address the identified data quality requirements, we explore the usage of a deep learning-based super-resolution model for generative quality enhancement of visual signals. Furthermore, we discuss how these can be applied to meet the growing requirements around data quality for learning-based vision systems. We also present a detailed analysis of the competitive quality generated by the proposed solution compared to cost-intensive hardware-based alternatives. This work aims to guide the development of efficient computer vision models in horticulture by overcoming existing data challenges and paving a pathway forward for contemporary data acquisition.

Expanding previous efforts to survey the visual opsin repertoires of the Hemiptera, this study confirms that homologs of the UV- and LW-opsin subfamilies are conserved in all Hemiptera, while the B-opsin subfamily is missing from the Heteroptera and subgroups of the Sternorrhyncha and Auchenorrhyncha, i.e. aphids (Aphidoidea) and planthoppers (Fulgoroidea), respectively. Unlike in the Heteroptera, which are characterized by multiple expansions of the LW-opsin subfamily, the lack of B-opsin correlates with the presence of tandem-duplicated UV-opsins in aphids and planthoppers. Available data on organismal wavelength sensitivities and retinal gene expression patterns lead to the conclusion that, in both groups, one UV-opsin paralog shifted from ancestral UV peak sensitivity to derived blue sensitivity, thereby compensating for the lost B-opsin. Two parallel bona fide tuning site substitutions compare to 18 non-corresponding amino acid replacements in the blue-shifted UV-opsin paralogs of aphids and planthoppers. Most notably, while the aphid blue-shifted UV-opsin clade is characterized by a replacement substitution at one of the best-documented UV/blue tuning sites (Rhodopsin site 90), the planthopper blue-shifted UV-opsin paralogs retained the ancestral lysine at this position. The combined findings identify aphid and planthopper UV-opsins as a new valuable data sample for studying adaptive opsin evolution.

Blood cell analysis is a crucial diagnostic process in medical practice. In particular, detecting white blood cells (WBCs) is essential for diagnosing of many diseases. The manual screening of blood films is a time-consuming and subjective process, which can lead to inconsistencies and errors. Therefore, automated detection of blood cells can improve the accuracy and efficiency of the screening process. In this study, an explainable Vision Transformer (ViT) model was proposed for the automatic detection of WBCs from blood films. The proposed model utilizes the self-attention mechanism to extract relevant features from the input images and leverages transfer learning by incorporating pre-trained model weights to improve its performance. The proposed model achieved a classification accuracy of 99.40% for five distinct types of WBCs and exhibited potential in reducing the time required for manual screening of blood films by pathologists. Upon examination of the misclassified test samples, it was observed that incorrect predictions were correlated with the presence or absence of granules in the cell samples. To validate this observation, the dataset was divided into two classes, namely Granulocytes and Agranulocytes, and a secondary training process was conducted. The resulting ViT model trained for binary classification achieved an accuracy of 99.70%, recall of 99.54%, precision of 99.32%, and F-1 score of 99.43% during the test phase. To ensure the reliability of the ViT model's multi-class classification of WBCs, the pixel areas that the model focuses on in its predictions are visualized through the Score-CAM algorithm.

Surface defect detection is a crucial step in the process of automotive wheel production. However, the task possesses challenges due to complex background and a wide range of defect types. In order to detect the defects on the wheel surface accurately and quickly, this paper proposes a YOLOv5-based algorithm for automotive wheel surface defect detection. The algorithm trains and tests the YOLOv5s model using the self-created automotive wheel surface defect dataset, which contains four kinds of defects: linear, dotted, sludge, pinhole. The extensive experimental results demonstrate that the deep learning network trained by our method can achieve an average accuracy of 71.7% and 57.14 FPS. Our findings prove that this detection algorithm performs better than other common target detection algorithms and meets the real-time requirements of industrial applications.

Computer vision models of salient object detection attempt to mimic the ability of the human visual system to select relevant objects in images. To this end, the development of deep neural networks on high-end computers has recently made it possible to achieve high performance. However, it remains a challenge to develop deep neural network models of the same performance for devices with much more limited resources. In this work, we propose a new approach for a lightweight salient object detection neural network model, inspired by the cone and spatial opponent processes of the primary visual cortex (V1), that inextricably link color and shape in human color perception. Our proposed model, namely CoSOV1net, is trained from scratch, without using backbones from image classification or other tasks. Experiments, on the most widely used and challenging datasets for salient object detection, show that CoSOV1Net achieves competitive performance (i.e. Fβ=0.931 on the ECSSD dataset) with state-of-the-art salient object detection models, while having low number of parameters (1.14M), low FLOPS (1.4G) and high FPS (211.2) on GPU (nvidia Geforce RTX 3090 TI) compared to the state-of-the-art in the salient object detection or lightweight salient object detection task. Thus, CoSOV1net turns out to be a lightweight salient object detection that can be adapted to mobile environments and resource-constrained devices.

With the advent of deep learning, significant progress has been made in low-light image enhancement methods. However, deep learning requires enormous paired training data, which is challenging to capture in real-world scenarios.To address this limitation, this paper presents a novel unsupervised low-light image enhancement method, which first introduces the frequency domain features of images in low-light image enhancement tasks. Our work is inspired by imagining a digital image as a spatially varying metaphoric “field of light”, then subjecting the influence of physical processes such as diffraction and coherent detection back onto the original image space via a frequency-domain to spatial-domain transformation (inverse Fourier transform). However, the mathematical model created by this physical process still requires complex manual tuning of the parameters for different scene conditions to achieve the best adjustment. Therefore, we proposed a dual-branch convolution network to estimate pixel-wise and high-order spatial interactions for dynamic range adjustment of the frequency feature of the given low-light image. Guided by the frequency feature from the “field of light” and parameter estimation networks, our method enables dynamic enhancement of low-light images. Extensive experiments have shown that our method performs well compared to state-of-the-art unsupervised, and its performance approximates the level of the state-of-the-art supervised methods qualitatively and quantitatively. At the same time, the light network structure design allows the proposed method to have an extremely fast inference speed(nearly 150 FPS on an NVIDIA 3090 Ti GPU for an image of size 600×400×3 ). Furthermore, the potential benefits of our method to object detection in the dark are discussed.

In recent years, the analysis of macro- and micro-expression has drawn the attention of researchers since they provide visual cues of an individual's emotions for a broad range of potential applications such as lie detection and criminal detection. In this paper, we address the challenge of spotting facial macro- and micro-expression from videos and present compelling results by using a deep learning approach to analyze the optical flow features. Different from other deep learning approaches that are mainly based on Convolutional Neural Networks (CNNs), we propose a Transformer-based deep learning approach that predicts a score indicating the probability of a frame being within an expression interval. In contrast to other Transformer-based models that achieve high performance by being pre-trained on large datasets, our deep learning model, called SL-Swin, which incorporates Shifted Patch Tokenization and Locality Self-Attention into the backbone network Swin Transformer, effectively spots macro- and micro-expressions by being trained from scratch on small-size expression datasets. Our evaluation outcomes surpass the MEGC 2022 spotting baseline result, obtaining an overall F1-score of 0.1366. Additionally, our approach also performs well in the MEGC 2021 spotting task with an overall F1-score of 0.1824 and 0.1357 on CAS(ME)^2 and SAMM Long Videos, respectively. The code is publicly available on GitHub (https://github.com/eddiehe99/pytorch-expression-spotting).

The mouse visual system recently became the most popular model to study the cellular and circuit mechanisms of sensory processing. However, the importance of eye movements in mice only started to be appreciated recently. Eye movements provide a basis for active sensing and deliver insights into various brain functions and dysfunctions. A plethora of knowledge on the central control of eye movements and their role in perception and behaviour arose from work on primates. However, an overview of the known eye movement types in mice and a comparison to primates is missing.Here, we review the eye movement types described to date in mice and compare them to those observed in primates. We discuss the central neuronal mechanisms for their generation and control. Furthermore, we review the mounting literature on eye movements in mice during head-fixed and freely moving behaviours. Finally, we highlight gaps in our understanding and suggest future directions for research.

The Kingdom of Saudi Arabia (KSA) is known for its high car ownership and usage as well its high GDP per capita. This is combined with low/ or no provision of public transportation (PT) systems, has been resulting in perceptual attitudes of full dependency on private car travel. The level of awareness of the benefits of reducing car use and increasing the travel by more sustainable options, has a great impact on social change and behaviour. The Kingdom is currently progressing towards a new phase of “national transfer” through implementation of strategic and sustainable measures and programs. The city of Riyadh is construction a massive metro-system in Riyadh, that is nearing completion and operation. The public is aware of the national agenda, aware of the newly constructed projects and aware of the needed social change to realize the new vision of the country. This paper aims to assess travel behaviour and attitudes towards public transportation of Saudi travellers’ who are witnessing the new transformation in the Kingdom and who are aware of the new sustainable projects. Depicted from the theory of random utility, a discrete choice model of the intent to use public transportation is calibrated as a function of social and attitudinal factors. An online survey was designed and carried out using social media means; a completed 399 questionnaires have been obtained and studied. The methodology includes examining attitudes and preferences of the participants towards using public transportation options against participants’ socio-economic data. The analysis was carried out using ordinal logistic regression analysis (OLR) which is an efficient technique that is derived from the theory of random utility. The results show a good support for PT; a higher support for public transportation modes, form participants who are young females, lower income groups and the university graduates were reported. The level of support seems higher with the higher level of awareness about the new PT system.

Existing methods for 3D human pose estimation mainly divide the task into two stages. The first stage identifies the 2D coordinates of the human joints in the input image, namely the 2D human joint coordinates. The second stage uses the results from the first stage as input to recover the depth information of human joints from the 2D human joint coordinates to achieve 3D human pose estimation. However, the recognition accuracy of the two-stage method relies heavily on the results of the first stage and includes too many redundant processing steps, which reduces the inference efficiency of the network. To address these issues, we propose the EDD, a fully End-to-end 3D human pose estimation method based on transformer architecture with Dual Decoders. By learning multiple human poses, the model can directly infer all 3D human poses in the image using a pose decoder, and then further optimize the recognition result using a joint decoder based on the kinematic relations between joints. With the attention mechanism, this method can adaptively focus on the most relevant features to the target joint, effectively overcoming the feature misalignment problem in the human pose estimation task and greatly improving the model performance. Any complex post-processing step, such as non-maximum suppression, is eliminated, further improving the efficiency of the model. The results show that the method achieves an accuracy of 87.4% on the MuPoTS-3D dataset, significantly improving the accuracy of end-to-end 3D human pose estimation methods.

This review paper provides a comprehensive overview of machine vision pose measurement algorithms. The paper focuses on the state-of-the-art algorithms and their applications. The paper is structured as follows: The introduction in Section 1 provides a brief overview of the field of machine vision pose measurement. Section 2 describes the commonly used algorithms for machine vision pose measurement. Section 3 discusses the factors that affect the accuracy and reliability of machine vision pose measurement algorithms. Section 4 presents the applications of machine vision pose measurement in various fields. The paper provides specific examples of how machine vision pose measurement is used in each of these fields. Finally, Section 5 summarizes the paper and provides future research directions. The paper highlights the need for more robust and accurate algorithms that can handle varying lighting conditions and occlusion. It also suggests that the integration of machine learning techniques may improve the performance of machine vision pose measurement algorithms. Overall, this review paper provides a comprehensive overview of machine vision pose measurement algorithms, their applications, and the factors that affect their accuracy and reliability. It provides a valuable resource for researchers and practitioners working in the field of computer vision.

Objectives: Understanding the intersection of age, ethnicity, and disability will become increasingly important as the global population ages and becomes more diverse. By 2060, Hispanics will comprise 28% of the U.S. population. This study examines critical associations between sensory impairment, social isolation, and cognitive functioning among Hispanic older adults. Methods: Our sample consisted of 557 Hispanic older adults that participated in Rounds 1-3 or Rounds 5-7 of the National Health and Aging Trends Study. Longitudinal mediation models across a three-year span were estimated using Mplus with vision, hearing, and dual sensory impairment predicting cognitive functioning directly and indirectly through social isolation. Results: Findings indicated that cognitive functioning was concurrently and, in certain cases, longitudinally predicted by vision and dual sensory impairment, and by social isolation. Contrary to expectations, vision and hearing impairment were not predictive of social isolation. Dual sensory impairment was associated with social isolation, yet no significant indirect associations were found for sensory impairments predicting cognitive functioning through social isolation. Discussion: The finding that social isolation did not mediate the relationship between sensory impairment and cognitive decline among Hispanic older adults in the U.S. is contrary to findings from other studies that were not specifically focused on this population. This finding may be evidence that culturally motivated family support and intergenerational living buffer the impact of sensory impairments in later life. Findings suggest that Hispanic older adults experiencing dual sensory impairments may benefit from interventions that foster social support and include family members.

Loss of photoreceptors in retinal degenerative diseases also impacts the inner retina: bipolar cell dendrites retract, neurons rewire and protein expression changes. ON-bipolar cells represent an attractive target for optogenetic vision restoration. However, above-described maladaptations may negatively impact the quality of restored vision. To investigate this question, we employed human post-mortem retinas and transgenic rd1_Opto-mGluR6 mice expressing the optogenetic construct Opto-mGluR6 in ON-bipolar cells and carrying the retinal degeneration rd1 mutation. We found significant changes in delayed rectifier potassium channel expression in ON-bipolar cells of degenerative retinas. In particular, we found an increase in Kv1.3 expression already in early stages of degeneration. Immunohistochemistry localized Kv1.3 channels specifically to OBC axons. In whole-cell patch-clamp experiments, ON-bipolar cells in the degenerated murine retina were less responsive, which could be reversed by application of the specific Kv1.3 antagonist Psora-4. Notably, Kv1.3 block significantly increased the amplitude and kinetics of Opto-mGluR6-mediated light responses in ON-bipolar cells of the blind retina and increased the signal-to-noise ratio of light-triggered responses in retinal ganglion cells. We propose that reduction of Kv1.3 activity in the degenerated retina, either by pharmacological block or by KCNA3 gene silencing, could improve the quality of restored vision.

The appropriate maintenance of highway roads is critical for the safe operation of road networks and conserves maintenance costs. Multiple methods have been developed to investigate the surface of roads for various types of cracks and potholes, among other damage. Like road surface damage, the condition of expansion joints in concrete pavement is important to avoid unexpected hazardous situations. Thus, in this study, a new system is proposed for autonomous expansion joint monitoring using a vision-based system. The system consists of the following three key parts: (1) a camera-mounted vehicle, (2) indication marks on the expansion joints, and (3) a deep learning-based automatic evaluation algorithm. With paired marks indicating the expansion joints in a concrete pavement, they can be automatically detected. An inspection vehicle is equipped with an action camera that acquires images of the expansion joints in the road. You Only Look Once (YOLO) automatically detects the expansion joints with indication marks, which has a performance accuracy of 95%. The width of the detected expansion joint is calculated using an image processing algorithm. Based on the calculated width, the expansion joint is classified into the following two types: normal and dangerous. The obtained results demonstrate that the proposed system is very efficient in terms of speed and accuracy.

Conducting related research is crucial since there are still many problems that need to be resolved in the research on soft robots in the areas of material selection, structure design and manufacture, and drive control. Soft manipulators, a subset of soft robots, are now a popular area of study for many researchers. In comparison to typical manipulators, soft manipulators feature a high degree of gripping flexibility and a basic morphological structure. They are composed of flexible materials. It has a wide range of potential applications in healthcare, rehabilitation, bionics, and detection, and it can compensate for the drawbacks of rigid manipulators in some use scenarios. A modular soft-body torsional gripping system is developed after a torsional and gripping actuator is conceived, constructed, and its performance is examined. The torsion actuator and the grasping actuator can be combined in the system in a modular fashion. With the help of RGB-D vision algorithms, this multi-modular setup makes it possible to combine soft actuators with various twisting degrees and achieve exact gripping. Through pneumatic control, the target object is precisely grasped and rotated at various angles, enabling rotation of the target object in three dimensions.

The point cloud is a form of three-dimensional data that comprises various detailed features at multiple scales. Due to this characteristic and its irregularity, point cloud analysis based on deep learning is challenging. While previous works utilize the sampling-grouping operation of PointNet++ for feature description and then explore geometry by means of sophisticated feature extractors or deep networks, such operations fail to describe multi-scale features effectively. Additionally, these techniques have led to performance saturation. And it is difficult for standard MLPs to directly "mine" point cloud. To address above problems, we propose the Detail Activation (DA) module, which encodes data based on Fourier transform after sampling and grouping. We activate the channels at different frequency levels from low to high in the DA module to gradually recover finer point cloud details. As training progresses, the proposed Point-MDA can uncover local and global geometries of point cloud progressively. Our experiments show that Point-MDA achieves superior classification accuracy, outperforming PointNet++ by 3.3% and 7.9% in terms of overall accuracy on the ModelNet40 and ScanObjectNN dataset, respectively. Furthermore, it accomplishes this without employing complicated operations, while exploring the full potential of PointNet++.

The abundance of information in academic articles, reports, and studies can make it challenging for researchers to gain insights from the existing literature. To address this issue, there is a growing demand for tools that can help researchers effectively parse and analyze large volumes of data. One such tool is DataDiscoveryLab, a software system that utilizes computer vision algorithms and NLP techniques to parse academic articles into text and figures, creating three separate databases. These databases allow researchers to quickly identify articles that may be relevant to their research questions, gain a deeper understanding of the research presented, and analyze visual data. The integration of article mining and computer vision in the DataDiscoveryLab software system provides researchers with a powerful tool for navigating the vast amount of scientific literature available today. Yet, as we will discuss in the latter papers these databases’ purpose is to create a bridge between researchers’ data and practically unlimited scientific publications. Yet, in this article, we will discuss how we plan to do that, and our efforts on integrating deep learning modes. After all, unlike already existing AI models, DataDiscoveryLab can be their combination and the first Generative AI in academia that can encompass every part of the natural sciences.

Newly build offshore wind farms (OWFs) render a collision risk between ships and installations. The paper proposed a real-time traffic monitoring method based on machine vision and deep learning technology to improve the efficiency and accuracy of the traffic monitoring system in the vicinity of offshore wind farms. Specifically, the method employs real automatic identification system (AIS) data to train a machine vision model, which is then used to identify passing ships in OWF waters. Furthermore, the system utilizes stereo vision techniques to track and locate the positions of passing ships. The method is tested in offshore water in China to validate its reliability. The results prove that the system sensitively detects the dynamic information of the passing ships, such as the distance between ships and OWFs, ship speed and course. Overall, this study provides a novel approach to enhancing the safety of OWFs, which is increasingly important as the number of such installations continues to grow. By employing advanced machine vision and deep learning techniques, the proposed monitoring system offers an effective means of improving the accuracy and efficiency of ship monitoring in challenging offshore environments.

Identifying a vestibular source of pathology in patients complaining of post-traumatic brain injury (TBI) dizziness can be difficult. We describe a possible new method utilizing the reduction of post-TBI symptom (including dizziness) with the use of a noise cancellation device (NCD). This retrospective case series included patients with TBI and dizziness presenting to a binocular vision specialty clinic, who were diagnosed with a vertical heterophoria (VH). If they did not respond adequately to microprism lenses and/or if they experienced hyperacusis, they were evaluated with an NCD. If there was marked reduction of TBI symptoms (including dizziness), the patients were referred to a neuro-otologist for vestibular diagnostic evaluation and treatment. Fourteen patients were identified and found to have abnormalities on vestibular testing consistent with Third Mobile Window Disorder (TMWD). All were treated with a 6-week medical protocol (diuretics, no straining, low sodium/no caffeine diet). Five responded positively, requiring no further treatment. Nine required surgical intervention and responded positively. In conclusion, 14 patients with post-concussive dizziness and VH, a positive response to NCD was associated with abnormal vestibular testing, a diagnosis of TMWD, and symptom reduction/resolution with a medical or surgical approach. The removal of sound resulting in reduction or resolution of vestibular symptoms represents an Inverse Tullio Phenomena.

Vision-based tactile sensors (VBTS) have become the de facto method of giving robots the ability to obtain tactile feedback from their environment. Unlike other solutions to tactile sensing, VBTS offers high spatial resolution feedback without compromising on instrumentation costs or incurring additional maintenance expenses. However, conventional cameras used in VBTS have a fixed update rate and output redundant data, leading to computational overhead downstream. In this work, we present a neuromorphic vision-based tactile sensor (N-VBTS) that employs observations from an event-based camera for contact angle prediction. Particularly, we design and develop a novel graph neural network, dubbed TactiGraph, that asynchronously operates on graphs constructed from raw N-VBTS streams exploiting their spatiotemporal correlations to perform predictions. Although conventional VBTS uses an internal illumination source, TactiGraph is reported to perform efficiently in both scenarios, with and without an internal illumination source. Rigorous experimental results revealed that TactiGraph achieved a mean absolute error of 0.62∘ in predicting the contact angle and was faster and more efficient than both conventional VBTS and other N-VBTS, with lower instrumentation costs. Specifically, N-VBTS requires only 5.5% of the compute-time needed by VBTS when both are tested on the same scenario.

Computer vision methods have shown to be effective in classifying garbage into recycling categories for waste processing, existing methods are costly, imprecise, and unclear. To tackle this issue, we introduce MWaste, a mobile application that uses computer vision and deep learning techniques to classify waste materials as trash, plastic, paper, metal, glass or cardboard. Its effectiveness was tested on various neural network architectures and real world images, achieving an average precision of 92% on the test set. This app can help combat climate change by enabling efficient waste processing and reducing the generation of greenhouse gases caused by incorrect waste disposal.

Increased screen time during the COVID-19 pandemic and the accelerated incorporation of technology into daily practices have directly contributed to the increased prevalence of eye disease. The human eye is not built for continual technological use and must adjust to exposure to digital screens. Overexposure to technology can result in eye strain and an increased risk of eye diseases such as myopia with complications later in life, including retinal tears, cataracts, and macular degeneration. As vision and ocular diseases become more prevalent, the study of eye diseases, including their causes and treatments, is especially relevant. There is a need to prevent ocular diseases before they become a burden to individuals, their families, and their communities by better understanding the pathophysiology of ocular diseases. Several ocular diseases are thought to be caused by imbalances in reactive oxygen species (ROS) and autophagy. ROS refers to a class of highly reactive oxygen-containing molecules that can undergo damaging reactions with other molecules in the cell. Autophagy is a cellular process of self-eating whereby damaged, harmful, or dead material in cells is broken down to maintain cellular homeostasis. This paper discusses the role of ROS and autophagy in the pathogenesis of computer vision syndrome (CVS), digital eye strain (DES), myopia, cataracts, glaucoma, diabetic retinopathy (DR), age-related macular degeneration (AMD), optic nerve crush injury (OCN), optic nerve gliomas, and retinoblastoma. Lifestyle changes such as limiting screen time, ensuring adequate exposure to evening sunlight, and using blue light protection measures or protective eyewear are important in the prevention of eye disease. Nutrition also contributes significantly to eye health. A balanced diet rich in vitamins and antioxidants may help to prevent ROS and autophagy imbalance-induced eye disease. Medical and surgical treatments become necessary when preventative measures fail. This paper also addresses how government measures to decrease the onset of eye disease, including targeted programs to increase access to fresh vegetables and fruits in food deserts, are a critical macro-level avenue to reduce eye disease prevalence in the US, which according to the National Eye Institute, costs $139 billion yearly.

Existing one-shot neural architecture search (NAS) methods have to conduct a search over a giant super-net, which leads to the huge computational cost. To reduce such cost, in this paper, we propose a method, called FTSO, to divide the whole architecture search into two sub-steps. Specifically, in the first step, we only search for the topology, and in the second step, we search for the operators. FTSO not only reduces NAS’s search time from days to 0.68 seconds, but also significantly improves the found architecture's accuracy. Our extensive experiments on ImageNet show that within 18 seconds, FTSO can achieve a 76.4% testing accuracy, 1.5% higher than the SOTA, PC-DARTS. In addition, FTSO can reach a 97.77% testing accuracy, 0.27% higher than the SOTA, with nearly 100% (99.8%) search time saved, when searching on CIFAR10.

Document deskewing is a fundamental problem in document image processing. While existing methods have limitations, such as Hough Line Transformation that can deskew images upside down, and Deep Learning models that require huge amounts of human labour and computational resources and still fail to deskew while taking care of orientation, OCR-based methods also struggle to read text when it is tilted. In this paper, we propose a novel, simple, cost-effective deep learning method for fixing the skew and orientation of documents. Our approach reduces the search space for the machine learning model to predict whether an image is upside down or not, avoiding the huge search space of predicting an angle between 0 and 360. We finetuned a MobileNetV2 model, which was pre-trained on imagenet, using only 200 images and achieve good results. This method is useful for automation-based tasks, such as data extraction using OCR technology, and can greatly reduce manual labour.

In the realm of computer vision, semantic segmentation is the task of recognizing objects in images at the pixel level. This is done by performing a classification of each pixel. The task is complex and requires sophisticated skills and knowledge about the context to identify objects’ boundaries. The importance of semantic segmentation in many domains is undisputed. In medical diagnostics, it simplifies the early detection of pathologies, thus mitigating the possible consequences. In this work, we provide a review of the literature on deep ensemble learning models for polyp segmentation and we develop new ensembles based on convolutional neural networks and transformers. The development of an effective ensemble entails ensuring diversity between its components. To this end, we combine different models (HarDNet-MSEG, Polyp-PVT, and HSNet) trained with different data augmentation techniques, optimization methods, and learning rates, which we experimentally demonstrate to be useful to form a better ensemble. Most importantly, we introduce a new method to obtain the segmentation mask which is more suitable for combining transformers in an ensemble. In our extensive experimental evaluation, the proposed ensembles exhibit state-of-the-art performance.