Vehicle logo detection plays a crucial role in various computer vision applications, such as vehicle classification and detection. In this research, we propose an improved vehicle logo detection method leveraging the MAMBA structure. The MAMBA structure integrates multiple attention mechanisms and bidirectional feature aggregation to enhance the discriminative power of the detection model. Specifically, we introduce the Multi-Head Attention for Multi-Scale Feature Fusion (MHAMFF) module to capture multi-scale contextual information effectively. Moreover, we incorporate the Bidirectional Aggregation Mechanism (BAM) to facilitate information exchange between different layers of the detection network. Experimental results on a benchmark dataset (VLD-45 dataset) demonstrate that our proposed method outperforms baseline models in terms of both detection accuracy and efficiency. Furthermore, extensive ablation studies validate the effectiveness of each component in the MAMBA structure. Overall, the proposed MAMBA-based vehicle logo detection approach shows promising potential for real-world applications in intelligent transportation systems.

The agricultural industry prioritizes crop protection, particularly against threats like pink bollworm infestations in cotton crops. This paper presents a comprehensive solution using the Robot Operating System (ROS) to deploy an autonomous robot for detecting pink bollworm infestations and tracking red illnesses. Drones collect multispectral photos of cotton fields, aiding path planning via Normalized Difference Vegetation Index (NDVI) computations. A bespoke dataset of RGB and multispectral images trained a YOLOv8 model for object detection, achieving a mean Average Precision (mAP) of 67.1%, Precision of 67.9%, and Recall of 61.8%. The robot was tested in two separate cotton fields, with rows around 20 meters long and the camera at 60 cm height for optimal real-time detection. The findings, compared to hand counts and confirmed against NDVI estimates, showed an average accuracy of 76.04% for B.T. Cotton and 67.54% for Organic Cotton fields. This study demonstrates the effectiveness of the proposed system in accurately detecting and managing crop threats. Combining ROS, drones, NDVI calculations, and a robust detection model, it provides a scalable approach for autonomous crop monitoring and protection, with the potential for broader agricultural adoption.

Reconstructing 3D indoor scenes from 2D images has always been an important task in computer vision and graphics applications. For indoor scenes, traditional 3D reconstruction methods suffer from the lack of surface details, poor reconstruction of large flat surface textures and areas with uneven lighting, and many falsely reconstructed floating debris noises in the reconstructed models. We add adaptive normal priors to the neural implicit reconstruction process to optimize the network, and improve the accuracy of volume density prediction by adding regularization terms to the neural radiation field to constrain the volume density obtained by weight distribution, and learn a smooth SDF surface from the network to obtain an explicit mesh model. Experiments show that the method proposed in this paper outperforms the state-of-the-art methods on ScanNet, Hypersim, and Replica datasets.

With the increasing demand for security surveillance, assisted driving, and remote sensing, target detection networks with rich environmental perception capabilities and high detection accuracy have become a research hotspot. However, detection techniques based on single-modality images face limitations in environmental adaptability, as they are easily affected by lighting conditions, as well as obstacles such as smoke, rain, and vegetation, leading to information loss and failing to meet the required accuracy. To address these issues, we propose a target detection network—IV-YOLO, which integrates features from both visible and infrared images. We built a dual-branch fusion network based on YOLOv8 (You Only Look Once v8) that combines infrared and visible images for target detection. On this foundation, we designed a Bidirectional Pyramid Feature Fusion structure (Bi-Fusion) to fully integrate complementary features from different modalities, reducing detection errors caused by feature redundancy and extracting fine-grained features of small targets through dual-modal fusion. Additionally, we developed the Shuffle-SPP structure, which combines channel and spatial attention mechanisms to enhance focus on deep features and further aggregate upsampling, extracting richer features. To improve the model’s expressive power, we designed a loss function tailored for multi-scale target detection boxes, accelerating the network’s convergence during training. Experimental results show that the proposed IV-YOLO network achieves an average precision (mAP) of 74.6% on the UAV remote sensing dataset, 75.4% on the KAIST pedestrian dataset, and 84.8% on the FLIR dataset. While ensuring detection accuracy, our model significantly reduces computational load, making it well-suited to meet real-time requirements. This architecture is expected to find broad application in fields such as autonomous driving, public safety, and others.

Crowd accident surveys have shown that regardless of the initial triggering factors, pedestrian fall behaviour is the most critical factor causing and aggravating crowd accidents in public traffic areas (PTAs). Application of pedestrian fall behaviour detection methods in PTAs is significant. Once deployed, it would prevent many pedestrians from losing life in crowed traffic areas accidents. However, most existing methods is still focused on medical assistance for the elderly. Therefore, this paper conducted bibliometric and content analyses, combining fall detection-related keywords to retrieve from internationally recognized literature databases and benchmark pedestrian behaviour datasets. Based on the analysis of the state-of-the-art (SOTA) achievements in fall detection methods, the fall detection methods were classified into different categories according to the research approach. Among them, it undertakes a comprehensive analysis of five predominant methods, namely computer vision, Internet of Things, smartphone, kinematic, and wearable device-based methods. Furthermore, the benchmark datasets including fall scenarios were introduced and compared. Finally, it provides a detailed discussion of existing fall detection methods, and possible future directions are identified considering application requirements in PTAs. This overview may help researchers understand the SOTA fall detection methods and devise new methodologies by improving and synthesizing the highlighted issues in PTAs.

In the world of digital avatars, three-dimensional (3D) human models are utilized to replicate real-world appearance and movements with a higher degree of realism compared to traditional 2D representations. Currently, extensive person-specific data capturing and 3D artists are required to create photorealistic avatars of existing people. Determining the 3D pose and articulation of an avatar from 2D images requires complex algorithms, a capability commonly found in the Visual Effects (VFX) industry but not readily available elsewhere. Hence, this study proposes an approach to generate 3D avatar from a 2D image of a person with uncanny resemblance and an accurate depiction of the subject’s likeness. The proposed approach, PIFu+CycleGAN, combines Pixel-aligned Implicit Function (PIFu) and cycleGAN for textured avatar construction. PIFu is specifically used to capture intricate details and arbitrary topology. Additionally, the hourglass architecture-based module is utilized for T-pose estimation for predicting the initial geometry and shape. Evaluating the proposed approach on the benchmark RenderPeople dataset, it outperforms the state-of-the-art models with values of 1.53 and 1.50 for the Chamfer and P2S distance, respectively. This indicates the creation of high-quality 3D meshes with promising textures, which are of exceptional quality and suitable for animation.

Background: Ice formation on power transmission lines presents significant risks, including structural damage and power outages. Traditional methods for detecting ice on these lines have limitations, particularly in challenging environmental conditions. This study aims to enhance ice detection accuracy by introducing a novel image segmentation and edge detection methodology using improved multi-scale Retinex and advanced morphological operations. Method: The proposed method integrates multi-stage image processing techniques, including image enhancement, grayscale conversion, thresholding, segmentation, object isolation, and edge detection. High-resolution binocular cameras were used to capture images of ice formations on transmission lines. The effectiveness of the method was assessed through a series of metrics, including accuracy, sensitivity, specificity, and precision, and compared with existing methodologies. Results: The proposed method demonstrated superior performance in both ice detection and thickness measurement. The method effectively identified and isolated ice formations under various conditions with an average accuracy of 98.35%, sensitivity of 91.63%, specificity of 99.42%, and precision of 96.03%. Additionally, the analysis of ice thickness measurements further highlighted the method’s accuracy, with a notably low root mean squared error (RMSE) of 1.20 mm and mean absolute error (MAE) of 1.10 mm, along with a strong R-squared value of 0.95. These statistical metrics show that the proposed method not only outperforms previous methods in consistency and alignment with manual measurements but also stands out as the most reliable option for tasks requiring precise measurements, offering significant advancements in accuracy and reliability over existing methodologies. Conclusions: The enhanced image segmentation and edge detection approach provides a more accurate and reliable method for monitoring ice formations on power transmission lines.

Detecting small objects in images poses significant challenges due to their limited pixel representation and the difficulty in extracting sufficient features, often leading to missed or false detections. To address these challenges and enhance detection accuracy, this paper presents an improved small object detection algorithm, CRL-YOLOv5. The proposed approach integrates the CBAM attention mechanism into the C3 module of the backbone network, which enhances the localization accuracy of small objects. Additionally, the Receptive Field Block (RFB) module is introduced to expand the model's receptive field, thereby fully leveraging contextual information. Furthermore, the network architecture is restructured to include an additional detection layer specifically for small objects, allowing for deeper feature extraction from shallow layers. When tested on the VisDrone2019 small object dataset, CRL-YOLOv5 achieved an mAP50 of 39.2%, representing a 5.4% improvement over the original YOLOv5, effectively boosting the detection precision for small objects in images.

Image captioning aims at generating meaningful verbal descriptions of a digital image. Our paper focuses on the classic encoder-decoder deep learning model that consists of several components – sub-networks, each performing a separate task that, combined, form an effective caption generator. We investigate image feature extractors, recurrent neural networks, word embedding models, and word generation layers and discuss how each component influences the captioning model’s overall performance. Our experiments are performed on the MS COCO 2014 dataset. The results help design efficient models with optimal combinations of their components.

Agriculture stands as the cornerstone of Africa’s economy, supporting over 60% of the continent’s labor force. Despite its significance, the quality assessment of agricultural products remains a challenging task, particularly at large scale, consuming valuable time and resources. The African plum is an agricultural fruit that is widely consumed across West and Central Africa but remains underrepresented in AI research. In this paper, we collected a dataset of 2,892 African plum samples from fields in Cameroon representing the first dataset of its kind for training AI models. The dataset contains images of plums annotated with quality grades. We then trained and evaluated various state-of-the-art object detection and image classification models, including YOLOv5, YOLOv8, YOLOv9, Fast R-CNN, Mask R-CNN, VGG16, Detectron-121, MobileNet and ResNet, on this African plum dataset. Our experimentation resulted in mean average precision scores ranging from 88.2% to 89.9% and accuracies between 86% and 91% for the object detection models and the classification models respectively. We then performed model pruning to reduce model sizes while preserving performance, achieving up to 93.6% mean average precision and 99.09% accuracy after pruning YOLOv5, YOLOv8 and ResNet by 10-30%. We deployed the high-performing YOLOv8 system in a web application, offering an accessible AI-based quality assessment tool tailored for African plums. To the best of our knowledge, this represents the first such solution for assessing this underrepresented fruit, empowering farmers with efficient tools. Our approach integrates agriculture and AI to fill a key gap.

In recent years, the High Dynamic Range (HDR) image has gained widespread popularity 1 across various domains such as security, multimedia, and biomedical fields, owing to its ability 2 to deliver an authentic visual experience. However, the extensive dynamic range and rich detail 3 in HDR images present challenges in assessing their quality. Therefore, current efforts involve 4 constructing subjective databases and proposing objective quality assessment metrics to achieve 5 efficient HDR Image Quality Assessment (IQA). Recognizing the absence of a systematic overview 6 of these approaches, this paper provides a comprehensive survey of existing HDR IQA methods. 7 Furthermore, this paper aims to serve as a valuable resource for researchers by discussing the 8 limitations of current methodologies and potential research directions in the future.

Form deviation generated during the milling profile process poses challenges to the precision and functionality of industrial fixtures and product manufacturing across various sectors. The inspection of contour profile quality often relies on commonly employed contact methods for measuring form deviation, but these methods frequently face limitations that can impact the reliability and overall accuracy of the inspection process. This paper introduces a novel approach, the novel intelligent inspection method (NIIM), developed to accurately inspect and categorize contour profiles in machined parts manufactured through the milling process by computer numerical control (CNC) machines. The NIIM integrates a calibration piece, a vision system (RAM−StarliteTM), and machine learning techniques to analyze the line profile and classify the quality of contour profile deformation generated during CNC milling. The calibration piece is specifically designed to identify form deviations in the contour profile during the milling process. The RAM−StarliteTM vision system captures contour profile images corresponding to curves, lines, and slopes. An algorithm generates a profile signature, extracting Fourier descriptor features from the contour profile to analyze form deviations when compared to an image reference. A feed-forward neural network is employed to classify contour profiles based on quality properties. Experimental evaluations involving 60 machined calibration pieces, resulting in 356 images for training and testing, demonstrate the accuracy and computational efficiency of the proposed NIIM for profile line tolerance inspection. Results demonstrate that the NIIM offers 96.99% accuracy, low computational requirements, 100% inspection capability, quality classification.

The computer program "Histropy" is an interactive Python program for the quantification of selected features of two-dimensional (2D) images/patterns (in either JPG/JPEG, PNG, GIF, BMP, or baseline TIF/TIFF formats) by means of calculations based on the pixel intensities in this data, their histograms, and user-selected sections of those histograms. The histograms of these images display pixel-intensity values along the x-axis (of a 2D Cartesian plot), with the frequency of each intensity value within the image represented along the y-axis. The images need to be of 8-bit or 16-bit information depth and can be of arbitrary size. (Up to 1024 pixels maximum on both sides is recommended as larger images tend to significantly slow the program’s performance). Histropy generates an image’s histogram surrounded by a graphical user interface that allows one to select any range of image-pixel intensity levels, i.e. sections along the histograms’ x-axis, using either the computer mouse or numerical text entries. The program subsequently calculates the (so-called Monkey Model) Shannon entropy and root-mean-square contrast for the selected section and displays them as part of what we call a "histogram-workspace-plot." To support the visual identification of small peaks in the histograms, the user can switch between a linear and log-base-10 display scale for the y-axis of the histograms. Pixel intensity data from different images can be overlaid onto the same histogram-workspace-plot for visual comparisons. The visual outputs of the program can be saved as histogram-workspace-plots in the PNG format for future usage. The source code of the program and a brief user manual are published in the supporting materials as well as on GitHub . Instead of taking only 2D images as inputs, the program’s functionality could be extended by a few lines of code to other potential uses employing data tables with one or two dimensions in the CSV format.

The advent of large models, also known as foundation models, has significantly transformed the AI research landscape, with models like Segment Anything (SAM) achieving notable success in diverse image segmentation scenarios. Despite its advancements, SAM encountered limitations in handling some complex low-level segmentation tasks like camouflaged object and medical imaging. In response, in 2023, we introduced SAM-Adapter, which demonstrated improved performance on these challenging tasks. Now, with the release of Segment Anything 2 (SAM2)—a successor with enhanced architecture and a larger training corpus—we reassess these challenges. This paper introduces SAM2-Adapter, the first adapter designed to overcome the persistent limitations observed in SAM2 and achieve new state-of-the-art (SOTA) results in specific downstream tasks including medical image segmentation, camouflaged (concealed) object detection, and shadow detection. SAM2-Adapter builds on the SAM-Adapter's strengths, offering enhanced generalizability and composability for diverse applications. We present extensive experimental results demonstrating SAM2-Adapter's effectiveness. We show the potential and encourage the research community to leverage the SAM2 model with our SAM2-Adapter for achieving superior segmentation outcomes. Code, pre-trained models, and data processing protocols are available at http://tianrun-chen.github.io/SAM-Adaptor/.

Time-of-Flight (ToF) cameras are subject to high levels of noise and errors due to Multi-Path-Interference (MPI). To correct these errors, algorithms and neuronal networks require training data. However, the limited availability of real data has led to the use of physically simulated data, which often involves simplifications and computational constraints. The simulation of such sensors is an essential building block for hardware design and application development. Therefore, the simulation data must capture the major sensor characteristics. This work presents a learning-based approach that leverages high-quality laser scan data to generate realistic ToF camera data. The proposed method employs MCW-Net (Multi-Level Connection and Wide Regional Non-Local Block Network) for domain transfer, transforming laser scan data into the ToF camera domain. Different training variations are explored using a real-world dataset. Additionally, a noise model is introduced to compensate for the lack of noise in the initial step. The effectiveness of the method is evaluated on reference scenes to quantitatively compare to physically simulated data.

Target detection technology plays a crucial role in fishery ecological monitoring, fishery diversity research, and intelligent aquaculture. Deep learning, with its distinct advantages, provides significant convenience to the fishery industry. However, it still faces various challenges in practical applications, such as significant differences in image species and image blurring. To address these issues, this study proposes a multi-scale, multi-level, and multi-stage cross-domain feature fusion model. In order to train the model more effectively, a new dataset called Fish52 (Multi-scene Fish dataset) was constructed, on which the model achieved an mAP of 82.57%. Furthermore, we compared prevalent one-stage and two-stage detection methods on the Lahatan (single-scene fish dataset) and Fish30 dataset and tested them on the F4k and FishNet dataset. The mAP of our proposed model on the Fish30, Lahatan, F4k, and FishNet datasets reaches 91.72%, 98.7%, 88.6%,and 81.5% respectively, outperforming existing mainstream models. Comprehensive empirical analysis indicates that our model possesses high generalization ability and has reached advanced performance levels. In this study, the depth of the model backbone is deepened, a novel neck structure is proposed, and anew module is embedded therein. To enhance the fusion ability of the model, a new attention mechanism module is introduced. In addition, in the adaptive decoupling detection header module, introducing classes with independent parameter sand regression adapters reduces interaction between different tasks. The proposed model can better monitor fishery resources and enhance aquaculture efficiency. It not only provides an effective approach for fish detection but also has certain reference significance for the identification of similar target sin other environments, and offers assistance for the construction of smart fisheries and digital fisheries.

In this paper, we study facial expression recognition (FER) using three modalities obtained from the Light Field camera; sub-aperture (SA), depth map, and all-in-focus (AiF) images. Our objective is to construct a more comprehensive and effective FER system by investigating multimodal fusion strategies. For this purpose, we employ EfficientNetV2-S, pre-trained on AffectNet, as our primary convolutional neural network. This model, combined with a BiGRU, is used to process SA images. We evaluate various fusion techniques at both decision and feature levels to assess their effectiveness in enhancing FER accuracy. Our findings show that the model using SA images surpasses state-of-the-art performance, achieving 88.13% ± 7.42% accuracy under the subject-specific evaluation protocol and 91.88% ± 3.25% under the subject-independent evaluation protocol. These results highlight our model's potential in enhancing FER accuracy and robustness, outperforming existing methods. Furthermore, our multimodal fusion approach, integrating SA, AiF and depth images, demonstrates substantial improvements over unimodal models. The decision-level fusion strategy, particularly using average weights, proved most effective, achieving 90.13% ± 4.95% accuracy under the subject-specific evaluation protocol and 93.33% ± 4.92% under the subject-independent evaluation protocol. This approach leverages the complementary strengths of each modality, resulting in a more comprehensive and accurate FER system.

To date, the development of unmanned technologies using autonomous underwater vehicles (AUV) has become an urgent demand for solving the problem of inspection of industrial sub-sea structures. A key issue here is precise localization of AUVs relative to underwater objects. However, the well-known challenges of recognizing and coordinating underwater objects (impossibility to use GPS, undercurrents, etc.) do not allow high-precision navigation if based solely on a standard suite of AUV navigation tools (sonars, etc.). An alternative is a technology of processing of optical images that, at short distances, can provide higher accuracy of AUV navigation compared to a technology of acoustic measurement processing. In this study, in the context of the problem of subsea production system inspection, we propose a method to recognize underwater objects and provide coordinate reference of AUV to them, based on stereo-image processing. Its distinctive features are the use of a non-standard technique for generating a geometric model of an object from its views (foreshortening) taken from positions of a pre-made overview trajectory, the use of various characteristic geometric elements when recognizing objects, and the original algorithms for comparing visual data of the inspection trajectory with an a priori model of the object.

The integration of sign language recognition systems into digital platforms has the potential to bridge communication gaps between the deaf community and the broader population. This paper introduces an advanced Israeli Sign Language (ISL) recognition system designed to interpret dynamic motion gestures, addressing a critical need for more sophisticated and fluid communication tools. Unlike conventional systems that focus solely on static signs, our approach incorporates both Deep Learning and Computer Vision techniques to analyze and translate dynamic gestures captured in real-time video. We provide a comprehensive account of our preprocessing pipeline, detailing every stage from video collection to the extraction of landmarks using MediaPipe, including the mathematical equations used for preprocessing these landmarks and the final recognition process. The dataset utilized for training our model is unique in its comprehensiveness and is publicly accessible, enhancing the reproducibility and expansion of future research. The deployment of our model on a publicly accessible website allows users to engage with ISL interactively, facilitating both learning and practice. We discuss the development process, the challenges overcome, and the anticipated societal impact of our system in promoting greater inclusivity and understanding.

Livestock marketplaces consist of infrastructure and rules through which information is exchanged and transactions are completed. Livestock trading incurs significant transaction costs, beyond just commission fees paid to marketplace owners (e.g., se-lecting, loading and transporting animals). In addition, animals being transported, held and transacted in a market setting increases the likelihood of injury or disease. The Covid-19 pandemic contributed to large-scale changes across typically physical marketplaces as stay-at-home orders and travel restrictions were invoked. Livestock marketplaces pivoted online rapidly, facilitated by the deployment of video feeds, al-lowing buyers to view/purchase animals via mobile phone. This shift offered the promise of mitigating many of the transaction costs associated with a fully physical marketplace. This study evaluated how computer vision (CV) approaches can con-tribute to the evolution of livestock marketplaces. It used livestock video data for 240 cattle (bulls) weighing 320-740 Kg from 16 different breeds to elicit additional trade-relevant data (livestock weight) and can be considered as an intermediary step to better understand how video data can be used to further digitalize livestock trade, price formation and transaction completion via technology solutions. The dataset comes from three sales held on different dates in a single mart location, which allows a consistent camera setup throughout. CV technologies were employed to solve the problem of cattle weight estimation and a distinctive achievement lies in curating a dataset cap-turing cattle in motion within the context of a cattle auction thereby injecting realism and complexity into the CV weight estimation task. Employing advanced object detection algorithms, the study created an intricate data pipeline to generate an appropriate modelling dataset and devised new multi-channel architectures for deeper analysis of diverse image inputs. The significance of sizable datasets and the challenge of man-aging inherent input variability for accurate CV regression is highlighted. Top-down view predictions demonstrate comparatively better performance, although failing to achieve benchmark precision. Challenges arise from camera angle variations, roaming behaviors, and breed diversity. Likewise, side-view predictions face challenges due to increased variability and lack of standardization. Despite this, these predictions offer insights into the value of incorporating Region of Interest (ROI) coordinates and data augmentation for enhanced performance. The study concludes with a multi-input ar-chitecture combining top-down and side-view predictions, demonstrating moderate success highlighting potential synergy between both perspectives. The central con-clusion resonates strongly – using standardized and multi-view images encompassing controlled movement and consistent poses could bolster accurate CV weight estima-tion.

The metaverse is gradually expanding. There is a growing number of photo and video recordings of metaverse virtual worlds being used in multiple domains, with growing collections. An essential element of the metaverse and its recordings are avatars. In this paper, we present a novel task of avatar detection in metaverse recordings, supporting semantic retrieval in collections of metaverse recordings and other use cases. Our work addresses the characterizations and definitions of avatars and presents a new model supporting avatar detection. The latest object detection algorithms are trained and tested on a variety of avatar types in metaverse recordings. Our work achieves a higher level of accuracy than existing models, which encourages further research in this field.

Accurate and robust deep learning models for facial expression recognition are challenging to achieve, given the diversity of human faces and variations in images, including different facial poses and lighting conditions. In this work, we proposed a clock-driven convolutional Spiking Neural Network (SNN) and STDP fine-tune architecture, meticulously calibrated its hyperparameters, and experimented with various optimization methods. The best model resulted was trained and evaluated on the Fer2013 and FER+ database, obtaining an accuracy of 61.87% and 79.97% without requiring auxiliary training data or face registration. To our best knowledge, the proposed SNN achieved comparable accuracy to CNNs of similar depth and possessed the advantages of low energy consumption and high computational efficiency. The computational efficiency of the proposed SNN is approximately three times that of CNNs. Along with this, we introduced the very recent cumulative spike guided encoder visualization technique and revealed the strong encoding capability of the proposed SNN.

With the development of educational technology, machine learning and deep learning provide technical support for traditional classroom observation assessment. However, in real classroom scenarios, the technique faces challenges such as lack of clarity of raw images, complexity of datasets, multi-target detection errors and complexity of character interactions.Based on the above problems, a student classroom behavior recognition network incorporating super-resolution and target detection is proposed. To cope with the problem of unclear original images in the classroom scenario, SRGAN (Super Resolution Generative Adversarial Network for Images) is used to improve the image resolution and thus the recognition accuracy.To address the dataset complexity and multi-targeting problems, feature extraction is optimized and multi-scale feature recognition is enhanced by introducing AKConv and LASK attention mechanisms into the Backbone module of YOLOv8s algorithm. To improve the character interaction complexity problem, the CBAM attention mechanism is integrated to enhance the recognition of important feature channels and spatial regions. Experiments show that it can detect six behaviors of students raising their hands, reading, writing, playing cell phones, looking down, and lying on the table in high-definition images. And the accuracy and robustness of this network are verified.Compared with small target detection algorithms such as Faster R-CNN, YOLOv5, and YOLOv8s, this network has better detection performance and can efficiently deal with the behavior recognition of multiple students.

The task of human action recognition (HAR) based on skeleton data is a challenging yet crucial technique owing to its wide-ranging applications in numerous domains, including patient monitoring, security surveillance, and observation of human-machine interactions. While numerous algorithms have been proposed in an attempt to distinguish between a myriad of activities, most practical applications necessitate highly accurate detection of specific activity types. This study proposes a novel and highly accurate spatiotemporal graph autoencoder network for HAR based on skeleton data. Furthermore, an extensive investigation was conducted employing diverse modalities. To this end, a spatiotemporal graph autoencoder was constructed to automatically learn both spatial and temporal patterns from human skeleton datasets. The powerful graph convolutional network, designated as GA-GCN, developed in this study, notably outperforms the majority of existing state-of-the-art methods when evaluated on two common datasets, namely NTU RGB+D and NTU RGB+D 120. On the first dataset, the proposed approach achieved accuracies of 92.3\% and 96.8\% for the cross-subject and cross-view evaluations, respectively. On the more challenging NTU RGB+D 120 dataset, GA-GCN attained accuracies of 88.8\% and 90.4\% for the cross-subject and cross-set evaluations, respectively.

The symptoms of multiple sclerosis (MS) are determined by the location of demyelinating lesions in the white matter of the brain and spinal cord. Currently, magnetic resonance imaging (MRI) is the most common tool for diagnosing MS, understanding the course of the disease, and analyzing the effects of treatments. However, undesirable components may appear during the generation of MRI, such as noise or intensity variations. Mathematical morphology (MM) is a powerful image analysis technique that helps to filter the image and extract relevant structures. Granulometry is an image measurement tool of MM that determines the size distribution of objects in an image without explicitly segmenting each object. While, several methods have been proposed for the automatic segmentation of MS lesions in MRI, in some cases only simple data preprocessing, such as image resizing to standardize the input dimensions, has been performed before the algorithm training. Therefore, this paper proposes an MRI preprocessing algorithm performing elementary morphological transformations in brain images of MS patients and healthy individuals to delete undesirable components and extract the relevant structures such as MS lesions. Also, the algorithm computes the granulometry in MRI to describe the size qualities of lesions and trains two artificial neural networks (ANN) to predict MS diagnosis. The computing of differences in granulometry measurements of an image with MS lesions and a reference image (without lesions) can determine the size characterization of the lesions. Then, the ANNs were evaluated with the validation set and the performance results (test accuracy = 0.9753, and cross-entropy loss = 0.0247) show the proposed algorithm can support the decision of specialists for diagnosing MS and estimating the disease progress, based on granulometry values.

The present work titled QRGB consists of the development of a Python application for generating QR codes using the additive color generation method (RGB). This innovative method increases the information density stored in QR codes by using three color layers (red, green, and blue), each representing a different data set. QRGB offers an efficient and secure solution for storing and transmitting large amounts of information in limited spaces, significantly improving the capabilities of traditional black-and-white QR codes.By using three color layers (red, green, and blue), QRGB codes can store up to three times more information in the same space. This technique not only increases storage capacity but also enhances information security, making it more difficult to counterfeit or manipulate the code. The superposition of multiple data layers allows for redundancy implementation, increasing the code's robustness against damage or reading errors.QRGB codes are especially useful in applications that require the transmission of large amounts of data in limited spaces, such as in the packaging industry, digital business cards, and interactive advertising. Additionally, they have great potential in areas such as document and ticket security, where the authenticity and integrity of information are crucial.These points provide a solid foundation for understanding the innovation and advantages of colored QR codes (QRGB) compared to traditional QR codes, highlighting their applicability and potential in various sectors.The Python application developed for QRGB utilizes advanced color image processing techniques, applying models and methods to generate and decode QR codes using the RGB color system, thus enhancing information density and security through the innovative use of additive color layers.

Color distortion in an image poses a challenge for classification and regression when the input data consists of pictures. Because of that, a new algorithm for color standardization of photos is proposed, which serves as a base for a deep neural network regression model. This approach utilizes a custom color template, which was developed based on an initial series of studies and digital imaging. Using the equalized histogram of R, G, B channels from the digital template and its photo, a color mapping strategy is computed. By applying this approach, the histograms are adjusted, and the colors of photos taken with a smartphone are standardized. The proposed algorithm has been developed for a series of photos where the entire surface roughly maintains a uniform color, and the differences in color between the photographs of individual objects are minor. The optimized approach was validated in the colorimetric determination procedure of vitamin C. The dataset for deep neural network in the regression variant was formed from photos of samples under two separate lighting conditions. For the concentration range of vitamin C from 0 to 87.72 µg·mL-1, the RMSE for the test set ranged between 0.75 and 1.95 µg·mL-1, in comparison to the non-standardized variant, where this indicator was at the level of 1.48-2.29 µg·mL-1. The consistency of the predicted concentration results with actual data expressed using R2 was between 0.9956 and 0.9999, for each of the standardized variants. This approach allows for the removal of light reflections on the shiny surface of solutions, which is a common problem in liquid samples. The color matching algorithm has a universal character, and its application scope is not limited.

Object detection algorithms for open water aerial images present challenges such as small object size, unsatisfactory detection accuracy, numerous network parameters, and enormous computational demands. Current detection algorithms struggle to meet the accuracy and speed requirements while being deployable on small mobile devices. This paper proposes DFLM-YOLO, a lightweight small-object detection network based on the YOLOv8 algorithm with multiscale feature fusion. Firstly, to solve the class imbalance problem of the SeaDroneSee dataset, we propose a data augmentation algorithm called Small Object Multiplication (SOM). SOM enhance dataset balance by increasing the number of objects in specific categories, thereby improving model accuracy and generalization capabilities. Secondly, we optimize the backbone network structure by implementing Depthwise Separable Convolution (DSConv) and the newly designed FC-C2f, which reduces the model's parameters and inference time. Finally, we design the Lightweight Multiscale Feature Fusion Network (LMFN) to address the challenges of multiscale variations by gradually fusing the four feature layers extracted from the backbone network in three stages. In addition, LMFN incorporates the Dilated Re-param Block structure to increase the effective receptive field and improve the model's classification ability and detection accuracy. Experimental results on the SeaDroneSea dataset indicate that DFLM-YOLO improves mAP by 12.4% compared to the original YOLOv8s, while reducing parameters by 67.2%. This achievement provides a new solution for UAVs to conduct object detection missions in open water efficiently.

This article explores the efficiency of various clustering methods for image segmentation under different luminosity conditions. Image segmentation plays a crucial role in computer vision appli-cations, and clustering algorithms are commonly used for this purpose. The search for an adaptive clustering mechanism aims to ensure maximum symmetry of real objects with objects/segments in their digital representations. However, clustering methods performance can fluctuate with varying lighting conditions during image capture. Therefore, we assess the performance of several clustering algorithms — including k-means, K-Medoids, fuzzy c-means, Possibilistic C-Means, Gus-tafson-Kessel, Entropy-based Fuzzy, Riddler-Calvard, Kohonen Self-Organizing Maps, and Meanshift — across images captured under different illumination conditions. Additionally, we develop an adaptive image segmentation system utilizing empirical data. Conducted experiments highlight varied performance among clustering methods under different luminosity conditions. This research enhance better understanding of luminosity's impact on image segmentation and aiding in method selection for diverse lighting scenarios.

In recent years, research on adversarial attack techniques for remote sensing object detection (RSOD) has made great progress. Still, most of the research nowadays is on end-to-end attacks, which mainly design adversarial perturbations based on the prediction information of the object detectors (ODs) to achieve the attack. These methods do not discover the common vulnerabilities of the ODs and thus the transferability is weak. Based on this, this paper proposes a foreground feature approximation (FFA) method to generate adversarial examples (AEs) that discover the common vulnerabilities of the ODs by changing the feature information carried by the image itself to implement the attack. Specifically, firstly, the high-quality predictions are filtered as attacked objects using the detector, after which a hybrid image without any target is made, and the hybrid foreground is created based on the attacked targets. The images' shallow features are extracted using the backbone network, and the features of the input foreground are approximated towards the hybrid foreground to implement the attack. In contrast, the model predictions are used to assist in realizing the attack. In addition, we have found the effectiveness of FFA for targeted attacks, and replacing the hybrid foreground with the targeted foreground can realize targeted attacks. Extensive experiments with seven rotating ODs on the RSOD datasets DOTA and UCAS-AOD show that FFA achieves both targetless and targeted attacks with a high success rate and transferability.

Traffic sign detection plays a pivotal role in autonomous driving systems. The intricacy of the detection model necessitates high-performance hardware. Real-world traffic environments exhibit considerable variability and diversity, posing challenges for effective feature extraction by the model. Therefore, it is imperative to develop a detection model that is not only highly accurate but also lightweight. In this paper, we proposed YOLO-ADual, a novel lightweight model. Our method leverages the C3Dual and Adown lightweight modules as replacements for CPS and CBL modules in YOLOv5. The Adown module effectively mitigates feature loss during downsampling while reducing computational costs. Meanwhile, C3Dual optimizes the processing power for kernel feature extraction, enhancing computation efficiency while preserving network depth and feature extraction capability. Furthermore, the inclusion of the CBAM module enables the network to focus on salient information within the image, thus augmenting its feature representation capability. Our proposed algorithm achieves a mAP@0.5 of 70.1% while significantly reducing the number of parameters and computational requirements to 51.83% and 64.73% of the original model, respectively. Compared to various lightweight models, our approach demonstrates competitive performance in terms of both computational efficiency and accuracy.

This paper highlights the importance of one-shot learning from prototype SKU images for efficient product recognition in retail and inventory management. Traditional methods require large supervised datasets to train deep neural networks, which can be costly and impractical. One-shot learning techniques mitigate this issue by enabling classification from a single prototype image per product class, thus reducing data annotation efforts. We introduce the variational prototyping-encoder (VPE), a novel deep neural network for one-shot classification. Utilizing a support set of prototype SKU images, VPE learns to classify query images by capturing image similarity and prototypical concepts. Unlike metric learning-based approaches, VPE pre-learns image translation from real-world object images to prototype images as a meta-task, facilitating efficient one-shot classification with minimal supervision. Our research demonstrates that VPE can significantly reduce the need for large datasets while accurately classifying query images into their respective categories, providing a practical solution for product classification tasks.

Traditional 3D reconstruction models have consistently encountered a challenge: attaining high recall of object edges while preserving precision. In this paper, we introduce a post-processing method Multi-Head Attention Refiner (MA-R) aimed at tackling this challenge by integrating a multi-head attention mechanism within the U-Net style refiner module. The 3D reconstruction model applying this method demonstrates excellence in parsing intricate image details, resulting in significant enhancement of boundary predictions and increased recall rates. In our experiments, our method significantly enhances the reconstruction performance of Pix2Vox++ when multiple images are utilized as input. Specifically, it achieves an enhanced Intersection Over Union score of approximately 1.1% compared to the original Pix2Vox++ model when 16-view images are employed.

Strip steel surface defect recognition research has important research significance in industrial production. Aiming at the problems of defect feature extraction difficulty, slow detection speed and insufficient dataset, YOLOv5 is improved on the basis of YOLOv5, and YOLO-LFPD (Lightweight fine particle detection) model is proposed. By introducing the RepVGG (Re-param VGG) module, the robustness of the model to the surface defect features of strip steel is enhanced, and the expressive ability of the model is improved. FasterNet is used to replace the backbone network, which ensures the accuracy and accelerates the inference speed, making the model more suitable for real-time monitoring. The use of pruning, GA genetic algorithm with OTA loss function method further reduces the model size while better learning the strip steel defect feature information, thus improving the generalisation ability and accuracy of the model. The experimental results show that the introduction of the RepVGG module and the use of FasterNet can well improve the model performance, with a reduction of 48% in the number of parameters, a reduction of 13% in the number of GFLOPs, an inference time of 77% of the original, and an optimal accuracy compared with the network models in recent years. The experimental results on the NEU-DET dataset show that the accuracy of YOLO-LFPD is improved by 3% to 81.2%, which is better than other models, and provides new ideas and references for the lightweight strip steel surface defect detection scenarios and application deployment.

Detecting cracks in structural health monitoring is crucial for ensuring infrastructure safety and longevity. Using drones to obtain crack images and automate processing can improve the efficiency of crack detection. To address the challenges posed by the limited computing resources of edge devices in practical applications, we propose CrackScopeNet, a lightweight segmentation network model that simultaneously considers local and global crack features, suitable for deployment on drone platforms with limited computational power and memory. CrackScopeNet features a multi-scale branch to improve sensitivity to cracks of varying sizes without substantial computational overhead and a stripe-wise context attention mechanism to enhance the capture of long-range contextual information, mitigating the interference from complex backgrounds. Experimental results on CrackSeg9k dataset demonstrate that CrackScopeNet leads to a significant improvement in prediction performance, with the highest mean intersection over union (mIoU) scores reaching 82.12%, while maintaining a lightweight architecture with only 1.05M parameters and 1.58G floating point operations (FLOPs). Besides, CrackScopeNet excels in inference speed on edge devices without GPU because of its low FLOPs. CrackScopeNet contributes to the development of efficient and effective crack segmentation networks, suitable for practical structural health monitoring applications using drone platforms.

Synthetic Aperture Radar has been extensively used in numerous fields and can gather a wealth of information about the area of interest. This large-scene data-intensive technology puts a high value on automatic target recognition (ATR) which can free the utilizers and boost the efficiency. Recent advances in artificial intelligence have made it possible to create a deep learning-based SAR ATR that can automatically identify target features from massive input data. In the last 6 years, intensive research has been conducted in this area, however, most papers in the current SAR ATR field used recurrent neural network (RNN) and convolutional neural network (CNN)-varied models to deepen the regime’s understanding of the SAR images. To equip SAR ATR with updated deep learning technology, this paper tries to apply a lightweight vision transformer (LViT)-based model to classify SAR images. The entire structure was verified by an open- accessed SAR data set and recognition results show that the final classification outcomes are robust and more accurate in comparison with referred traditional network structures without even using any convolutional layers.

The rapid advancements in satellite technology have led to a significant increase in high-resolution remote sensing (RS) images, necessitating advanced processing methods. Additionally, a patent analysis revealed a significant increase in deep learning and machine learning applications in remote sensing, highlighting the growing importance of these technologies. Therefore, this paper introduces the Kolmogorov-Arnold Network (KAN) model to remote sensing, aiming to enhance efficiency and performance in RS applications. We conducted several experiments to validate KAN's applicability, starting with the EuroSAT dataset, where we combined the KAN layer with multiple pretrained CNN models. The optimal performance was achieved with ConvNeXt, leading to the development of the KonvNeXt model. KonvNeXt was evaluated on the Optimal-31, AID, and Merced datasets for validation, and it achieved accuracies of 90.59%, 94.1%, and 98.1%, respectively. The model also showed fast processing speed, with the Optimal-31 and Merced datasets completed in 107.63 seconds each, while the bigger and more complicated AID dataset took 545.91 seconds. This result is meaningful since it achieved faster speeds and comparable accuracy compared to the existing study which utilized VIT and proved KonvNeXt's applicability for remote sensing classification tasks. Furthermore, we investigated the model's interpretability by utilizing Occlusion Sensitivity and by displaying the influential regions, it validated its potential use in a variety of domains including medical imaging and weather forecasting. This paper is meaningful in that it is the first use of KAN in remote sensing classification, proving its adaptability and efficiency.

Gastric cancer is the fifth most common and fourth deadliest cancer worldwide, with a bleak 5-year survival rate of about 20%. Despite significant research into its pathobiology, prognostic predictability remains insufficient due to pathologists’ heavy workloads and the potential for diagnostic errors. Consequently, there is a pressing need for automated and precise histopathological diagnostic tools. This study leverages Machine Learning and Deep Learning techniques to classify histopathological images into healthy and cancerous categories. By utilizing both handcrafted and deep features and shallow learning classifiers on the GasHisSDB dataset, we conduct a comparative analysis to identify the most effective combinations of features and classifiers for differentiating normal from abnormal histopathological images without employing fine-tuning strategies. Our methodology achieves an accuracy of 95% with the SVM classifier, underscoring the effectiveness of feature fusion strategies. Additionally, cross-magnification experiments produced promising results with accuracies close to 80% and 90% when testing the models on unseen testing images with different resolutions.

Public safety and intelligent surveillance systems critically depend on anomaly behavior detection for effective monitoring. In real-world pedestrian detection scenarios, prevalent challenges such as missed detections, complex background interference, and small target sizes hinder accurate anomaly identification. To address these issues, this study introduces YOLO-ABD, a lightweight method for anomaly behavior detection that integrates small target detection and channel shuffling. This method employs YOLOv8n as the baseline model, incorporating a small target detection mechanism in the Head part and utilizing GSConv convolution in the Backbone to enhance perceptual capability. Additionally, the SimAM attention mechanism is integrated to mitigate complex background interference, thereby improving target detection performance. Evaluation on the IITB-Corridor dataset demonstrated mAP50 and mAP50-95 scores of 89.3% and 60.6%, respectively. Generalization testing on the street-view-gdogo dataset further highlighted the superiority of YOLO-ABD over advanced detection algorithms, underscoring its effectiveness and generalizability. With a relatively small parameter count, YOLO-ABD presents an excellent lightweight solution for pedestrian anomaly behavior detection.

In this paper, we present a multi-task model that predicts disparities and confidence levels in deep stereo matching simultaneously. We do this by combining its successful model for each separate task and obtaining a multitask model that can be trained with a proposed loss function. We show the advantages of this model compared to training and predicting disparity and confidence sequentially. This method enables an improvement of 15% to 30% in the area under the curve (AUC) metric when trained parallel rather than sequential. In addition, the effect of weighting the components in the loss function on the stereo and confidence performance is investigated. By improving the confidence estimate, the practicality of stereo estimators for creating distance images is increased.

Infrared and visible light image fusion technology integrates image feature information from two different modalities to generate an image that integrates complementary information from the source images. However, in low-light scenarios, the degradation of lighting in visible light images leads to existing fusion methods being unable to effectively extract texture detail information from the scene. At this time, the target prominence information provided by infrared images alone is far from sufficient. To address this challenge, this paper proposes a lightweight infrared and visible light image fusion method based on low-light enhancement, named LLE-Fuse. Firstly, the method improves the MobileOne Block by embedding the Sobel operator in the Edge-MobileOne Block to perform feature extraction and downsampling on the source images. Then, intermediate features of different scales are obtained and fused through a cross-modal attention fusion module, followed by image reconstruction using a decoder with upsampling. Next, the CLAHE algorithm is used for image enhancement of infrared and visible light images, and the enhanced loss guides the network model to learn the capability of low-light enhancement. Finally, upon completion of the network training, the method optimizes the Edge-MobileOne Block into the direct connection structure of MobileNetV1 through structural re-parameterization, effectively reducing the computational resource consumption of the model network. Through extensive experimental comparisons, this paper's method demonstrates excellent performance in both visual appeal and evaluation metrics, providing satisfactory fusion results even in low-light scenarios.

Visualization of volumetric data holds great importance, particularly in medical and biomedical applications. Various imaging techniques, such as FMRI and 3D microscopy, are employed to generate volumetric data, used in medical practice, research, and teaching. Commonly utilized tools like 3D Slicer, Fiji, and MATLAB ® aid in rendering and analyzing 3D images. However, these tools may lack comprehensive rendering functionality and face challenges in handling computational demands as data sizes grow. To address these limitations, this work introduces a GPU-supported renderer with a MATLAB ® interface. This solution gives the user flexible control over rendering parameters and optimizes data transfer through a memory management system. By leveraging the computational power of NVIDIA GPUs, the renderer enables complex and high-quality renderings, enhancing speed and efficiency. It therefore facilitates the analysis and visualization of volumetric data within an integrated environment, namely MATLAB ®, streamlining their workflows. This advancement provides valuable opportunities for researchers and medical professionals to explore and comprehend volumetric data effectively.

Multi-threshold image segmentation is a relevant research field in further higher-level image preprocessing and computer vision. This paper adopts the Bayesian Forecasting Evolutionary Algorithm (BFEA) for natural scenery image segmentation using multilevel thresholding. An update method for the prediction vector is designed for image segmentation. The design idea of the BFEA algorithm for multilevel thresholding is based on the basic principle of Evolutionary Computation, the probability distribution of promising solutions, and the Bayesian theorem. BFEA could effectively solve the curse of dimensionality. Extensive experiments have demonstrated that the proposed algorithm dominated the state-of-the-art population-based thresholding method in term of the quality of image, function fitness value, optimal threshold value, SSIM value, PSNR value, computation time, dimensionality issue and convergence rate. The proposed method is very effective for color image segmentation.

In computer vision, human action recognition is a hot topic, popularized by the development of deep learning. Current models have achieved high accuracy results on public datasets. Despite this success, they require significant computational resources for training. Given that transfer learning based techniques allow reusing what other models have already learned and training models with less computational resources, in this work we propose using a transfer learning based approach for action recognition in videos. We describe a methodology for human action recognition using transfer learning techniques in a custom dataset. The proposed method consists of four stages: 1) human detection and tracking, 2) video preprocessing, 3) feature extraction (using pretrained models with ImageNet), and 4) action recognition using a two-stream model consisting of TCNs, LSTMs, and CNNs layers. The custom dataset is imbalanced with 189, 390, 490, 854, and 890 videos per class, respectively. For feature extraction, we analyzed the performance of seven pretrained models: Inception-v3, MobileNet-v2, MobileNet-v3-L, VGG-16, VGG-19, Xception, and ConvNeXt-L. We show that the best results were obtained with the last one. Finally, using pretrained models for feature extraction allowed training in a PC with a single GPU with an accuracy of 94.9%.

In this study, we investigate how dataset composition influences the performance and bias of age estimation models across different ethnic groups. We utilized pre-trained Convolutional Neural Networks (CNNs) such as VGG19, fine-tuning them on two datasets: UTKFace and APPA-REAL. UTKFace comprises 23,705 samples (12,391 males, 11,314 females) including 10,078 White, 4,526 Black, and 3,434 Asian individuals. APPA-REAL consists of 7,591 samples (3,818 males, 3,773 females) with 6,686 White, 231 Black, and 674 Asian individuals. We adjusted dataset composition by oversampling minority groups and reducing samples from the majority group to evaluate effects on model performance, measured using Mean Absolute Error (MAE) and standard deviation. Our findings demonstrate that oversampling alone does not guarantee equitable performance across ethnicities; reducing samples from the majority group often resulted in more balanced performance, evidenced by lower MAE standard deviations. These results underscore the importance of a nuanced approach to achieve fairness, including combining various sampling techniques. This study emphasizes the critical need for tailored dataset compositions to mitigate bias and suggests exploring more refined data processing methods and algorithmic adjustments for enhanced model fairness and accuracy in facial recognition technologies.

Drought stress severely affects the normal growth and development of wheat, leading to reduced yields and quality. To address the lag and limitations of traditional drought monitoring methods, this paper proposes a drought stress monitoring model for winter wheat during critical growth stages based on deep learning. By acquiring drought stress images of winter wheat during three critical growth stages: rise-jointing, heading-flowering and flowering-maturity, a dataset correlating these images with soil moisture monitoring data was established. The DenseNet121 network model was selected as the base network to extract phenotypic features of winter wheat under drought stress. Variables such as the model's training approach, changes in learning rate, and whether an attention mechanism was included, were used to conduct eight sets of model training and optimization strategy experiments. A drought stress monitoring model for winter wheat during critical growth stages based on an improved DenseNet-121 was constructed. The results showed that the average recognition accuracy of the eighth group in the test set reached 94.67%, indicating good application prospects in the recognition of drought stress levels during critical growth stages of wheat.

Real time camouflaged object detection so far using CAMO or COD10K datset has been able to achieve mAP50 less than 64%. We present a fourier transfom and Milkyway filament extraction based method extracting shape and edges of camouflaged object, achieving 88%+ mAP50 while continuing to deliver real time detection using YOLOv8s based models. This allows for search and rescue of wilelife and humans in smoke, fog, natural disasters possible by combining low compute Yolov8s model on headup display for rescue workers such as firefighters, first reponders etc.

Aiming at the problem that dynamic targets in indoor environments lead to low accuracy and large errors in the localization and position estimation of visual SLAM systems and the inability to build maps containing semantic information, a semantic visual SLAM algorithm based on the semantic segmentation network DeepLabV3+ and LK optical flow is proposed based on ORB-SLAM2 system. First, the dynamic target feature points are detected and rejected based on the lightweight semantic segmentation network DeepLabV3+ and LK optical flow method. second, the static environment occluded by the dynamic target is repaired using the time-weighted multi-frame fusion background repair technique. Lastly, the filtered static feature points are used for feature matching and position calculation. Meanwhile, the semantic labeling information of static objects obtained based on the lightweight semantic segmentation network DeepLabV3+ is fused with the static environment information after background repair to generate dense point cloud maps containing semantic information, and the semantic dense point cloud maps are transformed into semantic octree maps using the octree spatial segmentation data structure. The localization accuracy of the visual SLAM system and the construction of the semantic maps are verified using the TUM RGB-D widely used dataset and real scene data, respectively. The experimental results show that the proposed semantic visual SLAM algorithm can effectively reduce the influence of dynamic targets on the system, has a higher localization accuracy, and compared with other advanced algorithms, such as DynaSLAM, has the highest performance in indoor dynamic environments while considering both localization accuracy and real-time performance. In addition, semantic maps can be constructed so that the robot can better understand and adapt to the indoor dynamic environment.

This paper introduces a novel framework for zero-shot learning (ZSL), i.e., to recognize new categories that are unseen during training, by distilling knowledge from foundation models. Specifically, we first employ ChatGPT and DALL-E to synthesize reference images of unseen categories from text prompts. Then, the test image is aligned with text and reference images using CLIP and DINO. Finally, the predicted logits are aggregated according to their confidence to produce the final prediction.Experiments are conducted on multiple datasets, including CIFAR-10, CIFAR-100, and TinyImageNet. The results demonstrate that our model can significantly improve classification accuracy compared to previous approaches, achieving AUROC scores above 96\% across all test datasets. Our code is available at https://github.com/1134112149/MICW-ZIC.

Early disease detection is crucial for maximizingcrop yields and minimizing financial losses in agriculture. Inorder to categorise leaf diseases in pepper, tomato, and potato plants, this study suggests a revolutionary CNN design. The algorithm collects disease-specific features from input photos by using data augmentation techniques to artificially enlarge the training dataset. ReLU-activated convolutional layers gradually capture features at different levels, from low to high, while max pooling layers minimise spatial dimensionality. After thecharacteristics are extracted, a fully-connected layers with a softmax activation function classify them into illness groups. A well chosen dataset containing labelled healthy and sick leaves is used to train and validate the model. Generalizability to untesteddata is ensured via performance evaluation on an independent testing set. The field of deep learning for agricultural applications benefits from this research. This strategy has the ability to rev- olutionise agricultural practices by enabling automated disease identification. This might result in early interventions, better crop health, and eventually sustainable agricultural production.

Existing methods for simulating missile dynamics and training often rely on three degrees of freedom (3DOF) motions, limiting their ability to represent the complex maneuvers and targeting strategies of modern missiles. This paper proposes integrating virtual reality (VR) technology with six degrees of freedom (6DOF) dynamics to enhance the accuracy and efficiency of missile simulation and training. By incorporating 6DOF dynamics, simulations can capture the full range of missile motion, providing a comprehensive understanding of their capabilities and limitations. Through VR, users can immerse themselves in realistic missile scenarios, interact with dynamic behaviors in real-time, and refine their targeting and interception skills. The integration of VR with 6DOF dynamics also facilitates real-time feedback and evaluation, enhancing training efficiency and proficiency in missile operations. This approach aims to revolutionize missile training by bridging the gap between traditional simulation methods and real-world dynamics.

This paper presents computer and color vision research focusing on human color perception in VR environments. A VR art gallery with digital twins of original artworks is created for this experiment. In this research, the field of colorimetry and the application of CMYK and RGB color models are applied. The interrelationships of the two colors models are applied to create a color modification of the VR art gallery environment using C# Script procedures. This color-edited VR environment works with a smooth change of color tone in a given time interval. At the same time, a sudden change in the color of the RGB environment is defined in this interval. This experiment aims to record a user's reaction embedded in a VR environment and the effect of color changes on human perception in a VR environment. The research uses lie detector sensors that record the physiological changes of the user embedded in VR. Five sensors are used to record the signal. An experiment on the influence of the user's color perception in a VR environment using lie detector sensors has never been conducted. This research defines the basic methodology for analyzing and evaluating the recorded signals from the lie detector. The presented text thus provides a basis for further research in the field of colors and human color vision in a VR environment and lays an objective basis for use in many scientific and commercial areas.

Fluorescence microscopic images of cells contain a large number of morphological features that are used as an unbiased source of quantitative information about cell status, through which researchers can extract quantitative information about cells and study the biological phenomena of cells through statistical and analytical analysis. As an important research object of phenotypic analysis, images have a great influence on the research results. Saturation artifacts present in the image result in a loss of grayscale information that does not reveal the true value of fluorescence intensity. From the perspective of data post-processing, we propose a two-stage cell image recovery model based on generative adversarial network to solve the problem of phenotypic feature loss caused by saturation artifacts. The model is capable of restoring large areas of missing phenotypic features. In the experiment, we adopt the strategy of progressive restore to improve the robustness of the training effect, and add the contextual attention structure to enhance the stability of the restore effect. We hope to use deep learning methods to mitigate the effects of saturation artifacts to reveal how chemical, genetic, and environmental factors affect cell state, providing an effective tool for studying the field of biological variability and improving image quality in analysis.

While cycling presents environmental benefits and promotes a healthy lifestyle, the risks associated with overtaking maneuvers by motorized vehicles pose significant hindrances for many potential cyclists. Large-scale analysis of overtaking maneuvers could inform traffic researches and city planners on how to reduce these risks by better understanding these maneuvers. Drawing from the fields of sensor-based cycling research and of LiDAR-based traffic data sets, this paper provides a step towards addressing these safety concerns by introducing the Salzburg Bicycle 3d (SaBi3d) data set, consisting of LiDAR point clouds capturing car-to-bicycle overtaking maneuvers. The data set, collected using a LiDAR-equipped bicycle, facilitates detailed analysis of a large quantity of overtaking maneuvers without the need for manual annotation through enabling automatic labeling by a neural network. Additionally, a benchmark result for 3d object detection using a competitive neural network is provided as a baseline for future research. The SaBi3d data set is structured identically to the nuScenes data set, and therefore offers compatibility with numerous existing object detection systems. This work provides valuable resources for future researchers to better understand cycling infrastructure and mitigate risks, thus promoting cycling as a viable mode of transportation.

Computational animation of spatiotemporal models for 3-D shape and motion is considered as core of dynamic real-time vision. This is based on sets of features evaluated from multi-focal image streams. Subjects are defined as special objects capable of sensing environmental parameters and of initiating own actions in combination with stored knowledge. Object/subject recognition and scene understanding are achieved on different levels. Multiple objects are tracked individually for perceiving their actual state (‘here and now’). Fast saccadic jumps in gaze direction allow flexible concentration on objects or parts of actual interest. By analyzing motion of relevant objects/subjects over a larger time scale on the level of state variables and documenting them in the ‘scene tree representation’ (computer graphics), the situation with respect to decision making is assessed. Various behavioral capabilities of subjects are represented on an abstract level for characterizing their potential behaviors. These are generated by stereotypical feed-forward and feedback control applications on a separate level close to the actuator hardware with corresponding methods. This dual representation on one level for decision making and one on the implementation level allows for flexibility and easy adaptation or extension. Results are shown for road vehicle guidance based on three cameras on a gaze-controlled platform.

The fusion of global contextual information with local cropped block details is crucial for segmenting ultra-high resolution images. In this study, we introduce a novel fusion mechanism termed Global-Local Deep Fusion (GL-Deep Fusion) based on an enhanced transformer architecture, which efficiently integrates global contextual information and local details. Specifically, we propose the Global-Local Synthesis Networks (GLSNet), a dual-branch network where one branch processes the entire original image, while the other branch handles cropped local patches as input. The feature fusion of different branches in GLSNet is achieved through GL-Deep Fusion, significantly enhancing the accuracy of ultra-high resolution image segmentation. Particularly effective is GLSNet in identifying tiny overlapping items. To optimize GPU memory utilization, we meticulously design a dual-branch architecture that proficiently leverages the features it extracts, seamlessly integrating them into the enhanced transformer framework of GL-Deep Fusion. Extensive experiments conducted on challenging benchmarks, including DeepGlobe and Vaihingen datasets, demonstrate that GLSNet achieves a new state-of-the-art performance in terms of GPU memory utilization and segmentation accuracy tradeoff.

This paper presents a novel segmentation algorithm specially developed for applications in 3D point clouds with high variability and noise, particularly suitable for heritage building 3D data. The method can be categorized within the segmentation procedures based on edge detection. In addition, it uses a graph-based topo-logical structure generated from the supervoxelization of the 3D point clouds, which is used to make the clo-sure of the edge points and to define the different segments. The algorithm provides a valuable tool for generating results that can be used in subsequent classification tasks and broader computer applications dealing with 3D point clouds. One of the characteristics of this segmentation method is that it is unsupervised, which makes it particularly advantageous for heritage applications where labelled data is scarce. It is also easily adaptable to different edge point detection and supervoxelization algorithms. Finally, the results show that the 3D data can be segmented into different architectural elements, which is important for further classification or recognition. Extensive testing on real data from historic buildings demonstrates the effectiveness of the method. The results show superior performance compared to three other segmentation methods, both globally and in the segmentation of planar and curved zones of historic buildings.

Sign language recognition technology can help people with hearing impairments to communicate with those who are hearing impaired. At present, with the rapid development of society, deep learning technology also provided certain technical support for sign language recognition work. In sign language recognition tasks, the use of traditional convolutional neural networks to extract spatio-temporal features from sign language videos suffers from insufficient feature extraction, resulting in low recognition rates. Nevertheless, video-based sign language datasets are very large and require a lot of computational resources for training and generalisation must be ensured, which poses a challenge for recognition. In this paper, we have presented a video-based sign language recognition method based on resnet and lstm. As the number of network layers increases, the ResNet network can effectively solve the granularity explosion problem and obtain better time series features. We use the Resnet convolutional network as the backbone model. At the initialisation stage, we obtain sign language features using ResNet; then, the learned feature space is used as the input of LSTM network to obtain long sequence features. The experimental results show that the accuracy of the above model is better than the mainstream model, and it can effectively extract the spatio-temporal features in sign language videos and improve the recognition rate of sign language actions.

Fire is a serious security threat that can lead to casualties, property damage, and environmental damage. However, despite the availability of object detection algorithms, challenges persist in detecting fires and smoke. These challenges include slow convergence speed, poor performance in detecting small targets, and high computational cost limiting deployments. In this paper, Fire smoke and human detection based on ConvNeXt and Mixed encoder (FCM-DETR), an end-to-end object detection algorithm based on Deformable DETR, is proposed. Firstly, we introduce ConvNeXt to take the place of Resnet, which greatly reduces the amount of computation and improves the ability to extract irregular flame features. Secondly, to effectively process multi-scale features, the original encoder is decoupled into two modules. Then Mixed encoder, an innovative encoder structure, is proposed, resulting in excellent performance of multi-scale fire and smoke features fusion. What can’t be overlooked is that the encoder block is compatible with any DETR-based models. Finally, the convergence speed is accelerated and the mean of average precision is improved by applying a novel loss function called Powerful IoU v2. The experimental results indicate that our model achieves the best detection accuracy compared to other models, with mAP reaching 66.7%, mAPs and Accuracyfire achieving impressive 50.2% and 98.05%, respectively.

Some of the barriers preventing Virtual Reality (VR) from being widely adopted are the cost and unfamiliarity of VR systems. Here we propose that in many cases the specialized controllers shipped with most VR head-mounted displays can be replaced by a regular smartphone, cutting the cost of the system, and allowing users to interact in VR using a device they are already familiar with. To achieve this, we developed SmartVR Pointer, an approach that uses smartphones as a replacement for the specialized controllers for two essential operations in VR: selection and navigation by teleporting. In SmartVR Pointer a camera mounted on the head-mounted display (HMD) is tilted downwards so that it points to where the user will naturally be holding their phone in front of them. SmartVR Pointer supports three selection modalities: tracker-based,gaze-based, and combined/hybrid. In the tracker-based SmartVR pointer selection we use image-based tracking to track a QR code displayed on the phone screen and them map the phone’s position to a pointer shown within the field of view of the camera in the Virtual Environment. In the gaze-based selection modality the user places the pointer using their gaze and taps on the phone for selection. The combined technique is a hybrid between head-gaze-based interaction in VR and smartphone-based Augmented Reality. It allows the user to control a VR pointer that behaves like a mouse pointer by moving their smartphone to select objects within the virtual environment, and to interact with the selected objects using the smartphone’s touch screen. The touchscreen is used for selection and dragging. SmartVR Pointer is simple and requires no calibration and no complex hardware assembly or disassembly. We demonstrate successful interactive applications of SmartVR Pointer in a VR environment with a demo where the user navigates in the virtual environment using teleportation points on the floor and then solves a Tetris-style key-and-lock challenge.

The event camera generates an event stream based on changes in brightness, retaining only the characteristics of moving objects, addresses the high power consumption associated with using high-frame-rate cameras for high-speed eye tracking tasks. However, the asynchronous incremental nature of event camera output has not been fully utilized, and there are also issues related to missing event datasets. Combining the temporal information encoding and state-preserving properties of spiking neural network (SNN) with event camera, a near-range eye tracking algorithm is proposed as well as a novel event-based dataset for validation and evaluation. According to experimental results, the proposed solution outperforms artificial neural network (ANN) algorithms, while computational time remains only 12.5\% of that of traditional SNN algorithms. Furthermore, the proposed algorithm allows for self-adjustment of time resolution, with a maximum achievable resolution of 0.081ms, enhancing tracking stability while maintaining accuracy.

Medical image segmentation is a critical step in various healthcare applications, aiding in diagnosis, treatment planning, and disease monitoring. In this study, we investigate the efficacy of a segmentation approach based on the 2D wavelet transform. Leveraging a dataset comprising 10 diverse medical images, we evaluate the performance of our segmentation method using three key metrics: accuracy, precision, and recall. Our findings demonstrate that the proposed approach enhances segmentation accuracy, offering promising results compared to existing methods. By harnessing the multi-resolution feature extraction capabilities of the 2D wavelet transform, our method achieves improved delineation of medical image structures, paving the way for more accurate and efficient healthcare interventions.

Real-time object detection remains a pivotal topic in the realm of computer vision. Balancing the accuracy and speed of object detectors poses a formidable challenge for both academic researchers and industry practitioners. While recent transformer-based models have showncase the prowess of the attention mechanism, delivering remarkable performance improvements over CNNs, their computational demands can hinder their effectiveness in real-time detection tasks. In this paper, we elect to use YOLOX as our robust starting point and introduce a series of effective enhancements, culminating in a new high-performance detector named YOLOAX. To further exploit the power of the attention mechanism, we devise multi-dimensional attention-based modules that activate CNNs, emphasizing the most pertinent regions and bolstering the capacity to learn the most informative image representations from feature maps. Moreover, we introduce a novel leading label assignment strategy called STA, along with a groundbreaking loss function named GEIOU Loss, to further refine our detector's performance. We provide extensive ablation studies on the COCO and PASCAL VOC 2012 detection datasets to validate our proposed methods. Remarkably, our YOLOAX is trained solely on the MS-COCO dataset from scratch, without leveraging any prior knowledge, which achieves an impressive 54.2% AP on the COCO 2017 test set while maintaining a real-time speed of 72.3 fps, surpassing YOLOX by a significant margin of 3.0% AP. Our source code and pre-trained models are openly accessible at https://github.com/KejianXu/yoloax.

In response to the burgeoning global demand for premium agricultural products, particularly within the competitive nut market, this paper introduces an innovative methodology aimed at enhancing the grading process for almonds and their shells. Leveraging state-of-the-art Deep Convolutional Neural Networks (CNNs), specifically the AlmondNet-20 architecture, our study achieves exceptional accuracy exceeding 99%, facilitated by the utilization of a 20-layer CNN model. To bolster robustness in differentiating between almonds and shells, data augmentation techniques are employed, ensuring the reliability and accuracy of our classification system. Our model, meticulously trained over 1000 epochs, demonstrates remarkable performance, boasting an accuracy rate of 99% alongside a minimal loss function of 0.0567. Rigorous evaluation through test datasets further validates the efficacy of our approach, revealing impeccable precision, recall, and F1-score metrics for almond detection. Beyond its technical prowess, this advanced classification system offers tangible benefits to both industry experts and non-specialists alike, ensuring globally reliable almond classification. The application of deep learning algorithms, as showcased in our study, not only enhances grading accuracy but also presents opportunities for product patents, thereby contributing to the economic value of our nation. Through the adoption of cutting-edge technologies such as the AlmondNet-20 model, we pave the way for future advancements in agricultural product classification, ultimately enriching global trade and economic prosperity.

Within environmental perception, automatic navigation and object detection, computer vision is a crucial and demanding field with many applications in modern industries, such as multi-target long-term visual tracking in automated production, defect detection, and driverless robotic vehicles with sensing ability. The performance of computer vision has greatly improved recently thanks to developments in deep learning algorithms and hardware computing capabilities, which has spawned the creation of a large number of related applications. This paper presents the results of a detailed review of over 50 papers published over the course of two decades (1999–2024), with a primary focus on the technical advancement of computer vision. To elucidate the foundational principles, an examination of typical visual algorithms based on traditional correlation filtering was initially conducted. Subsequently, a comprehensive overview of the most recent advancements in deep learning-based computer vision techniques was compiled. Furthermore, a comparative analysis between conventional and novel algorithms was undertaken to discuss the future trends and directions of computer vision. Lastly, the feasibility of employing visual SLAM (Simultaneous Localization and Mapping) algorithms in the context of autonomous vehicles with sensing capability was explored. Furthermore, we explored a thorough and efficient architecture for utilizing these techniques in autonomous robotic vehicles into unmanned green and low-carbon smart factories, underlining novel advances as well as potential prospects for future research.

The existing segmentation-based scene text detection methods mostly need complicated post-processing, and the post-processing operation is separated from the training process, which greatly reduces the detection performance. The previous method, DBNet successfully simplified post-processing and integrated the post-processing into a segmentation network. However, the training process of the model took a long time for 1200 epochs and the sensitivity to texts of various scales was lacking, leading to some text instances being missed. Considering the above two problems, we design the text detection Network with Binarization of Hyperbolic Tangent(HTBNet). First of all, we propose Binarization of Hyperbolic Tangent (HTB), optimized along with which, the segmentation network can expedite the initial convergent speed by reducing the amount of epochs from 1200 to 600. Because features of different channels in the same scale feature map focus on the information of different regions in the image, to better represent the important features of all objects in the image, we devise the Multi-Scale Channel Attention(MSCA). Meanwhile considering that multi-scale objects in the image cannot be simultaneously detected, we propose a novel module named Fused Module with Channel and Spatial(FMCS), which can fuse the multi-scale feature maps from channel and spatial dimension. Finally we adopt cross entropy as the loss function, which measures the difference between predicted values and ground truths. The experimental results show that HTBNet compared with lightweight models has achieved competitive performance and speed on Total-Text(F-measure:86.0%, FPS:30) and MSRA-TD500 (F-measure:87.5%, FPS:30).

Over the last few decades, advancements in astrophysics have been closely linked to the development of powerful machine-learning models that can accurately classify celestial bodies. At the same time, however, many astronomical datasets are filled with new features collected by increasingly powerful telescopes. These features can cause overfitting, clouding predictive abilities, and dampening the ability of many models to classify images. Therefore, motivation to design more efficient models has skyrocketed—aiming to optimize for lower run times and high accuracies, even with fewer provided features. In our project, we seek to optimize a convolutional neural network model using a technique known as wavelet analysis. This technique allows us to pick the key features of an astronomical image and accentuate niche details, saving time and boosting accuracy. We applied it to the MiraBest Dataset, a dataset compiled from the FIRST sky survey using a virtual telescope. In the end, after training our neural network on the original images and the five filters (approximation, horizontal, vertical, diagonal, and combined), we found that with fewer features and less overfitting, the vertical Daubechies-family wavelet filter outperformed the original runs with the unaltered images by over 10%. Our findings suggest that wavelet analysis can help harvest the most valuable features in images of celestial bodies–leading to enhanced predictions in astronomical applications and perhaps bolstering modern astrophysical theory.

In high-risk railway construction, personal protective equipment monitoring is critical but challenging due to small and frequently obstructed targets. We propose YOLO-EA, an innovative model that enhances safety measure detection by integrating ECA into its backbone's convolutional layers, improving discernment of minuscule objects like hardhats. YOLO-EA further refines target recognition under occlusion by replacing GIoU with EIoU loss. YOLO-EA's effectiveness was empirically substantiated using a dataset derived from real-world railway construction site surveillance footage. It outperforms YOLOv5, achieving 98.9% precision and 94.7% recall, up 2.5% and 0.5% respectively, while maintaining real-time performance at 70.774 fps. This highly efficient and precise YOLO-EA holds great promise for practical application in intricate construction scenarios, enforcing stringent safety compliance during complex railway construction projects.

Machine learning and computer vision have proven to be valuable tools for farmers to streamline their resource utilization to lead to more sustainable and efficient agricultural production. These techniques have been applied to strawberry cultivation in the past with limited success. To build on this past work, in this study two separate sets of strawberry images, along with their associated diseases, were collected and subjected to, resizing, and augmentation. Subsequently, a combined dataset consisting of 9 classes was utilized to fine-tune three distinct pre-trained models: Vision Transformer (ViT), MobileNetV2, and ResNet18. To address the imbalanced class distribution in the dataset, each class was assigned weights to ensure nearly equal impact during the training process. To enhance the outcomes, new images were generated by removing backgrounds, reducing noise, and flipping them. The performances of ViT, MobileNetV2, and ResNet18 were compared after being selected. Customization specific to the task was applied to all three algorithms, and their performances were assessed. Throughout this experiment, none of the layers were frozen, ensuring all layers remained active during training. Attention heads were incorporated into the first 5 and last 5 layers of MobileNetV2 and ResNet18, while the architecture of ViT was modified. The results indicated accuracy factors of 98.4%, 98.1%, and 97.9% for ViT, MobileNetV2, and ResNet18, respectively. Despite the data being imbalanced, the precision, which indicates the proportion of correctly identified positive instances among all predicted positive instances, approached nearly 99% with the ViT. MobileNetV2 and ResNet18 demonstrated similar results. Overall, the analysis revealed that the Vision Transformer model exhibited superior performance in strawberry ripeness and disease classification. The inclusion of attention heads in the early layers of ResNet18 and MobileNet18, along with the inherent attention mechanism in ViT, improved the accuracy of image identification. These findings offer the potential for farmers to enhance strawberry cultivation through passive camera monitoring alone, promoting the health and well-being of the population.

Breast cancer stands as a predominant health concern for women globally. As mammography is the primary screening tool for breast cancer detection, improving the detection of breast cancer at screening could save more lives. This mammography re-view paper provides a systematic review of computer-aided techniques during a specific time frame for the segmentation and classification of microcalcification, evaluating image processing, machine learning, and deep learning techniques. The systematic review is meticulously carried out, adhering closely to the preferred reporting items for Systematic Reviews and Meta-Analysis (PRISMA) guidelines. This article focuses on mammographic breast cancer detection approaches based on automated systems, discussed chronologically from 1970 through 2023. This article encompasses the breadth of artificial intelligence-based methods from the most primitive to the most sophisticated models. Image processing and machine learning-based methods are compre-hensively reviewed. The evaluation of a deep learning architecture based on self-extracted features for classification tasks demonstrated outclass performance. Large-scale datasets required for a broader and in-depth analysis of novel methods for breast cancer detection are also discussed in this article. This research work is aligned with the United Nations' sustainability de-velopment goals.

Laser-targeted weeding methods further enhance the sustainable development of green agriculture, with one key technology being the improvement of weed localization accuracy. Here, we propose an improved YOLOv8 instance segmentation based on bidirectional feature fusion and deformable convolution (BFFDC-YOLOv8-seg) to address the challenges of insufficient weed localization accuracy in complex environments with resource-limited laser weeding devices. Initially, by training on extensive datasets of plant images, the most appropriate model scale and training weights are determined, facilitating the development of a lightweight network. Subsequently, the introduction of the Bidirectional Feature Pyramid Network (BiFPN) during feature fusion effectively prevents the omission of weeds. Lastly, the use of Dynamic Snake Convolution (DSConv) to replace some convolutional kernels enhances flexibility, benefiting the segmentation of weeds with elongated stems and irregular edges. Experimental results indicate that the BFFDC-YOLOv8-seg model achieves a 4.9% increase in precision, an 8.1% increase in recall rate, and a 2.8% increase in mAP50 value to 98.8% on a vegetable weed dataset compared to the original model. It also shows improved mAP50 over other typical segmentation models such as Mask R-CNN, YOLOv5-seg, and YOLOv7-seg by 10.8%, 13.4%, and 1.8%, respectively. Furthermore, the model achieves a detection speed of 24.8 FPS on the Jetson Orin nano standalone device, with a model size of 6.8MB that balances between size and accuracy. The model meets the requirements for real-time precise weed segmentation, suitable for complex vegetable field environments and resource-limited laser weeding devices.

In the field of visualization, understanding users’ analytical process is important to determine the effectiveness of a designed visualization application. To understand the users’ analytical process, numerous studies have been performed to capture and analyze user interactions. Although many studies have emphasized the importance of analyzing user interactions to understand users’ analytical reasoning processes, few have successfully linked these interactions to users’ reasoning processes. This paper introduces an approach that bridges this gap by correlating semantic user interactions with analysis decisions through an interactive wire transaction analysis system and a visual state transition matrix. With the designed analysis system, interactive analysis can be performed to evaluate financial fraud in wire transactions. It also allows the mapping of captured user interactions and analytical decisions back onto the visualization, revealing users’ distinct results. The visual state transition matrix further aids to help understanding users’ analytical flows, revealing their decision-making processes. Classification machine learning algorithms are applied to assess the effectiveness of our approach in understanding analysts’ strategies by connecting them to their decisions. From the study, we observed an average of 72% accuracy in clearly classifying the semantic user interactions. For classifying individual decisions, we observed an average of 70% classification accuracy. This emphasizes the importance of capturing semantic user interactions to understand users’ analytical processes. Overall, it is determined that the proposed approach not only enhances the understanding of analytical behaviors but also offers a robust approach for evaluating user interactions in visualization tools.

Outliers seriously affect the accuracy in geometric model fitting. Previous works in coping with outliers involve threshold selection and scale estimation. However most scale estimators suppose the inlier distribution as a Gaussian model, which usually poorly meets the cases in geometric model fitting. Outliers, considered as the points with big residuals to all the true models, share common items with big values in quantized residual preferences, thus making the outliers gather away from the inliers in quantized residual preference space. In this paper we make use of this outlier consensus in quantized residual preference space by extending the usage of energy minimization to combine the model error and the spatial smoothness in quantized residual preference space for outlier detection. The energy minimization based outlier detection process follows an alternate sampling and labeling framework. After the outlier detection, an ordinary energy minimization method is employed to optimize the inlier labels, which also follows the framework of alternate sampling and labeling. The experimental results in this study show that the energy minimization based outlier detection method can detect most of the outliers in the data. Furthermore, the proposed energy minimization based inlier segmentation process segments the inliers into different models quite accurately. Overall, the performance of the proposed method is better than most of the state-of-the-art methods.

In this paper, we present a method for estimating GPS signals from visual information captured by a monocular camera mounted on a fixed-wing Unmanned Aerial Vehicle. The main challenge in visual odometry from aerial images is the computation of scale due to irregularities in the elevation of the terrain. That is, it is not possible to accurately convert from pixels in the image to meters in space, and the error accumulates. The contribution of this work is to reduce the accumulated error by comparing the images from the camera with satellite images. The algorithm has been tested in real-time experiments, and the results show that it is possible to eliminate the accumulated error using satellite images.

We introduce a novel microscopic image dataset augmented with segmentation and detection labels specifically designed for microplastic analysis in sewage environments. Recognizing the increasing concern over microplastics — particles of synthetic polymers smaller than 5mm — and their detrimental effects on marine ecosystems and human health, our research focuses on enhancing detection and analytical methodologies through advanced computer vision and deep learning techniques. The dataset comprises high-resolution microscopic images of microplastics collected from sewage, meticulously labeled for both segmentation and detection tasks, aiming to facilitate accurate and efficient identification and quantification of microplastic pollution. In addition to dataset development, we present example deep learning models optimized for segmentation and detection of microplastics within complex sewage samples. The models demonstrate significant potential in automating the analysis of microplastic contamination, offering a scalable solution to environmental monitoring challenges. Furthermore, we ensure the accessibility and reproducibility of our research by making the dataset and model codes publicly available, accompanied by detailed documentation on GitHub and LabelBox. The dataset and example deep learning models are publicly available at the following GitHub link. https://anonymous.4open.science/r/Microplastics-in-Sewage-1BEF

This paper investigates the efficacy of YOLOv8 variants for vehicle detection and license plate detection within smart parking applications, emphasizing performance under varying ambient lighting conditions. The proposed system is to seize full video frames, extracts regions of interest containing vehicles, and feeds them into separate, pre-trained YOLOv8 models – one dedicated to vehicle detection and another for license plate detection. Four YOLOv8 variants, nano, small, medium, and large, are evaluated. As a pre-processing step, the images are processed with the help of OpenCV and Pillow libraries to adjust the luminosity and increase the images’ DPI so that they would be easy to perceive by the Tesseract OCR engine. Sixteen potential combinations arise from pairing the four YOLOv8 models for vehicle and license plate detection tasks. To identify the most suitable combinations, we employ the Multi-Criteria Decision-Making method, specifically Technique for Order of Preference by Similarity to Ideal Solution (TOPSIS) analysis. This analysis considers 4 critical metrics: Precision, mean average precision at 95% Intersection of Union threshold, Recall and Total Inference Time. The objective is to achieve an optimal balance between high accuracy and real-time processing. Following the selection of optimal YOLOv8 combinations through TOPSIS analysis, we assess their performance under varying ambient light intensity (measured in lux). This evaluation aims to identify the most robust model combinations that ensure accurate vehicle and license plate recognition across the diverse lighting conditions encountered in real-world environment.

We propose a novel unsupervised semantic segmentation method for fast and accurate flood area detection utilizing color images acquired from Unmanned Aerial Vehicles (UAVs). To the best of our knowledge, this is the first fully unsupervised method for flood area segmentation in color images captured by UAVs, without the need of pre-disaster images. The proposed framework addresses the problem of flood segmentation based on parameter-free calculated masks and unsupervised image analysis techniques. First, a fully unsupervised algorithm gradually excludes areas classified as non-flood utilizing calculated masks over each component of the LAB colorspace, as well an RGB vegetation index and the detected edges of the original image. Unsupervised image analysis techniques, such as distance transform, are then applied, producing a probability map for the location of flooded areas. Finally, flood detection is obtained by applying the hysteresis thresholding segmentation. The proposed method is tested and compared with variations, and other supervised methods in two public datasets, consisting of 953 color images in total, yielding high-performance results, with 87.4% and 80.9% overall accuracy and F1-Score, respectively. The results and computational efficiency of the proposed method show that it is suitable for on board data execution and decision-making during UAVs flight.

High-power laser facilities necessitate predicting incremental damage to final optics to identify evolving damage trends. In this study, we propose a surface damage detection method utilizing image segmentation employing ResNet-18, and a damage area estimation network employing U-Net++. Paired sets of online and offline images of optics obtained from a large laser facility is used to train the network. The trends of varying damage could be identified by incorporating additional experimental parameters. A key advantage of the proposed method is that the network can be trained end-to-end on small samples, eliminating the need for manual labeling or feature extraction. The software developed based on these models can facilitate the daily inspection and maintenance of optics in large laser facilities.

In Indonesia, types of meat are still identified manually due to increasing prices for beef and buf-falo ahead of Idul Fitri. Traders mix beef with pork to thwart the beef scam. Haar wavelet and GLCM, GLCM with angles of 0°, 45°, 90°, and 135°, as well as matrices using contrast, correlation, energy, homogeneity, and entropy, are used in feature extraction. The following are the results of meat image classification testing using k-NN, Haar wavelet, and GLCM: The k-NN algorithm shows superiority in identifying fresh (99%), frozen (99%), and rotten (96%) meat texture images with the most fantastic accuracy results in every situation. Meanwhile, GLCM routinely provides good results, especially regarding the texture image of fresh meat (183.21) and rotten meat (115.79). However, despite delivering less throughput than k-NN and GLCM, Haar wavelets are still helpful, especially when dealing with fresh meat texture images (89,96).

Visualising textual content could be helpful to professionals as well as amateurs across several fields. However, training a text-to-image generator in the mainstream domain requires large amounts of paired text-image and data, which is too expensive to collect since labeling millions of images and videos can be tiresome. GANs like StackGAN and StyleGAN can be considered as solutions to generate images from text. But the images generated may be of low accuracy and resolution, and the entire processing can be highly time-consuming. Moreover, image generation is a notion that is still being researched. Hence, the process of developing a Video Generation model necessitates substantial research. Despite the need for such a model, modern technology has lagged behind the solutions to this problem. This proposal suggests combining two methods, Text modification for Action Definition (TexAD) and SeQuential Image Generation for Video Synthesis (SQIGen). The proposed solution synthesises a sequence of images from textual information feeds and combines these images to create a video. TexAD uses Natural Language Processing and Deep Learning techniques to process, classify and modify text data. SQIGen is an extension of the VQGAN+CLIP neural network architecture that generates a sequence of images from the modified text data

Camouflage evaluation typically involves human visual search and detection experiments that are time-consuming and expensive. Hence, there is a need for models that compute camouflage effectiveness from digital imagery. Convolutional neural networks are a powerful tool for automatic object detection and recognition. We investigated whether such a network (YOLO) can also provide a measure of camouflage effectiveness that is related to human perception. To this end, human performance measures of camouflage effectiveness such as detection time and target conspicuity were obtained in observer experiments and compared with the performance of the (standard, i.e. pre-trained) YOLO-V4-tiny algorithm. YOLO provides the probability that a detected object is correctly classified, and this is adopted as our measure of camouflage effectiveness. We compared the YOLO-predicted classification probability for a person in camouflage clothing moving through rural and urban scenes to human detection performance. Overall, camouflage effectiveness predicted by YOLO correlates with human observer performance. At close distances, YOLO’s classification performance appears sensitive to high-contrast shape-breaking elements of a camouflage pattern. This suggests that YOLO may be adapted to assess camouflage effectiveness for a wide range of applications, such as evaluating or optimizing one’s signature and predicting optimal hiding locations in a given environment. Further research is required to fully establish YOLO’s limitations and applicability for this purpose.

Generalized zero-shot learning (GZSL) aims to simultaneously recognize both seen classes and unseen classes by training only on seen class samples and auxiliary semantic descriptions. Recent state-of-the-art methods infer unseen classes based on semantic information or synthesize unseen classes using generative models based on semantic information, all of which rely on the correct alignment of visual-semantic features. However, they often overlook the inconsistency between original visual features and semantic attributes. Additionally, due to the existence of cross-modal dataset biases, the visual features extracted and synthesized by the model may also mismatch with some semantic features, which could hinder the model from properly aligning visual-semantic features. To address this issue, this paper proposes a GZSL framework that enhances the consistency of visual-semantic features using self-distillation and disentanglement network (SDDN). The aim is to utilize self-distillation and disentanglement network to obtain semantically consistent refined visual features and non-redundant semantic features to enhance the consistency of visual-semantic features. Firstly, SDDN utilizes self-distillation technology to refine the extracted and synthesized visual features of the model. Subsequently, the visual-semantic features are then disentangled and aligned using a disentanglement network to enhance the consistency of the visual-semantic features. Finally, the consistent visual-semantic features are fused to jointly train a GZSL classifier. Extensive experiments demonstrate that the proposed method achieves more competitive results on four challenging benchmark datasets (AWA2, CUB, FLO, and SUN).

Few-shot object detection (FSOD) aims to address the challenge of requiring a substantial amount of annotations for training in conventional object detection, which is very labor-intensive. However, existing few-shot methods achieve high precision with the sacrifice of time-consuming for exhaustive fine-tuning, or take poor performance in novel-class adaptation. We presume the major reason is that the valuable correlation feature among different categories is insufficiently exploited, hindering the generalization of knowledge from base to novel categories for object detection. In this paper, we propose Few-Shot object detection via Correlation-RPN and Transformer Encoder-Decoder (CRTED), a novel training network to learn object-relevant features of inter-class correlation and intra-class compactness while suppressing object-agnostic features in the background with limited annotated samples. And we also introduce a 4-way tuple-contrast training strategy to positively activate the training progress of our object detector. Experiments over two few-shot benchmarks (Pascal VOC, MS-COCO) demonstrate that, our proposed CRTED without further fine-tuning can achieve comparable performance with current state-of-the-art fine-tuned works. The codes and pre-trained models will be released.

Open-vocabulary learning has recently gained prominence as a means to enable image segmentation for arbitrary categories based on textual descriptions. This advancement has extended the applicability of segmentation systems to a broader range of generally purpose scenarios. However, current methods often revolve around specialized architectures and parameters tailored to specific segmentation tasks, resulting in a fragmented landscape of segmentation models. In response to these challenges, we introduce OVAMTSeg, a versatile framework designed for Open-Vocabulary and Multitask Image Segmentation. OVAMTSeg harnesses adaptive prompt learning to empower the model to capture category-sensitive concepts, enhancing its robustness across diverse multi-task and scenario contexts. Text prompts are employed to effectively capture semantic and contextual features of the text, while cross-attention and cross-modal interactions enable the fusion of image and text features. Furthermore, a transformer-based decoder is incorporated for dense prediction. Extensive experimental results underscore the effectiveness of OVAMTSeg, showcasing its state-of-the-art performance and superior generalization capabilities across three segmentation tasks. Notable achievements include a 47.5 mIoU in referring expression segmentation, 51.6 mIoU on Pascal-VOC with four unseen classes, 46.6 mIoU on Pascal-Context in zero-shot segmentation, 65.9 mIoU on Pascal-5i, and 35.7 mIoU on COCO-20i datasets for one-shot segmentation.

Facial recognition systems are widely used systems around the globe. Their applications in real world scenarios like law enforcement, surveillance, ID card identification, information security, access control and many more has made them very important in today’s world. These systems have shown promising results and excellent performance; however, these systems need tremendous number of facial images for training before they are implemented in the real-world scenarios. The problem arises when we face lack of training images due to which the performance of such systems decreases exponentially. The problem becomes even worse when there is only one image per subject for training our facial recognition system, the probe image might contain variation like illumination, expression and disguise which are hard to be accurately recognized. Generative models have achieved great success for handling the lack of training data by generating synthetic data that resembles the original data. We have created a new method by fine tuning the basic Conditional Generative Adversarial Network (CGAN) to generate synthetic facial images provided only single image per subject. These synthetic facial images contain six variations that are smile, anger, stare, illumination and disguise. We increased the size of the dataset and then applied a convolutional neural network for facial recognition, which improved the accuracy of our system from 0.76 to 0.99.

The state-of-the-art smart city has been calling for an economic but efficient energy management over large-scale network, especially for the electric power system. It is a critical issue to monitor, analyze and control electric loads of all users in system. In this paper, we employ the popular computer vision techniques of AI to design a non-intrusive load monitoring method for smart electric energy management. First of all, we utilize both signal transforms (including wavelet transform and discrete Fourier transform) and Gramian Angular Field (GAF) methods to map one-dimensional current signals onto two-dimensional color feature images. Second, we propose to recognize all electric loads from color feature images using a deep neural network with multi-scale feature extraction and attention mechanism. Third, we design our method as a cloud-based, non-intrusive monitoring of all users, thereby saving energy cost during electric power system control. Experimental results on both public and our private datasets have demonstrated our method achieves superior performances than its peers, and thus supports efficient energy management over large-scale Internet of Things.

The task of weakly supervised point cloud semantic segmentation has received widespread attention and also has been widely used in autonomous driving, robotics, and modern industries. Due to the high dimensionality and large volume of point cloud data, many technical difficulties appeal to weakly supervised semantic processing, especially segmentation. However, weakly-supervised schemes usually provide only partial labelling information of the underlying point cloud data and thus need to effectively extract local features, geometric information, and contextual relationships only using these limited labelled data for supervised learning. To improve weakly-supervised semantic segmentation, we propose a novel segmentation network through the boundary-based feature aggregation based on a K-NN algorithm with down-sampling operation, and also introduce the smoothness loss and Siamese loss for effective segmentation. The experiments on public datasets demonstrate that our presented segmentation method exceeds most of the existing fully supervised and weakly-supervised methods in terms of mIoU. Specifically, our network has high segmentation accuracy on the labels of objects with similar geometrical structures, such as ceiling, wall, floor, chair and table, reaching 91.2%, 98.8%, 83.3%, 75.3% and 80.2%, respectively. Extensive experiments also illustrate its robustness, effectiveness, and generalization of the proposed weakly supervised segmentation network.

This paper introduces PatchFusion, an innovative approach for nonrigid tracking and reconstruction of deformable objects using a single RGB-D sensor. Existing methods face challenges in accurately capturing the rapid deformations of soft and flexible objects, thereby limiting their utility in diverse scenarios. Our approach overcomes this challenge by employing a dynamic patch-based framework that adapts to rapid inter-frame motions. Firstly, patch-wise rigid transformation fields for non-overlapping patches are solved via Iterative Closest Point (ICP) by incorporating geometric features as additional similarity constraints, thereby enhancing robustness and accuracy. Secondly, deformation optimization based on a nonrigid solver is applied to refine the coarse transformation fields. In order to enable simultaneous tracking and reconstruction of deformable objects, the patch-based rigid solver is designed to run in parallel with the nonrigid solver, serving as a plug-and-play module requiring minimal modifications for integration while enabling real-time performance. Following a comprehensive evaluation, PatchFusion showcases superior performance in effectively dealing with rapid inter-frame deformations when compared to existing techniques, rendering it a promising solution with broad applicability across domains such as robotics, computer vision, and human-computer interaction.

This research presents the creation of a Virtual Reality experience for the Balzi Rossi museum, commemorating the centenary of Prince Albert I Grimaldi's archaeological work at the site. The project, a collaboration between the University of Genoa's 3D Lab Factory, Eurodrone, and the Balzi Rossi archaeological museum, aims to preserve and convey the site's heritage through advanced VR technology. Initially employing Unreal Engine 4, the team used photogrammetry for 3D reconstruction of the "Caviglione" and "Florestano" caves, notable for their ancient rock engravings. This first VR model, despite hardware limitations, provided an interactive exploration of the site. The subsequent development phase utilized Unreal Engine 5, incorporating Nanite technology for enhanced visual fidelity. This advancement resulted in a more detailed and immersive VR experience, presenting the Balzi Rossi cliff across different historical periods, including the Würm glaciation. Key to this phase was optimizing the VR experience for performance, focusing on stable frame rates and minimizing motion sickness, and integrating a multi-lingual interface for broader accessibility. Since November 2023, the VR setup at the Balzi Rossi Museum has been an educational and interactive feature, enabling visitors to explore the site's history virtually. This research demonstrates the potential of VR in cultural heritage preservation and education, providing a foundation for future technological improvements in the field.

The segmentation of abnormal regions is vital in smart manufacturing. However, the existing segmentation system for detecting sauce-packet leakage on intelligent sensors encounters an issue of imaging blurring caused by uneven illumination. This issue adversely affects segmentation performance, thereby impeding the rapid production of industrial assembly lines. To alleviate this issue, we propose the two-stage Illumination-aware Sauce-packet Leakage Segmentation (ISLS) method for intelligent sensors. The ISLS comprises two main stages: Illumination-aware region enhancement and leakage region segmentation. In the first stage, YOLO-Fastestv2 is employed to capture the Region of Interest (ROI), which reduces redundancy computations. Additionally, we propose an image enhancement to relieve the impact of uneven illumination, enhancing the texture details of ROI. In the second stage, we propose a novel feature extraction network. Specifically, we propose the Multi-scale Feature Fusion Module (MFFM) and the Sequential Self-Attention Mechanism (SSAM) to capture discriminative representations of leakage. The multi-level features are fused by MFFM with a small number of parameters, which capture leakage semantics at different scales. The SSAM realizes the enhancement of valid features and the suppression of invalid features by adaptive weighting of spatial and channel dimensions. Furthermore, we generated a self-built dataset of sauce-packets, including 606 images with various leakage areas. Comprehensive experiments demonstrate that our ISLS method shows better results than several state-of-the-art methods, with additional performance analyses deployed on intelligent sensors to affirm the effectiveness of our proposed method. Our code is available at https://github.com/LSJ5106/SauceDetect.

Abstract: Rationale. Object tracking has significance in many applications ranging from control of unmanned vehicles to autonomous monitoring of specific situations and events, especially when providing safety for patients with certain adverse conditions such as epileptic seizures. Conventional tracking methods face many challenges, such as the need for dedicated attached devices or tags, influence by high image noise, complex object movements, and intensive com-putational requirements. We have developed earlier computationally efficient algorithms for global optical flow reconstruction of group velocities that provide means for convulsive seizure detection and have potential applications in fall and apnea detection. Here, we address the chal-lenge of using the same calculated group velocities for object tracking in parallel. Methods. We propose a novel optical flow-based method for object tracking. It utilizes real-time image se-quences from the camera and directly reconstructs global motion-group parameters of the con-tent. These parameters can steer a rectangular region of interest surrounding the moving object to follow the target. The method successfully applies to multi-spectral data, further improving its effectiveness. Results. Experimental results on simulated tests and complex real-world data demonstrate the method's capabilities. The proposed optical flow reconstruction can provide accurate, robust, and faster results compared to current state-of-the-art approaches.

Cloud-induced atmospheric extinction and occlusion have a major effect on the effectiveness and quality of telescope observations. Real-time cloud cover distribution and long-term statistical data are essential for astronomical siting and telescope operations. At ground-based astronomical telescope sites, cloud cover distribution is currently analyzed using manual observation methods. However, the main disadvantages of manual observation methods are human subjective, heavy workloads and poor real-time performance. Therefore, a real-time automatic cloud images classification method is desperately needed. This paper presents a new cloud identification method, which is named PSO+XGBoost model by combining eXtreme Gradient Boosting (XGBoost) and Particle Swarm Optimization (PSO). The entire cloud image is divided into 37 sub-regions in order to more precisely identify the distribution of the clouds. 19 features are then extracted, including the sky background, star density, lighting conditions, and subregion grayscale values. The experimental results have shown that the overall classification accuracy of 96.91% and our model is able to outperform several state-of-the-art baseline methods. Our approach achieves high accuracy in comparison with the manual observation methods. Moreover, this method meets telescope real-time scheduling requirements.

The research of object classification and part segmentation is a hot topic in computer vision. A considerable number of studies have been carried out about deep learning on 3D point clouds. However,it is challenging to achieve effective feature learning due to sparsity of point clouds. Recently, a variety of Transformers have been adopted to improve point cloud processing and display great potential. Nevertheless, large numbers of Transformer layers tend to incur huge computational and memory costs. PointNet++ is one of the most influential neural architectures for point cloud understanding. Although the accuracy of PointNet++ has been largely surpassed by recent networks, this does not mean that PointNet++ has no potential.Thus, this paper offer two major contributions that significantly improve PointNet++ performance. Firstly, we introduce a novel contextual feature extraction (CFE) block that significantly enhances the feature extraction capabilities of PointNet++ networks. Secondly, to further enhance feature fusion, we seamlessly integrate a mutual learning (ML) block into the network architecture. By embedding these two innovative blocks within each layer of the network, we not only enrich the network's functionality but also impart it with greater robustness and adaptability. The specific experiments were conducted on the S3DIS (6-fold cross-validation) and Modelnet40 datasets with 86.5% and 92.7% accuracy, respectively, which proved that our model is comparable or even better than most existing methods for classification and segmentation tasks.

Accurate classification of objects in 3D point clouds is a significant problem in several applications, such as autonomous navigation and augmented/virtual reality scenarios, which has become a research hot spot. In this paper, we presented a deep learning strategy for 3D object classification in augmented reality. The proposed approach is a combination of the GRU and LSTM. LSTM networks learn longer dependencies well, but due to the number of gates, it takes longer to train; on the other hand, GRU networks have a weaker performance than LSTM, but their training speed is much higher than GRU, which is The speed is due to its fewer gates. The proposed approach used the combination of speed and accuracy of these two networks. The proposed approach achieved an accuracy of 0.99 in the 4,499,0641 points dataset, which includes eight classes (unlabeled, man-made terrain, natural terrain, high vegetation, low vegetation, buildings, hardscape, scanning artifacts, cars). Meanwhile, the traditional machine learning approaches could achieve a maximum accuracy of 0.9489 in the best case.

As technology and multimedia production have advanced, there has been a significant rise in attacks on digital media, resulting in duplicated, fraudulent, and altered data and the infringement of copyright laws. This paper presents a robust and secure digital image watermarking technique that has been implemented in the spatial domain and exploits the erratic and chaotic behaviour of the powerful elementary cellular automata rule-30. The crucial characteristics of the watermarking system, i.e., imperceptibility, capacity, and robustness, have been perfectly balanced by the suggested blind watermarking technique. In this approach, prior to embedding, the grayscale watermark image is downsized to its two Most Significant Bits (MSBs). Then, the 2-MSBs watermark is encrypted using an ECA rule-30 so as to level up the security attribute of the system. Then, the host image is scrambled using ECA rule-30 to distribute the watermark pixels throughout the host image and thus achieve the highest robustness against geometrical attacks. Finally, the encrypted watermark data is embedded into the scrambled host image using the ECA rule-30-based embedding key. The proposed method performs better in terms of imperceptibility, capacity, and robustness when compared to several systems with similar competencies. The simulation's findings demonstrate strong imperceptibility as evaluated by the Peak Signal-to-Noise Ratio (PSNR), which has an average value of 58.3735 dB and a high payload. The experimental outcomes, observed across a diverse range of standardized attack scenarios, unequivocally establish the ascendancy of the proposed algorithm over competing methodologies in the realm of image watermarking.

Image stitching is a crucial aspect of image processing. However, factors like perspective and environment often lead to irregular shapes in stitched images. Cropping or completion methods typically result in substantial loss of information. This paper proposes a method for rectifying irregularly images into rectangles using deformable meshes and residual networks. The method utilizes a convolutional neural network to quantify rigid structures of images. Choosing the most suitable mesh structure based on the extraction results, offering options such as triangular, rectangular, and hexagonal. Subsequently, the irregularly image, predefined mesh structure, and predicted mesh structure are input into a wide residual neural network for regression. The loss function comprises local and global, aimed at minimizing the loss of image information within the mesh and global target. This method not only significantly reduces information loss during rectification but also adapting to different images with various rigid structures. Validation on the DIR-D dataset shows this method outperforms state-of-the-art methods in image rectification.

In the era of Artificial Intelligence (AI), comprehending and responding to non-verbal communication is increasingly vital. This research extends AI's reach in bridging communication gaps, notably benefiting American Sign Language (ASL) and Taiwan Sign Language (TSL) communities. It focuses on employing various AI models, especially the Hierarchical Vision Transformer with Shifted Windows (Swin), for recognizing diverse sign language datasets. The study assesses Swin architecture's adaptability to different sign languages, aiming to create a universal platform for Unvoiced communities. Utilizing deep learning and transformer technologies, hybrid application prototypes have been developed for ASL-to-English translations and vice versa, with plans to expand this to multiple sign languages. The Swin models, trained on varied dataset sizes, show considerable accuracy, indicating their flexibility and effectiveness. This research underscores major advancements in sign language recognition and underlines a commitment to inclusive communication in the digital era. Future work will focus on enhancing these models and broadening their scope to include more sign languages, integrating multimodality and Large Language Models (LLMs), thereby fostering global inclusivity.

Artistic image completion is fundamental for the preservation and restoration of invaluable art paintings and it has experienced significant progress through the implementation of deep learning methodologies. Despite these advancements, challenges persist, even with sophisticated approaches like Generative Adversarial Networks (GANs), particularly in achieving optimal results for high-resolution paintings. Small-scale texture synthesis and the inference of missing information from distant contexts present persistent issues, leading to distortions in lines and unnatural colors, especially in art paintings with complicated structures and textures. Concurrently, patch-based image synthesis has evolved by incorporating global optimization on the image pyramid to enhance structural coherence and details. However, methods relying on gradient-based synthesis encounter obstacles related to directionality, inconsistency, and the heavy computational burdens associated with solving the Poisson equation in non-integrable gradient fields. This paper introduces a groundbreaking approach, integrating Weighted Similarity-Confidence Laplacian Synthesis, to comprehensively address these challenges and advance the field of artistic image completion. This proposal addresses challenges not only in high-resolution artistic image completion but also makes a significant contribution to the broader field of patch-based synthesis by utilizing the Laplacian pyramid for enhanced edge-aware correspondence search. Experimental result confirms the effectiveness of our approach, offering promising outcomes for the preservation and restoration of art paintings with complicated details and irregular missing regions.

A human eye has about 120 million rod cells and 6 million cone cells. This huge number of light sensing cells inside a human eye will continuously produce a huge quantity of visual signals which flow into a human brain for daily processing. However, the real-time processing of these visual signals does not cause any fatigue to a human brain. This fact tells us the truth which is to say that human-like vision processes do not rely on complicated formulas to compute depth, displacement, and colors, etc. On the other hand, a human eye is like a PTZ camera. Here, PTZ stands for pan, tilt and zoom. We all know that in computer vision, each set of PTZ parameters (i.e., coefficients of pan, tilt and zoom) requires a dedicated calibration to determine a camera’s projection matrix. Since there is an infinite number of PTZ parameters which could be produced by a human eye, it is unlikely that a human brain stores an infinite number of calibration matrices for each human eye. Therefore, it is an interesting question for us to answer, which is to say whether simpler formulas of computing depth and displacement exist or not. Moreover, these formulas must be calibration friendly (i.e., easy process on the fly or on the go). In this paper, we disclose an important discovery of a new solution to 3D projection in a human-like binocular vision system. The purpose of doing 3D projection in binocular vision is to undertake forward and inverse transformations (or mappings) between coordinates in 2D digital images and coordinates in a 3D analogue scene. The formulas underlying the new solution are accurate, easily computable, easily tunable (i.e., to be calibrated on the fly or on the go) and could be easily implemented by a neural system (i.e., a network of neurons). Experimental results have validated the discovered formulas.

Object detection is complex and involved diverse requirements for different applications. In critical application such as visual impaired navigation guide and real-time surveillance for instance, identifying a particular object of instance is required and this involved a complex approach for realization. Literatures were reviewed, starting with the application of deep learning techniques for object detection; their gaps were identified and addressed using real-time object detection models literature review. From the review gaps were also detected and addressed using literature review on occlusion detection techniques. Object of instance detection in clustered scene with similar object of interest was identified as an open gap not addressed in the existing literature; even through it is vital for many applications. This research recommended an Occlusion Based Object of Instance Detection (OBOID) technique which used the spatial information to identify instant object of interest in clustered scene with many similarly object of interest. Limitation of the recommended OBOID is that it requires only system where position and distance of object is necessary to inform other decision.