Face swapping is an intriguing and intricate task in the field of computer vision. Currently, most mainstream face swapping methods employ face recognition models to extract identity features and inject them into the generation process. Nonetheless, such methods often struggle to effectively transfer identity information, result in generated results failing to achieve a high identity similarity with the source face. Furthermore, if we can accurately disentangle identity information, we can achieve controllable face swapping, thereby providing more choices to users. In pursuit of this goal, we propose a new face swapping framework (ControlFace) based on the disentanglement of identity information. We disentangle the structure and texture of the source face, encoding and characterizing them in the form of feature embeddings separately. According to the semantic level of each feature representation, we inject them into the corresponding feature mapper and fuse them adequately in the latent space of StyleGAN. Owing to such disentanglement of structure and texture, we are able to controllablely transfer parts of the identity features. Extensive experiments and comparisons with state-of-the-art face swapping methods demonstrate the superiority of our face swapping framework in terms of transferring identity information, producing high-quality face images and controllable face swapping.

Tracking information diffusion is a non-trivial task and it has been widely studied across different domains and platforms. The advent of social media has led to even more challenges, given the higher speed of information propagation and the growing impact of social bots and anomalous accounts. Nevertheless, it is crucial to derive a trustworthy information diffusion graph, which is capable of highlighting the importance of specific nodes in spreading the original message. The paper introduces the interaction strength, a novel metric to model retweet cascade graphs by exploring users’ interactions. Initial findings show the soundness of the approaches based on this new metric with respect to the state-of-the-art model, and its ability to generate a denser graph, revealing crucial nodes that participated in the retweet propagation. Reliable retweet graph generation will enable a better understanding of the diffusion path of a specific tweet.

The Independent Component Analysis (ICA) has been recently introduced as a reliable alternative to identify canonical scattering mechanisms within Polarimetric Synthetic Aperture Radar (PolSAR) images. This manuscript addresses two important aspects when applying such methods on real data, namely speckle filtering and statistical classification with ICA. A novel PolSAR data processing framework is introduced by adjusting the Lee's sigma filter to the particular nature of the Touzi's polarimetric decomposition. In its current form, it allows the use of the ICA mixing matrix in the derived speckle filter. An extension of the Fromont at al. iterative segmentation is introduced, equally. This proposed framework is tested using P band airborne PolSAR data acquired for the ESA campaign TropiSAR campaign.

Due to the inclusion of both the principles and divination methods, the study of Yijing (Yijing) becomes more complex. The complex data types, diverse semantic relationships, and unclear influencing mechanisms in Yijing pose challenges to its scientific research. Knowledge graph, as an advanced way of organizing knowledge, can provide technical support in exploring the hidden connections and knowledge structure within text resources. This paper proposes a method of constructing a Yijing knowledge graph based on Neo4j. By analyzing the concepts, entities, and various related relationships in Yijing, the method discusses the basic architecture, data model, and implementation steps of constructing the Yijing knowledge graph. It also extracts, integrates, and structures the Yijing knowledge, allowing for inquiries, associations, and reasoning about the knowledge and its various elements. By adopting a top-down approach, the paper reconstructs the fundamental knowledge system of the Yijing, including yin-yang, the five elements, generation and restriction, 64 hexagrams, and six lines(yao). It establishes a knowledge graph pattern layer and a data layer that encompass concepts, attributes, and relationships. With the technical support of the Neo4j platform, the visualization and retrieval of Yijing knowledge are realized, providing new ideas and methods for studying the mechanisms of the Yijing. The proposed method of constructing a Yijing knowledge graph can be expanded to other types of ancient scripture knowledge graph research and can be applied to the field of digital humanities, promoting in-depth cross-disciplinary research and integration.

One of the most leading causes of death worldwide is Colorectal cancer(CRC). Polyp segmentation is the most important detected measure for preventing CRC. However, there is still a missing rate for diminutive polyps and multiple ones. In order to solve the phenomenon, we propose to introduce auxiliary attention module(AAM) that can enhance the learning of features related to multiple and diminutive polyps by focusing more on the located and detailed information. Meanwhile, we design to decrease missed rate of multiple and diminutive polyps by implementing an area adaptive loss(AAL) which adapts the weight according to the area and the number of polyps. Our proposed novel AAM and AAL concentrates on training with hard examples and localized information. To evaluate the effectiveness and generalization ability of our proposed model, We utilize three different datasets of variable sizes and a cross dataset. Our proposed method achieves the best results on the Kvasir-SEG dataset, the CVC-ClinicDB dataset and the cross dataset, particularly for the Kvasir-Sessile dataset consisting of small,flat and diminutive polyps. Extensive experimental results show that our proposed DoubleAANet surpass the performance of all existing state-of-the-art segmentation methods.

The presence of fog in the background can prevent small and distant objects from being detected, let alone tracked. Under safety-critical conditions, multi-object tracking models require faster-tracking speed while maintaining high object-tracking accuracy. The original DeepSORT algorithm used YOLOv4 for the detection phase, and a simple neural network for deep appearance descriptor. Consequently, the feature map generated loses relevant details about the track being matched with a given detection in fog. Targets with a high degree of appearance similarity on the detection frame are more likely to be mismatched, resulting in identity switches or track failures in heavy fog. We propose an improved multi-object tracking model based on the DeepSORT algorithm to im-prove tracking accuracy and speed under foggy weather conditions. First, we employed our camera-radar fusion network (CR-YOLOnet) in the detection phase for faster and more accurate object detection. We proposed an appearance feature network to replace the basic convolutional neural network. We incorporated GhostNet to take the place of the traditional convolutional layers to generate more features and reduce computational complexities and cost. We adopted a segmentation module and fed the semantic labels of the corresponding input frame to add rich semantic information to the low-level appearance feature maps. Our proposed method outperformed YOLOv5 + DeepSORT with a 35.15% increase in multi-object tracking accuracy, a 32.65% increase in multi-object tracking precision, the speed increased by 37.56%, and identity switches decreased by 46.81%.

Non-rigid registration presents a significant challenge in the domain of point cloud processing. The general objective is to model complex non-rigid deformations between two or more overlapping point clouds. Applications are diverse and span multiple research fields, including registration of topographic data, scene flow estimation, and dynamic shape reconstruction. To provide context, we begin with a general introduction to the topic of point cloud registration, including a categorization of methods. Next, we introduce a general mathematical formulation for point cloud registration and extend it to address non-rigid registration. A detailed discussion and categorization of existing approaches to non-rigid registration follows. We then introduce our own method where the usage of piece-wise tricubic polynomials for modeling non-rigid deformations is proposed. Our method offers several advantages over existing methods. These advantages include easy control of flexibility through a small number of intuitive tuning parameters, a closed-form optimization solution, and an efficient transformation of huge point clouds. We demonstrate our method through multiple examples that cover a broad range of applications, with a focus on remote sensing applications - namely, the registration of Airborne Laser Scanning (ALS), Mobile Laser Scanning (MLS), and Terrestrial Laser Scanning (TLS) point clouds. The implementation of our algorithms is open source and can be found on GitHub.

A simple undirected graph $\Gamma=(V_\Gamma,E_\Gamma)$ admits an $H$-covering if every edge in $E_\Gamma$ belongs to at least one subgraph of $\Gamma$ that isomorphic to a graph $H$. For any graph $\Gamma$ admitting $H$-covering, a total Labelling $\beta: V_\Gamma \cup E_\Gamma \longrightarrow \{1,2,\dots,p\}$ is called an $H$-irregular total $p$-labelling of $\Gamma$ if every two different subgraphs $H_1$ and $H_2$ of $\Gamma$ isomorphic to $H$ have distinct weights where the weight $w_\beta(K)$ of subgraph $K$ of $\Gamma$ is defined as $w_f(K):=\displaystyle \sum_{v\in V_K} f(v) + \sum_{e\in E_K} f(e)$. The smallest number $p$ for which a graph $\Gamma$ admits an $H$-irregular total $p$-labelling is called the total $H$-irregularity strength of $\Gamma$ and is denoted by $ths(\Gamma)$. In this paper, we determine the total $H$-irregularity strength of edge comb product of two graphs.

Accuracy and computational cost are the main challenges of deep neural networks in image recognition. This paper proposes an efficient ranking reduction to binary classification approach using a new feed-forward network and feature selection based on ranking the image pixels. Preference net (PN) is a novel deep ranking learning approach based on Preference Neural Network (PNN), which uses new ranking objective function and positive smooth staircase (PSS) activation function to accelerate the image pixels’ ranking. PN has a new type of weighted kernel based on spearman ranking correlation instead of convolution to build the features matrix. The PN employs multiple kernels that have different sizes to partial rank image pixels’ in order to find the best features sequence. PN consists of multiple PNNs’ have shared output layer. Each ranker kernel has a separate PNN. The output results are converted to classification accuracy using the score function. PN has promising results comparing to the latest deep learning (DL) networks using the weighted average ensemble of each PN models for each kernel on CFAR-10 and Mnist-Fashion datasets in terms of accuracy and less computational cost.

Although deep learning has received extensive attention and achieved excellent performances in various of scenarios, it suffers from adversarial examples to some extent. Especially, physical attack poses more threats than digital attack. However, existing researches pay less attention to physical attack of object detection in remote sensing images (RSIs). In this work, we systematically analyze the universal adversarial patch attack for multi-scale objects in the remote sensing field. There are two challenges for adversarial attack in RSIs. On one hand, the number of objects in remote sensing images is more than that of natural images. Therefore, it is difficult for adversarial patch to show adversarial effect on all objects when attacking a detector of RSIs. On the other hand, the wide range of height of photography platform causes that the size of objects diverse a lot, which brings challenges for generating universal adversarial perturbation for multi-scale objects. To this end, we propose an adversarial attack method on object detection for remote sensing data. One of the key ideas of the proposed method is the novel optimization of adversarial patch. We aim to attack as many objects as possible by formulating a joint optimization problem. Besides, we raise a scale factor to generate a universal adversarial patch that adapts to multi-scale objects, which ensures the adversarial patch is valid for multi-scale objects in the real world. Extensive experiments demonstrate the superiority of our method against state-of-the-art methods on YOLO-v3 and YOLO-v5. In addition, we also validate the effectiveness of our method in real-world applications.

New notion of dimension as set, as two optimal numbers including metric number, dimension number and as optimal set are introduced in individual framework and in formation of family. Behaviors of twin and antipodal are explored in fuzzy(neutrosophic) graphs. Fuzzy(neutrosophic) graphs, under conditions, fixed-edges, fixed-vertex and strong fixed-vertex are studied. Some classes as path, cycle, complete, strong, t-partite, bipartite, star and wheel in the formation of individual case and in the case, they form a family are studied in the term of dimension. Fuzzification(neutrosofication) of twin vertices but using crisp concept of antipodal vertices are another approaches of this study. Thus defining two notions concerning vertices which one of them is fuzzy(neutrosophic) titled twin and another is crisp titled antipodal to study the behaviors of cycles which are partitioned into even and odd, are concluded. Classes of cycles according to antipodal vertices are divided into two classes as even and odd. Parity of the number of edges in cycle causes to have two subsections under the section is entitled to antipodal vertices. In this study, the term dimension is introduced on fuzzy(neutrosophic) graphs. The locations of objects by a set of some junctions which have distinct distance from any couple of objects out of the set, are determined. Thus it’s possible to have the locations of objects outside of this set by assigning partial number to any objects. The classes of these specific graphs are chosen to obtain some results based on dimension. The types of crisp notions and fuzzy(neutrosophic) notions are used to make sense about the material of this study and the outline of this study uses some new notions which are crisp and fuzzy(neutrosophic). Some questions and problems are posed concerning ways to do further studies on this topic. Basic familiarities with fuzzy(neutrosophic) graph theory and graph theory are proposed for this article.

With the continuous development of deep learning, the application of object detection based on deep neural network in the coal mine has been expanding. Simultaneously, as the production applications demand higher recognition accuracy, most research chooses to enlarge the depth and parameters of the network to improve accuracy. However, due to the limited computing resources in the coal mining face, it is challenging to meet the computation demands of a large number of hardware resources. Therefore, this paper proposes a lightweight object detection algorithm designed specifically for coal mining face, referred to as CM-YOLOv8. The algorithm introduces adaptive predefined anchor boxes tailored to the coal mining face dataset to enhance the detection performance of various targets. Simultaneously, a pruning method based on the L1 norm is designed, significantly compressing the model's computation and parameter volume without compromising accuracy. The proposed algorithm is validated on the coal mining dataset DsLMF+, achieving a compression rate of 40% on the model volume with less than a 1% drop in accuracy. Comparative analysis with other existing algorithms demonstrates its efficiency and practicality in coal mining scenarios. The experiments confirm that CM-YOLOv8 significantly reduces the model's computational requirements and volume while maintaining high accuracy.

Wearing gloves while operating machinery in workshops is an essential precaution to prevent mechanical injuries and burns from high temperatures, among other potential hazards. Ensuring workers are properly equipped with gloves, which is a crucial measure in accident prevention. Glove images often occupy a minimal proportion of the frame and are easily obscured by cluttered backgrounds, especially with limited edge computing resources. Consequently, this study proposes a glove detection algorithm called YOLOv8-AFPN-M-C2f based on YOLOv8, offering swifter detection speeds, lower computational demands, and enhanced accuracy for workshop scenarios. This research innovates by substituting the head of YOLOv8 with the AFPN-M-C2f network, amplifying the pathways for feature vector propagation, and mitigating semantic discrepancies between non-adjacent feature layers. Additionally, the introduction of a superficial feature layer enriches surface feature information, augmenting the model's sensitivity to smaller objects. To assess the performance of the YOLOv8-AFPN-M-C2f model, we conducted multiple experiments using a factory glove detection dataset compiled for this study. The results indicate that the enhanced YOLOv8 model surpasses other network models. Compared to the baseline YOLOv8 model, the refined version shows a 2.6% increase in mAP@50%, a 90.1% rise in FPS, and a 13% reduction in the number of parameters. This research contributes an effective solution for the detection of glove adherence.

The agricultural industry places a high priority on crop protection, especially when it comes to protecting cotton crops against pink bollworm infestations. Rapid disease identification is essential for effective crop management at the same time. In order to overcome these difficulties, this study offers a thorough process for utilizing the Robot Operating System (ROS) to deploy an autonomous robot. The robot is intended to identify infestations of pink bollworm in addition to counting and tracking red Disease that impact cotton harvests. Drones are used to check the condition of cotton fields, and the autonomous robot's path planning component is closely related to this process. Large fields are photographed by these drones in multispectral mode, and the robot's course planning technique is done by calculating the Normalized Difference Vegetation Index (NDVI) based on these images. This approach ensures targeted surveillance and intervention, optimizing the use of resources. A customized dataset was created especially for this application to improve the robot's detecting abilities. The dataset was utilized to train a YOLOV8 model, a state-of-the-art object detection architecture. Performance characteristics of the trained model are impressive; it has a mean Average Precision (mAP) of 67.1%, Precision of 67.9%, and Recall of 61.8%. These metrics highlight how well the model works to precisely locate and measure interesting occurrences in the cotton fields. In order to address the particular challenges presented by pink bollworm infestations and crop diseases in the context of cotton cultivation, this research contributes a comprehensive solution for autonomous crop monitoring and protection by seamlessly integrating ROS, drones, NDVI calculations, and a robust detection model.

Transportation of goods is as old as human civilizations : past networks and their evolution shed light on long term trends. Transportation impact on climate change is measured as major, as well as the impact on spreading a pandemic. These two reasons motivate the importance of providing relevant and reliable historical geographic datasets of these networks. This paper focuses on reconstructing the railway network in France at its maximal extent, a century ago. The active stations and lines are well documented by the French SNCF, in open public data. However, that information ignores past stations (ante 1980), which represent probably more than what is recorded in public data. Additional open data, individual or collaborative (eg. Wikipedia) are particularly valuable, but they are not always geo-coded, and two more sources are necessary to completing that geo-coding: ancient maps and aerial photography. Therefore, remote sensing and volunteer geographic information are the two pillars of past railway reconstruction. The methods developed are adapted to the extraction of information from these sources: automated parsing of Wikipedia Infoboxes, data extraction from simple tables, even from simple text. That series of sparse procedures can be merged into a comprehensive computer-assisted process. Beyond this, a huge effort in quality control is necessary when merging these data: automated wherever possible, or finally visually controlled by observation of remote sensing information. The main output is a reliable dataset, under ODbl, of more than 9100 stations, which can be combined with the information about the 35000 communes of France, for a large variety of studies. This work demonstrates two thesis: (a) it is possible to reconstruct transport network data from the past, and generic computer assisted methods can be developed; (b) the value of remote sensing and volunteered geo info is considerable (what archeologists already know).

This manuscript presents a system that can generate a 2D grid map and a 3D voxel model of an environment by fusing multiple sensors integrated into a robot. The fusion of these sensors is done by utilizing Robot Operating System (ROS) and some of its libraries. The sensors that are fused in the robot are a Light Detection and Ranging (LIDAR), Inertial Measurement Units (IMU), wheel encoders, and a Time-of-Flight camera. With the fusion of the point cloud data information obtained with the Time-of-Flight camera, the laser information of the LIDAR sensor, in addition to the odometry generated with the other sensors, a 2D grid map and a 3D model of two different environments were created, thus testing the performance of the system.

Virtual reality software might be challenging to utilize for beginners and unskilled professionals who do not have a programming or 3D modeling background. Concurrently, there is a knowledge gap in software project design for intuitive virtual reality authoring tools, which were supposed to be easier to use. These tools are frequently insufficient due to a lack of support and standard operating procedures. Adopting the Design Science Research paradigm, this study aims to evaluate the validity of fourteen design guidelines for the development of intuitive virtual reality authoring tools as an artifact. While a previous study completed the first steps of the Design Science Research, by identifying problems, defining solution objectives, and developing and demonstrating the design guidelines, this work seeks to qualitatively evaluate their application in a practical experiment. A group of engineering students with no prior experience in creating virtual worlds were tasked with examining the design guidelines while using the NVIDIA Omniverse Enterprise as an exemplary use case and responding to a questionnaire and a focus group interview about how they perceived these guidelines. A correlation analysis confirmed that most guidelines scores behaved as expected and were ranked according to the use-case functionality. The participants understood the guidelines' definition and could decide if they agreed or disagreed with their presence during the experiment. We conclude that, in accordance with the Design Science Research, the proposed artifact is useful, i.e., the design guidelines for virtual reality authoring tools perform what they are designed to do and are operationally reliable in accomplishing their goals.

Fog, haze, or smoke are usual atmospheric phenomena that dramatically compromise the overall visibility of any scene, critically affecting features such as illumination, contrast, and contour detection of objects. The decrease in visibility compromises the performance of computer vision algorithms such as pattern recognition and segmentation, some of them very relevant for decision-making for the security or autonomous vehicle industries. Several dehazing methods have been proposed, however, to the best of our knowledge, all proposed metrics compare the defogged image to its ground truth for evaluation of the defogging algorithms, or need to estimate parameters through physical models. This fact hinders progress in the field as obtaining proper ground truth images is costly and time-consuming, and physical parameters greatly depend on the scene conditions. This paper aims to tackle this issue by proposing a contour-based metric for image defogging evaluation that does not need a ground truth image. The proposed metric only requires the original hazy RGB image and the RGB image after the defogging procedure. A comparison of the proposed metric with metrics currently used in the NTIRE 2018 defogging challenge is performed to prove its effectiveness in a general situation, showing comparable results to conventional metrics.

In recent years, significant advancements in the field of machine learning have influenced the domain of image restoration. While these technological advancements present prospects for improving the quality of images, they also present difficulties, particularly the proliferation of manipulated or counterfeit multimedia information on the internet. The objective of this paper is to provide a comprehensive review of existing inpainting algorithms and forgery detections, with a specific emphasis on techniques that are designed for the purpose of removing objects from digital images. In this study, we will examine various techniques encompassing conventional texture synthesis methods, as well as those based on neural networks. Furthermore, we will explore the artifacts associated with the identification of modified photos and present the artifacts frequently introduced by the inpainting procedure and assess the state-of-the-art technology for detecting such modifications. Lastly, we shall look at the available datasets and how the methods compare with each other. Having covered all of the above, the final outcome of this study is to provide a comprehensive perspective on the abilities and constraints to detect images for which an inpainting object removal method was applied.

Vision loss can be avoided if diabetic retinopathy (DR) is diagnosed and treated promptly. Following are the main 5 DR stages: none, moderate, mild, proliferate, and severe. In this study, a deep learning (DL) model is presented that diagnoses all 5 stages of DR with more accuracy than previous methods. The suggested method presents two scenarios: case 1 with image enhancement using contrast limited adaptive histogram equalization (CLAHE) filtering algorithm in conjunction with an Enhanced Super-resolution generative adversarial network (ESRGAN), and case 2 without image enhancement; augmentation techniques are then performed to generate a balanced dataset utilizing the same parameters for both cases. Using Inception-V3 applied to the Asia Pacific Tele-Ophthalmology Society (APTOS) datasets, the developed model achieved an accuracy of 98.7% for case 1 and 80.87% for case 2, which is greater than existing methods for detecting the five stages of DR. It was demonstrated that using CLAHE and ESRGAN improves a model's performance and learning ability.

In this paper, we give some upper and lower bounds for the multiplicative Randic index, reduced reciprocal Randic index, Narumi-Katayama index and symmetric division index a graph using solely the vertex degrees. Then we obtain upper and lower bounds for these indices for the complete graphs, path graphs and Fibonacci-sum graphs. Finally, we compared the bounds of these indices for a general graph and some special graphs.

Accurate weight measurement is pivotal for monitoring the growth and well-being of cattle. However, the conventional weighing process, which involves physically placing cattle on scales, is labor-intensive and distressing for the animals. Hence, the development of automated cattle weight prediction techniques assumes critical significance. This study proposes a weight prediction approach for Korean cattle using 3D segmentation-based feature extraction and regression machine learning techniques from incomplete 3D shapes acquired from real farm environments. In the initial phase, we generated mesh data of 3D Korean cattle shapes using a multiple-camera system. Subsequently, deep learning-based 3D segmentation with the PointNet network model was employed to segment two dominant parts of the cattle. From these segmented parts, three crucial dimensions of Korean cattle were extracted. Finally, we implemented five regression machine learning models (CatBoost regression, LightGBM, Polynomial regression, Random Forest regression, and XGBoost regression) for weight prediction. To validate our approach, we captured 270 Korean cattle in various poses, totaling 1190 poses of 270 cattle. The best result was achieved with mean absolute error (MAE) of 25.2 kg and mean absolute percent error (MAPE) of 5.81% using the random forest regression model.

Drowsiness detection is an important task in road safety and other areas that require sustained attention. In this article, an approach to detect drowsiness in drivers is presented, focusing on the eye region, since eye fatigue is one of the first symptoms of drowsiness. The method used for the extraction of the eye region is Mediapipe, chosen for its high accuracy and robustness. Three neural networks were analyzed based on: InceptionV3, VGG16 and ResNet50V2 that implement deep learning. The database used is NITYMED, which contains videos of drivers with different levels of drowsiness. The three networks were evaluated in terms of accuracy, precision and recall in detecting drowsiness in the eye region. The results of the study show that all three convolutional neural networks have high accuracy in detecting drowsiness in the eye region. In particular, the Resnet50V2 network achieved the highest accuracy, with a rate of 99.71% on average. For better visualization of the data, the Grad-CAM technique is used, with which we get a better understanding of the performance of the algorithms in the classification process.

Leveraging the open-world understanding capacity of large-scale visual-language pre-trained models has become a hot-spot in point cloud classification. Recent approaches rely on transferable visual-language pre-trained models, classifying point clouds by projecting them into 2D images and evaluating consistency with textual prompts. These methods benefit from the robust open-world understanding capabilities of visual-language pre-trained models and require no additional training. However, they face several challenges summarized as prompt ambiguity, image domain gap, view weights confusion, and feature deviation. In response to these challenges, we propose PointBLIP, a zero-training point cloud classification network based on the recently introduced BLIP-2 visual-language model. PointBLIP is adept at processing similarities between multi-images and multi-prompts. We separately introduce a novel method for point cloud zero-shot and few-shot classification, which involves comparing multiple features to achieve effective classification. Simultaneously, we enhance the input data quality for both the image and text sides of PointBLIP. In point cloud zero-shot classification tasks, we outperform state-of-the-art methods on three benchmark datasets. For few-shot classification tasks, to the best of our knowledge, we present the first zero-training few-shot point cloud method, surpassing previous works under the same conditions and showcasing comparable performance to full-training methods.

Automatic and real-time pose estimation is important in monitoring animal behavior, health and welfare. In this paper, we utilized pose estimation for monitoring farrowing process to prevent piglet mortality and preserve the health and welfare of sow. State-of-the-art Deep Learning (DL) methods have lately been used for animal pose estimation. The aim of this paper was to probe the generalization ability of five common DL networks (ResNet50, ResNet101, MobileNet, EfficientNet and DLCRNet) for sow and piglet pose estimation. These architectures predict body parts of several piglets and the sow directly from input video sequences. Real farrowing data from a commercial farm was used for training and validation of the proposed networks. The experimental results demonstrated that MobileNet was able to detect seven body parts of the sow with median test error of 0.61 pixels.

We present a near real-time solution for 3D reconstruction from aerial images captured by consumer UAVs. Our core idea is to simplify the multi-view stereo problem into a series of two-view stereo matching problems. Our method applies to UAVs equipped with only one camera and does not require special stereo-capturing setups. We found that the neighboring two video frames taken by UAVs flying at a mid-to-high cruising altitude can be approximated as left and right views from a virtual stereo camera. Leveraging GPU-accelerated real-time stereo estimation, efficient PnP correspondence solving algorithms and extended Kalman filter, our system simultaneously predicts scene geometry and camera position/orientation from the virtual stereo cameras. Also, this method allows user selection of varying baseline lengths, which provides more flexibility given the trade-off between camera resolution, effective measuring distance, flight altitude, and mapping accuracy. Our method outputs dense point clouds at a constant speed of 25 frames per second and is validated on a variety of real-world datasets with satisfactory results.

In the era of Artificial Intelligence (AI), comprehending and responding to non-verbal communication is increasingly vital. This research extends AI's reach in bridging communication gaps, notably benefiting American Sign Language (ASL) and Taiwan Sign Language (TSL) communities. It focuses on employing various AI models, especially the Hierarchical Vision Transformer with Shifted Windows (Swin), for recognizing diverse sign language datasets. The study assesses Swin architecture's adaptability to different sign languages, aiming to create a universal platform for Unvoiced communities. Utilizing deep learning and transformer technologies, hybrid application prototypes have been developed for ASL-to-English translations and vice versa, with plans to expand this to multiple sign languages. The Swin models, trained on varied dataset sizes, show considerable accuracy, indicating their flexibility and effectiveness. This research underscores major advancements in sign language recognition and underlines a commitment to inclusive communication in the digital era. Future work will focus on enhancing these models and broadening their scope to include more sign languages, integrating multimodality and Large Language Models (LLMs), thereby fostering global inclusivity.

Recent advancements in Artificial Intelligence (AI), Deep Learning (DL), and computer vision have revolutionized various industrial processes through image classification and object detection. State-of-the-art Optical Character Recognition (OCR) and Object Detection (OD) technologies, such as YOLO and PaddleOCR, have emerged as powerful solutions for addressing challenges in recognizing textual and non-textual information on printed stickers. However, a well-established framework integrating these cutting-edge technologies for industrial applications still needs to be discovered. In this paper, we propose an innovative framework that combines advanced OCR and OD techniques to automate visual inspection processes in an industrial context. Our primary contribution is a comprehensive framework adept at detecting and recognizing textual and non-textual information on printed stickers within a company, harnessing the latest AI tools and technologies for sticker information recognition. Our experiments reveal an overall macro accuracy of 0.88 for the sticker OCR across three distinct patterns. Furthermore, the proposed system goes beyond traditional Printed Character Recognition (PCR) by extracting supplementary information, such as barcodes and QR codes present in the image, significantly streamlining industrial workflows and minimizing manual labor demands.

This paper introduces an innovative approach to lipreading, leveraging a web application designed to generate subtitles for videos where the speaker's mouth is visible and a comprehensive literature review that precedes the discussion, encompassing a thorough examination of various lipreading methods employed over the past decade. Our method employs a powerful deep learning model, featuring a 3D-convolution network and bidirectional LSTM, enabling accurate sentence-level predictions based solely on visual lip movements. With an impressive accuracy of 97%, our model is trained using pre-segmented lips regions, transformed into animated GIFs for effective pre-training. This work stands as a significant contribution to the evolving landscape of lipreading research, offering a practical and accurate solution for real-world applications.

3D human pose estimation has made significant advancements through the integration of deep learning techniques. This survey provides a comprehensive review of recent 3D human pose estimation methods, with a focus on monocular images, videos, and multi-view cameras. Our approach stands out through a systematic literature review methodology, ensuring an up-to-date and meticulous overview. Unlike many existing surveys that categorize approaches based on learning paradigms, our survey offers a fresh perspective, delving deeper into the subject. For image-based approaches, we not only follow existing categorizations but also introduce and compare significant 2D models. Additionally, we provide a comparative analysis of these methods, enhancing the understanding of image-based pose estimation techniques. In the realm of video-based approaches, we categorize them based on the types of models used to capture inter-frame information. Furthermore, in the context of multi-person pose estimation, our survey uniquely differentiates between approaches focusing on relative poses and those addressing absolute poses. Our survey aims to serve as a pivotal resource for researchers, highlighting state-of-the-art deep learning strategies and identifying promising directions for future exploration in 3D human pose estimation.

In view of the problem of cracking easily and partial distortion of images after encryption or decryption, a novel image encryption and decryption algorithm based on Arnod Transform and fractional chaotic is proposed. To begin with, the Arnold transform is used to encrypt. So that the spatial confidence of the original image has been comprehensively disturbed. Secondly, the XOR involving the fractional order chaotic sequence is used to encrypt. The key sequence is dynamically generated to ensure the randomness and difference of key generation. When decryption is required, the first decryption is performed using the key and XOR. Then the second decryption is carried out by using the inverse Arnold transform, and finally the decrypted image is obtained. Experimental results show that the improved algorithm has achieved better performance in encryption and decryption.

Human Action Recognition is widely used in multiple fields to recognize activities and extract spatial and temporal information. This paper focuses on analyzing various methods as well as provides extensive knowledge of the foundational concepts of the HAR. A dataset is also very crucial for any research study so we discussed the popular dataset and its features in this paper.

This study presents a novel approach for automating the body condition scoring of dairy cows, leveraging advancements in 3D imaging technology and deep neural networks. The primary objective was to design and implement a system capable of accurately and efficiently assessing the body condition of dairy cows, a critical metric in livestock management for optimizing health and productivity. To achieve this, a 3D camera was employed to capture detailed point cloud data, reconstructing the three-dimensional morphology of individual cows. The obtained data were then fed into a deep neural network, specifically tailored for the task of ranking body condition. The neural network was trained on a diverse dataset of annotated body condition score representing varying degrees of body condition, ensuring robust performance across different physiological states. The results demonstrate the efficacy of the proposed system in automatically and objectively scoring the body condition of dairy cows. The automated process not only expedites the assessment but also reduces the subjectivity associated with manual scoring methods. This innovative approach holds promise for improving the efficiency of dairy farm management by providing timely and accurate body condition assessments. The integration of 3D imaging and deep learning techniques paves the way for future advancements in precision livestock farming, contributing to enhanced animal welfare, optimized production, and sustainable agriculture practices.

To solve the problem that the common long-tailed classification method does not use the semantic features of the original label text of the image, and the difference between the classification accuracy of most classes and minority classes is large, the long-tailed image classification method based on enhanced contrast visual language trains the head class and tail class samples separately, uses text image to pre-train the information, and uses enhanced momentum contrast loss function and RandAugment enhancement to improve the learning of tail class samples. On the ImageNet-LT long-tailed dataset, the enhanced contrastive visual-language based long-tailed image classification method has improved all class accuracy, tail class accuracy, middle class accuracy, and F1 values by 3.4%, 7.6%, 3.5%, and 11.2%, respectively, compared to the BALLAD method. The difference in accuracy between the head class and tail class is reduced by 1.6% compared to the BALLAD method. The results of three comparative experiments indicate that the long-tailed image classification method based on enhanced contrastive visual-language has improved the performance of tail classes and reduced the accuracy difference between majority and minority classes.

Quality control is a critical component in industrial manufacturing, directly influencing efficiency, product reliability, and ultimately, customer satisfaction. In the dynamic environment of industrial manufacturing, traditional methods of inspection may not adequately meet the evolving complexity, necessitating innovative approaches to bolster precision and productivity. In this study, we explore the application of mixed reality (MR) technology for real-time quality control in the assembly process. Our methodology involved the integration of smart glasses with a server-based image recognition system, designed to conduct real-time component analysis. The innovative aspect of our study lies in the harmonization of MR and computer vision algorithms, providing immediate visual feedback to inspectors and thereby improving the speed and accuracy of defect detection. YOLOv8 have been adopted in this study for detection object model. The project implementation occurred in a controlled environment to enable a comprehensive evaluation of the system functionality, the identification of possible problems and improvements in the system performance. The results indicated the viability of mixed reality as a powerful tool for enhancing traditional inspection processes. The fusion of MR and computer vision offers possibilities for future advancements in industrial quality control, paving the way for more efficient and reliable manufacturing ecosystems.

A major challenge of augmented and mixed reality applications is identifying the context and semantics of the real environment. Studies on object and action recognition were developed based on the improvement of machine learning techniques, allowing them to be annotated and recognized. This study aims to characterize current knowledge on the use of machine learning for recognizing objects and actions in augmented and mixed reality environments, increasing context awareness. Therefore, a systematic literature review of works related to these topics was made, using the Scopus and Web of Science knowledge bases. We searched articles and conference reviews or papers published between 2018 and 2022 and selected fifteen studies to be reviewed. The results indicate that there is a great demand for using machine learning to immersive technologies in factories, engineering, entertainment, education, health, among other application domains. However, these real-time interactive systems still have challenges and limitations to be solved, involving network communication, prediction time and the creation of a model that recognize objects and actions in broad contexts. Furthermore, additional research is needed to investigate how object and action recognition can increase context awareness in augmented reality applications.

Deep learning heavily relies on statistical correlations to drive artificial intelligence (AI) innovations, particularly in computer vision applications like autonomous driving and robotics. However, despite providing a solid foundation for deep learning, these statistical correlations can be vulnerable to unforeseen and uncontrolled factors. The lack of prior knowledge guidance can result in spurious correlations, introducing confounding factors and affecting the model's robustness. To address this challenge, recent research efforts have focused on integrating causal theory into deep learning methodologies. By modelling the inherent and unbiased causal structure, causal theory can potentially mitigate the impact of spurious correlations effectively. Hence, this paper explores the basics of causal methodologies in image classification.

We compute explicitly traces of one-dimensional diffusion processes. The obtained trace forms can be regarded as Dirichlet forms on graphs. Then we discuss conditions ensuring the trace forms to be conservative. Finally, the obtained results are applied to the one-dimensional diffusion related to the Bessel’s process of order $\nu$.

A single paragraph of about 200 words maximum. For research articles, abstracts should give a pertinent overview of the work. We strongly encourage authors to use the following style of structured abstracts, but without headings: (1) Background: place the question addressed in a broad context and highlight the purpose of the study; (2) Methods: describe briefly the main methods or treatments applied; (3) Results: summarize the article’s main findings; (4) Conclusions: indicate the main conclusions or interpretations. The abstract should be an objective representation of the article, it must not contain results which are not presented and substantiated in the main text and should not exaggerate the main conclusions.

A topic of growing interest in urban remote sensing is the automated extraction of geometrical building information for 3D city modeling. Roof geometry information is useful for applications such as urban planning, solar potential estimation and telecommunication installation planning, and wind flow simulations for pollutant diffusion analysis. Recent research has proven that the advance in remote sensing technologies and deep learning methods offer the prospects of deriving the roof structure information accurately and efficiently. In this study, we propose a Vectorized Roof Extractor- method based on Fully Convolutional Networks (FCNs) and advanced polygonization method to extract roof structure from aerial imagery and a normalized Digital Surface Models (nDSM) in a regularized vector format. The roof structure consists of building outlines, external edges of the building roof, inner rooflines, internal intersections of the main roof planes. The methodology is comprised of segmentation, vectorization and post-processing for outer rooflines, external edges of the building roof, and inner rooflines, and internal intersections of the main roof planes. For the comparison, we adapt the Frame field Learning (FFL) method originally designed to extract building polygons [1]. Our experiments are conducted on a custom data set derived for the city of Enschede, The Netherlands, using aerial imagery, nDSM and manually digitized training polygons. The results show that the proposed Vectorized Roof Extractor outperformed adapted FFL on PoLiS distance with values of 3.5 m and 1.2 m for outlines and inner rooflines, respectively. Furthermore, the model surpassed the adapted FFL on PoLiS-thresholded F-score for outlines and inner rooflines, with 0.31 and 0.57, respectively. The Vectorized Roof Extractor produced adequate visual results, with straighter walls and fewer missed inner roofline detections. It can predict buildings with common walls thanks to skeleton graph computation. To summarize, the proposed method is suitable for urban applications and has the potential to be improved further.

Purpose: This study investigates the impact of different intensity normalization (IN) methods on the overall survival (OS) radiomics models’ performance of MR sequences in primary (pHGG) and recurrent high-grade glioma (rHGG). Methods: MR scans acquired before radiotherapy were retrieved from two independent cohorts (rHGG C1: 197, pHGG C2: 141) from multiple scanners (15, 14). The sequences are T1 weighted (w), contrast-enhanced T1w (T1wce), T2w, and T2w-FLAIR. Sequence-specific significant features (SF) associated with OS, extracted from the tumour volume, were derived after applying 15 different IN methods. Survival analyses were conducted using Cox proportional hazard (CPH) and Poisson regression (POI) models. A ranking score was assigned based on the 10-fold cross-validated (CV) concordance index (C-I), mean square error (MSE), and the Akaike information criterion (AICs), to evaluate the methods’ performance. Results: Scatter plots of the 10-CV C-I and MSE against the AIC showed an impact on the survival predictions between the IN methods and MR sequences (C1/C2 C-I range: 0.62-0.71/0.61-0.72, MSE range: 0.20-0.42/0.13-0.22). White stripe showed stable results for T1wce (C1/C2 C-I: 0.71/0.65, MSE: 0.21/0.14). Combat (0.68/0.62, 0.22/0.15) and histogram matching (HM, 0.67/0.64, 0.22/0.15) showed consistent prediction results for T2w models. They were also the top-performing methods for T1w in C2 (Combat: 0.67, 0.13; HM: 0.67, 0.13), however, only HM achieved high predictions in C1 (0.66, 0.22). After eliminating IN impacted SF using Spearman’s rank-order correlation coefficient, a mean decrease in the C-I and MSE of 0.05 and 0.03 was observed in all four sequences. Conclusion: The IN method impacted the predictive power of survival models. Thus, performance is sequence-dependent.

Covid-19 is a lung disease caused by a Coronavirus family virus. Due to its extraordinary prevalence and death rates, it has spread quickly to every country in the world. Thus, achieving peaks and outlines and curing different types of relapses is extremely important. Given the worldwide prevalence of Coronavirus and the participation of physicians in all countries, Information has been gathered regarding the properties of the virus, its diverse types, and the means of analyzing it. Numerous approaches have been used to identify this evolving virus. It is generally considered the most accurate and acceptable method of examining the patient's lungs and chest through a CT scan. As part of the feature extraction process, a method known as fractional Fourier transform (FrFT) has been applied as one of the time-frequency domain transformations. The proposed method was applied to a database consisting of 2481 CT images. Following the transformation of all images into equal sizes and the removal of non-lung areas, multiple combination windows are used to reduce the number of features extracted from the images. In this paper, the results obtained for KNN and SVM classification have been obtained with accuracy values of 99.84% and 99.90%, respectively.

The Spiking Neural Network (SNN) has attracted more and more attention recently. It adopts binary spike signals to transmit information. Benefitting from the information passing paradigm of SNNs, the multiplications of activations and weights can be replaced by additions, which are more energy-efficient. However, its ``Hard Reset" mechanism for the firing activity would ignore the difference among membrane potentials when the membrane potential is above the firing threshold, causing information loss. Meanwhile, quantifying the membrane potential to 0/1 spikes at the firing instants will inevitably introduce the quantization error thus bringing about information loss too. To address these problems, we propose to use the ``Soft Reset" mechanism for the supervised training-based SNNs, which will drive the membrane potential to a dynamic reset potential according to its magnitude, and Membrane Potential Rectifier (MPR) to reduce the quantization error via redistributing the membrane potential to a range close to the spikes. Results show that the SNNs with the ``Soft Reset" mechanism and MPR outperform their vanilla counterparts on both static and dynamic datasets.

Vegetation indexes help perform precision farming because they provide useful information regarding moisture, nutrient content, and crop health. Primary sources of those indexes are satellites and unmanned aerial vehicles equipped with expensive multispectral sensors. Reducing the price of obtaining such information would increase the availability of precision farming for small farms. Several studies have proposed deep neural network methods to estimate the indexes from RGB color images. However, these methods report relatively large errors for mature plants when highly non-linear relationships of images and vegetation indexes arise. One could apply multilayer random forest-based models (Deep Forests) to solve this problem, but the discriminative power of such models is limited: they cannot catch complex dependencies between image features. In this paper, we propose a method that combines ideas of deep forests, random forests of kernel trees, and global pruning of random forests to tackle the problem. As a result, the method considers the properties of objects with a complex structure: the presence of relationships between groups of features, displacement, and scaling of objects. The experimental results show that the proposed method outperforms neural network-based solutions in several datasets.

Pure monocular 3D reconstruction is an ill-posed problem that has attracted the research community's interest due to the affordability and availability of RGB sensors. SLAM, VO, and SFM are disciplines formulated to solve the 3D reconstruction problem and estimate the camera’s ego-motion, so many methods have been proposed. However, most of these methods were not evaluated in large datasets, under various motion patterns, had not been tested under the same metrics, and most of them had not been evaluated following a taxonomy, making their comparison and selection difficult. In this research, we performed a comparison of ten publicly available SLAM and VO methods following a taxonomy, including one method for each category of the primary taxonomy, three machine learning-based methods, and two updates of the best methods, to identify the advantages and limitations of each category of the taxonomy and test if the addition of machine learning or the updates made on those methods improved them significantly. Thus, we evaluated each algorithm under the TUM-Mono benchmark and performed an inferential statistical analysis to identify significative differences through its metrics. Results determined that sparse-direct methods significantly outperformed the rest of the taxonomy, and fusing them with machine learning techniques significantly improves the performance of geometric-based methods from different perspectives.

Digital twins are gaining in popularity for simulating complex natural and urban environments. In this context, accurate segmentation of objects within 3D urban environments is of crucial importance. The aim of this project is to develop a methodology for extracting buildings from textured 3D meshes. To this end, PicassoNet-II, a semantic segmentation architecture is employed. The methodology also incorporates Markov field-based contextual analysis to assess post-segmentation features. In addition, building instantiation is performed using cluster analysis algorithms. Training this model to fit various datasets requires a large amount of annotated data, both from Quebec City, Canada, and from simulated data. Experimental results show that the use of simulated data improves segmentation accuracy, and the DBScan algorithm proves effective in extracting isolated buildings. This project paves the way for improved applications in 3D urban modeling, offering opportunities in fields ranging from urban planning to resource management. The positive results with simulated data reinforce the impact of this research on improving digital models of our ever-changing urban environments.

CNN-based off-the-shelf features have shown themselves as a good baseline for trademark retrieval. However, in recent years, the computer vision area was transitioning from CNNs to a new architecture – Vision Transformer. In this paper, we investigate the performance of off-the-shelf features extracted with vision transformers and explore the effects of pre, post-processing, and pre-training on big datasets. We propose a method of joint usage of global and local features, which leverages the best aspects of both approaches. Experimental results on METU Trademark Dataset show that off-the-shelf features extracted with ViT-based models outperform off-the-shelf features from CNN-based models. The proposed method achieves the mAP value of 31.23, surpassing previous state-of-the-art results. We assume that the proposed approach for the trademark similarity evaluation will allow one to improve the protection of such data with the help of artificial intelligence methods. Moreover, this approach will allow one to identify cases of unfair use of such data and form an evidence base for litigation.

Image registration (IR) is a process that deforms images to align them with respect to a reference space, making it easier for medical practitioners to examine various medical images in a standardized reference frame, such as having the same rotation and scale. This document introduces image registration using a simple numeric example. It provides a definition of image registration along with a space-oriented symbolic representation. This review covers various aspects of image transformations, including affine, deformable, invertible, and bidirectional transformations, as well as medical image registration algorithms such as Voxelmorph, Demons, SyN, Iterative Closest Point, and SynthMorph. It also explores atlas-based registration and multistage image registration techniques, including coarse-fine and pyramid approaches. Furthermore, this survey paper discusses medical image registration taxonomies, datasets, evaluation measures, such as correlation-based metrics, segmentation-based metrics, processing time, and model size. It also explores applications in image-guided surgery, motion tracking, and tumor diagnosis. Finally, the document addresses future research directions, including the further development of transformers.

Target detection and segmentation in synthetic aperture radar (SAR) images are vital steps for many remote sensing applications. In the era of data-driven deep learning, this task is extremely challenging due to the limited labeled data. Few-shot Learning has the ability to learn quickly from few samples with supervised information. Inspired by this, a few-shot learning framework named MSG-FN is proposed to solve the segmentation of ship targets in heterologous SAR images with few annotated samples. The proposed MSG-FN adopts a dual-branch network consisting of a support branch and a query branch. The support branch is used to extract features with an encoder, and the query branch uses a U-shaped encoder-decoder structure to segment the target in the query image. The encoder of each branch is composed of well-designed residual blocks combined with filter response normalization to capture robust and domain-independent features. A multi-scale similarity guidance module is proposed to improve the scale adaptability of detection by applying hand-on-hand guidance of support features to query features of various scales. In addition, a SAR dataset named SARShip-4i is built to evaluate the proposed MSG-FN and the experimental results show that the proposed method achieves superior segmentation results compared with the state-of-the-arts.

In present paper, spatial Minkowski Pythagorean Hodograph (MPH) curves are characterized with Rational Rotation Minimizing Frames (RRMFs). We define Euler-Rodrigues Frame (ERF) for MPH curves and by means of this concept, we reach the definition of MPH curves of type (n, m). Expressing the conditions provided by these curves in the form of Minkowski-Hopf map that we define, it is aimed to establish a connection with the Lorentz force which occurs during the process of Computer Numerical Control (CNC) type sinker Electronic Discharge Machines (EDMs). This approach is reinforced by split quaternion polynomials. Finally, we give conditons satisfied by MPH curves of low degree to be type (n, m) and construct illustrative examples.

Neural radiance fields (NeRF) leverage a neural representation to encode scenes, obtaining photo-realistic rendering of novel views. However, NeRF has notable limitations. A significant drawback is that it does not capture surface geometry and only renders the object surface colors. Furthermore, the training of NeRF is exceedingly time-consuming. We propose Depth-NeRF as a solution to these issues. Specifically, our approach employs a fast depth completion algorithm to denoise and complete the depth maps generated by RGB-D cameras. These improved depth maps guide the sampling points of NeRF to be distributed closer to the scene's surface, benefiting from dense depth information. Furthermore, we have optimized the network structure of NeRF and integrated depth information to constrain the optimization process, ensuring that the termination distribution of the ray is consistent with the scene’s geometry. Compared to NeRF, our method accelerates the training speed by 18% and significantly reduces the RMSE between the rendered scene depth and the ground truth depth, which indicates that our method can better capture the geometric information of the scene. With these improvements, we can train the NeRF model more efficiently and achieve more accurate rendering results.

In recent years, there has been a growing interest in the prediction of facial age due to its diverse applications in fields such as security, entertainment, and healthcare. The current scourge of underaged voting in Nigeria is a problem and this research delves into the realm of facial age prediction by employing four well-known convolutional neural network (CNN) architectures, namely VGG16, ResNet50, Mobile Net, and VGG19 to predict chronological ages from facial pictures of person. The objective is to achieve precise age estimation from facial images, utilizing two datasets: UTKFace and CASIA African Facial Datasets. The results of this investigation are noteworthy. Specifically, the VGG16 model which demonstrated remarkable performance, yielding a Mean Absolute Error (MAE) of 1.76 when applied to the UTK-Face dataset. Additionally, when utilizing Mobile-Net, an unprecedented MAE of 1.10 was achieved for the Casia Africa Face Dataset. Notably, this marks the first instance of employing the dataset for facial age detection with CNNs, and this approach outperformed previous works, yielding the lowest MAE among all the studies reviewed.

Binarization of degraded documents is an important preprocessing task for various document analysis such as OCR and historical document analysis. Existing studies have applied various convolutional neural network (CNN) models and generative models for document binarization, but they do not show generalized performance for noise that the model has not seen and it suffers from extracting elaborate text strokes. In this paper, to overcome these challenges, we utilize latent diffusion model (LDM), which is known for high-quality image generation model, for the first time in document binarization. By utilizing the iterative diffusion-denoising process in latent space, it shows high-quality cleaned binarized image generation and high generalized performance through using both data distribution and time step while training. Additionally, we apply gated U-Net to the backbone network to preserve text strokes using trainable gating value. Gated convolution can extract elaborate text stroke by allowing the model to focus on text region by combining gating value and feature. Furthermore, we maximize the effectiveness of the proposed model by training it with a combination of LDM loss and pixel-level loss, which is suitable for the model structure. Experiments on H-DIBCO and DIBCO benchmark datasets show that the proposed model outperforms existing methods.

Digital images have become an important carrier for people to access information in the information age. However, with the development of the technology, digital images are vulnerable to illegal access and tampering, to the extent that they pose a serious threat to personal privacy, social order and national security. Therefore, image forensic techniques have become an important research topic in the field of multimedia information security. In recent years, deep learning technology has been widely applied in the field of image forensics and the performance achieved has significantly exceeded the conventional forensic algorithms. This survey compares the state-of-the-art image forensic techniques based on deep learning in recent years. The image forensic techniques are divided into passive and active forensics. In passive forensics, forgery detection techniques are reviewed, and the basic framework, evaluation metrics and commonly used datasets for forgery detection are presented. The performance, advantages and disadvantages of existing methods are also compared and analyzed according to different types of detection. In active forensics, robust image watermarking techniques are overviewed, the evaluation metrics and basic framework of robust watermarking techniques are presented. The technical characteristics and performance of existing methods are analyzed based on the different types of attacks on images. Finally, future research directions and conclusions are given to provide useful suggestions for people in image forensics and related research fields.

Person Re-identification is the task of recognizing comparable subjects across a network of nonoverlapping cameras. This is typically achieved by extracting from the source image a vector of characteristic features of the specific person captured by the camera. Learning a good set of robust, invariant and discriminative features is a complex task, often leveraging contrastive learning. In this article, we explore a different approach, learning the representation of an individual as the conditioning information required to generate images of the specific person starting from random noise. In this way we decouple the identity of the individual from any other information relative to a specific instance (pose, background, etc.), allowing interesting transformations from one identity to another. As generative models, we use the recent diffusion models that have already proven their sensibility to conditioning in many different contexts. The results presented in this article serve as a proof-of-concept. While our current performance on common benchmarks is lower than state-of-the-art techniques, the approach is intriguing and rich of innovative insights, suggesting a wide range of potential improvements along various lines of investigation.

As we all know, many more dynamic objects appear almost continuously in real world, which are immensely able to impair the performance of the majority of vision-based SLAM systems that are based on the static-world assumption. In order to improve the robustness and accuracy of visual SLAM in the high dynamic environment, a real-time and robust stereo SLAM system for dynamic scenes was proposed. To weaken influences of dynamic content, the moving object detection method was put forward in our visual odometry, then the semantic segmentation network was combined into our stereo SLAM to extract pixel-level contours of dynamic objects. Then influences of dynamic objects were extremely weaken and the performance of our system was increased markedly in dynamic, complex and crowed city spaces. Experiment with both on KITTI Odometry dataset and in a real-life scene, the results show that our method can dramatically decrease the tracking error or drift, improve the robustness and stability of our stereo SLAM in high dynamic outdoor scenarios.

There is a considerable amount of research in online social networks, most of which focuses on the structural analysis of social graphs. The interpersonal relationships of social networks, especially friend circle, can solve the cold start and sparsity problems, and through the relationship between social networks can effectively recommend users' favorite items (items), such as music , videos, brands/products, preferred tags, location, services, etc. User relationships in social networks are diverse and there are many different perspectives on different social networks. Associations among users can form multi-layered composite networks, and multi-layered social networks present new challenges and opportunities. Different relationships can influence users' preferences to different degrees, which in turn affects their behavior. Therefore, fusing multiple social networks is an effective way to improve recommendation. Although some studies have started to address multiple social network recommendations, simple linear superposition cannot reflect the coupling and nonlinear association between multiple social networks. In this paper, we propose a graph neural network recommendation model under social relationships based on this background. We first propose to compute the 2nd order collaborative signals and their intensities directly from the neighboring matrix for updating the node embedding of the graph convolution layer. Secondly by embedding historical evaluations, various social networks constituting different dimensions, the attention integration of user preferences by different social networks is achieved, and its effectiveness and scalability are demonstrated in theoretical derivation and experimental validation. The theoretical derivation and experimental validation demonstrate its effectiveness and scalability.

Infrared dim and small target detection is a key technology of various detection tasks. However, due to the lack of shape, texture, and other information, it is a challenging task to detect dim and small targets. Recently, since many traditional algorithms ingore global information of infrared images, they generate some false alarms in complicated environments. To address this problem, in this paper, a coarse-to-fine deep learning-based method is proposed to detect dim and small targets. Firstly, a coarse-to-fine detection framework integrating deep learning and background prediction is applied for detecting targets. The framework contains of coarse detection module and fine detection module. In coarse detection stage, Region Proposal Network (RPN) is employed to generate masks in target candidate regions. Then, for further optimize the result, inpainting is utilized to predict the background by the global semnantic of images. In this paper, inpainting algorithm with a mask aware dynamic filtering module is incorporated into the fine detection stage to estimate background in candidate target areas. Finally, compared with existing algorithms, the experimental results indicate that the proposed framework have effective detection capability and robustness for complex surroundings.

Forest fires are one of the world's deadliest natural disasters. Early detection of forest fires can help minimize the damage to ecosystems and forest life. In this paper, we propose an improved fire detection method YOLOv5-IFFDM for YOLOv5. Firstly, the fire and smoke detection accuracy and the network perception accuracy of small targets are improved by adding attention mechanism in the backbone network. Secondly, the loss function is improved and the SoftPool pyramid pooling structure is used to improve the regression accuracy and detection effect of the model and the robustness of the model. In addition, random Mosaic augmentation technique is used to enhance the data to increase the generalization ability of the model, and re-clustering of flame and smoke detection a priori frames are used to improve the accuracy and speed. Finally, the parameters of the convolutional and normalization layers of the trained model are homogeneously merged to further reduce the model responsibility to improve the detection speed. Experimental results on homemade forest fire and smoke datasets show that this algorithm has high detection accuracy and fast detection speed, with average accuracy of fire up to 90.5% and smoke up to 84.3%, and detection speed up to 75 FPS (frames per second transmission), which can meet the requirements of real-time and efficient fire detection.

Infrared detection plays an important role in the military, aerospace, and other fields, which has the advantages of all-weather, high stealth, and strong anti-interference. However, infrared dim small target detection suffers from complex backgrounds, low signal-to-noise ratio, blurred targets with small area percentages, and other challenges. In this paper, we proposed a multiscale YOLOv5-AFAM algorithm to realize high-accuracy and real-time detection. Aiming at the problem of target intra-class feature difference and inter-class feature similarity, the Adaptive Fusion Attention Module - AFAM was proposed to generate feature maps that are calculated to weigh the features in the network and make the network focus on small targets. This paper proposed a multiscale fusion structure to solve the problem of small and variable detection scales in infrared vehicle targets. In addition, the downsampling layer is improved by combining Maxpool and convolutional downsampling to reduce the number of model parameters and retain the texture information. For multiple scenarios, we constructed an infrared dim and small vehicle target detection dataset, ISVD. The multiscale YOLOv5-AFAM was conducted on the ISVD dataset, compared to YOLOv7, [email protected] achieves a small improvement while the parameters are only 17.98% of it. By contrast with the YOLOv5s model, [email protected] was improved by 4.3% with a 6.6% reduction in the parameters. Experiments results demonstrate that the multiscale YOLOv5-AFAM has a higher detection accuracy and detection speed on infrared dim and small vehicles.

In emergency situations, every second counts for an ambulance navigating through traffic. Efficient use of traffic light systems can play a crucial role in minimizing response time. This paper introduces a novel automated Real-Time Ambulance in an Emergency Detector (RTAIAED). The proposed system uses special Lookout Stations (LS) suitably positioned at a certain distance from each involved traffic light (TL), to obtain timely and safe transitions to green lights as the Ambulance in an Emergency (AIAE) approaches. The RTAIAED is particularly pertinent on one-way roads, addressing the challenge of regulating the sequence of traffic-lights-signals so as to minimize the time needed to safely grant the green signal to the AIAE. The proposed solution leverages a part-based model made of elementary detectors for video analysis, specifically realized with a customized YOLOv8 model, and for audio analysis, thanks to an additional neural network based on Mel Frequency Cepstral Coefficients (MFCCs). This way, the RTAIAED ensures the accurate and robust identification of an AIAE heading towards a traffic light in time to ensure a green light thanks to the strategic positioning of the LS detectors. Extensive experiments demonstrate the robustness of the approach and its reliable application in real-world scenarios thanks to its predictions in real-time, showcasing the ability to detect AIAEs even in challenging conditions such as noisy environments, nighttime, or adverse weather conditions, provided a suitable-quality camera is appropriately positioned. The proposed system can also find application in traffic flow management.

Underwater target detection is widely used in various applications such as underwater search and rescue, underwater environment monitoring, and Marine resources survey. However, the visibility of the underwater environment and the accuracy of target detection can be affected by complex underwater light changes and unpredictable background noise. To address these issues, we propose an improved underwater target detection algorithm based on YOLOv8n. Our algorithm focuses on three aspects. Firstly, we replace the original C2f module with Deformable Convnets v2 to enhance the adaptive ability of the target region in the convolution check feature map and extract the target region's features more accurately. Secondly, we introduce SimAm, a non-parametric attention mechanism, which can deduce and assign three-dimensional attention weights without adding network parameters. Lastly, we optimize the loss function by replacing the CIOU loss function with the Wise-IOU loss function. To conduct our experiments, we create our own dataset of underwater target detection for experimentation. Meanwhile, we also utilized the Pascal VOC dataset to evaluate our approach. The [email protected] and [email protected]:0.95 of the original YOLOv8n algorithm on the underwater target detection were 88.6% and 51.8%, respectively, and the improved algorithm [email protected] and [email protected]:0.95 can reach 91.8% and 55.9%. The original YOLOV8n algorithm was 62.2% and 45.9% [email protected] and [email protected]:0.95 on the Pascal VOC dataset, respectively. The improved YOLOV8n algorithm [email protected] and [email protected]:0.95 were 65.7% and 48.3%, respectively. The floating-point computation volume of the model is reduced by about 6%. The above experimental results prove the effectiveness of our method.

Augmented reality (AR) offers an accessible, inexpensive, and rich user experience that has the potential to engage end-users in an immersive environment. Along with vivid visualizations coupled with virtual agents, this technology further develops learning interest in end-users, guides them in various tasks, and boosts motivation and productivity. In this study, we leverage the deep penetration of mobile phones in daily lives and their advanced features to design, develop and demonstrate an AR application (CirculAR) that offers unique user-environment interaction in a gamified way. CirculAR combines learning with enjoyment to help end-users understand fundamental sustainability and circular economy principles. The application has been showcased in a controlled environment to heterogenous audiences and has been shown to improve end-user engagement and motivation. First, participants older than 18 were recruited to showcase the technology acceptance and engagement towards circular economy principles through AR. Then, students aged 5 to 15 years old, along with their parents and educators, were invited to a treasure hunt game where our virtual agent ARis guided them through a map full of virtual experiences. Assessment and evaluation were performed through a survey and a questionnaire. The outcome of their analysis showcased an increase in the dedication and enjoyment of the performed activities, engagement and learning attributes given the AR virtual agent supporting functionalities. Observations during showcasing reported a need for more commitment from the younger audience compared to the older one. This application contributes to the discourse on mobile AR as a tool for the education of novel concepts with a high impact on our daily lives and decisions and aims to shed light on the design principles of educative tools.

We prove a version of Ekeland Variational Principle (EkVP) in a weighted graph $G$ and its equivalence to Caristi fixed point theorem and to Takahashi minimization principle. The usual completeness and topological notions are replaced with some weaker versions expressed in terms of the graph $G$. The main tool used in the proof is the OSC property for sequences in a graph. Converse results, meaning the completeness of graphs for which one of these principles holds is also considered.

This research paper presents a deep learning approach to early detection of skin cancer using image augmentation techniques. The authors propose a two-stage image augmentation technique that involves the use of geometric augmentation and generative adversarial network (GAN) to classify skin lesions as either benign or malignant. This research utilized the public HAM10000 dataset to test the proposed model. Several pre-trained models of CNN were employed, namely Xception, Inceptionv3, Resnet152v2, EfficientnetB7, InceptionresnetV2, and VGG19. Our approach achieved accuracy, precision, recall, and F1-score of 96.90%, 97.07%, 96.87%, 96.97%, respectively, which is higher than the performance achieved by other state-of-the-art methods. The paper also discusses the use of SHapley Additive exPlanations (SHAP), an interpretable technique for skin cancer diagnosis, which can help clinicians understand the reasoning behind the diagnosis and improve trust in the system. Overall, the proposed method presents a promising approach to automated skin cancer detection that could improve patient outcomes and reduce healthcare costs.

The integration of deep learning methods, especially Convolutional Neural Networks (CNN), and Synthetic Aperture Radar Automatic Target Recognition (SAR ATR) has been widely deployed in the field of radar signal processing. Nevertheless, these methods are frequently regarded as black-box models due to the limited visual interpretation of their internal feature representation and parameter organization. In this paper, we propose an innovative approach named C-RISE, which builds upon the RISE algorithm to provide a post-hoc interpretation technique for black-box models used in SAR Images Target Recognition. C-RISE generates saliency maps that effectively visualize the significance of each pixel. Our algorithm outperforms RISE by clustering masks that capture similar fusion features into distinct groups, enabling more appropriate weight distribution and increased focus on the target area. Furthermore, we employ Gaussian blur to process the masked area, preserving the original image structure with optimal consistency and integrity. C-RISE has been extensively evaluated through experiments, and the results demonstrate superior performance over other interpretation methods based on perturbation when applied to neural networks for SAR image target recognition. Furthermore, our approach is highly robust and transferable compared to other interpretable algorithms, including white-box methods.

Existing methods for 3D human pose estimation mainly divide the task into two stages. The first stage identifies the 2D coordinates of the human joints in the input image, namely the 2D human joint coordinates. The second stage uses the results from the first stage as input to recover the depth information of human joints from the 2D human joint coordinates to achieve 3D human pose estimation. However, the recognition accuracy of the two-stage method relies heavily on the results of the first stage and includes too many redundant processing steps, which reduces the inference efficiency of the network. To address these issues, we propose the EDD, a fully End-to-end 3D human pose estimation method based on transformer architecture with Dual Decoders. By learning multiple human poses, the model can directly infer all 3D human poses in the image using a pose decoder, and then further optimize the recognition result using a joint decoder based on the kinematic relations between joints. With the attention mechanism, this method can adaptively focus on the most relevant features to the target joint, effectively overcoming the feature misalignment problem in the human pose estimation task and greatly improving the model performance. Any complex post-processing step, such as non-maximum suppression, is eliminated, further improving the efficiency of the model. The results show that the method achieves an accuracy of 87.4% on the MuPoTS-3D dataset, significantly improving the accuracy of end-to-end 3D human pose estimation methods.

Multispectral image (MSI) and hyperspectral image (HSI) fusion (MHIF) aims to address the challenge of acquiring high-resolution (HR) HSI images. This field combines a low-resolution (LR) HSI with an HR-MSI to reconstruct HR-HSI. Existing methods directly utilize transformers to perform feature extraction and fusion. Despite the demonstrated success, there exist two limitations: 1) Employing the entire transformer model for feature extraction and fusion fails to fully harness the transformer’s potential in integrating the spectral of the HSI and spatial information of the MSI. 2) HSI has a strong spectral correlation and exhibits sparsity in the spatial domain. Existing transformer-based models do not optimize this physical property, which makes their methods prone to spectral distortion. To accomplish these issues, this paper introduces a novel framework for MHIF called Sparse Mix-Attention Transformer (SAMformer). Specifically, to fully harness the advantages of the Transformer architecture, we propose a Spectral Mix Attention Block (SMAB), which concatenates the keys and values extracted from LR-HSI and HR-MSI to create a new multi-head attention module. This design facilitates the extraction of detailed long-range information across spatial and spectral dimensions. Besides, to address the spatial sparsity inherent in HSI, we incorporated a sparse mechanism within the core of SMAB called Sparse Spectral Mix Attention Block (SSMAB). In the SSMAB, we compute attention maps from queries and keys and select the K highly correlated values as the sparse attention map. This approach enables us to achieve a sparse representation of spatial information while eliminating spatially disruptive noise. Extensive experiments conducted on three benchmark datasets, namely Cave, Harvard, and Pavia Center, demonstrate the SMAformer method outperforms state-of-the-art methods.

Abstract: The Micro-Electro-Mechanical System (MEMS) LiDAR point cloud in autonomous vehicles has a large deflection range, which results in slow registration speed and poor applicability. To maximize speed, an improved NDT (Normal Distribution Transform) method that integrates point cloud density features has been proposed. First, the point cloud is reduced using a modified voxel filter and a pass-through filter. Next, the ISS (Intrinsic Shape Signature) algorithm is utilized to analyze the point cloud features and extract keypoints, the 4PCS (4-Point Congruent Sets) algorithm is then employed to calculate the initial pose under the constraints of the keypoint set to complete the coarse registration. Finally, the self-adaptive segmentation model is constructed by using KD-tree to obtain the density features of keypoints, and the NDT algorithm is combined with this model to form an SSM-NDT algorithm, which is used for fine registration. Each algorithm was compared on the autonomous vehicle dataset PandaSet and actual collected data sets, the results show that the novel method increases the speed by at least 60% and takes into account good registration accuracy and strong anti-interference.

Water area segmentation in remote sensing is of great importance for flood monitoring. To overcome some challenges in this task, we construct the Water Index and Polarization Information (WIPI) multi-modality dataset and propose a multi-Modality Fusion and Gated multi-Filter U-Net (MFGF-UNet) convolutional neural network. The WIPI dataset enhances the water information and reduces the data dimension. In particular, the Cloud-Free label provided in the database effectively mitigates the issue of scarcity of labeled samples Since a single form or uniform kernel size cannot handle the variety in size and shape of water bodies, we proposed the gated multi-filter inception (GMF-Inception) module in our MFGF-UNet. Moreover, we utilize the attention mechanism by introducing the Gated Channel Transform (GCT) skip connection and integrating GCT into GMF- Inception to further improve model performance. Extensive experiments on three benchmarks, including WIPI, Chengdu and GF2020 datasets demonstrate that our method achieves favorable performance with lower complexity and better robustness against six competing approaches. For example, on WIPI, Chengdu and GF2020 datasets, the proposed MFGF-UNet model achieves F1 of 0.9191, 0.7410, and 0.8421, respectively, which average F1 on three datasets is 0.0045 higher than the U-Net model and reduces 62% average GFLOPs. New WIPI dataset, code and trained models have been released on GitHub

Grapes are a globally popular fruit, with grape cultivation worldwide being second only to citrus. This article focuses on the low efficiency and accuracy of traditional manual grading of red grape external appearance and proposes a small-sample red grape external appearance grading model based on transfer learning with convolutional neural networks (CNNs). Initially, the CNN transfer learning method was used to transfer the pre-trained AlexNet, VGG16, GoogleNet, InceptionV3, and ResNet50 network models on the ImageNet image dataset to the red grape image grading task. By comparing the classification performance of the CNN models of these five different network depths with fine-tuning, ResNet50 with a learning rate of 0.001 and a loop number of 10 was determined to be the best feature extractor for red grape images. Moreover, given the small number of red grape image samples in this study, different convolutional layer features output by the ResNet50 feature extractor were analyzed layer by layer to determine the effect of deep features extracted by each convolutional layer on SVM classification performance. This analysis helped to obtain a ResNet50+SVM red grape external appearance grading model based on the optimal ResNet50 feature extraction strategy. Experimental data showed that the classification model constructed using the feature parameters extracted from the 10th node of the ResNet50 network achieved an accuracy rate of 95.08% for red grape grading. These research results provide a reference for the online grading of red grape clusters based on external appearance quality and have certain guiding significance for the quality and efficiency of grape industry circulation and production.

The digital orthophoto is an image with geometric accuracy and no distortion. It is acquired through a top view of the scene and finds widespread applications in map creation, planning, and related fields.This paper classifies the algorithms for digital orthophoto generation into two groups: explicit methods and implicit methods. Explicit methods rely on traditional geometric methods, obtaining geometric structure presented by explicit parameters with Multi-View Stereo (MVS) theories, as seen in our proposed Top view constrained Dense Matching (TDM). Implicit methods rely on neural rendering, obtaining implicit neural representation of scenes through the training of neural networks, as exemplified by Neural Radiance Fields (NeRF). Both of them obtain digital orthophoto by rendering from a top view perspective. In addition, this paper conducts an in-depth comparative study between explicit and implicit methods. The experiments demonstrate that both algorithms meet the measurement accuracy requirements and exhibit a similar level of quality in terms of generated results. Importantly, the explicit method shows a significant advantage in terms of efficiency, with a time consumption reduction of two orders of magnitude under our latest CUDA version TDM algorithm. Although explicit and implicit methods differ significantly in their representation forms, they share commonalities in the implementation across algorithmic stages. These findings highlight the potential advantages of explicit methods in orthophoto generation, while also providing beneficial references and practical guidance for the fast digital orthophoto generation using implicit methods.

The point cloud is a form of three-dimensional data that comprises various detailed features at multiple scales. Due to this characteristic and its irregularity, point cloud analysis based on deep learning is challenging. While previous works utilize the sampling-grouping operation of PointNet++ for feature description and then explore geometry by means of sophisticated feature extractors or deep networks, such operations fail to describe multi-scale features effectively. Additionally, these techniques have led to performance saturation. And it is difficult for standard MLPs to directly "mine" point cloud. To address above problems, we propose the Detail Activation (DA) module, which encodes data based on Fourier transform after sampling and grouping. We activate the channels at different frequency levels from low to high in the DA module to gradually recover finer point cloud details. As training progresses, the proposed Point-MDA can uncover local and global geometries of point cloud progressively. Our experiments show that Point-MDA achieves superior classification accuracy, outperforming PointNet++ by 3.3% and 7.9% in terms of overall accuracy on the ModelNet40 and ScanObjectNN dataset, respectively. Furthermore, it accomplishes this without employing complicated operations, while exploring the full potential of PointNet++.

There has been considerable progress in implicit neural representation to upscale an image to any arbitrary resolution. However, existing methods are based on defining a function to predict the RGB value from just four specific loci. Relying on just four loci is insufficient as it leads to losing fine details from the neighboring region(s). We show that by taking into account the semi-local region leads to an improvement in performance. In this paper, we propose applying a new technique called Overlapping Windows on Semi-Local Region (OW-SLR) to an image to obtain any arbitrary resolution by taking the coordinates of the semi-local region around a point in the latent space. This extracted detail is used to predict the RGB value of a point. We illustrate the technique by applying the algorithm to the Optical Coherence Tomography-Angiography (OCT-A) images and show that it can upscale them to random resolution. This technique outperforms the existing state-of-the-art methods when applied to the OCT500 dataset. OW-SLR provides better results for classifying healthy and diseased retinal images such as diabetic retinopathy and normals from the given set of OCT-A images. The project page is available at https://rishavbb.github.io/ow-slr/index.html

A close-range photogrammetric approach and its implementation using a CNC device and a macro camera are proposed. A tailored image acquisition approach is proposed that is implemented using this device. To increase reconstruction robustness and accuracy, the key point features detected in the acquired images are tracked across multiple views from multiple viewpoints at multiple distances. This approach reduces spurious correspondences and, as a result, the estimation accuracy of calibration parameters is increased and reconstruction errors are reduced. Qualitative and quantitative evaluation demonstrate the efficacy and accuracy of the proposed approach, which exhibits micrometre resolution and low implementation cost.

People are drawn to woodcut-style designs due to their striking visual impact and strong contrast. However, traditional woodcut prints and previous computer-aided methods have not addressed the issues of dwindling design inspiration, lengthy production times, and complex adjustment procedures. We propose a novel network framework, the Woodcut-style Design Assistant Network (WDANet), to tackle these challenges. Notably, our research is the first to utilize diffusion models to streamline the woodcut-style design process. We've curated the Woodcut-62 dataset, featuring works from 62 renowned historical artists, to train WDANet in absorbing and learning the aesthetic nuances of woodcut prints, offering users a wealth of design references. Based on a noise reduction network, our dual cross-attention mechanism effectively integrates text and woodcut-style image features. This allows users to input or slightly modify a text description to quickly generate accurate, high-quality woodcut-style designs, saving time and offering flexibility. As confirmed by user studies, quantitative and qualitative analyses show that WDANet outperforms the current state-of-the-art in generating woodcut-style images and proves its value as a design aid.

In recent years, the analysis of macro- and micro-expression has drawn the attention of researchers since they provide visual cues of an individual's emotions for a broad range of potential applications such as lie detection and criminal detection. In this paper, we address the challenge of spotting facial macro- and micro-expression from videos and present compelling results by using a deep learning approach to analyze the optical flow features. Different from other deep learning approaches that are mainly based on Convolutional Neural Networks (CNNs), we propose a Transformer-based deep learning approach that predicts a score indicating the probability of a frame being within an expression interval. In contrast to other Transformer-based models that achieve high performance by being pre-trained on large datasets, our deep learning model, called SL-Swin, which incorporates Shifted Patch Tokenization and Locality Self-Attention into the backbone network Swin Transformer, effectively spots macro- and micro-expressions by being trained from scratch on small-size expression datasets. Our evaluation outcomes surpass the MEGC 2022 spotting baseline result, obtaining an overall F1-score of 0.1366. Additionally, our approach also performs well in the MEGC 2021 spotting task with an overall F1-score of 0.1824 and 0.1357 on CAS(ME)^2 and SAMM Long Videos, respectively. The code is publicly available on GitHub (https://github.com/eddiehe99/pytorch-expression-spotting).

This paper presents the development of an underwater crack detection system for structural integrity assessment of submerged structures, like offshore oil and gas installations, underwater pipelines, underwater foundations for bridges, dams etc. Focus is on use of machine learning based approaches. First a detailed literature review of state of the current methods for underwater surface crack detection is presented highlighting challenges and opportunities. An overview of image augmentation approach for creation of underwater optical effects is also presented. Experimental results using standard network based machine learning approach, used for surface crack detection in onshore environment, is presented. Series of Test cases are presented where existing networks performance are improved using augmented images for underwater conditions. The experimental results demonstrate the effectiveness and accuracy of the proposed system in detecting cracks in underwater structures. The system has the potential to improve the safety and reliability of underwater structures and prevent catastrophic failures.

During the creation and integration of a health care system based on medical knowledge graphs, it is necessary to review and select the vocabularies and definitions that best fit the information requirements of the system being developed. This implies the reuse of medical knowledge graphs; however, full importation of knowledge graphs is not a tractable solution in terms of memory requirements. In this paper we present a modularization-based method for knowledge graph reuse. A case study of graph reuse is presented by transforming the original model into a lighter one.

When the image quality is evaluated, the human visual system (HVS) infers the details in the image through its internal generative mechanism. In this process, the HVS integrates both local and global information of the image, utilizes contextual information to restore the original image information, and compares it with the distorted image information for image quality evaluation. Inspired by this mechanism, a no-reference image quality assessment method is proposed based on a multitask image restoration network. The multitask image restoration network generates a pseudo-reference image as the main task and produces structural similarity index measure map as an auxiliary task. By mutually promoting the two tasks, a higher quality pseudo-reference image is generated. In addition, when predicting the image quality score, both the quality restoration features and the difference features between the distorted and reference images are used, thereby fully utilizing the information from the pseudo-reference image. To enable the model to focus on both global and local features, a multi-scale feature fusion module is proposed. Experimental results demonstrate that the proposed method achieves excellent performance on both synthetically and authentically distorted databases.

Object detection is complex and involved diverse requirements for different applications. In critical application such as visual impaired navigation guide and real-time surveillance for instance, identifying a particular object of instance is required and this involved a complex approach for realization. Literatures were reviewed, starting with the application of deep learning techniques for object detection; their gaps were identified and addressed using real-time object detection models literature review. From the review gaps were also detected and addressed using literature review on occlusion detection techniques. Object of instance detection in clustered scene with similar object of interest was identified as an open gap not addressed in the existing literature; even through it is vital for many applications. This research recommended an Occlusion Based Object of Instance Detection (OBOID) technique which used the spatial information to identify instant object of interest in clustered scene with many similarly object of interest. Limitation of the recommended OBOID is that it requires only system where position and distance of object is necessary to inform other decision.

The images captured by UAVs during inspection often contain numerous small targets related to transmission lines, which are critical and vulnerable elements for ensuring the safe operation of these lines. However, due to various factors such as the small size of the targets, low resolution, complex background, and potential line aggregation, achieving accurate and real-time detection becomes challenging. To address these issues, this paper proposes a detection algorithm called P2-ECA-EIOU-YOLOv5 (P2E-YOLOv5). Firstly, in order to address the challenges posed by the complex background and environmental interference that impact small targets, an ECA attention module is integrated into the network. This module effectively enhances the network's focus on small targets while concurrently mitigating the influence of environmental interference. Secondly, considering the characteristics of small target size and low resolution, a new high-resolution detection head is introduced, which is more sensitive to small targets. Lastly, the network utilizes the EIOU-Loss as the regression Loss function to improve the positioning accuracy of small targets, as they tend to aggregate. Experimental results demonstrate that the proposed P2E-YOLOv5 detection algorithm achieves an accuracy P of 96.0% and an average accuracy (mAP) of 97.0% for small target detection in transmission lines.

Human action recognition (HAR) based on skeleton data is a challenging yet important technique because of its wide range of applications in many fields, including patient monitoring, security surveillance, and observing human-machine interactions. Many algorithms that attempt to distinguish between many types of activities have been proposed. However, most practical applications require highly accurate detection of specific types of activities. In this study, a novel and highly accurate spatiotemporal graph autoencoder network for HAR based on skeleton data is proposed. Furthermore, an extensive study was conducted using different modalities. For this purpose, a spatiotemporal graph autoencoder that automatically learns spatial as well as temporal patterns from human skeleton datasets was built. The powerful graph convolutional network named GA-GCN, developed in this study, notably outperforms most of the existing state-of-the-art methods based on two common datasets, namely NTU RGB+D and NTU RGB+D 120 . On the first dataset, we achieved an accuracy of 92.3% and 96.7% based on the cross-subject and cross-view evaluations, respectively. On the other more challenging dataset (i.e. NTU RGB+D 120), GA-GCN achieved 88.8% and 90.4% based on the cross-subject and cross-set evaluation, respectively.

In the realm of autonomous vehicle environment perception, the primary objective of point cloud target detection is to swiftly and accurately identify three-dimensional objects from point cloud data. To meet this requirement, the prevalent network architecture employed in the industry is the voxel-based PointPillars model. Nonetheless, this model faces challenges in maintaining detection accuracy when objects are obscured or diminutive in size. In response to this issue, we introduce AGPNet, a novel model that seamlessly integrates four key modules: Data Augmentation, Dynamic Graph CNN, Pillar Feature Net, and Detection Head (SSD). The Data Augmentation module enhances the adaptability of point cloud data to complex and ever-changing real-world environments. The Dynamic Graph CNN module endows the network structure with geometric features, which encapsulate not only the point itself but also its adjacent points. The Pillar Feature Net module translates three-dimensional point cloud data into pseudo-image data through the utilization of voxels. Subsequently, the Detection Head (SSD) module leverages this pseudo-image data to conduct target detection of three-dimensional objects. Our experiments, conducted on the KITTI dataset, demonstrate that our proposed method boosts object detection accuracy by 6-7 percentage points compared to the PointPillars model, while maintaining similar detection times.

Deep learning has been introduced to single-image super-resolution (SISR) in the last decade. These techniques have taken over the benchmarks of SISR tasks. Nevertheless, most architectural designs necessitate substantial computational resources, leading to a prolonged inference time on embedded systems or rendering them infeasible for deployment. This paper presents a comprehensive survey of plausible solutions and optimization methods to address this problem. Then, we propose a pipeline that aggregates the latter in order to enhance the inference time without significantly compromising the perceptual quality. We investigate the effectiveness of the proposed method on a lightweight GAN-based perceptual-oriented model as a case study. The experimental results show that our proposed method leads to significant improvement in the inference time on both Desktop and Jetson Xavier NX, especially for higher resolution input sizes on the latter, thereby making it deployable in practice.

Computer vision models of salient object detection attempt to mimic the ability of the human visual system to select relevant objects in images. To this end, the development of deep neural networks on high-end computers has recently made it possible to achieve high performance. However, it remains a challenge to develop deep neural network models of the same performance for devices with much more limited resources. In this work, we propose a new approach for a lightweight salient object detection neural network model, inspired by the cone and spatial opponent processes of the primary visual cortex (V1), that inextricably link color and shape in human color perception. Our proposed model, namely CoSOV1net, is trained from scratch, without using backbones from image classification or other tasks. Experiments, on the most widely used and challenging datasets for salient object detection, show that CoSOV1Net achieves competitive performance (i.e. Fβ=0.931 on the ECSSD dataset) with state-of-the-art salient object detection models, while having low number of parameters (1.14M), low FLOPS (1.4G) and high FPS (211.2) on GPU (nvidia Geforce RTX 3090 TI) compared to the state-of-the-art in the salient object detection or lightweight salient object detection task. Thus, CoSOV1net turns out to be a lightweight salient object detection that can be adapted to mobile environments and resource-constrained devices.

In the past few decades, the transmission of data over an unsecure channel resulted in an increased rate of hacking. Therefore, the requirement to make multimedia data more secure is increasing day by day. Numerous algorithms are developed to improve efficiency and robustness. In this article, a novel and secure image encryption algorithm is presented. It is based on a modified chaotic logistic map (CLM) that provides the advantage of having less computational time to encrypt an input image. The encryption algorithm is based on Shannon’s idea of using substitution-permutation and one-time-pad network to achieve ideal secrecy. The CLM is used for substitution and permutation to improve randomness and increase dependency on the encryption key. Various statistical tests are conducted, such as Key Space Analysis, Complexity Analysis, Sensitivity Analysis, Strict Avalanche Criteria (SAC), Histogram Analysis, Entropy Analysis, Mean of Absolute Deviation (MAD) Analysis, Correlation Analysis, Contrast Analysis, and Homogeneity to give a comparative analysis of the proposed algorithm and verify its security. As a result of various statistical tests, it is evident that the proposed algorithm is more efficient and robust as compared to previous ones.

In this paper, an intelligent blind guide system based on 2D LiDAR and RGB-D camera sensing is proposed, and the system is mounted on the smart cane. The intelligent guide system relies on 2D LiDAR, RGB-D camera, IMU, GPS, Jetson nano B01, STM32 and other hardware. The main advantage of the intelligent guide system proposed by us is that the distance between the smart cane and obstacles can be measured by 2D LiDAR based on Cartographer algorithm, thus achieving Simultaneous localization and mapping (SLAM). At the same time, through the improved yolov5 algorithm, pedestrians, vehicles, pedestrian crosswalks, traffic lights, warning posts, stone piers, tactile paving and other objects in front of the visually impaired can be quickly and effectively identified. Laser SLAM and improved yolov5 obstacle identification tests were carried out inside a teaching building on the campus of Hainan Normal University and on a pedestrian crossing on Longkun South Road in Haikou City, Hainan Province. The results show that the intelligent guide system developed by us can drive the wheels and omnidirectional wheels at the bottom of the smart cane, and give the smart cane a self-leading blind guide function like a "guide dog", which can effectively guide the visually impaired to avoid obstacles and reach the predetermined destination, and can quickly and effectively identify the obstacles on the way out. The laser SLAM speed of this system is 25~31FPS, which can realize the short-distance obstacle avoidance and navigation function both in indoor and outdoor envionments. The improved yolov5 helps to identify 86 types of objects, the recognition rate for pedestrian crosswalks and for vehicles are 84.6% and 71.8%, respectively; the overall recognition rate for 86 types of objects is 61.2%, and the obstacle recognition rate of the intelligent guide system is 25-26FPS.

Previous camera self-calibration methods have exhibited certain notable shortcomings. On one hand, they either exclusively emphasized scene cues or solely focused on vehicle-related cues, resulting in a lack of adaptability to diverse scenarios and a limited number of effective features. Furthermore, these methods either solely utilized geometric features within traffic scenes or exclusively extracted semantic information, failing to comprehensively consider both aspects. This limited the comprehensive feature extraction from scenes, ultimately leading to a decrease in calibration accuracy. Additionally, conventional vanishing point-based self-calibration methods often required the design of additional edge-background models and manual parameter tuning, thereby increasing operational complexity and the potential for errors. Given these observed limitations, and in order to address these challenges, we propose an innovative roadside camera self-calibration model based on the Transformer architecture. This model possesses a unique capability to simultaneously learn scene features and vehicle features within traffic scenarios while considering both geometric and semantic information. Through this approach, our model can overcome the constraints of prior methods, enhancing calibration accuracy and robustness while reducing operational complexity and the potential for errors. Our method outperforms existing approaches on both real-world dataset scenarios and publicly available datasets, demonstrating the effectiveness of our approach.

This paper proposes an algorithm for transmitting and reconstructing the estimated point cloud using temporally estimating a dynamic point cloud sequence. When a non-rigid 3D point cloud sequence (PCS) is input, the sequence is divided into groups of point cloud frames (PCF), and a key PCF is selected. After the 3D skeleton is predicted through 3D pose estimation, the motion of the skeleton is estimated by analyzing the joints and bones of the 3D skeleton. For the deformation of the non-rigid human PC, the 3D PC model is transformed into a mesh model, and the key PCF is rigged using the 3D skeleton. After deforming the key PCF into the target PCF utilizing the motion vector of the estimated skeleton, the residual PC between the motion compensation PCF and the target PCF is generated. If there is a key PCF, the motion vector of the target PCF, and a residual PC, the target PCF can be reconstructed. Just as compression is performed using pixel correlation between frames in a 2D video, this paper compresses 3D PCF by estimating the non-rigid 3D motion of a 3D object in a 3D PC. The proposed algorithm can be regarded as an extension of the 2D motion estimation of a rigid local region in a 2D plane to the 3D motion estimation of a non-rigid object (human) in 3D space. Experimental results show that the proposed method can successfully compress 3D PC sequences. If it is used together with a PC compression technique such as MPEG PCC (point cloud compression) in the future, a system with high compression efficiency may be configured.

This paper introduces PatchFusion, an innovative approach for nonrigid tracking and reconstruction of deformable objects using a single RGB-D sensor. Existing methods face challenges in accurately capturing the rapid deformations of soft and flexible objects, thereby limiting their utility in diverse scenarios. Our approach overcomes this challenge by employing a dynamic patch-based framework that adapts to rapid inter-frame motions. Firstly, patch-wise rigid transformation fields for non-overlapping patches are solved via Iterative Closest Point (ICP) by incorporating geometric features as additional similarity constraints, thereby enhancing robustness and accuracy. Secondly, deformation optimization based on a nonrigid solver is applied to refine the coarse transformation fields. In order to enable simultaneous tracking and reconstruction of deformable objects, the patch-based rigid solver is designed to run in parallel with the nonrigid solver, serving as a plug-and-play module requiring minimal modifications for integration while enabling real-time performance. Following a comprehensive evaluation, PatchFusion showcases superior performance in effectively dealing with rapid inter-frame deformations when compared to existing techniques, rendering it a promising solution with broad applicability across domains such as robotics, computer vision, and human-computer interaction.

Bone Scan Index (BSI) is an image biomarker for quantification on bone metastasis of cancers. To compute BSI, not only the hotspots (metastasis) but also the bones have to be segmented. Most related researches focus on binary classification on bone scintigraphy: having metastasis or none. Rare studies focus on pixelwise segmentation. In this study we compare three advanced convolutional neural network (CNN) based models to explore the bone segmentation on dataset in house. The best model is Mask R-CNN, which reaches the precision, sensitivity and F1-score: 0.93, 0.87, 0.90 for prostate cancer patients and 0.92, 0.86, 0.88 for breast cancer patients, respectively. The results are the average of 10-fold cross-validation, which reveals the reliability for clinical use on bone segmentation.

The research of object classification and part segmentation is a hot topic in computer vision. A considerable number of studies have been carried out about deep learning on 3D point clouds. However,it is challenging to achieve effective feature learning due to sparsity of point clouds. Recently, a variety of Transformers have been adopted to improve point cloud processing and display great potential. Nevertheless, large numbers of Transformer layers tend to incur huge computational and memory costs. PointNet++ is one of the most influential neural architectures for point cloud understanding. Although the accuracy of PointNet++ has been largely surpassed by recent networks, this does not mean that PointNet++ has no potential.Thus, this paper offer two major contributions that significantly improve PointNet++ performance. Firstly, we introduce a novel contextual feature extraction (CFE) block that significantly enhances the feature extraction capabilities of PointNet++ networks. Secondly, to further enhance feature fusion, we seamlessly integrate a mutual learning (ML) block into the network architecture. By embedding these two innovative blocks within each layer of the network, we not only enrich the network's functionality but also impart it with greater robustness and adaptability. The specific experiments were conducted on the S3DIS (6-fold cross-validation) and Modelnet40 datasets with 86.5% and 92.7% accuracy, respectively, which proved that our model is comparable or even better than most existing methods for classification and segmentation tasks.

Vein segmentation and projection correction constitute the core algorithms of auxiliary venipuncture device, responding to the accurate venous positioning to assist puncture and reduce the number of punctures and pain of patients. This paper proposes an improved U-Net for segmenting vein and a coaxial correction for image alignment in the self-build vein projection system. The proposed U-Net is embedded by Gabor convolution kernels in the shallow layers to enhance segmentation accuracy. Additionally, to mitigate the semantic information loss caused by channel reduction, the network model is lightweighted by the mean of replacing conventional convolutions with inverted residual blocks. During the visualization process, a method that combining coaxial correction and homography matrix is proposed to address the non-planarity of the dorsal hand in this paper. First, use a hot mirror to adjust the light paths of both projector and camera to be coaxial, and then align the projected image with dorsal hand using a homography matrix. Using this approach, the device requires only a single calibration before use. With the implementation of the improved segmentation method, an accuracy rate of 95.12% is achieved by the dataset. The intersection over union ratio between the segmented and original images is reached at 90.07%. The entire segmentation process is completed in a 0.09 second, and the largest distance error of vein projection onto dorsal hand is 0.53mm. Experiments show that the device has reached practical accuracy and has the value of research and application.

In the Versatile Video Coding (VVC) standard, affine motion models have been applied to enhance the resolution of complex motion patterns. However, due to the high computational complexity involved in affine motion estimation, real-time video processing applications face significant challenges. This paper focuses on optimizing affine motion estimation algorithms in the VVC environment and proposes a fast gradient iterative algorithm based on edge detection for efficient computation. Firstly, we establish judging conditions during the construction of affine motion candidate lists to streamline the redundant judging process. Secondly, we employ the Canny edge detection method for gradient assessment in the affine motion estimation process, thereby enhancing the iteration speed of affine motion vectors. Experimental results demonstrate that our affine motion estimation algorithm reduces encoding time by approximately 15%-35% while maintaining video bitrate and quality.

The Colony-Forming Unit (CFU) counting problem remains a complex issue without a universal solution in biomedical and food safety domains. A multitude of sophisticated heuristics and segmentation-driven approaches have been proposed by numerous researchers. Among those, U-Net is the most frequently cited and popular Deep Learning method. The latter approach provides a segmentation output map and requires an additional counting procedure which accounts for unique segmented regions and detected microbial colonies. However, because of pixel-based targets it tends to generate irrelevant artifacts or errant pixels, leading to inaccurate and mixed post-processing results. In response to these challenges, we propose a novel hybrid counting approach, incorporating a multi-loss U-Net reformulation and a post-processing Petri dish localization algorithm. First of all, our unique innovation lies in the multi-loss U-Net reformulation. We introduce an additional loss term at the bottleneck U-Net layer, focusing on delivering an auxiliary signal indicating where to look for distinct CFUs. Second, our novel localization algorithm accurately incorporates an agar plate and its bezel into the CFU counting routines. Finally, our proposition is further enhanced by the integration of a fully automated solution. This comprises a specially designed uniform Petri dish illumination system and a counting web application. The latter application is capable of directly receiving images from the camera, which are subsequently processed, and the segmentation results are sent back to the user. This feature provides an opportunity to correct the CFU counts, offering a feedback loop that contributes to the continued development of the Deep Learning model. Through extensive experimentation, we have found that all probed multi-loss U-Net architectures incorporated in our hybrid approach consistently outperform their single-loss counterparts which utilize exclusively the combination of Tversky and Cross-Entropy training losses at the output U-Net layer. We report further significant improvements by the means of our novel localization algorithm. This reaffirms the effectiveness of our proposed hybrid solution in addressing contemporary challenges of the precise in-vitro CFU counting.

Drones have been used in a variety of scenarios such as atmospheric monitoring, fire rescue, agricultural irrigation, etc., in which accurate environmental perception is of crucial importance for both decision-making and control. Among the drone sensors, the RGB camera is indispensable for capturing rich visual information for vehicle navigation but encounters a grand challenge in high-dynamic-range scenes that occur frequently in real applications. Specifically, the recorded frames suffer from under-exposure and over-exposure simultaneously and degenerate the successive vision tasks. To solve the problem, we take object tracking as an example and leverage the superior response of event cameras over a large intensity range to propose an event-assisted object tracking algorithm that can achieve reliable tracking under large intensity variations. Specifically, we propose to pursue feature matching from dense event signals, and based on which to (i) design a UNet-based image enhancement algorithm to balance the RGB intensity with the help of neighboring frames in the time domain, and then (ii) construct a dual-input tracking model to track the moving objects from intensity balanced RGB video and event sequence. The proposed approach is comprehensively validated in both simulation and real experiments.

We present a Deep Learning generative model specialized to work with graphs with a regular geometry. It is build on a Variational Autoencoder framework and employs Graph convolutional layers in both encoding and decoding phases. We also introduce a pooling technique (ReNN-Pool), used in the encoder, that allows to downsample graph nodes in a spatially uniform and highly interpretable way. In the decoder, a symmetrical un-pooling technique is used to retrieve the original dimensionality of graphs. Performance of the model are tested on the standard Sprite benchmark dataset, a set of 2D images of video game characters, adequately transforming images data into graphs, and on the more realistic use-case of a dataset of cylindrical-shaped graph data that describe the distributions of the energy deposited by a particle beam in a medium.

Sign language recognition is essential in hearing-impaired people’s communication. Sign language recognition is an important concern in computer vision and has been developed with rapid progress in image recognition technology. However, sign language recognition using a general monocular camera has problems with occlusion and recognition accuracy in sign language recognition. In this research, we aim to improve accuracy by using a 2-axis bending sensor as an aid in addition to image recognition. We aim to achieve higher recognition accuracy by acquiring hand keypoint information of sign language actions captured by a monocular RGB camera and adding sensor assist. To improve sign language recognition, we need to propose new AI models. In addition, the amount of dataset is small because it uses the original data set of our laboratory. To learn using sensor data and image data, we used MediaPipe, CNN, and BiLSTM to perform sign language recognition. MediaPipe is a method for estimating the skeleton of the hand and face provided by Google. In addition, CNN is a method that can learn spatial information, and BiLSTM can learn time series data. Combining the CNN and BiLSTM methods yields higher recognition accuracy. We will use these techniques to learn hand skeletal information and sensor data. Additionally, the 2-axis Bending sensor glove data support training AI model. Using these methods, we aim to improve the recognition accuracy of sign language recognition by combining sensor data and hand skeleton data. Our method performed better than using skeletal information, achieving 96.5% accuracy in Top-1.

Plant phenotype plays an important role in crop breeding and planting. Leaf phenotype is an important part of plant phenotype. In order to analyze the leaf phenotype, the target leaf is required to be segmented from the complex background image. In this paper, an automatic soybean leaf segmentation method based on object detection and interactive segmentation models is proposed. Firstly, the Libra R-CNN object detection algorithm is used to detect all soybean leaves in the image. Then, based on the idea that the target soybean leaf is located in the center of the image and the area is large, the detection bounding box of the target leaf is selected. In order not to destroy the segmentation result, the bounding box is optimized to completely enclose the whole leaf. Finally, according to the optimized bounding box, the prior channels of foreground and background are constructed using Gaussian model. The two channels together with the original image are as the input of the interactive object segmentation with inside-outside guidance model to segment the target soybean leaf. A large number of qualitative and quantitative experimental results show that the method has high segmentation accuracy and strong generalization capacity.