Beyond semantic segmentation,3D instance segmentation(a process to delineate objects of interest and also classifying the objects into a set of categories) is gaining more and more interest among researchers since numerous computer vision applications need accurate segmentation processes(autonomous driving, indoor navigation, and even virtual or augmented reality systems…) This paper gives an overview and a technical comparison of the existing deep learning architectures in handling unstructured Euclidean data for the rapidly developing 3D instance segmentation. First, the authors divide the 3D point clouds based instance segmentation techniques into two major categories which are proposal based methods and proposal free methods. Then, they also introduce and compare the most used datasets with regard to 3D instance segmentation. Furthermore, they compare and analyze these techniques performance (speed, accuracy, response to noise…). Finally, this paper provides a review of the possible future directions of deep learning for 3D sensor-based information and provides insight into the most promising areas for prospective research.

Foraminifera are single-celled marine organisms that construct shells that remain as fossils in the marine sediments. Classifying and counting these fossils are important in e.g. paleo-oceanographic and -climatological research. However, the identification and counting process has been performed manually since the 1800s and is laborious and time-consuming. In this work, we present a deep learning-based instance segmentation model for classifying, detecting, and segmenting microscopic foraminifera. Our model is based on the Mask R-CNN architecture, using model weight parameters that have learned on the COCO detection dataset. We use a fine-tuning approach to adapt the parameters on a novel object detection dataset of more than 7000 microscopic foraminifera and sediment grains. The model achieves a (COCO-style) average precision of 0.78±0.00 on the classification and detection task, and 0.80±0.00 on the segmentation task. When the model is evaluated without challenging sediment grain images, the average precision for both tasks increases to 0.84±0.00 and 0.86±0.00, respectively. Prediction results are analyzed both quantitatively and qualitatively and discussed. Based on our findings we propose several directions for future work, and conclude that our proposed model is an important step towards automating the identification and counting of microscopic foraminifera.

Interaction between multiple phases especially liquid-gas is one of the most common phenomena in industrial systems. Computer Vision enables modern systems to derive an automated and robust image-processing technique to address this issue. However, diverse experimental conditions still make it a challenging task to overcome. In Deep Learning, instance segmentation has created a niche in various computer vision applications as it aims to provide different instance IDs to objects belonging to the same class. In this paper, we propose a novel approach employing the YOLO (You Look Only Once) an end-to-end deep learning architecture for the detection and segmentation of bubbles in fluid flows. YOLOv5 offers significant improvements in both speed and accuracy compared to its predecessors like YOLOv3. Therefore, we train the YOLOv5 network on a dataset consisting of annotated images representing instances of bubbles across varied fluid flow environments. This paper provides both qualitative and quantitative evaluation of three YOLOv5 networks (small, medium, and extra-large) in detecting and segmenting bubbles with high accuracy and efficiency. Based on the metrics all three YOLOv5 models achieved high precision above 0.98 for bounding box detection and 0.97 for segmentation mask, and high recall value greater than 0.97 for box and 0.96 for mask quality was achieved. The achieved results indicate that the models accurately localize bubbles within the image frame, and distinguish bubble pixels from the background. Also, mean average precision (mAP50-95) consistently exceeds 0.84 across all models for the bounding box and 0.73 for the segmentation mask. The ascertained results emphasize the potential of YOLOv5 architecture as a powerful tool for automated bubble analysis in real-world applications. Our research contributes to advancing the field of deep neural networks in fluid systems and provides a fresh approach for future developments in automated bubble analysis.

Identification and segmentation of rock particles is very important in geology, mining, engineering construction, and environmental monitoring. Traditional methods for recognizing the rock particle characteristics are time-consuming, labor-intensive, and limited by the geographical scope of the sampling sites, which makes it more difficult to represent the spatial distribution of the rock particles accurately. Recent advancements in deep learning techniques offer promising solutions to automate this process. This study aims at utilizing state-of-the-art (SOTA) Mask R-CNN deep learning algorithm from the Detectron2 framework, to capture global features of rocks that will enable the recognition of rock particles more efficiently, just from having the rock images. These images are subject to image processing techniques such as a Gaussian filter, to denoise and smoothen the image, and an Illumination Adaptive Transformer (IAT) framework, to adjust the lighting exposure; all of which add to improve the quality of the image dataset. Afterward, the Mask R-CNN and Detectron2 models are utilized in training the images using transfer learning. Experimental results from evaluating the performance of the proposed algorithms showcase the effectiveness of the approach, thereby highlighting its potential to revolutionize rock particle segmentation across various domains.

Vehicle identification is an important task in traffic monitoring because it allows for efficient inference and provides a cause for action. Vehicle classification via deep learning and other approaches such as segmentation is a critical tool for re-identification. In this paper, instance segmentation is used to identify vehicle makes with license plate detection, allowing for better unique vehicle recognition for re-identification. A dataset is annotated and modified, for example, by segmenting it with polygonal bounding boxes that capture the vehicle's unique frontal features. In addition, license plate localization is performed. The results showed improved classification as well as a high mAP for the dataset when compared to previous approaches based on CNN and deformed CNN. Furthermore, a deep residual network and fully connected layer-based classification were utilized as the backbone for feature representation. Instance segmentation detects objects by segmenting and classifying regions of interest. The imbalance in the dataset is resolved using a mosaic-tiled approach, which produces greater precision than other approaches.

Fire outbreaks continue to cause damage despite the improvements in fire-detection tools and algorithms. It is still challenging to implement a well performing and optimized approach, which is sufficiently accurate, and has tractable complexity and low false alarming rate. Small amount of fire and identification of fire from a long distance is also a challenge in previously proposed techniques. In this study, we propose a novel hybrid model based on Convolutional Neural Networks (CNN) to detect and analyze fire intensity. 21 convolutional layers, 24 Rectified Linear Unit (ReLU) layers, 6 pooling layers, 3 fully connected layers, 2 dropout layers, and a softmax layer are included in the proposed 57-layer CNN model. Our proposed model performs instance segmentation in order to distinguish between fire and non-fire events. To reduce the intricacy of the proposed model, we also propose a key-frame extraction algorithm. The proposed model uses Internet of Things (IoT) devices to alert the relevant person by calculating the severity of fire. Our proposed model is tested on a publicly available dataset having fire and normal videos. The achievement of 95.25 % classification accuracy, 0.09% False Positive Rate (FPR), 0.65 percent False Negative Rate (FNR), and a prediction time of 0.08 seconds validates the proposed system.

The prediction of drug-target interactions plays a fundamental role in facilitating drug discovery, where the goal is to find prospective drug candidates. With the increase in the number of drug-protein interactions, machine learning techniques, especially deep learning methods, have become applicable for drug-target interaction discovery because they significantly reduce the required experimental workload. In this paper, we present a spontaneous formulation of the drug-target interaction prediction problem as an instance of multi-instance learning. We address the problem in three stages, first organizing given drug and target sequences into instances via a private-public mechanism, then identifying the predicted scores of all instances in the same bag, and finally combining all the predicted scores as the output prediction. A comprehensive evaluation demonstrates that the proposed method outperforms other state-of-the-art methods on three benchmark datasets.

The study addresses the challenge of efficiently peeling pineapples, which have a distinct elliptical form, thick skin, and small eyes that are difficult to detect with conventional automated methods. This results in significant flesh waste. To improve the process, the researchers developed an integrated system combining an enhanced BlendMask method, termed SAAF-BlendMask, and a PCP method. SAAF-BlendMask improves the detection of small pineapple eyes, while PCP ensures accurate posture adjustment for precise path planning. The system uses 3D vision and deep learning technologies, achieving an average precision (AP) of 73.04% and a small object precision (APs) of 62.54% in eye detection, with a path planning success rate reaching 99%. The fully automated electromechanical system was tested on 110 real pineapples, demonstrating a reduction in flesh waste by 11.7% compared to traditional methods. This study highlights the potential of advanced machine vision and robotics in enhancing the efficiency and precision of food processing.

This study proposes a novel hybrid machine learning approach for modeling of rainfall-induced shallow landslides. The proposed approach is a combination of an instance-based learning algorithm (k-NN) and Rotation Forest (RF), state of the art machine techniques that have seldom explored for landslide modeling. The Lang Son city area (Vietnam) is selected as a case study. For this purpose, a spatial database for the study area was constructed, and then, was used to build and evaluate the hybrid model. Performance of the model was assessed using Receiver Operating Characteristic (ROC), area under the ROC curve (AUC), success rate and prediction rate, and several statistical evaluation metrics. The results showed that the model has high performance with both the training data (AUC = 0.948) and the validation data (AUC = 0.848). The results were compared with those obtained from soft computing techniques i.e. Random Forest, J48 Decision Trees, and Multilayer Perceptron Neural Networks. Overall, the performance of the proposed model is better than those obtained from the above methods. Therefore, the proposed model is a promising tool for landslide modeling. The research result can be highly useful for land use planning and management in landslide prone areas.

(1) Background: Evidence-based policymaking requires data about the local population's socioeconomic status (SES) at detailed geographical level, however, such information is often not available, or is too expensive to acquire. Researchers have proposed solutions to estimate SES indicators by analyzing Google Street View images, however, these methods are also resource-intensive, since they require large volumes of manually labeled training data. (2) Methods: We propose a methodology for automatically computing surrogate variables of SES indicators using street images of parked cars and deep multiple instance learning. Our approach does not require any manually created labels, apart from data already available by statistical authorities, while the entire pipeline for image acquisition, parked car detection, car classification, and surrogate variable computation is fully automated. The proposed surrogate variables are then used in linear regression models to estimate the target SES indicators. (3) Results: We implement and evaluate a model based on the proposed surrogate variable at 30 municipalities of varying SES in Greece. Our model has $R^2=0.76$ and a correlation coefficient of $0.874$ with the true unemployment rate, while it achieves a mean absolute percentage error of $0.089$ and mean absolute error of $1.87$ on a held-out test set. Similar results are also obtained for other socioeconomic indicators, related to education level and occupational prestige. (4) Conclusions: The proposed methodology can be used to estimate SES indicators at the local level automatically, using images of parked cars detected via Google Street View, without the need for any manual labeling effort.

Ligand-based virtual screening (LBVS) is a promising approach for rapid and low-cost screening of potentially bioactive molecules in the early stage of drug discovery. Compared with traditional similarity-based machine learning methods, deep learning frameworks for LBVS can more effectively extract high-order molecule structure representations from molecular fingerprints or structures. However, the 3D conformation of a molecule largely influences its bioactivity and physical properties, and has rarely been considered in previous deep learning-based LBVS methods. Moreover, the relative bioactivity benchmark dataset is still lacking. To address these issues, we introduce a novel end-to-end deep learning architecture trained from molecular conformers for LBVS. We first extracted molecule conformers from multiple public molecular bioactivity data and consolidated them into a large-scale bioactivity benchmark dataset, which totally includes millions of endpoints and molecules corresponding to 954 targets. Then, we devised a deep learning-based LBVS called EquiVS to learn molecule representations from conformers for bioactivity prediction. Specifically, graph convolutional network (GCN) and equivariant graph neural network (EGNN) are sequentially stacked to learn high-order molecule-level and conformer-level representations, followed by attention-based deep multiple-instance learning (MIL) to aggregate these representations and then predict the potential bioactivity for the query molecule on a given target. We conducted various experiments to validate the data quality of our benchmark dataset, and confirmed EquiVS achieved better performance compared with 10 traditional machine learning or deep learning-based LBVS methods. Further ablation studies demonstrate the significant contribution of molecular conformation for bioactivity prediction, as well as the reasonability and non-redundancy of deep learning architecture in EquiVS. Finally, a model interpretation case study on CDK2 shows the potential of EquiVS in optimal conformer discovery. The overall study shows that our proposed benchmark dataset and EquiVS method have promising prospects in virtual screening applications.

The pursuit of Artificial Intelligence (AI) that emulates human cognitive processes is a cornerstone of ethical AI development, ensuring that emerging technologies can integrate seamlessly into societal frameworks requiring nuanced understanding and decision-making. Zero-Shot Instance Retrieval (ZSIR) stands at the forefront of this endeavour, potentially providing a robust platform for AI systems, particularly large visual language models, to demonstrate and refine cognition-aligned learning without the need for direct experience. In this paper, we critically evaluate current cognition alignment methodologies within traditional zero-shot learning paradigms, using visual attributes and word embedding generated by large AI models. We propose a unified similarity function that quantifies the cognitive alignment level, bridging the gap between AI processes and human-like understanding. Through extensive experimentation, our findings illustrate that this similarity function can effectively mirror the visual-semantic gap, steering the model towards enhanced performance in zero-shot instance retrieval. This work not only benchmarks the cognition alignment of AI, but also sets a new precedent for the development of visual language models attuned to the complexities of human cognition.}

Object detection is complex and involved diverse requirements for different applications. In critical application such as visual impaired navigation guide and real-time surveillance for instance, identifying a particular object of instance is required and this involved a complex approach for realization. Literatures were reviewed, starting with the application of deep learning techniques for object detection; their gaps were identified and addressed using real-time object detection models literature review. From the review gaps were also detected and addressed using literature review on occlusion detection techniques. Object of instance detection in clustered scene with similar object of interest was identified as an open gap not addressed in the existing literature; even through it is vital for many applications. This research recommended an Occlusion Based Object of Instance Detection (OBOID) technique which used the spatial information to identify instant object of interest in clustered scene with many similarly object of interest. Limitation of the recommended OBOID is that it requires only system where position and distance of object is necessary to inform other decision.

The objective of this study was to develop and evaluate artificial intelligence (AI) models for the detection and instance segmentation of vertebrae and spinal metastases in computer tomography (CT) scans. The models were trained on datasets consisting of patients with polytrauma and relatively undamaged spines, as well as patients diagnosed with spinal metastases. Our results indicate that the models achieved high performance in vertebra segmentation, with F-beta scores ranging from 0.88 to 0.96 across all vertebra classes. For spinal metastases, the model attained F-beta scores of 0.68 for lytic type and 0.57 for sclerotic type metastases. Additionally, the models demonstrated the capability to detect isolated metastatic nodes in other bones, highlighting their robustness and potential for broader clinical application. Despite these promising results, the study faced limitations including a relatively small and homogeneous dataset, variability in segmentation mask quality, and the need for real-world clinical validation. Future clinical trials are necessary to evaluate the practical utility and effectiveness of these AI models in improving patient outcomes. Our study emphasizes the potential of AI-assisted detection and segmentation models in enhancing diagnostic accuracy and efficiency in clinical practice.

Artificial Intelligence is dominated by Artificial Neural Networks (ANNs). Currently, the Batch Gradient Descent (BGD) is the only solution to train ANN weights when dealing with large datasets. In this article, a modification to the BGD is proposed which significantly reduces the training time and improves the convergence. The modification, called Instance Eliminating Back Propagation (IEBP), eliminates correctly-predicted-instances from the Back Propagation. The speedup is due to the elimination of unnecessary matrix multiplication operations from the Back Propagation. The proposed modification does not add any training hyperparameter to the existing ones and reduces the memory consumption during the training.

Multiple mRNA isoforms of the same gene are produced via alternative splicing, a biological mechanism that regulates protein diversity while maintaining genome size. Alternatively spliced mRNA isoforms of the same gene may sometimes have very similar sequence, but they can have significantly diverse effects on cellular function and regulation. The products of alternative splicing have important and diverse functional roles, such as response to environmental stress, regulation of gene expression, human heritable and plant diseases. The mRNA isoforms of the same gene, such as the apoptosis associated CASP3 gene, can have dramatically different functions. The shorter mRNA isoform product CASP3-S inhibits apoptosis, while the longer CASP3-L mRNA isoform promotes apoptosis. Despite the functional importance of mRNA isoforms, very little has been done to annotate their functions. The recent years have however seen the development of several computational methods aimed at predicting mRNA isoform level biological functions. These methods use a wide array of proteo-genomic data to develop machine learning-based mRNA isoform function prediction tools. In this review, we discuss the computational methods developed for predicting the biological function at the individual mRNA isoform level.