Early-season crop-type data are required for a variety of agricultural monitoring and decision-making applications. The early season in this study referred to the prophase and middle of a growth season. Early-season detection of crop types remains challenging because of limited discriminative features, especially for landscapes that are characterized by complex cropping patterns. In fact, different remote-sensing satellites can increase the frequency of data acquisition, which can provide more information in the early season. Moreover, optical and radar sensors have different degrees of sensitivity to crop parameters. Therefore, the integration and application of two types of multitemporal data are of great significance to improve the accuracy and timeliness of crop type detection.In deep learning (DL), convolutional neural network (CNN) and recurrent neural network (RNN) models have great potential for temporal feature extraction. Compared with RNNs, CNNs usually have fewer parameters and are more conducive to determining early-season detection dates of different crop types, which requires a lot of training because of the need to model on different dates. Nevertheless, revisit dates and temporal intervals of different satellites are usually different, resulting in different data acquisition time series; these data cannot be directly used as input for the same convolutional layer. To address this challenge, a Dual-1DCNN was bulit based on the CNN model in this study. Moreover, an incremental training method was used to attain the network on each data acquisition date and obtained the best detection date for each crop type in the early season. A case study for Hengshui City in China was conducted using time series of Sentinel-1A (S1A) and Sentinel-2 (S2). To verify this method, classical methods support vector machine (SVM) and random forest (RF) were implemented. The results demonstrated the following: (1) the Dual-1DCNN extracted discriminative features from S1A and S2 time series at the early season by producing the highest overall accuracy (OA: 87.23%); (2) for summer maize, cotton, and common yam rhizome, the Dual-1DCNN achieved F1 values of 92.39%, 87.71%, and 84.38%, respectively, at the early season (moreover, the early seasons were almost 40, 70, and 80 days before the end of the growth seasons, respectively). These findings suggested that the Dual-1DCNN is promising for the accurate and timely detection of crop types.

The root system plays an irreplaceable role in plant growth. Its improvement can increase crop productivity. However, such system is still mysterious for us. The underlying mechanism has not been fully uncovered. The investigation on proteins related to the root system is an important means to complete this task. In the previous time, lack of root-related proteins makes it impossible to adopt machine learning methods for designing efficient models for the discovery of novel root-related proteins. Recently, a public database on root-related proteins was set up and machine learning methods can be applied in this field. In this study, we proposed a machine learning based model, named Graph-Root, for identification of root-related proteins. The features derived from protein sequences and one network were extracted, where the former features were processed by graph convolutional neural network and multi-head attention, and the later features abstracted the linkage between proteins. These features were fed into the fully connected layer to make prediction. The 5-fold cross-validation and independent tests suggested its good performance. It also outperformed the only one previous model, SVM-Root. Furthermore, the importance of each feature type and component in the proposed model was investigated.

This paper presents a study of an automated system for identifying planktic foraminifera at the species level. The system uses a combination of deep learning methods, specifically Convolutional Neural Networks (CNNs), to analyze digital images of foraminifera taken at different illumination angles. The dataset is composed of 1437 groups of sixteen grayscale images, one group for each foraminifer, that are then converted to RGB images with various processing methods. These RGB images are fed into a set of CNNs, organized in an Ensemble Learning (EL) environment. The ensemble is built by training different networks using different approaches for creating the RGB images. The study finds that an ensemble of CNN models trained on different RGB images improves the system's performance compared to other state-of-the-art approaches. The proposed system was also found to outperform human experts in classification accuracy.

To drive safely, the driver must be aware of the surroundings, pay attention to the road traffic, and be ready to adapt to new circumstances. Most studies on driving safety focus on detecting anomalies in driver behavior and monitoring the cognitive capabilities of drivers. In our study, we proposed a classifier for basic activities in driving a car, based on a similar approach that could be applied to the recognition of basic activities in daily life, that is, using electrooculographic (EOG) signals and a one-dimensional convolutional neural network (1D CNN). Our classifier achieved an accuracy of 80% for the 16 primary and secondary activities. The accuracy related to primary activities in driving, including crossroad, parking, roundabout was 97.9%, 96.8%, 97.4%, and 99.5%, respectively. The F1 score for secondary driving actions (0.99) was higher than for primary driving activities (0.93–0.94). Furthermore, using the same algorithm, it was possible to distinguish four secondary activities related to activities of daily life and secondary when driving a car.

The automatic diagnosis of cardiovascular diseases has received much attention in the deep learning field. In this context, the segmentation of the left ventricle endocardium constitutes a major task in diagnosing heart conditions such as health failure and hypertrophic cardiomyopathy. The objective of this paper is to propose a "deep convolution network" for segmenting the internal cavity of the left ventricle (endocardium) using MRI images. In particular, we design an improved UNET model which handles additional inception modules for efficiently segmenting the internal cavity of the left ventricle. Our approach has been validated on the Sunnybrook Cardiac Data (SCD) dataset and has showed promising results in terms of precision. More specifically, the improved UNET largely outperforms the baseline UNET model and many existing state-of-the-art methods.

Fault detection is an important step for subsurface interpretation and reservoir characterization from 3D seismic images. Due to the numerous and complicated faulting structures of seismic images, manual seismic interpretation is time taking and need intensive work. To overcome this problem, geoscientists are coming up with productive computer-aided techniques for assisting in interpreter science for many years. However, in this paper, we used a pre-trained CNN model to predict faults from the 3D seismic volume of the Poseidon field of Browse Basin, Australia. Basically, this field is highly structured with complex normal faulting throughout the targeted Plover Formations. So, our motivation for this work in this field is to compare machine learning-based fault prediction to user interpreted faults identification with supported by seismic variance attribute. We found very satisfying result using DL with having an improved fault probability volume that outperform variance technology. Therefore, we propose that this workflow could reduce time and able to predict fault quite accurately in any field area.

When extract building from high resolution remote sensing image with meter/sub-meter accuracy, the shade of trees and interference of roads are the main factors of reducing the extraction accuracy. Proposed a Bayesian Convolutional Neural Networks(BCNET) model base on standard fully convolutional networks(FCN) to solve these problems. First take building with no shade or artificial removal of shade as Sample-A, woodland as Sample-B, road as Sample-C. Set up 3 sample libraries. Learn these sample libraries respectively, get their own set of feature vector; Mixture Gauss model these feature vector set, evaluate the conditional probability density function of mixture of noise object and roofs; Improve the standard FCN from the 2 aspect:(1) Introduce atrous convolution. (2) Take conditional probability density function as the activation function of the last convolution. Carry out experiment using unmanned aerial vehicle(UVA) image, the results show that BCNET model can effectively eliminate the influence of trees and roads, the building extraction accuracy can reach 97%.

During planetary exploration mission operations, one of the key responsibilities of the instrument teams to determine data viability for subsequent analysis. During the 2019 CanMoon Lunar Sample Return Analogue Mission, the Lead Raman Specialist manually examined each spectra to provide quality assurance/validation. This non-trivial process requires years of experience to complete accurately. With the proven efficacy of Convolutional Neural Networks (CNNs) in classification tasks, and the increased use of automation and control loops on planetary space platforms for navigation and science targeting, an opportunity presents itself to approach this validation problem utilising CNNs. We present the Generalised Raman Validation Network (GRaVN), an neural network focused specifically on extracting the generalised structure of Raman spectra for quality assurance/validation. This work demonstrates the viability of utilising a CNN network in validation activities for Raman spectroscopy. Utilising only two hidden layers, a configuration was developed that provided good levels of accuracy on a manually curated dataset. This indicates that such a system could be useful as part of an autonomous control loop during planetary exploration activities.

The continuous and precise mapping of glacier calving fronts is essential for monitoring and understanding rapid glacier changes in Antarctica and Greenland, which have the potential for significant sea level rise within the current century. This effort has been mostly restricted to the slow and painstaking manual digitalization of the calving front positions in thousands of satellite imagery products. Here, we have developed a machine learning toolkit to robustly and automatically detect glacier calving front margins in satellite imagery. The toolkit is based on semantic image segmentation using Convolutional Neural Networks (CNN) with a modified U-Net architecture to isolate the calving fronts from satellite images after having been trained with a dataset of images and their corresponding manually-determined calving fronts. As a case study we train our neural network on a varied set Landsat images with lowered resolutions from Jakobshavn, Sverdrup, and Kangerlussuaq glaciers, Greenland and test the results on novel images from Helheim glacier, Greenland to evaluate the performance of the approach. The neural network is able to identify the calving front in new images with a mean deviation of 96.3 m from the true fronts, equivalent to 1.97 pixels on average, while the corresponding error for manually-determined fronts on the same resolution images is 92.5 m. We find that the trained neural network significantly outperforms common edge detection techniques, and can be used to continuously map out calving-ice fronts with a variety of data products.

The interpretation of land use and land cover (LULC) is an important issue in the fields of high-resolution remote sensing (RS) image processing and land resource management. Fully training a new or existing convolutional neural network (CNN) architecture for LULC classification requires a large amount of remote sensing images. Thus, fine-tuning a pre-trained CNN for LULC detection is required. To improve the classification accuracy for high resolution remote sensing images, it is necessary to use another feature descriptor and to adopt a classifier for post-processing. A fully connected conditional random fields (FC-CRF), to use the fine-tuned CNN layers, spectral features, and fully connected pairwise potentials, is proposed for image classification of high-resolution remote sensing images. First, an existing CNN model is adopted, and the parameters of CNN are fine-tuned by training datasets. Then, the probabilities of image pixels belong to each class type are calculated. Second, we consider the spectral features and digital surface model (DSM) and combined with a support vector machine (SVM) classifier, the probabilities belong to each LULC class type are determined. Combined with the probabilities achieved by the fine-tuned CNN, new feature descriptors are built. Finally, FC-CRF are introduced to produce the classification results, whereas the unary potentials are achieved by the new feature descriptors and SVM classifier, and the pairwise potentials are achieved by the three-band RS imagery and DSM. Experimental results show that the proposed classification scheme achieves good performance when the total accuracy is about 85%.

With the development of deep learning technology, various structures and research methods for super-resolution restoration of natural images and document images have been introduced. In particular, a number of recent studies have been conducted and developed in image restoration using generative adversarial network. Super-resolution restoration is ill-posed problem because of some complex restraints such as a lot of high-resolution images being restored for the same low-resolution image and also difficulty in restoring noises like edges, light smudging, and blurring. In this study, we utilized the spatially adaptive de-normalization (SPADE) structure for document image restoration to solve previous problems such as edge unclearness, hardness to catch features of texts, and the image color transition. Consequently, it can be confirmed that the edge of the character and the ambiguous stroke are restored more clearly when contrasting with the other previously suggested methods. Also, the proposed method’s PSNR and SSIM scores are geting 8% and 15% higher, respectively, compared to the previous methods.

In recent times, using image-based deep learning techniques has been very useful for classifying fruits. However, mango is a particular challenge because its shelf life is directly related to the degree of maturity at the time of harvest, which is determined mainly through the visual assessment of the operators when sorting them, doing it incorrectly, coupled with the blows and compressions exposed during storage and transport, trigger the deterioration in their quality, hence the importance of this operation. In this sense, the purpose of this study focuses on designing a computational model based on Deep Learning that uses a convolutional neural network (CNN) parameterized to select mangoes according to their color tone based on their maturity stage. A dataset was established from 201 labeled images, then a CNN architecture was built, obtaining a model on which 3 criteria were evaluated: the number of convolution and grouping layers, the resolution of the input images and the number of training epochs required. As a result, an input image resolution of 32 x 32 pixels was achieved, the process included a convolution layer followed by pooling that fed another layer that processed the results and ended in an output layer with 2 neurons under the One Hot Encoding representation. The model required 20 epochs to perfect its training, achieving 96.04 % accuracy in mango image selection. This study supports the potential of CNNs in quality analysis through mango color selection, even in uncertainty scenarios.

In the remote sensing domain, it is crucial to complete semantic segmentation on the raster images, e.g., river, building, forest, etc, on raster images. A deep convolutional encoder--decoder (DCED) network is the state-of-the-art semantic segmentation method for remotely sensed images. However, the accuracy is still limited, since the network is not designed for remotely sensed images and the training data in this domain is deficient. In this paper, we aim to propose a novel CNN for semantic segmentation particularly for remote sensing corpora with three main contributions. First, we propose applying a recent CNN called a global convolutional network (GCN), since it can capture different resolutions by extracting multi-scale features from different stages of the network. Additionally, we further enhance the network by improving its backbone using larger numbers of layers, which is suitable for medium resolution remotely sensed images. Second, "channel attention'' is presented in our network in order to select the most discriminative filters (features). Third, "domain-specific transfer learning'' is introduced to alleviate the scarcity issue by utilizing other remotely sensed corpora with different resolutions as pre-trained data. The experiment was then conducted on two given datasets: (i) medium resolution data collected from Landsat-8 satellite and (ii) very high resolution data called the ISPRS Vaihingen Challenge Dataset. The results show that our networks outperformed DCED in terms of $F1$ for 17.48% and 2.49% on medium and very high resolution corpora, respectively.

In the field of maneuvering target tracking, the combined observations of azimuth and Doppler may cause weak observation or non-observation in the application of traditional target tracking algorithms. Additionally, traditional target-tracking algorithms require pre-defined multiple mathematical models to accurately capture the complex motion states of targets, while model mismatch and unavoidable measurement noise lead to significant errors in target state prediction. To address those above challenges, in recent years, the target-tracking algorithms based on neural networks, such as recurrent neural networks (RNNs), long short-term memory (LSTM) networks, and Transformer architectures, have been widely used for their unique advantages to achieve accurate predictions. To better model the nonlinear relationship between the observation time series and the target state time series, as well as the contextual relationship among time series points, we present a deep learning algorithm called recursive downsample-convolve-interact neural network (RDCINN) based on convolutional neural network (CNN) that downsamples time series into sub-sequences and extracts multi-resolution features to enable the modeling of complex relationships between time series, which overcomes the shortcomings of traditional target-tracking algorithms in using observation information inefficiently due to weak observation or non-observation. The experimental results show that our algorithm outperforms other existing algorithms in the scenario of strong maneuvering target tracking with the combined observations of azimuth and Doppler.

In the age of social media, images come from unreliable sources that may change their content to fit a narrative. Automated detection of forged images is a complex task, especially considering recent technological advances in image manipulation software. However, in the existing literature, it is possible to identify two general approaches adopted by detection methods, active and passive. Active techniques preemptively act on an image by inserting structures before any manipulation is made that can verify its authenticity. In contrast, passive methods analyze the content of an image in search of traces of manipulation. In this way, this research proposes a novel approach to image manipulation detection, combining two passive methods through neural networks, creating a generalist approach capable of detecting with greater accuracy than the methods that compose it. Furthermore, this work used a combination of four datasets available in the literature for training and evaluation. After training, the merged approach obtained an accuracy of 89.59% in the set of validation images, significantly higher than the model trained with only unaltered images, which obtained 78.64%, and the two other models trained using images with a feature selection method applied to enhance inconsistencies that obtained 68.02% for Error-Level Analysis images and 50.70% for the method using Discrete Wavelet Transform. In addition to the performance improvement, the proposed approach’s accuracy variation was lower than the other models.

Insulator defect detection is of great significance to compromise the stability of the power transmission line. The state-of-the-art network of object detection, YOLOv5, has been widely used on insulator and defect detection. However, YOLOv5 network has some limitations like poor detection rate and high computational loads in detecting small insulator defects. To solve these problems, we proposed a light-weight network for insulator and defect detection. In this network, we introduced Ghost module into YOLOv5 backbone and neck to reduce the parameters and model size to enhance the performance in unmanned aerial vehicles (UAVs). Besides, we added small object detection anchors and layers for small defect detection. In addition, we optimized the backbone of YOLOv5 by applying convolutional block attention module (CBAM) to focus on critical information for insulator and defect detection and suppress uncritical information. The experiment result shows the mean average precision (mAP) 0.5 and the mAP0.5:0.95 of our model can reach 99.4% and 91.7%, the parameters and model weight are reduced to 3807372 and 8.79M, which can easily deploy to embedded devices like UAVs. And the speed of detection can reach 10.9ms/image, which can meet the real-time detection requirement.

The small satellite market continues to grow year after year. A compound annual growth rate of 17% is estimated during the period between 2020 and 2025. Low-cost satellites can send a vast amount of images to be post-processed at the ground to improve the quality and extract detailed information. In this domain lies the resolution enhancement task, where a low-resolution image is converted to a higher resolution automatically. Deep learning approaches to Super-Resolution (SR) reached the state-of-the-art in multiple benchmarks; however, most of them were studied in a single-frame fashion. With satellite imagery, multi-frame images can be obtained at different conditions giving the possibility to add more information per image and improve the final analysis. In this context, we developed and applied to the PROBA-V dataset of multi-frame satellite images a model that recently topped the European Space Agency’s Multi-frame Super Resolution (MFSR) competition. The model is based on proven methods that worked on 2D images tweaked to work on 3D: the Wide Activation Super Resolution (WDSR) family. We show that with a simple 3D CNN residual architecture with WDSR blocks and a frame permutation technique as data augmentation better scores can be achieved than with more complex models. Moreover, the model requires few hardware resources, both for training and evaluation, so it can be applied directly from a personal laptop.

Iron ore processing involves critical steps that affect the quality of the final product. Determining the diameter of the pellets is the necessary initial step for volume measurement and, ultimately, for porosity and bulk density measurement, crucial characteristics for optimizing the burning process in the blast furnace. Traditional measurement methods using mercury present issues related to operator safety and environmental preservation, while the practices described in ISO 4698 standard require lengthy preparation and execution time. In light of environmental needs, operator safety, and the time consumed in tests, implementing a new method based on digital image processing and convolutional neural networks for measuring the diameter of burnt iron ore pellets is proposed. The methodology involves capturing images of the pellets using a high-resolution camera and utilizing digital image processing and neural networks capable of performing pixel-by-pixel object segmentation in the images, providing precise information about the pellets to calculate their volume automatically. The results were compared with those obtained using traditional methods, evaluating their conformity with the ISO 4698 standard. In conclusion, this study offers a new approach to measuring the volume of burnt iron ore pellets, providing accurate and reliable results in compliance with safety and environmental preservation standards.

Speech Emotion Recognition (SER) is an interesting and difficult problem to handle. In this paper, we deal with it through the implementation of deep learning networks. We have designed and implemented six different deep learning networks, a Deep Belief Network (DBN), a simple deep neural network (SDNN), a LSTM network (LSTM), a LSTM network with the addition of an attention mechanism (LSTM-ATN), a Convolutional neural network (CNN), and a Convolutional neural network with the addition of an attention mechanism (CNN-ATN), having in mind, apart from solving the SER problem, to test the impact of attention mechanism to the results. Dropout and Batch Normalization techniques are also used to improve the generalization ability (prevention of overfitting) of the models as well as to speed up the training process. The Surrey Audio-Visual Expressed Emotion database (SAVEE), and the Ryerson Audio-Visual Database (RAVDESS) database were used for training and evaluation of our models. The results showed that networks with the addition of the attention mechanism did better than the others. Furthermore, they showed that CNN-ATN was the best among tested networks, achieving an accuracy of 74% for the SAVEE and 77% for the RAVDESS dataset, and exceeded existing state-of-the-art systems for the same datasets.

In recent years, data providers are generating and streaming a large number of images. More particularly, processing images that contain faces have received great attention due to its numerous applications, such as entertainment and social media apps. The enormous amount of images shared on these applications presents serious challenges and requires massive computing resources to ensure efficient data processing. However, images are subject to a wide range of distortions in real application scenarios during the processing, transmission, sharing, or combination of many factors. So, there is a need to guarantee acceptable delivery content, even though some distorted images do not have access to their original version. In this paper, we present a framework developed to estimate the images' quality while processing a large number of images in real-time. Our quality evaluation is measured using an integration of a deep network with random forests. In addition, a face alignment metric is used to assess the facial features. Experimental results have been conducted on two artificially distorted benchmark datasets, LIVE and TID2013. We show that our proposed approach outperforms the state-of-art methods, having a Pearson Correlation Coefficient (PCC) and Spearman Rank Order Correlation Correlation Coefficient (SROCC) with subjective human scores of almost 0.942 and 0.931 while minimizing the processing time from 4.8ms to 1.8ms.

Convolutional Neural Networks combined with autonomous drones are increasingly seen as enablers of partially automating the aircraft maintenance visual inspection process. Such an innovative concept can have a significant impact on aircraft operations. Through supporting aircraft maintenance engineers detect and classify a wide range of defects, the time spent on inspection can significantly be reduced. Examples of defects that can be automatically detected include aircraft dents, paint defects, cracks and holes, and lightning strike damage. Additionally, this concept could also increase the accuracy of damage detection and reduce the number of aircraft inspection incidents related to human factors like fatigue and time pressure. In our previous work, we have applied a recent Convolutional Neural Network architecture known by MASK R-CNN to detect aircraft dents. MASK-RCNN was chosen because it enables the detection of multiple objects in an image while simultaneously generating a segmentation mask for each instance. The previously obtained F1 and F2 scores were 62.67% and 59.35% respectively. This paper extends the previous work by applying different techniques to improve and evaluate prediction performance experimentally. The approaches uses include (1) Balancing the original dataset by adding images without dents; (2) Increasing data homogeneity by focusing on wing images only; (3) Exploring the potential of three augmentation techniques in improving model performance namely flipping, rotating, and blurring; and (4) using a pre-classifier in combination with MASK R-CNN. The results show that a hybrid approache combining MASK R-CNN and augmentation techniques leads to an improved performance with an F1 score of (67.50%) and F2 score of (66.37%)

Although automated Acute Lymphoblastic Leukemia (ALL) detection is essential, it is challenging due to the morphological correlation between malignant and normal cells. The traditional ALL classification strategy is arduous, time-consuming, often suffers inter-observer variations, and necessitates experienced pathologists. This article has automated the ALL detection task, employing deep Convolutional Neural Networks (CNNs). We explore the weighted ensemble of deep CNNs to recommend a better ALL cell classifier. The weights are estimated from ensemble candidates' corresponding metrics, such as accuracy, F1-score, AUC, and kappa values. Various data augmentations and pre-processing are incorporated for achieving a better generalization of the network. We train and evaluate the proposed model utilizing the publicly available C-NMC-2019 ALL dataset. Our proposed weighted ensemble model has outputted a weighted F1-score of 88.6%, a balanced accuracy of 86.2%, and an AUC of 0.941 in the preliminary test set. The qualitative results displaying the gradient class activation maps confirm that the introduced model has a concentrated learned region. In contrast, the ensemble candidate models, such as Xception, VGG-16, DenseNet-121, MobileNet, and InceptionResNet-V2, separately produce coarse and scatter learned areas for most example cases. Since the proposed ensemble yields a better result for the aimed task, it can experiment in other domains of medical diagnostic applications.

Automated lip reading is a research problem that has developed considerably in recent years. Lip reading is evaluated both visually and audibly in some cases. The lip reading model is a field of use for detecting specific words using images from security cameras, but it is not possible to use audio-visual databases in this situation. It is not possible to obtain the sound input of the pronounced word in all cases. We collected a new Turkish dataset with only the image in this study. The new dataset is produced using Youtube videos, which is an uncontrolled environment. For this reason, images have difficult parameters in terms of environmental factors such as light, angle, color, and personal characteristics of the face. Despite the different features on the human face such as mustache, beard, and make-up, the visual speech recognition problem was developed on 10 classes including single words and two-word phrases using Convolutional Neural Networks (CNN) without any intervention on the data. The proposed study using only-visual data obtained a model which is automated visual speech recognition with a deep learning approach. In addition, since this study uses only-visual data, the computational cost and resource usage is less than in multi-modal studies. It is also the first known study to address the lip reading problem with a deep learning algorithm using a new dataset belonging to the Ural-Altaic languages.

Plant leaf diseases can affect the plants’ leaves to an extent that the plants can collapse and die completely. These diseases may drastically drop the supply of vegetables and fruits to the market, and result in a low agricultural economy. In the literature, different laboratory methods of plant leaf disease detection have been used. These methods were time consuming and could not cover large areas for the detection of leaf diseases. This study infiltrates through the facilitated principles of the Convolutional Neural Networks (CNN) in order to model a network for image recognition and classification of these diseases. Neuroph was used to perform the training of a CNN network that recognized and classified images of the maize leaf diseases that were collected by use of a smart phone camera. A novel way of training and the methodology used, expedite a quick and easy implementation of the system in practice. The developed model was able to recognize 3 different types of maize leaf diseases out of healthy leaves. The Northern Corn Leaf Blight (Exserohilum), Common Rust (Puccinia sorghi) and Gray Leaf Spot (Cerospora) diseases were chosen for this study as they affect most parts of Southern Africa’s maize fields.

Interest in drone solutions in forestry applications is growing. Using drones, datasets can be captured flexibly and at high spatial and temporal resolutions when needed. In forestry applications, fundamental tasks include the detection of individual trees, tree species classification, bio-mass estimation, etc. Deep Neural Networks (DNN) have shown superior results when comparing with conventional machine learning methods such as Multi-Layer Perceptron (MLP) in cases of huge input data. The objective of this research was to investigate 3D convolutional neural networks (3D-CNN) to classify three major tree species in a boreal forest: pine, spruce, and birch. The proposed 3D-CNN models were employed to classify tree species in a test site in Finland. The classifiers were trained with a dataset of 3039 manually labelled trees. Then the accuracies were assessed by employing independent datasets of 803 records. To find the most efficient set of feature combination, we compare the performances of 3D-CNN models trained with hyperspectral (HS) channels, RGB channels, and canopy height model (CHM), separately and combined. It is demonstrated that the proposed 3D-CNN model with RGB and HS layers produces the highest classification accuracy. The producer accuracy of the best 3D-CNN classifier on the test dataset were 99.6%, 94.8%, and 97.4% for pines, spruces, and birches, respectively. The best 3D-CNN classifier produced ~5% better classification accuracy than the MLP with all layers. Our results suggest that the proposed method provides excellent classification results with acceptable performance metrics for HS datasets. Our results show that pine class was detectable in most layers. Spruce was most detectable in RGB data, while birch was most detectable in the HS layers. Furthermore, the RGB datasets provide acceptable results for many low-accuracy applications.

Reliable prediction of infectious disease is an essential need for public health organizations to minimize or prevent the disease's spread. With the increase of data growth in the healthcare sector, accurate analysis of such data helps for early disease detection and better patient care. With the availability of huge data, it is a great deal for predicting and handling an epidemic occurrence. To control the epidemic disease, a new Artificial Intelligence called Manhattan Hessian Linear Feature Embedding Based Tucker Congruence Deep Convolutional Network (MHLFE-TCDCN) is introduced with higher accuracy and minimum time consumption. The MIoT device is used to collect the data from the patient body. The proposed MHLFE-TCDCN includes two major processes namely feature selection and classification with multiple layers such as input, two hidden layers, and one output layer. First, the feature selection process is performed using the Manhattan Hessian Locally Linear Embedding technique in the first hidden layer to minimize the dimensionality and time complexity. After selecting the feature, the classification is performed using the Tucker coefficient of congruence regression. The Tucker coefficient of congruence measures the similarity between the training and testing disease data. A positive similarity level is used to accurately identify the disease at the output layer based on the similarity measure. Experimental assessment is carried out with various parameters such as accuracy, precision, recall, F-measure, and computational time concerning several numbers of patient data. The quantitative results and discussion that the proposed MHLFE-TCDCN technique achieves higher accuracy with minimal computation time as compared to the conventional methods.

The difficulty in detecting tumors in earlier stages is the major cause of mortalities of patients, despite the advancements in treatment and research regarding ovarian cancer. Deep Learning algorithms are applied to serve the purpose of a diagnostic tool by applying them on CT scan images of the ovarian region. The images go through a series of pre-processing techniques and further the tumor is segmented using the UNet model. Instances are then classified into two categories – benign and malignant tumors. Classification is performed using Deep Learning models like CNN, ResNet, DenseNet, Inception-ResNet, VGG16 and Xception along with Machine Learning models such as Random Forest, Gradient Boosting, AdaBoosting, XGBoosting. DenseNet 121 emerges as the best model on this dataset even after applying optimization on the Machine Learning models by obtaining an accuracy of 95.7%. The current work demonstrates the comparison of multiple CNN architectures among themselves and with common Machine Learning algorithms, with and without optimization techniques applied.

Military medical service is an important component of the Armed Forces of any country [1]. In wartime, healthcare performs critical functions of medical support for military operations, providing effective and qualified care to the wounded and sick. Consequently, it is imperative to equip military hospitals with state-of-the-art materials and technical resources commensurate with the predefined requirements. In this study, we examine a series of enhancements applied to the TurtleBot 3 Waffle Pi-based medical assistant robot, tailored for deployment in military hospital settings.

One of the critical tools for early detection and subsequent evaluation of the incidence of lung diseases is chest radiography. At a time when the speed and reliability of results, especially for COVID-19 positive patients, is important, the development of applications that would facilitate the work of untrained staff involved in the evaluation is also crucial. Our model takes the form of a simple and intuitive application, into which you only need to upload X-rays: tens or hundreds at once. In just a few seconds, the physician will determine the patient's diagnosis, including the percentage accuracy of the estimate. While the original idea was a mere binary classifier that could tell if a patient was suffering from pneumonia or not, in this paper we present a model that distinguishes between a bacterial disease, a viral infection, or a finding caused by COVID-19. The aim of this research is to demonstrate whether pneumonia can be detected or even spatially localized using a uniform, supervised classification.

Object segmentation on remotely-sensed images: aerial (or very high resolution, VHS) images and satellite (or high resolution, HR) images, has been applied to many application domains, especially road extraction in which the segmented objects are served as a mandatory layer in geospatial databases. Several attempts in applying deep convolutional neural network (DCNN) to extract roads from remote sensing images have been made; however, the accuracy is still limited. In this paper, we present an enhanced DCNN framework specifically tailored for road extraction on remote sensing images by applying landscape metrics (LMs) and conditional random fields (CRFs). To improve DCNN, a modern activation function, called exponential linear unit (ELU), is employed in our network resulting in a higher number of and yet more accurate extracted roads. To further reduce falsely classified road objects, a solution based on an adoption of LMs is proposed. Finally, to sharpen the extracted roads, a CRF method is added to our framework. The experiments were conducted on Massachusetts road aerial imagery as well as THEOS satellite imagery data sets. The results showed that our proposed framework outperformed Segnet, the state-of-the-art object segmentation technique on any kinds of remote sensing imagery, in most of the cases in terms of precision, recall, and F1.

Microscopic vision plays an important role in automated micro-assembly. However, some uncertain factors in the assembly process, such as occlusion and stains can lead to the mistakes of feature extraction. Herein, to solve the problem, the deep learning techniques are introduced into the feature recognition tasks, focusing on the attention mechanism and visualizing CNNs for DL-based microscopic vision. The main contributions are summarized as follows: The CBAM attention mechanism is combined with the YOLOv5 algorithm to improve the accuracy and robustness of feature extraction. The micropart feature occlusion experiment results show that at 70% occlusion degree, YOLOV5-CBAM can reach 97.9% mAP@0.5, which is 4.6% higher than the original one. Visualization analysis of DL-based model is conducted using Grad-CAM to make the decision result more transparent and avoid potential visual detection risks during assembly. The heatmap matching degree between GT area and high-light area is increased by 27.81% on average, which further verify the effectiveness of attention mechanism in micropart feature localization. Additionally, micropart surface stain and droplet quality classification models based on ResNet50 are trained to replace the manual sorting. The visual results are consistent with human eye discernment and judgement, confirming the reliability of parts and droplets sorting.

Arrhythmia is a cardiac condition characterized by an irregular heart rhythm that hinders the proper circulation of blood, posing a severe risk to individuals’ lives. Globally, arrhythmias are recognized as a significant health concern, accounting for nearly 12 percent of all deaths. As a result, there has been a growing focus on utilizing artificial intelligence for the detection and classification of abnormal heartbeats. In recent years, self-operated heartbeat detection research has gained popularity due to its cost-effectiveness and potential for expediting therapy for individuals at risk of arrhythmias. However, building an efficient automatic heartbeat monitoring approach for arrhythmia identification and classification comes with several significant challenges. These challenges include addressing issues related to data quality, determining the range for heart rate segmentation, managing data imbalance difficulties, handling intra- and inter-patient variations, distinguishing supraventricular irregular heartbeats from regular heartbeats, and ensuring model interpretability. In this study, we propose the Reseek-Arrhythmia model, which leverages deep learning techniques to automatically detect and classify heart arrhythmia diseases. The model combines different convolutional blocks and identity blocks, along with essential components such as convolution layers, batch normalization layers, and activation layers. To train and evaluate the model, we utilized the MIT-BIH and PTB datasets. Remarkably, the proposed model achieves outstanding performance with an accuracy of 99.35% and 93.50% and an acceptable loss of 0.688 and 0.2564, respectively.

In the current industrial landscape, increasingly pervaded by technological innovations, the adoption of optimized strategies for asset management is becoming a critical key success factor. Among the various strategies available, the “Prognostics and Health Management” strategy is able to support maintenance management decisions more accurately, through continuous monitoring of equipment health and “Remaining Useful Life” forecasting. In the present study, Convolutional Neural Network-based Deep Neural Network techniques are investigated for the Remaining Useful Life prediction of a punch tool, whose degradation is caused by working surface deformations during the machining process. Surface deformation is determined using a 3D scanning sensor capable of returning point clouds with micrometric accuracy during the operation of the punching machine, avoiding both downtime and human intervention. The 3D point clouds thus obtained are transformed into bidimensional image-type maps, i.e., maps of depths and normal vectors, to fully exploit the potential of convolutional neural networks for extracting features. Such maps are then processed by comparing 15 genetically optimized architectures with the transfer learning of 19 pre-trained models, using a classic machine learning approach, i.e., Support Vector Regression, as a benchmark. The achieved results clearly show that, in this specific case, optimized architectures provide performance far superior (MAPE=0.058) to that of transfer learning which, instead, remains at a lower or slightly higher level (MAPE=0.416) than Support Vector Regression (MAPE=0.857).

Hyperspectral image and LiDAR image fusion plays a crucial role in remote sensing by capturing spatial relationships and modeling semantic information for accurate classification and recognition. However, existing methods, like Graph Convolutional Networks (GCNs), face challenges in constructing effective graph structures due to variations in local semantic information and limited receptiveness to large-scale contextual structures. To overcome these limitations, we proposed a invariant attribute-driven binary bi-branch classification (IABC) method which is a unified network that combines binary Convolutional Neural Network (CNN) and GCN with invariant attributes. Our approach utilizes a joint detection framework that can simultaneously learn features from small-scale regular regions and large-scale irregular regions, resulting in an enhanced structured representation of HSI and LiDAR images in the spectral-spatial domain. This approach not only improves the accuracy of classification and recognition but also reduces storage requirements and enables real-time decision-making, which is crucial for effectively processing large-scale remote sensing data. Extensive experiments demonstrates the superior performance of our proposed method in hyperspectral image analysis tasks. The combination of CNNs and GCNs allows for accurate modeling of spatial relationships and effective construction of graph structures. Furthermore, the integration of binary quantization enhances computational efficiency, enabling real-time processing of large-scale data. Therefore, our approach presents a promising opportunity for advancing remote sensing applications using deep learning techniques.

Direct measurement of strain within muscle is important for understanding muscle function in health and disease. Current technology (kinematics, dynamometry, electromyography) provides limited ability to measure strain within muscle. Regional fiber orientation and length are related with active/passive strain within muscle. Currently, ultrasound imaging provides the only non-invasive means of observing regional fiber orientation within muscle during dynamic tasks. Previous attempts to automatically estimate fiber orientation from ultrasound are not adequate, often requiring manual region selection, feature engineering, providing low-resolution estimations (one angle per muscle), and deep muscles are often not attempted. Here, we propose deconvolutional neural networks (DCNN) for estimating fiber orientation at the pixel-level. Dynamic ultrasound images sequences of the calf muscles were acquired (25 Hz) from 8 healthy volunteers (4 male, ages: 25–36, median 30). A combination of expert annotation and interpolation/extrapolation provided labels of regional fiber orientation for each image. We then trained DCNNs both with and without dropout using leave one out cross-validation. Our results demonstrated robust estimation of regional fiber orientation with approximately 3° error, which was an improvement on previous methods. The methods presented here provide new potential to study muscle in disease and health.

In this paper, we present a framework to study the spatial structure of noctilucent clouds formed by ice particles in the upper atmosphere at mid and high latitude during summer. We study noctilucent cloud activity in optical images taken from three different locations and under different atmospheric conditions. In order to identify and distinguish noctilucent cloud activity from other objects in the scene, we employ linear discriminant analysis (LDA) with feature vectors ranging from simple metrics to higher-order local autocorrelation (HLAC), and histogram of oriented gradients (HOG). Finally, we propose a Convolutional Neural Networks (CNN) based method for the detection of noctilucent clouds. The results clearly indicate that the CNN based approach outperforms LDA based methods used in this article. Furthermore, we outline suggestions for future research directions to establish a framework that can be used for synchronizing the optical observations from ground based camera systems with echoes measured with radar systems like EISCAT in order to obtain independent additional information on the ice clouds.

Deep Learning is an extremely important research topic in Earth Observation. Current use-cases range from semantic image segmentation, object detection to more common problems found in computer vision such as object identification. Earth Observation is an excellent source for different types of problems and data for Machine Learning in general and Deep Learning in particular. It can be argued that both Earth Observation and Deep Learning as fields of research will benefit greatly from this recent trend of research. In this paper we take several state of the art Deep Learning network topologies and provide a detailed analysis of their performance for semantic image segmentation for building footprint detection. The dataset used is comprised of high resolution images depicting urban scenes. We focused on single model performance on simple RGB images. In most situations several methods have been applied to increase the accuracy of prediction when using deep learning such as ensembling, alternating between optimisers during training and using pretrained weights to bootstrap new models. These methods although effective, are not indicative of single model performance. Instead, in this paper, we present different topology variations of these state of the art topologies and study how these variations effect both training convergence and out of sample, single model, performance.

This work addresses the challenge of EEG visual multiclass classification into 40 classes for Brain-Computer interface applications, by using deep learning architectures. The visual multiclass classification approach offers BCI applications a significant advantage, since it allows the supervision of more than one BCI interaction, considering that each class label supervises a BCI task. However, because of the nonlinearity and nonstationarity of EEG signals, performing multiclassifiers based on EEG features remains a significant challenge for BCI systems. In the present work, mutual information-based discriminant channel selection and Minimum-Norm Estimate algorithms are implemented to select discriminant channels and enhance EEG data. Hence, deep EEGNet and Convolutional-recurrent neural networks are implemented separately to classify EEG data of image visualization into 40 labels. By using the k-fold cross-validation approach, average classification accuracies of 94.8% and 89.8% were obtained by implementing the aforementioned network architectures, respectively. Satisfactory results obtained with this method offer a new implementation opportunity for multi-task BCI applications by utilizing a reduced number of channels (<50%), compared to those presented in the related literature where the whole set of channels is used.

To effectively preserve marine environments and manage endangered species, it is necessary to employ efficient, precise, and scalable solutions for environmental monitoring. Ecoacoustics provides several benefits as it enables non-intrusive, prolonged sampling of environmental sounds, making it a promising tool for conducting biodiversity surveys. However, analyzing and interpreting acoustic data can be time-consuming and often demands substantial human supervision. This challenge can be addressed by harnessing contemporary methods for automated audio signal analysis, which have exhibited remarkable performance due to advancements in deep learning research. This paper introduces a research investigation into developing an automatic computerized system to detect dolphin whistles. The proposed method utilizes a fusion of various resnet50 networks integrated with data augmentation techniques. Through extensive experiments conducted on a publically available benchmark, our findings demonstrate that our ensemble yields significant performance enhancements across all evaluated metrics. The MATLAB/PyTorch source code is freely available at: https://github.com/LorisNanni/

Existing Knowledge Graph (KG) models for commonsense question answering present two challenges: (i) existing methods retrieved entities related to questions from the knowledge graph, which may extract noise and irrelevant nodes, and (ii) lack of interaction representation between questions and graph entities. However, current methods mainly focus on retrieving relevant entities with some noisy and irrelevant nodes. In this paper, we propose a novel Retrieval-augmented Knowledge Graph (RAKG) model, which solves the above issues through two key innovations. First, we leverage the density matrix to make the model reason along the corrected knowledge path and extract an enhanced knowledge graph subgraph. Second, we fuse representations of questions and graph entities through a bidirectional attention strategy, in which two representations fuse and update by Graph Convolutional Network (GCN). To evaluate the performance of our method, we conduct experiments on two widely-used benchmark datasets CommonsenseQA and OpenBookQA. The case study gives insight into findings that the augmented subgraph provides reasoning along the corrected knowledge path for question answering.

The detection performance of small objects in remote sensing images has not been satisfactory compared to large objects, especially in low-resolution and noisy images. A generative adversarial network (GAN)-based model called enhanced super-resolution GAN (ESRGAN) showed remarkable image enhancement performance, but reconstructed images usually miss high-frequency edge information. Therefore, object detection performance showed degradation for small objects on recovered noisy and low-resolution remote sensing images. Inspired by the success of edge enhanced GAN (EEGAN) and ESRGAN, we applied a new edge-enhanced super-resolution GAN (EESRGAN) to improve the quality of remote sensing images and used different detector networks in an end-to-end manner where detector loss was backpropagated into the EESRGAN to improve the detection performance. We proposed an architecture with three components: ESRGAN, EEN, and Detection network. We used residual-in-residual dense blocks (RRDB) for both the ESRGAN and EEN, and for the detector network, we used a faster region-based convolutional network (FRCNN) (two-stage detector) and a single-shot multibox detector (SSD) (one stage detector). Extensive experiments on a public (car overhead with context) dataset and another self-assembled (oil and gas storage tank) satellite dataset showed superior performance of our method compared to the standalone state-of-the-art object detectors.

Amidst the burgeoning global concerns surrounding skin cancer, the quest for precise and efficient diagnostic methodologies has intensified. Conventional diagnostic approaches relying on subjective visual inspection by dermatologists often grapple with subjectivity and resource constraints. This predicament underscores the critical role of Artificial Intelligence (AI) in revolutionizing diagnostic paradigms by significantly enhancing accuracy and accessibility. This study embarks on an extensive exploration of skin cancer classification, specifically targeting the intricate task of 8-class skin cancer classification. Employing cutting-edge deep learning models on the ISIC2019 dataset, a comprehensive analysis was conducted utilizing a diverse array of pre-trained ImageNet architectures and Vision Transformer models. To counteract the inherent class imbalance in skin cancer datasets, a pioneering "Naturalize" augmentation technique was introduced, leading to the creation of indispensable tools—the Naturalized 2.4K ISIC2019 and groundbreaking Naturalized 7.2K ISIC2019 datasets—catalyzing advancements in classification accuracy. The pivotal role of AI in mitigating the risks of misdiagnosis and under-diagnosis in skin cancer becomes evident through this research. AI’s proficiency in analyzing vast datasets and discerning subtle patterns augments the diagnostic prowess of dermatologists significantly. Integration of AI-powered systems in skin cancer diagnosis holds immense promise in facilitating early detection and substantially improving patient outcomes. The meticulous evaluation employed quantitative measures such as confusion matrices, classification reports, and visual analyses using Score-CAM across diverse dataset variations. Astonishingly, the culmination of these endeavors culminated in an unprecedented achievement—100% average accuracy, precision, recall, and F1-Score—within the groundbreaking Naturalized 7.2K ISIC2019 dataset. This remarkable feat underscores the transformative potential of AI-driven methodologies, exemplifying their prowess in redefining the landscape of skin cancer diagnosis and management. In summary, this research represents a pivotal stride towards redefining dermatological diagnosis, showcasing the remarkable impact of AI-powered solutions in surmounting the challenges inherent in skin cancer diagnosis. The attainment of 100% across crucial metrics within the Naturalized 7.2K ISIC2019 dataset serves as a testament to the transformative capabilities of AI-driven approaches in reshaping the trajectory of skin cancer diagnosis and patient care.

With the fast development of the mobile Internet, the online platforms of social networks have rapidly been developing for the purpose of making friends, sharing information, etc. In these online platforms, users being related to each other forms social networks. Literature reviews have shown that social networks have community structure. Through the studies of community structure, the characteristics and functions of networks structure and the dynamical evolution mechanism of networks can be used for predicting user behaviours and controlling information dissemination. Therefore, this study proposes a deep community detection method which includes (1) matrix reconstruction method, (2) spatial feature extraction method and (3) community detection method. The original adjacency matrix in social network is reconstructed based on the opinion leader and nearer neighbors for obtaining spatial proximity matrix. The spatial eigenvector of reconstructed adjacency matrix can be extracted by an auto-encoder based on convolution neural network for the improvement of modularity. In experiments, four open datasets of practical social networks were selected to evaluate the proposed method, and the experimental results show that the proposed deep community detection method obtained higher modularity than other methods. Therefore, the proposed deep community detection method can effectively detect high quality communities in social networks.

Convolutional neural network architectures have become increasingly complex, which has improved the performance slowly on well-known benchmark datasets in the recent years. In this research, we have analyzed the true need for such complexity. We have introduced G-Net light, a lightweight modified GoogleNet with improved filter count per layer to reduce feature overlaps and complexity. Additionally, by limiting the amount of pooling layers in the proposed architecture, we have exploited the skip connections to minimize the spatial information loss. The investigations on the proposed architecture are evaluated on three retinal vessel segmentation publicly available datasets. The proposed G-Net light outperforms other vessel segmentation architectures by reducing the number of trainable parameters..

Chickpea is one of the most widely consumed pulses globally because of its high protein content. The morphological features of chickpea seed, such as colour, texture, are observable and play a major role in classifying different chickpea varieties. This process is often carried out by human experts, and is time-consuming, inaccurate, and expensive. The objective of the study was to design an automated chickpea classifier using an RGB colour image-based model by considering the morphological features of chickpea seed. As part of the data acquisition process, five hundred and fifty images were collected per variety for four varieties of chickpea (CDC-Alma, CDC-Consul, CDC-Cory, and CDC-Orion) using an industrial RGB camera and a mobile phone camera. Three CNN-based models such as NasNet-A (mobile), MobileNetV3 (small), and EfficientNetB0 were evaluated using a transfer learning-based approach. The classification accuracy was 97%, 99%, and 98% for NasNet-A (mobile), MobileNetV3 (small), and EfficientNetB0 models, respectively. The MobileNetV3 model was used for further deployment on an Android mobile and Raspberry Pi 4 devices based on its higher accuracy and light-weight architecture. The classification accuracy for the four chickpea varieties was 100% while the MobileNetV3 model was deployed on both Android mobile and Raspberry Pi 4 platforms.

Convolutional neural networks (CNNs) have proven to be a very efficient class of machine learning (ML) architectures for handling multidimensional data by maintaining data locality, especially in the field of computer vision. Data pooling, a major component of CNNs, plays a crucial role for extracting important features of the input data and downsampling its dimensionality. Multidimensional pooling, however, is not efficiently implemented in existing ML algorithms. In particular, quantum machine learning (QML) algorithms have a tendency to ignore data locality for higher dimensions by representing/flattening multidimensional data as simple one-dimensional data. In this work, we propose using the quantum Haar transform (QHT) and quantum partial measurement for performing generalized pooling operations on multidimensional data. We present the corresponding decoherence-optimized quantum circuits for the proposed techniques along with their theoretical circuit depth analysis. Our experimental work was conducted using multidimensional data, ranging from 1-D audio data, to 2-D image data, to 3-D hyperspectral data, to demonstrate the scalability of the proposed methods. In our experiments, we utilized both noisy and noise-free quantum simulations on a state-of-the-art quantum simulator from IBM Quantum. We also show the efficiency of our proposed techniques for multidimensional data by reporting the fidelity of results.

Object detection is a popular image processing technique, widely used in numerous applications for detecting and locating objects in images or videos. While being one of the fastest algorithms for object detection, Single-Shot Multibox Detection (SSD) networks are also computationally very demanding, which limits their usage in real-time edge applications. Even though the SSD post-processing algorithm is not the most complex segment of the overall SSD object detection network, it is still computationally demanding and can become a bottleneck with respect to processing latency and power consumption, especially in edge applications with limited resources. When using hardware accelerators to accelerate backbone CNN processing, the SSD post-processing step implemented in software can become critical for high-end applications where high frame rates are required, as this paper shows. To overcome this problem, we propose Puppis, an architecture for hardware acceleration of SSD post-processing algorithm. As experiments will show, our solution will lead to an average SSD post-processing speedup of 34.42 when compared with a software implementation. Furthermore, execution of a complete SSD network will be on average 45.39 times faster than software implementation when the proposed Puppis SSD hardware accelerator is used together with some existing CNN accelerators.

A long time has passed since COVID-19 was discovered and widely disseminated. Both machine learning and deep learning have moved towards the research of diagnosing COVID-19. Compared to deep learning, traditional machine learning-based methods can also achieve good diagnostic results if they are improved based on innovative points. And deep learning remains a more popular research object. Based on the deep neural network in the field of deep learning, the convolutional neural network, recurrent neural network, long and short-term memory network, and Transformer model have been extended. These models improve the performance and processing power of neural networks by introducing new structures and algorithms. This paper will introduce the basic concepts of machine learning and deep learning, as well as the details of using the relevant methods of both to help diagnose COVID-19.

Objective A trained T1 class Convolutional Neural Network (CNN) model will be used to examine its ability to successfully identify motor imagery when fed pre-processed electroencephalography (EEG) data. In theory, and if the model has been trained accurately, it should be able to identify a class and label it accordingly. The CNN model will then be restored and used to try and identify the same class of motor imagery data using much smaller sampled data in an attempt to simulate live data. Approach PyCharm, a Python platform, will be used to house and process the CNN. The raw data used for the training of the CNN will be sourced from the PhysioBank website. The EEG signal data will then be pre-processed using Brainstorm software that is a toolbox used in conjunction with MATLAB. The sample data used to validate and test the trained CNN, will be also be extracted from Brainstorm but in a much smaller size compared to the training data which is comprised of thousands of images. The sample size would be comparable to a person wearing a Brain Computer Interface (BCI), offering approximately 20 seconds of motor imagery signal data. Results The raw EEG data was successfully extracted and pre-processed. The deep learning model was trained using the extracted image data along with their corresponding labels. After training, it was able to accurately identify the T1 class label at 100 percent. The python coding was then modified to restore the trained model and feed it test sample data in which it was found to recognise 6 out of 10 lines of T1 signal image data. The result suggested that the initial training of the model required a different, more varying approach, so that it would be able to detect varying sample signal image data. The outcome of which could mean that the model could be used in applications for multiple patients wearing the same BCI hardware to control a device or interface.

The use of convolutional neural networks (CNNs) in image classification has become the standard method of approaching computer vision problems. Here we apply pre-trained networks to classify images of non-breaking, plunging and spilling breaking waves. The CNNs are used as basic feature extractors and a classifier is then trained on top of these networks. The dynamic nature of breaking waves is exploited by using image sequences to gain extra information and improve the classification results. We also see improved classification performance in using pre-computed image features such as the optical flow between image pairs. The inclusion of the dynamic information improves the classification between breaking wave classes. We also provide corrections to the methodology from the article from which the data originates to achieve a more accurate assessment of performance.

Due to the recent trend of unmanned economy, retail stores have gradually reduced their service and cashier manpower. The retail product recognition becomes one of key issues for unmanned shopping. Although the success of deep neural networks has made object recognition feasible in a variety of applications, it still struggles to perform well on a large number of classes of retail products. In this paper, we propose an improved ensemble learning-based approach for retail product recognition. In the proposed approach, the object classification network is first improved through feature extraction and block attention. An ensemble model is then built by integrating multiple network models and using loss selection as model weights. In the experiments, the feasibility of our ensemble learning-based approach has been evaluated through many production items. The results demonstrate the effectiveness of the proposed approach compared with previous retail product recognition methods.

This paper introduces a multi-class hand gesture recognition model developed to identify a set of defined hand gesture sequences in two-dimensional RGB video recordings. The work presents an action detection classifier that looks at both appearance and spatiotemporal parameters of consecutive frames. The classifier utilizes a convolutional-based network combined with a long-short-term memory unit. To leverage the need for a large-scale dataset, the model uses an available dataset to then adopt a technique known as transfer learning to fine-tune the model on the hand gestures of relevance. Validation curves performed over a batch size of 64 indicate an accuracy of 93.95% (± 0.37) with a mean Jaccard index of 0.812 (± 0.105) for 22 participants. The presented model illustrates the possibility of training a model with a small set of data (113,410 fully labelled frames). The proposed pipeline embraces a small-sized architecture that could facilitate its adoption.

Automated weeding is an important research area in agrorobotics. Weeds can be removed mechanically or with the precise usage of herbicides. Deep Learning techniques achieved state of the art results in many computer vision tasks, however their deployment on low-cost mobile computers is still challenging. These paper present an advanced version of the system presented in [1]. The described system contains several novelties, compared both with its previous version and related work. It is a part of a project of the automatic weeding machine, developed by Warsaw University of Technology and MCMS Warka Ltd. The obtained model reaches satisfying accuracy at over 10~FPS on the Raspberry Pi 3B+ computer. It was tested for four different plant species at different growth stadiums and lighting conditions. The system performing semantic segmentation is based on Convolutional Neural Networks. Its custom architecture mixes U-Net, MobileNets, DenseNet and ResNet concepts. Amount of needed manual ground truth labels was significantly decreased by the usage of knowledge distillation process, learning final model to mimic an ensemble of complex models on the large database of unlabeled data. Further decrease of the inference time was obtained by two custom modifications: in the usage of separable convolutions in DenseNet block and in the number of channels in each layer. In the authors’ opinion, described novelties can be easily transferred to other agrorobotics tasks.

Obesity and overweight is a foremost concern around the globe for each group of age. This can be accelerated by the current imposed lockdown. However, excessive weight gain may result in other chronic diseases. This study has been considering the age group of 25 to 55 years as the sample populations and monitoring them from July, 2020 to November, 2020. The lifestyle of this population, food habit, mental health conditions are explored using deep learning based framework. All these parameters need to be monitored as these have close relation with currently imposed constraints due to COVID-19. A predictive model is constructed using deep learning techniques to predict the risk of gaining weight. The predictive model hybridizes the convolutional layer and gated recurrent neural networks as a unified entity for achieving the objective of early weight gain prediction. The result obtained by this model exhibits an encouraging predictive efficiency of 93.7%.

In August 2020, the World Health Assembly launched a global initiative to eliminate cervical cancer by 2030, setting three primary targets. One key goal is to achieve a 70% screening coverage rate for cervical cancer, primarily relying on the precise analysis of Papanicolaou (Pap) or digital Pap smears. However, the responsibility of reviewing Pap smear samples to identify potentially cancerous cells primarily falls on pathologists, a task known to be exceptionally challenging and time-consuming. This paper proposes a solution to address the shortage of pathologists for cervical cancer screening. It leverages the OpenAI-GYM API to create a deep reinforcement learning environment utilizing liquid-based Pap smear images. By employing the Proximal Policy Optimization algorithm, autonomous agents navigate Pap smear images, identifying cells with the aid of rewards, penalties, and accumulated experiences. Furthermore, the use of a pre-trained convolutional neuronal network like Res-Net54 enhances the classification of detected cells based on their potential for malignancy. The ultimate goal of this study is to develop a highly efficient, automated Papanicolaou analysis system, ultimately reducing the need for human intervention in regions with limited pathologists.

The mutton sheep breeding industry has transformed significantly in recent years, from traditional grassland free-range farming to a more intelligent approach. As a result, automated sheep face recognition systems have become vital to modern breeding practices and have gradually replaced ear tagging and other manual tracking techniques. Although sheep face datasets have been introduced in previous studies, they have often involved pose or background restrictions (e.g., fixing of the subject’s head, cleaning of the face), which restrict data collection and have limited the size of available sample sets. As a result, a comprehensive benchmark designed exclusively for the evaluation of individual sheep recognition algorithms is lacking. To address this issue, this study develops a large-scale benchmark dataset, Sheepface-107, comprised of 5,350 images acquired from 107 different subjects. Images were collected from each sheep at multiple angles, including front and back views, in a diverse collection that provides a more comprehensive representation of facial features. In addition to the dataset, an assessment protocol is developed by applying multiple evaluation metrics to the results produced by three different deep learning models: VGG16, GoogLeNet, and ResNet50, which achieved F1-scores of 83.79%, 89.11%, and 93.44%, respectively. A statistical analysis of each algorithm suggested that accuracy and the number of parameters were the most informative metrics for use in evaluating recognition performance.

Recently, deep learning models have been widely applied to modulation recognition, which have become a hot topic due to their excellent end-to-end learning capabilities. However, current methods are mostly based on uni-modal inputs, which suffer from incomplete information and local optimization. To complement the advantages of different modalities, we focus on the multi-modal fusion method. Therefore, we introduce an iterative dual-scale attentional fusion (iDAF) method to integrate multimodal data. Firstly, two feature maps with different receptive field sizes are constructed using local and global embedding layers. Secondly, the feature inputs are iterated into the Iterative Dual Scale Attention Module (iDCAM), where the two branches capture the details of high-level features and the global weights of each modal channel, respectively. The iDAF not only extracts the recognition characteristics of each specific domains, but also complements the strengths of different modalities to obtain a fruitful view. Our iDAF achieves a recognition accuracy of 93.5\% at 10dB and 0.6232 at full SNR. The comparative experiments and ablation studies effectively demonstrate the effectiveness and superiority of the iDAF.

Nowadays, power sector is an area that gather great scientific interest, due to events such as the increase in electricity prices in the wholesale energy market and new investments due to technological development in various sectors. These new challenges have in turn created new needs, such as the accurate prediction of the electrical load of the end users. On the occasion of the new challenges, Artificial Neural Networks approaches have become increasingly popular due to their ability to adopt efficiently to time-series predictions. In this paper, it is presented the development of a model which, through an automated process, will provide an accurate prediction of electrical load for the island of Thira in Greece. Through an automated application, deep learning load forecasting models have been created, such as Multilayer Perceptron, Long Short-Term Memory (LSTM), Convolutional Neural Network One Dimensional (CNN-1D), CNN-LSTM, Temporal Convolutional Network (TCN) and a proposed hybrid model called Convolutional LSTM Encoder-Decoder. The results in terms of prediction accuracy show satisfactory performances for all models, with the proposed hybrid model achieving the best accuracy.

It is very difficult to accurately divide farmland and woodland in Gaofen 2 (GF-2) remote sensing image, because their single plant coverage is very small, and their spectra are very similar. The ratio of spatial resolution and one plant’s coverage area must be fully taken into account when designing the Convolutional Neural Network structure for extracting them from GF-2 image. We establish a Convolutional Encode Neural Networks model (CENN), The first layer has two sets of convolution kernels to learn the characteristics of farmland and woodland respectively, while the second layer is the encoder to encode the characteristics by transfer function, which can map the results to the corresponding category number. In the training stage, samples of farmland, woodland, and other categories are categorically used to train CENN, as soon as training is accomplished, CENN would acquire enough ability to accurately extract farmland and woodland from remote sensing images. The final extraction result is obtained by implementing per-pixel segmentation of images used to train the CENN. CENN is compared and analyzed with others such as Deep Belief Network (DBN), Full Convolutional Network (FCN), Deeplab Model. The results of experiments show that CENN can more accurately mine the characteristics of farmland and woodland, and it achieves its goal of extracting farmland and woodland with high precision from GF-2 images.

Banks are normally offered two kinds of deposit accounts. It consists of deposits like current/saving account and term deposits like fixed or recurring deposits. For enhancing the maximized profit from bank as well as customer perspective, term deposit can accelerate uplifting of finance fields. This paper focuses on likelihood of term deposit subscription taken by the customers. Bank campaign efforts and customer detail analysis can influence term deposit subscription chances. An automated system is approached in this paper that works towards prediction of term deposit investment possibilities in advance. This paper proposes deep learning based hybrid model that stacks Convolutional layers and Recurrent Neural Network (RNN) layers as predictive model. For RNN, Gated Recurrent Unit (GRU) is employed. The proposed predictive model is later compared with other benchmark classifiers such as k-Nearest Neighbor (k-NN), Decision tree classifier (DT), and Multi-layer perceptron classifier (MLP). Experimental study concludes that proposed model attains an accuracy of 89.59% and MSE of 0.1041 which outperform well other baseline models.

With the continuous progress of remote sensing image object detection tasks in recent years, researchers in this field have gradually shifted the focus of their research from horizontal object detection to the study of object detection in arbitrary directions. It is worth noting that some properties are different from the horizontal object detection during oriented object detection that researchers have yet to notice much. This article presents the design of a straightforward and efficient arbitrary-oriented detection system, leveraging the inherent properties of the orientation task, including the rotation angle and box aspect ratio. In the detection of low aspect ratio objects, the angle is of little importance to the orientation bounding box, and it is even difficult to define the angle information in extreme categories. Conversely, in the detection of objects with high aspect ratios, the angle information plays a crucial role and can have a decisive impact on the quality of the detection results. By exploiting the aspect ratio of different targets, this letter proposes a ratio-balanced angle loss that allows the model to make a better trade-off between low-aspect ratio objects and high-aspect ratio objects. The rotation angle of each oriented object, which we naturally embed into a two-dimensional Euclidean space for regression, thus avoiding an overly redundant design and preserving the topological properties of the circular space. The performance of the UCAS-AOD, HRSC2016, and DLR-3K datasets show that the proposed model in this paper achieves a leading level in terms of both accuracy and speed. The code is released at https://github.com/minghuicode/Periodic-Pseudo-Domain.

Background: Processing whole-slide images (WSI) to train neural networks can be intricate and laborious. We developed an open-source library covering recurrent tasks in processing of WSI and in evaluating the performance of the trained networks for classification tasks. Methods: Two histopathology use-cases were selected. First we aimed to train a CNN to distinguish H&E-stained slides obtained from neuropathologically classified low-grade epilepsy-associated dysembryoplastic neuroepithelial tumor (DNET) and ganglioglioma (GG). The second project we trained a convolutional neural network (CNN) to predict the hormone expression of pituitary adenoms only from hematoxylin and eosin (H&E) stained slides. In the same approach, we addressed the issue to also predict clinically silent corticotroph adenoma. We included four clinico-pathological disease conditions in a multilabel approach. Results: Our best performing CNN achieved an area under the curve (AUC) of 0.97 for the receiver operating characteristic (ROC) for corticotroph adenoma, 0.86 for silent corticotroph adenoma and 0.98 for gonadotroph adenoma. Our DNET-GG classifier achieved an AUC of 1.00 for the ROC curve. All scores were calculated with the help of our library on predictions on a case basis. Conclusions: Our comprehensive library is most helpful to standardize the work-flow and minimize the work-burden in training CNN. It is also compatible with fastai. Indeed, our new CNNs reliably extracted neuropathologically relevant information from the H&E staining only. This approach will supplement the clinico-pathological diagnosis of brain tumors, which is currently based on cost-intense microscopic examination and variable panels of immunohistochemical stainings.

For the Remotely Piloted Aircraft Systems (RPAS) market to continue its current growth rate, cost-effective "Detect and Avoid" systems that enable safe beyond visual line of sight (BVLOS) operations are critical. We propose an audio-based "Detect and Avoid" system, composed of microphones and an embedded computer, which performs real-time inferences using a sound event detection (SED) deep learning model. Two state-of-the-art SED models, YAMNet and VGGish, are fine-tuned using our dataset of aircraft sounds and their performances are compared for a wide range of configurations. YAMNet, whose MobileNet architecture is designed for embedded applications, outperformed VGGish both in terms of aircraft detection and computational performance. YAMNet's optimal configuration, with > 70% true positive rate and precision, results from combining data augmentation and undersampling with the highest available inference frequency (i.e. 10 Hz). While our proposed "Detect and Avoid" system already allows the detection of small aircraft from sound in real time, additional testing using multiple aircraft types is required. Finally, a larger training dataset, sensor fusion, or remote computations on cloud-based services could further improve system performance.

An autonomous concrete crack inspection system is necessary for preventing hazardous incidents arising from deteriorated concrete surfaces. In this paper, we represent a concrete crack detection framework to aid the process of automated inspection. The proposed approach employs a deep convolutional neural network architecture for crack segmentation from concrete image. The proposed network alleviates the effect of gradient vanishing problem present in deep neural network architectures. A feature silencing module is incorporated in the crack detection framework, for eliminating unnecessary feature maps from the network. The overall performance of the network significantly improves as a result. Experimental results support the benefit of incorporating feature silencing within a convolutional neural network architecture for improving the network’s robustness, sensitivity, and specificity. An added benefit of the proposed architecture is its ability to accommodate for the trade-off between specificity (positive class detection accuracy) and sensitivity (negative class detection accuracy) with respect to the target application. Furthermore, the proposed framework achieves a high precision rate and processing time than crack detection architectures present in literature.

Intelligent fault diagnosis techniques have replaced the time-consuming and unreliable human analysis, increasing the efficiency of fault diagnosis. Deep learning model can improve the accuracy of intelligent fault diagnosis with the help of its multilayer nonlinear mapping ability. This paper has proposed a novel method named Deep Convolutional Neural Networks with Wide First-layer Kernels (WDCNN). The proposed method uses raw vibration signals as input (data augmentation is used to generate more inputs), and uses the wide kernels in first convolutional layer for extracting feature and suppressing high frequency noise. Small convolutional kernels in the preceding layers are used for multilayer nonlinear mapping. AdaBN is implemented to improve the domain adaptation ability of the model. The proposed model addresses the problem that currently, the accuracy of CNN applied to fault diagnosis is not very high. WDCNN can not only achieve 100% classification accuracy on normal signals, but also outperform state of the art DNN model which is based on frequency features under different working load and noisy environment.

Using smartphone especially android platform has already got eighty percent market shares, due to aforementioned report, it becomes attacker’s primary goal. There is a growing number of private data onto smart phones and low safety defense measure, attackers can use multiple way to launch and to attack user’s smartphones.(e.g. Using different coding style to confuse the software of detecting malware). Existing android malware detection methods use multiple features, like safety sensor API, system call, control flow structure and data information flow, then using machine learning to check whether its malware or not. These feature provide app’s unique property and limitation, that is to say, from some perspectives it might suit for some specific attack, but wouldn’t suit for others. Nowadays most malware detection methods use only one aforementioned feature, and these methods mostly analysis to detect code, but facing the influence of malware’s code confusion and zero-day attack, aforementioned feature extraction method may cause wrong judge. So, it’s necessary to design an effective technique analysis to prevent malware. In this paper, we use the importance of word from apk, because of code confusion, some malware attackers only rename variables, if using general static analysis wouldn’t judge correctly, then use these importance value to go through our proposed method to generate picture, finally using convolutional neural network to see whether the apk file is malware or not.

Generally, when cognitive radio users are in Underlay mode, through the analysis of the electromagnetic spectrum in time, frequency, space, and field of multiple dimensional data accounts for judging the inherent correlation of the radio spectrum used/idle state, this is the realization of cognitive radio users (secondary users, SUs) efficient access to the foundation of the limited spectrum resources. Therefore, how to efficiently use of spectrum status of each SU implementation of reception multidimensional combination forecasting is the core of this paper addressing the problem. In this paper, we propose a deep-learning hybrid model called TensorGCN-LSTM based on the tensor data structure. The model treats SUs deployed at different spatial locations under the same frequency and the spectrum status of SUs themselves under different frequencies in the task area as nodes and constructs two types of graph structures. Graph convolutional operations are used to sequentially extract corresponding spatial-domain and frequency-domain features from the two types of graph structures. Then, the long short-term memory (LSTM) model is used to fuse the spatial, frequency, and temporal features of the cognitive radio environment data. Finally, the prediction task of the spectrum distribution situation is accomplished through fully connected layers. Specifically, the model constructs a tensor graph based on the spatial similarity of SUs' locations and the frequency correlation between different frequency signals received by SUs, which describes the electromagnetic wave's dependency relationship in spatial and frequency domains. LSTM is used to capture the electromagnetic wave's dependency relationship in the temporal domain. To evaluate the effectiveness of the model, we conducted ablation experiments on LSTM, GCN, GC-LSTM, and TensorGCN-LSTM models using simulated data. The experimental results show that our model achieved better prediction performance in RMSE, and the correlation coefficient R2 of 0.8753 also confirmed the feasibility of the model.

During geomagnetic storms, which are a result of solar wind interaction with the Earth’s magnetosphere, geomagnetically induced currents (GICs) begin to flow in the long-distance high-voltage power grids on the Earth’s surface. It causes a number of negative phenomena that affect the normal operation of the entire electric power system. To investigate the nature of the phenomenon and its possible effects on transformers, a GIC monitoring system was created in 2011, the devices of which were installed at five substations of the Kola-Karelian power transit in northwestern Russia. Over 12 years of operating the system a large amount of data has been accumulated, which cannot be analyzed manually within a reasonable amount of time. To analyze the constantly growing volume of recorded data effectively, a method for automatic classification of GIC in autotransformer neutral was proposed. The method is based on a continuous wavelet transform of the neutral current data combined with a convolutional neural network (CNN) to classify the obtained scalogram images. As the result of comparing four CNNs with different architectures, a model that showed excellent GIC classification performance on the validation set (100.00% accuracy and loss of 0.0115) was chosen.

Ultrasound (US) is commonly used for the diagnosis of liver masses. Ensemble learning has been widely used for image classification, but its methods have not been fully optimized. This study was performed to investigate the usefulness of ensemble learning and compare a number of ensemble learning techniques using multiple convolutional neural network (CNN)-trained models for image classification of liver masses in US images. The US imaging data set was classified into four categories: benign liver tumor (BLT, 6320 images), liver cyst (LCY, 2320 images), metastatic liver cancer (MLC, 9720 images), and primary liver cancer (PLC, 7840 images). In this study, 250 test images were randomly selected for each class, for a total of 1000 images, and the remaining images were used for training. Sixteen different CNNs were used for to train and test the US images. All four types of ensemble learning—soft voting (SV), stacking (ST), weighted average voting (WAV), and weighted hard voting (WHV)—showed greater accuracy than the single CNN. All four types also showed significantly better deep learning performance than ResNeXt101 alone. For image classification of liver masses using US images, ensemble learning improved the performance of deep learning over a single CNN.

Melanoma is considered to be the most aggressive form of skin cancer. Due to the similar shape of malignant and benign cancerous lesions, doctors spend considerably more time when diagnosing these findings. At present, the evaluation of malignancy is performed primarily by invasive histological examination of the suspicious lesion. Developing an accurate classifier for early and efficient detection can minimize and monitor the harmful effects of skin cancer and increase patient survival rates. This paper proposes a multi-class classification task using the CoAtNet architecture, a hybrid model that combines the depthwise convolution matrix operation of traditional convolutional neural networks with the strengths of Transformer models and self-attention mechanics to achieve better generalization and capacity. The proposed multi-class classifier achieves an overall precision of 0.901, recall 0.895, and AP 0.923, indicating high performance compared to other state-of-the-art networks.

Diabetic retinopathy (DR) is the primary factor leading to vision impairment and blindness in diabetic people. Uncontrolled diabetes can damage the retinal blood vessels. Initial detection and prompt medical intervention are vital in preventing progressive vision impairment. Today's growing medical field poses more significant workload and diagnostic demands on medical professionals. In the proposed study, a convolutional neural network (CNN) is employed to detect the stages of DR. This research is crucial for studying DR because of its innovative methodology incorporating two different public datasets. This strategy enhances the model's capacity to generalize unseen DR images, as each dataset encompasses unique demographics and clinical circumstances. The network can learn and capture complicated hierarchical image features with asymmetric weights. Each image undergoes preprocessing using contrast-limited adaptive histogram equalization (CLAHE) and the discrete wavelet transform (DWT). The model is trained and validated using the combined datasets of Dataset for Diabetic Retinopathy (DDR) and the Asia Pacific Tele-Ophthalmology Society (APTOS). The CNN model is tuned with different learning rates and optimizers. The CNN model with Adam optimizer achieved 70% accuracy and an area under curve (AUC) score of 0.82. The recommended study results may reduce diabetes-related vision impairment by early DR severity identification.

Accurate and efficient medicinal plant image classification is of utmost importance as these plants produce a wide variety of bioactive compounds that offer therapeutic benefits. With a long history of medicinal plant usage, different parts of plants, such as flowers, leaves, and roots, have been recognized for their medicinal properties and are used for plant identification. However, leaf images are extensively used due to their convenient accessibility and are a major source of information. In recent years, transfer learning and fine-tuning, which use pre-trained deep convolutional networks to extract pertinent features, has emerged as an extremely effective approach for image identification problems. This study is leveraging the power of three component deep convolutional neural networks, namely VGG16, VGG19 and DenseNet201, to derive features from the input images of the medicinal plant dataset, containing leaf images of 30 classes. The models were compared and ensembled to make four hybrid models to enhance the predictive performance by utilizing the averaging and weighted averaging strategies. Quantitative experiments were carried out to evaluate the models on the Mendeley medicinal leaf dataset. The resultant ensemble of VGG19+DensNet201 with fine-tuning showcases enhanced capability in identifying medicinal plant images with an improvement of 7.43% and 5.8% compared with VGG19 and VGG16. Furthermore, VGG19+DensNet201 can outperform its standalone counterparts by achieving an accuracy of 99.12% on the test set. A thorough assessment with metrics such as accuracy, recall, precision, and f1-score firmly establishes the effectiveness of the ensemble strategy.

Deep learning has recently attracted much attention due to its excellent performance in processing audio, image, and video data. However, few studies are devoted to the field of automatic modulation classification (AMC). It is one of the most well-known research topics in communication signal recognition, which remains challenging for traditional methods due to the complex disturbance from other sources. This paper proposes a heterogeneous deep model fusion (HDMF) method to solve the problem in a unified framework. The contributions include: 1) The convolutional neural network (CNN) and long short-term memory (LSTM) are combined by two different ways without prior knowledge involved; 2) A large database, including eleven types of single-carrier modulation signals with various noises as well as a fading channel, is collected with various signal-to-noise ratios (SNRs) based on a real geographical environment; and 3) Experimental results demonstrate that HDMF is super capable of copping with the AMC problem, and achieves much better performance when compared with the independent network. The source code and the database will be publically available.

Human activity recognition (HAR) using wearable sensors is gaining popularity in the healthcare domain, enabling continuous monitoring and timely interventions. The conventional HAR systems are centralised and need data sharing with a server for processing and model training. However, this data sharing leads to various challenges, such as privacy concerns, increased communication and storage costs, inefficient computational algorithms, data scarcity due to unwillingness to share, and difficulty achieving high accuracy and personalised models simultaneously. To overcome these challenges, this paper introduces a privacy and energy-aware federated learning (FL) framework for model training without data sharing. The proposed scheme incorporates spiking neural networks (SNNs), which are particularly suited for resource-constrained devices due to their event-driven information processing. The key idea is to synergise the strengths of SNNs and long short-term memory (LSTM) to create a hybrid model for robust HAR. The proposed hybrid spiking LSTM (S-LSTM) model uses the LSTM as an input layer to capture temporal dependencies efficiently, followed by spiking layers that replace the typical activation using leaky integrate-and-fire neurons. We employ the combination of surrogate gradient learning and backpropagation through time (BPTT) to update the weights of SNN layers in a supervised manner. The performance of the proposed model is evaluated using two publicly available datasets for both indoor and outdoor scenarios. The results show that the proposed S-LSTM performs much better than spike convolutional neural networks (S-CNN) and simple CNN, achieving the accuracy of 97% and 87% for indoor and outdoor datasets, respectively. Personalisation, also known as fine-tuning the global model using local data, is crucial in HAR. It allows the model to adapt to individual behaviour and preferences, making predictions more accurate. The results also show that personalisation improves the performance of participants up to 9% on average. Additionally, analysis with 50% random client participation reduces communication costs by 50% with minimal impact on accuracy.

Tail-biting convolutional codes extend the classical zero-termination convolutional codes: Both encoding schemes force the equality of start and end states, but under the tail-biting each state is a valid termination. This paper proposes a machine-learning approach to improve the state-of-the-art decoding of tail-biting codes, focusing on the widely employed short length regime as in the LTE standard. This standard also includes a CRC code. First, we parameterize the circular Viterbi algorithm, a baseline decoder that exploits the circular nature of the underlying trellis. An ensemble combines multiple such weighted decoders, each decoder specializes in decoding words from a specific region of the channel words' distribution. A region corresponds to a subset of termination states; the ensemble covers the entire states space. A non-learnable gating satisfies two goals: it filters easily decoded words and mitigates the overhead of executing multiple weighted decoders. The CRC criterion is employed to choose only a subset of experts for decoding purpose. Our method achieves FER improvement of up to 0.75dB over the CVA in the waterfall region for multiple code lengths, adding negligible computational complexity compared to the circular Viterbi algorithm in high SNRs.

The intracranial pressure (ICP) signal, as monitored on patients in intensive care units, contains pulses of cardiac origin where P1 and P2 subpeaks can often be observed. When calculable, the ratio of their relative amplitudes is an indicator of the patient’s cerebral compliance. This characterization is particularly informative on the overall state of the cerebrospinal system. We developed a recurrent neural network-based framework for P2/P1 ratio computation that only takes a raw ICP signal as an input. Two tasks are performed, namely pulse classification and subpeak designation. Pulse classification was achieved with an area under the curve of 0.90 on a 4,344-pulse testing dataset, while the peak designation algorithm identified pulses with a P2/P1 ratio > 1 with a 97.3% accuracy. Although it still needs to be evaluated on a larger number of labeled recordings, our automated P2/P1 ratio calculation framework appears to be a promising tool for improving bedside monitoring devices.

Cervical cancer is a worldwide public health problem with a high rate of illness and mortality among women. In this study, we proposed a novel framework based on Faster RCNN-FPN ar-chitecture for the detection of abnormal cervical cells in cytology images from cancer screening test. We extended the Faster RCNN-FPN model by infusing deformable convolution layers into the feature pyramid network (FPN) to improve scalability. Furthermore, we introduced a global contextual aware module alongside the Region Proposal Network (RPN) to enhance the spatial correlation between the background and the foreground. Extensive experimentations with the proposed deformable and global context aware (DGCA) RCNN were carried out using the cer-vical image dataset of “Digital Human Body" Vision Challenge from the Alibaba Cloud TianChi Company. Performance evaluation based on the mean average precision (mAP) and receiver operating characteristic (ROC) curve has demonstrated considerable advantages of the proposed framework. Particularly, when combined with tagging of the negative image samples using tra-ditional computer-vision techniques, 6-9% increase in mAP has been achieved. The proposed DGCA-RCNN model has potential to become a clinically useful AI tool for automated detection of cervical cancer cells in whole slide images of Pap smear.

Efficient management of basal cell carcinoma (BCC) requires reliable assessments of both tumors and post-treatment scars. We aimed to estimate image similarity metrics that account for BCC's perceptual color and texture deviation from perilesional skin. 176 clinical photographs of BCC were assessed by six physicians using a visual deviation scale. Internal consistency and inter-rater agreement were estimated using Cronbach's α, weighted Gwet's AC2, and quadratic Cohen's Kappa. The mean visual scores were used to validate a range of similarity metrics employing different color spaces, distances, and image embeddings from pre-trained VGG16 neural network. The calculated similarities were transformed into discrete values using ordinal logistic regression models. The Bray-Curtis distance in the YIQ color model and rectified embeddings from the 'fc6' layer minimized the mean squared error and demonstrated strong performance in representing perceptual similarities. Box plot analysis and Wilcoxon rank-sum test were used to visualize and compare the levels of agreement, conducted on a random validation round between the two groups: ‘Human-System’ and ‘Human-Human’. The proposed metrics were comparable in terms of internal consistency and agreement with human raters. The findings suggest that the proposed metrics offer a robust, and cost-effective approach to monitoring BCC treatment outcomes in clinical settings.

Detailed flow distributions in vascular systems are the key to identifying hemodynamic risk factors for the development and progression of vascular diseases. Although computational fluid dynamics (CFD) has been widely used in bioengineering research on hemodynamics predictions, not only are high-fidelity CFD simulations time-consuming and computing-expensive, but also not friendly to clinical applications due to the difficulty of comprehensive numerical calculations. Machine learning (ML) algorithums to estimate the flow field in vascular systems based on the angiographic images of the blood flow using existed diagnostic tools are emerging as a new pathway to facilitate the mapping of hemodynamics. In present work, the dye injection in a water flow was simulated as an analogy of the contrast perfusion in blood flow using CFD. In the simulation, the light passes through the flow field and generates projective images, as an analogy of X-ray imaging. The simulations provide both the ground truth velocity field and the projective images of the flow with dye patterns. A rough velocity field was estimated using the optical flow method (OFM) based on projective images. ML algorithums are then trained using the ground truth CFD data and the OFM velocity estimation as the input. Finally, the interpretable (logistic regression) and deep (neural networks, convolutional neural networks, long short term memory) machine learning models are validated by using parallel in vitro experiments on the same flow setup. The validation results showed that the employed ML model significantly reduced the error rate from 53.5% to 2.5% in average for the v-velocity estimation.

The continuously developing project aims to build an informatics system enabling analysis of spatial and temporal parameters of movement activities occurring in the sport of speed climbing. The monitoring system (climbing information speed system – CISS) is to be used for conducting comprehensive scientific research in the field of speed climbing. The system enables the evaluation of the training process of climbers at various levels of competition. The study analysis was based on video. The video recording with a camera positioned at a short distance (10 m) from the wall. The marker was positioned closest to the centre of mass (gravity) BMC. Results: development of a system for data collection and analysis of the climbing run based on video recording (application of the Kanade-Lucas-Tomasi (KLT) algorithm). Our results showed that used devices can measure a wide range of specific internal and external variables during speed climbing. Some of the analyzed parameters were significantly correlated with speed climbing time. These results could be a theoretical basis for future research and for training program’s preparation.

Epilepsy is a chronic neurological disorder characterized by abnormal electrical activity in the brain, often leading to recurrent seizures. With approximately 50 million people worldwide affected by epilepsy, there is a pressing need for efficient and accurate methods to detect and diagnose seizures. Electroencephalogram (EEG) signals, which record the brain’s electrical activity, have emerged as a valuable tool in detecting epilepsy and other neurological disorders. Traditionally, the process of analyzing EEG signals for seizure detection has relied on manual inspection by experts, which is time-consuming, labor-intensive, and susceptible to human error. To address these limitations, researchers have turned to machine learning and deep learning techniques to automate the seizure detection process. In this work, we propose a novel method for epileptic seizure detection, leveraging the power of 1-D Convolutional layers in combination with Bidirectional Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU) and Average pooling Layer as a unit. The performance of the proposed model is verified on the Bonn dataset. Our proposed model offers significant advancements over existing approaches for the detection of epileptic seizures, particularly for 2, 3, 4, and 5-class classification tasks. To assess the robustness and generalizability of our proposed architecture, we employ 5-fold cross-validation. By dividing the dataset into five subsets and iteratively training and testing the model on different combinations of these subsets, we obtain robust performance measures, including accuracy, sensitivity, and specificity. We demonstrate the effectiveness of the proposed architecture in accurately detecting epileptic seizures from EEG signals by using EEG signals of varying lengths. The results indicate its potential as a reliable and efficient tool for automated seizure detection, paving the way for improved diagnosis and management of epilepsy.

Accurate detection of forest restoration sites using deep learning is crucial for effective ecosystem management, optimizing reforestation efforts, and ensuring environmental sustainability. While various deep learning-based systems have been developed for this purpose, the influence of different neural network architectures on model performance during the identification of forest restoration sites remains underexplored. This study aimed to address this gap by exploring the optimal methodology for extracting and classifying candidate sites for forest restoration based on Convolutional Neural Networks (CNNs), specialized for image recognition. Four categories—arable land, road and barren, quarry, and forest—were defined as candidate sites for forest restoration. A dataset comprising 17,043 samples was divided into training (11,929) and validation (5,114) sets at a ratio of 7:3 for model learning. Model accuracy was evaluated using pixel accuracy (PA) and mean intersection over union (mean IoU). The Resnet152-U-Net model demonstrated a pixel accuracy of 95.2% and a mean IoU of 61.3%, indicating excellent performance in extracting candidate sites for forest restoration. This approach offers spatial and temporal advantages over conventional field surveys or aerial image-based assessments, potentially serving as valuable data for selecting future forest restoration sites.

Cardiovascular disease is the leading cause of death worldwide. A key factor in assessing the risk of cardiovascular disease is left ventricular functional evaluation. Left ventricular (LV) systolic function is evaluated by measuring the left ventricular ejection fraction (LVEF) using echocardiography data. Therefore, quick and accurate left ventricle segmentation is important for estimating the LVEF. However, it is difficult to accurately segment the left ventricle due to changes in the shape and area of the left ventricle during cardiac cycles. In this study, we proposed a framework that considers changes in the shape and area of the left ventricle during the cardiac cycle by applying the convolutional long short-term memory (CLSTM) approach. In addition, we evaluated the left ventricular segmentation and multidimensional quantification of the proposed system in comparison to manual and automated segmentation methods. In addition, to assess the validity of CLSTM, the values of multi-dimensional quantification metrics were compared and analyzed using graphs and Bland–Altman plots on a frame-by-frame basis. We demonstrated that the CLSTM method effectively segments the left ventricle by considering the LV activity. In conclusion, we demonstrated that LV segmentation based on our framework may be utilized to accurately estimate LVEF values.

With the rapid expansion of intelligent resource-constrained devices and high-speed communication technologies, Internet of Things (IoT) has earned a wide recognition as the primary standard for low-power lossy networks (LLNs). Nevertheless, IoT infrastructures are vulnerable to cyber-attacks due to the constraints in computation, storage, and communication capacity of the endpoint devices. From one side, the majority of newly developed cyber-attacks are formed by slightly mutating formerly established cyber-attacks to produce a new attack tending to be treated as a normal traffic through the IoT network. From the other side, the influence of coupling the deep learning techniques with cybersecurity field has become a recent inclination of many security applications due to their impressive performance. In this paper, we provide a comprehensive development of a new intelligent and autonomous deep learning-based detection and classification system for cyber-attacks in IoT communication networks leveraging the power of convolutional neural networks, abbreviated as (IoT-IDCS-CNN). The proposed IoT-IDCS-CNN makes use of the high-performance computing employing the robust CUDA based Nvidia GPUs and the parallel processing employing the high-speed I9-Cores based Intel CPUs. In particular, the proposed system is composed of three subsystems: Feature Engineering subsystem, Feature Learning subsystem and Traffic classification subsystem. All subsystems are developed, verified, integrated, and validated in this research. To evaluate the developed system, we employed the NSL-KDD dataset which includes all the key attacks in the IoT computing. The simulation results demonstrated more than 99.3% and 98.2% of cyber-attacks’ classification accuracy for the binary-class classifier (normal vs anomaly) and the multi-class classifier (five categories) respectively. The proposed system was validated using k-fold cross validation method and was evaluated using the confusion matrix parameters (i.e., TN, TP, FN, FP) along with other classification performance metrics including precision, recall, F1-score, and false alarm rate. The test and evaluation results of the IoT-IDCS-CNN system outperformed many recent machine-learning based IDCS systems in the same area of study.

Contrapose the highly integrated, multiple types of faults and complex working conditions of aircraft Electro Hydrostatic Actuator (EHA), to effectively identify its typical faults, we propose a fault diagnosis method based on the fusion convolutional neural networks (FCNN). First, the aircraft EHA fault data is encoded by GADF to obtain the fault feature images. Then we build an FCNN model that integrates the 1DCNN and 2DCNN, where the original 1D fault data is the input of the 1DCNN model, and the feature images obtained by GADF transformation are used as the input of 2DCNN. Multiple convolution and pooling operations are performed on each of these inputs to extract the features, next these feature vectors are spliced in the convergence layer, and the fully connected layers and the Softmax layers are finally used to attain the classification of aircraft EHA faults. Furthermore, the multi-strategy hybrid particle swarm optimization (MSPSO) algorithm is applied to optimize the FCNN to obtain a better combination of FCNN hyperparameters; MSPSO incorporates various strategies, including an initialization strategy based on homogenization and randomization, and an adaptive inertia weighting strategy, etc. The experimental result indicates that the FCNN model optimized by MSPSO achieves an accuracy of 96.86% for identifying typical faults of the aircraft EHA, respectively higher than the 1DCNN and the 2DCNN about 16.5% and 5.7%. Additionally, the FCNN model improved by MSPSO has a higher accuracy rate when compared to PSO.

Semantic image segmentation has recently witnessed considerable progress by training deep convolutional neural networks (CNNs). The core issue of this technique is the limited capacity of CNNs to depict visual objects. Existing approaches tend to utilize approximate inference in a discrete domain or additional aides and do not have a global optimum guarantee. We propose the use of the multi-label manifold ranking (MR) method in solving the linear objective energy function in a continuous domain to delineate visual objects and solve these problems. We present a novel embedded single stream optimization method based on the manifold ranking (MR) model to avoid approximations without sacrificing expressive power. In addition, we propose a novel network, which we refer to as dual multi-scale manifold ranking (\textbf{DMSMR}) network, that combines the dilated, multi-scale strategies with the single stream MR optimization method in the deep learning architecture to further improve the performance. Experiments on high resolution images, including close-range and remote sensing datasets, demonstrate that the proposed approach can achieve competitive accuracy without additional aides in an end-to-end manner.

Reemergence of deep Neural Networks (CNNs) has lead to high-performance supervised learning algorithms for the Electro-Optical (EO) domain classification and detection problems. This success is possible because generating huge labeled datasets has become possible using modern crowdsourcing labeling platforms such as Amazon Mechanical Turk that recruit ordinary people to label data. Unlike the EO domain, labeling the Synthetic Aperture Radar (SAR) domain data can be a lot more challenging and for various reasons using crowdsourcing platforms is not feasible for labeling the SAR domain data. As a result, training deep networks using supervised learning is more challenging in the SAR domain. In the paper,we present a new framework to train a deep neural network for classifying Synthetic Aperture Radar (SAR) images by eliminating the need for huge labeled dataset. Our idea is based on transferring knowledge from a related EO domain problem, where labeled data is easy to obtain. We transfer knowledge from the EO domain through learning a shared invariant cross-domain embedding space that is also discriminative for classification. To this end, we train two deep encoders that are coupled through their last year to map data points from the EO and the SAR domains to the shared embedding space such that the distance between the distributions of the two domains is minimized in the latent embedding space. We use the Sliced Wasserstein Distance (SWD) to measure and minimize the distance between these two distributions and use a limited number of SAR label data points to match the distributions class-conditionally. As a result of this training procedure, a classifier trained from the embedding space to the label space using mostly the EO data would generalize well on the SAR domain. We provide theoretical analysis to demonstrate why our approach is effective and validate our algorithm on the problem of ship classification in the SAR domain by comparing against several other learning competing approaches.

Macula fovea detection is a crucial prerequisite towards screening and diagnosing macular diseases. Without early detection and proper treatment, any abnormality involving the macula may lead to blindness. However, with the ophthalmologist shortage and time-consuming artificial evaluation, neither accuracy nor effectiveness of the diagnose process could be guaranteed. In this project, we proposed a deep learning approach on ultra-widefield fundus (UWF) images for macula fovea detection. This study collected 2300 ultra-widefield fundus images from Shenzhen Aier Eye Hospital in China. Methods based on U-shape network (Unet) and Fully Convolutional Networks (FCN) are implemented on 1800 (before amplifying process) training fundus images, 400 (before amplifying process) validation images and 100 test images. Three professional ophthalmologists were invited to mark the fovea. A method from the anatomy perspective is investigated. This approach is derived from the spatial relationship between macula fovea and optic disc center in UWF. A set of parameters of this method is set based on the experience of ophthalmologists and verified to be effective. Results are measured by calculating the Euclidean distance between proposed approaches and the accurate grounded standard, which is detected by Ultra-widefield swept-source optical coherence tomograph (UWF-OCT) approach. Through a comparation of proposed methods, we conclude that, deep learning approach of Unet outperformed other methods on macula fovea detection tasks, by which outcomes obtained are comparable to grounded standard method.

This paper presents the Advanced Syntactic-Graph Convolutional Model (ASGCM), a pioneering approach in Aspect-Based Sentiment Analysis (ABSA) that integrates syntactic dependency features within a graph convolution framework. ASGCM stands out for its novel use of dependency edge encoding and tag-based graph convolutions, providing a fine-grained analysis of sentiments associated with specific aspects in text. This model meticulously captures the intricacies of syntactic structures, thereby offering enhanced precision in sentiment analysis. Notably, ASGCM incorporates a dual-layer graph convolution system: one layer processes syntactic dependencies (edges), while the other interprets semantic roles (tags), ensuring a comprehensive understanding of both structural and contextual elements in text. We rigorously tested ASGCM on multiple datasets, including both English and Chinese languages, and our findings reveal a significant improvement in sentiment classification accuracy compared to existing models. The versatility of ASGCM makes it a robust tool for diverse linguistic environments, setting a new standard for ABSA methodologies.

Convolutional Neural Networks (CNNs) are largely hand-crafted, which leads to inefficiency in the constructed network. Various other algorithms have been proposed to address this issue, but the inefficiencies resulting from human intervention have not been addressed. Our proposed EvolveNet algorithm is a task-agnostic evolutionary search algorithm that can find optimal depth and width scales automatically in an efficient way. The optimal configurations are not found using grid search, instead evolved from an existing network. This eliminates inefficiencies that emanate from hand-crafting, thus reducing the drop in accuracy. The proposed algorithm is a framework to search through a large search space of subnetworks until a suitable configuration is found. Extensive experiments on the ImageNet dataset demonstrate the superiority of the proposed method by outperforming the state-of-the-art methods.

Model understanding is critical in many domains, particularly those involved in high-stakes decisions, i.e., medicine, criminal justice, and autonomous driving. Explainable AI (XAI) methods are essential for working with black-box models such as Convolutional Neural Networks. This paper evaluates the traffic sign classifier of Deep Neural Network (DNN) from the Programmable Systems for Intelligence in Automobiles (PRYSTINE) project for explainability. The results of explanations were further used for the CNN PRYSTINE classifier vague kernels` compression. After all, the precision of the classifier was evaluated in different pruning scenarios. The proposed classifier performance methodology was realised by creating the original traffic sign and traffic light classification and explanation code. First, the status of the kernels of the network was evaluated for explainability. For this task, the post-hoc, local, meaningful perturbation-based forward explainable method was integrated into the model to evaluate each kernel status of the network. This method enabled distinguishing high and low-impact kernels in the CNN. Second, the vague kernels of the classifier of the last layer before the fully connected layer were excluded by withdrawing them from the network. Third, the network's precision was evaluated in different kernel compression levels. It is shown that by using the XAI approach for network kernel compression, the pruning of 5% of kernels leads only to a 1% loss in traffic sign and traffic light classification precision. The proposed methodology is crucial where execution time and processing capacity prevail.

The paper concludes the first research on mouth-based Emotion Recognition (ER), adopting a Transfer Learning (TL) approach. Transfer Learning results paramount for mouth-based emotion ER, because a few data sets are available, and most of them include emotional expressions simulated by actors, instead of adopting a real-world categorization. Using TL we can use fewer training data than training a whole network from scratch, thus more efficiently fine-tuning the network with emotional data and improving the convolutional neural network accuracy in the desired domain. The proposed approach aims at improving the Emotion Recognition dynamically, taking into account not only new scenarios but also modified situations with respect to the initial training phase, because the image of the mouth can be available even when the whole face is visible only in an unfavourable perspective. Typical applications include automated supervision of bedridden critical patients in an healthcare management environment, or portable applications supporting disabled users having difficulties in seeing or recognizing facial emotions. This work takes advantage from previous preliminary works on mouth-based emotion recognition using CNN deep-learning, and has the further benefit of testing and comparing a set of networks on large data sets for face-based emotion recognition well known in literature. The final result is not directly comparable with works on full-face ER, but valorizes the significance of mouth in emotion recognition, obtaining consistent performances on the visual emotion recognition domain.

Multilabel learning goes beyond standard supervised learning models by associating a sample with more than one class label. Among the many techniques developed in the last decade to handle multilabel learning best approaches are those harnessing the power of ensembles and deep learners. This work proposes merging both methods by combining a set of gated recurrent units, temporal convolutional neural networks, and long short-term memory networks trained with variants of the Adam optimization approach. We examine many Adam variants, each fundamentally based on the difference between present and past gradients, with step size adjusted for each parameter. We also combine Incorporating Multiple Clustering Centers and a bootstrap-aggregated decision trees ensemble, which is shown to further boost classification performance. In addition, we provide an ablation study for assessing the performance improvement that each module of our ensemble produces. Multiple experiments on a large set of datasets representing a wide variety of multilabel tasks demonstrate the robustness of our best ensemble, which is shown to outperform the state-of-the-art. The MATLAB code for generating the best ensembles in the experimental section will be made available at https://github.com/LorisNanni.

Generative models for images, audio, text and other low-dimension data have achieved great success in recent years. Generating artificial human movements can also be useful for many applications, including improvement of data augmentation methods for human gesture recognition. The object of this research is to develop a generative model for skeletal human movement, allowing to control the action type of generated motion while keeping the authenticity of the result and the natural style variability of gesture execution. We propose to use a conditional Deep Convolutional Generative Adversarial Network (DC-GAN) applied to pseudo-images representing skeletal pose sequences using Tree Structure Skeleton Image format. We evaluate our approach on the 3D-skeleton data provided in the large NTU RGB+D public dataset. Our generative model can output qualitatively correct skeletal human movements for any of its 60 action classes. We also quantitatively evaluate the performance of our model by computing Frechet Inception Distances, which shows strong correlation to human judgement. Up to our knowledge, our work is the first successful class-conditioned generative model for human skeletal motions based on pseudo-image representation of skeletal pose sequences.

This paper presents a non-random weight initialization method in convolutional layers of neural networks examined with the Fast Gradient Sign Method (FSGM) attack. This paper's focus is convolutional layers, and are the layers that have been responsible for better than human performance in image categorization. The proposed method induces earlier learning through the use of striped forms, and as such has less unlearning of the existing random number speckled methods, consistent with the intuitions of Hubel and Wiesel. The proposed method provides a higher performing accuracy in a single epoch, with improvements of between 3-5% in a well known benchmark model, of which the first epoch is the most relevant as it is the epoch after initialization. The proposed method is also repeatable and deterministic, as a desirable quality for safety critical applications in image classification within sensors. That method is robust to Glorot/Xavier and He initialization limits as well. The proposed non-random initialization was examined under adversarial perturbation attack through the FGSM approach with transferred learning, as a technique to measure the affect in transferred learning with controlled distortions, and finds that the proposed method is less compromised to the original validation dataset, with higher distorted datasets.

Melanoma is widely recognized as one of the most lethal forms of skin cancer, with its incidence showing an upward trend in recent years. Nonetheless, the timely detection of this malignancy substantially enhances the likelihood of patients’ long-term survival. Several computer-based methods have recently been proposed in the pursuit of diagnosing skin lesions at their early stages. Despite achieving some level of success, there still remains a margin of error that the machine learning community considers to be an unresolved research challenge. This study presents a novel framework for the classification of skin lesions. The framework incorporates deep features to generate a highly discriminant feature vector, while also maintaining the integrity of the original feature space. Recent deep models including Darknet53, DenseNet201, InceptionV3, and InceptionResNetV2 are employed in our study for the purpose of feature extraction. Additionally, transfer learning is leveraged to enhance the performance of our approach. In the subsequent phase, the extracted feature information from the chosen pre-existing models is combined, with the aim of preserving maximum information, prior to undergoing the process of feature selection using a novel entropy-controlled grey wolf optimization (ECGWO) algorithm. The integration of fusion and selection techniques is employed to initially incorporate the feature vector with a high level of information and subsequently eliminate redundant and irrelevant feature information. The efficacy of our design is substantiated through the evaluation on three benchmark dermoscopic datasets, namely PH2, ISIC-MSK, and ISIC-UDA. In order to validate the proposed methodology, a comprehensive evaluation is conducted, including a rigorous comparison with established techniques in the field.

Augmented reality (AR) provides an opportunity for mobile fitness applications to show users real-time feedback on their current fitness activity. For such applications, it is essential to accurately track the user’s current fitness activity using available mobile sensors, such as inertial measurement units (IMUs). Convolutional neural networks (CNNs) have been shown to produce strong results in different time series classification tasks, including the recognition of activities of daily living. However, fitness activities can present unique challenges to the human activity recognition task (HAR), including greater similarity between individual activities and fewer available data for model training. In this paper, we evaluate the applicability of CNNs to the fitness activity recognition task (FAR) using IMU data and determine the impact of input data size and sensor count on performance. For this purpose, we adapted three existing CNN architectures to the FAR task and designed a fourth CNN variant, which we call the scaling fully convolutional network (Scaling-FCN). We designed a preprocessing pipeline and recorded a running exercise data set with 20 participants, in which we evaluated the respective recognition performances of the four networks, comparing them with three traditional machine learning methods commonly used in HAR. On our data set, all CNN architectures significantly outperformed traditional machine learning methods, reaching up to 97.14±1.36% classification accuracy on the test set for ResNet. Whereas a reduced input data size invariably led to a performance loss, removing specific sensors improved the performance of all CNN architectures, with our Scaling-FCN reaching the highest accuracy of 99.86±0.11%. Our results suggest that CNNs are generally well suited for fitness activity recognition, and noticeable performance improvements can be achieved if sensors are dropped selectively.

Craters are the most prominent geomorphological features on the surface of celestial bodies, which is playing a crucial role in studying the formation and evolution of celestial bodies, as well as in landing and planning for surface exploration. Currently, the main automatic crater detection models and datasets focus on the detection of large and medium craters. In this paper, we created 23 small lunar crater datasets for model training based on the Chang’E-2 (CE-2) DOM, DEM, Slope, and integrated data with 7 kinds of visualization stretching methods. And then, we proposed the YOLO-Crater model for Lunar and Martian small crater detection by replacing EioU and VariFocal loss to solve the crater samples imbalance problem and introducing a CBAM attention mechanism to mitigate interference from the complex extraterrestrial environment. The results show that the accuracy (P = 87.86%, R = 66.04%, and F1 = 75.41%) of the Lunar YOLO-Crater model based on the DOM-MMS (Maximum-Minimum Stretching) dataset is the highest and better than that of YOLOX model. And the Martian YOLO-Crater, trained by the Martian dataset from the 2022 GeoAI Martian Challenge, achieves good performance with P = 88.37%, R = 69.25%, and F1 = 77.65%. It indicates that the YOLO-Crater model has a strong transferability and generalization capability, which can be applied to detect small craters on the Moon and other celestial bodies.

Object classification is a crucial task in deep learning, which involves the identification and categorization of objects in images or videos. Although humans can easily recognize common objects, such as cars, animals, or plants, performing this task on a large scale can be time-consuming and error-prone. Therefore, automating this process using neural networks can save time and effort while achieving higher accuracy. Our study focuses on the classification step of human chromosome karyotyping, an important medical procedure that helps diagnose genetic disorders. Traditionally, this task is performed manually by expert cytologists, which is a time-consuming process that requires specialized medical skills. Therefore, automating it through deep learning can be immensely useful. To accomplish this, we implemented and adapted existing preprocessing and data augmentation techniques to prepare the chromosome images for classification. We used ResNet-50 convolutional neural networks and an ensemble approach to classify the chromosomes, obtaining state-of-the-art performance in the tested dataset.

Optical neural networks (ONNs), or optical neuromorphic hardware accelerators, have the potential to dramatically enhance the computing power and energy efficiency of mainstream electronic processors, due to their ultra-large bandwidths of up to 10’s of terahertz together with their analog architecture that avoids the need for reading and writing data back-and-forth. Different multiplexing techniques have been demonstrated to demonstrate ONNs, amongst which wavelength-division multiplexing (WDM) techniques make sufficient use of the unique advantages of optics in terms of broad bandwidths. Here, we review recent advances in WDM-based ONNs, focusing on methods that use integrated microcombs to implement ONNs. We present results for human image processing using an optical convolution accelerator operating at 11 Tera operations per second. The open challenges and limitations of ONNs that need to be addressed for future applications are also discussed.