The precise determination of the State-of-Health (SOH) of lithium-ion batteries is critical in the domain of battery management systems. The proposed model in this research paper emulates any deep learning or machine learning model by utilizing a Look Up Table (LUT) memory to store all activation inputs and their corresponding outputs. The operation that follows the completion of training is referred to as the LUT memory preparation procedure. The lookup operation performed on this method simply substitutes for the inference process. This is achieved by discretizing the input data and features before binarizing them. The term for the aforementioned operation is the LUT inference method. The procedure was evaluated in this study using two distinct neural network architectures: a bidirectional long short-term memory (LSTM) architecture and a standard fully connected neural network (FCNN). It is anticipated that considerably greater efficiency and velocity will be achieved during the inference procedure when the pre-trained deep neural network architecture is inferred directly. The principal aim of this research is to construct a lookup table that effectively establishes correlations between the SOH of lithium-ion batteries and ensures a degree of imprecision that is tolerable. According to the results obtained from the NASA PCoE lithium-ion battery dataset, the proposed methodology exhibits performance that is largely comparable to that of the initial machine learning models. Utilising the error assessment metrics RMSE, MAE, and (MAPE), the accuracy of SOH prediction has been quantitatively evaluated. The indicators mentioned above demonstrate a significant degree of accuracy when predicting SOH.

Following the occurrence of a typhoon, quick damage assessment related to residents can facilitate quick dispatch of house repair and disaster insurance works. Employing a deep learning method, this study used aerial photos of the Chiba prefecture obtained following the Typhoon Faxai in 2019 to automatically detect and evaluate the roof damage. This study comprised three parts: training deep learning model, detecting the roof damage using trained model, and classifying the level of roof damage. The detection object comprised roof outline, blue tarps, and roof completely destroyed. The roofs were divided into three categories: roof without damage, roof with blue tarps and roof completely destroyed. the F value obtained using the proposed method was higher than those obtained using other methods. In addition, it can be further divided into 5 levels from level 0 to 4. Finally, the spatial distribution of the roof damage was analyzed using ArcGIS tools. The proposed method is expected to provide certain reference for the real-time detection of the roof damage after the occurrence of a typhoon.

Machine learning (ML) methods has recently contributed very well in the advancement of the prediction models used for energy consumption. Such models highly improve the accuracy, robustness, and precision and the generalization ability of the conventional time series forecasting tools. This article reviews the state of the art of machine learning models used in the general application of energy consumption. Through a novel search and taxonomy the most relevant literature in the field are classified according to the ML modeling technique, energy type, perdition type, and the application area. A comprehensive review of the literature identifies the major ML methods, their application and a discussion on the evaluation of their effectiveness in energy consumption prediction. This paper further makes a conclusion on the trend and the effectiveness of the ML models. As the result, this research reports an outstanding rise in the accuracy and an ever increasing performance of the prediction technologies using the novel hybrid and ensemble prediction models.

In this perspective article, we ponder topographical enhancements of artificial neural networks. In our recent paper in JMLR, we proved a quasi-equivalence between the network width and depth and also discussed the power of intra-links, which can be viewed as network height. In 1982, Hopfield published a network to model human associative memory, which contains many loops for dynamic evolution toward fixed points. Based on noising-denoising loops, diffusion models are recently developed to enable Bayesian modeling and inference with big data. Furthermore, we envision development of multi-AI-agent systems through “netware” engineering as a quantum leap of software engineering for emergent behaviors and autonomous AI at individual and population levels. We believe that the novel use of links and loops in space and time via multi-scale coupling would catalyze the next-generation neural networks.

Raman spectroscopy (RS) turns out to be a promising tool for cancer di- agnosis. In particular, in the last years several studies have demonstrated how the diagnostic performances of Raman Spectroscopy can be significantly improved by employing Machine Learning (ML) algorithms for the interpre- tation of Raman-based data. In line with these findings, in this paper, we demonstrated how RS coupled with ML allowed to accurately distinguish the three grades of Chondrosarcoma and to distinguish Chondrosarcoma and a benign counterpart called Enchondroma. We obtained such results by ana- lyzing a dataset composed of a relatively small number of Raman spectra, collected in a previous study. Such spectra were acquired from micromet- ric tissue sections with a Confocal Raman Microscope. In particular, we tested the classification performances of a Support Vector Machine and a Random Forest Classifier, as representative of Machine Learning, and two versions of the Multi-Layer Perceptron, as representative of Deep Learning (DL). These models showed excellent classification performances, especially those belonging to DL, with accuracy reaching 99.7%. This outcome makes the aforementioned models a promising route for future improvements of di- agnostic devices focused on detecting bone cancerous tissues. In addition, we highlighted how the ML models studied resulted in slightly worse classifi- cation performances in comparison to DL. Alongside the diagnostic purpose, the aforementioned approach allowed to identify characteristic molecules, i. e. amino acids, nucleic acids and bioapatites, relevant for the obtainment of the final diagnostic response, through the analysis of the so-called Per- mutation Feature Importance. Permutation Feature Importance could hence represent a promising parameter for the understanding of the biochemical processes on the basis of the tumor progression. In turn, the spectral bands highlighted by Permutation Feature Importance could represent precious in- dicators in the attempt to restrict the spectral acquisition to specific Raman bands. This last objective could help to reduce the amount of experimental data needed to obtain an accurate final grading outcome, with consequent reduction of the computational cost.

Given the landscape of intricate and constantly evolving cyber threats, organizations demand refined strategies to deploy a Security Information and Event Management to support the management of a Cyber Security Operations Center. The dynamic nature of cyber threats complicates the efficient allocation of the location of network intrusion detection sensors, a critical component of a robust cybersecurity framework. Our research introduces an approach that integrates the precision of biomimetic optimization algorithms with the adaptability of Deep Q–Learning. By employing different biomimetic algorithms enhanced with deep learning, we aim to refine the deployment of sensors in network infrastructures, balancing the network security imperative against deployment costs. The results of computational tests demonstrate that the improved iterations leveraging Deep Q–Learning have outperformed their native counterparts. These findings underscore the importance of reinforcement learning, specifically through Deep Q–Learning, as a powerful tool to enhance the effectiveness of metaheuristics in addressing optimization challenges.

While machine learning (ML) has been quite successful in the field of structural health monitoring (SHM), its practical implementation has been limited. This is because ML model training requires data containing a variety of distinct damaged instances captured from a real structure and the experimental generation of such data is challenging. One way to tackle this issue is by generating training data through numerical simulations. However, simulated data cannot capture the bias and variance of experimental uncertainty. To overcome this problem, this work proposes a deep learning-based domain transformation method for transforming simulated data to the experimental domain. Use of this technique has been demonstrated for debonding location and size prediction of stiffened panels using a vibration-based method. The results are satisfactory for both, debonding location and size prediction. This domain transformation method can be used in any field where experimental data for training machine learning models is scarce.

Deep learning techniques are increasingly being recognized as effective image classifiers. Aside from their successful performance in past studies, the accuracies have varied in complex environments in comparison with the popularly applied machine learning classifiers. This study seeks to explore the feasibility for using a U-Net deep learning architecture to classify bi-temporal high resolution county scale aerial images to determine the spatial extent and changes of land cover classes that directly or indirectly impact tidal marsh. The image set used in the analysis is a collection of a 1-m resolution collection of National Agriculture Imagery Program (NAIP) tiles from 2009 and 2019 covering Beaufort County, South Carolina. The U-net CNN classification results were compared with two machine learning classifiers, the Random Trees (RT) and the Support Vector Machine (SVM). The results revealed a significant accuracy advantage in using the U-Net classifier (92.4%) as opposed to the SVM (81.6%) and RT (75.7%) classifiers for overall accuracy. From the perspective of a GIS analyst or coastal manager, the U-Net classifier is now an easily accessible nad powerful tool for mapping large areas. Change detection analysis indicated little areal change on marsh extent, though increased land development throughout the county has the potential to negatively impact the health of the marshes. Future work should explore applying the constructed U-Net classifier to coastal environments in large geographic areas, while also implementing other data sources (e.g., LIDAR, multispectral data) to enhance classification accuracy.

In the era of digital advancements, the escalation of credit card fraud necessitates the development of robust and efficient fraud detection systems. This paper delves into the application of machine learning models, specifically focusing on ensemble methods, to enhance credit card fraud detection. Through an extensive review of existing literature, we identified limitations in current fraud detection technologies, including issues like data imbalance, concept drift, false positives/negatives, limited generalisability, and challenges in real-time processing. To address some of these shortcomings, we propose a novel ensemble model that integrates a Support Vector Machine (SVM), K-Nearest Neighbor (KNN), Random Forest (RF), Bagging, and Boosting classifiers. This ensemble model tackles the dataset imbalance problem associated with most credit card datasets by implementing under-sampling and the Synthetic Over-sampling Technique (SMOTE) on some machine learning algorithms. The evaluation of the model utilises a dataset comprising transaction records from European credit card holders, providing a realistic scenario for assessment. The methodology of the proposed model encompasses data pre-processing, feature engineering, model selection, and evaluation, with Google Colab computational capabilities facilitating efficient model training and testing. Comparative analysis between the proposed ensemble model, traditional machine learning methods, and individual classifiers reveals the superior performance of the ensemble in mitigating challenges associated with credit card fraud detection. Across accuracy, precision, recall, and F1-score metrics, the ensemble outperforms existing models. This paper underscores the efficacy of ensemble methods as a valuable tool in the battle against fraudulent transactions. The findings presented lay the groundwork for future advancements in the development of more resilient and adaptive fraud detection systems, crucial as credit card fraud techniques continue to evolve.

Clients are increasingly looking for fast and effective means to quickly and frequently survey and communicate the condition of their buildings so that essential repairs and maintenance work can be done in a proactive and timely manner before it becomes too dangerous and expensive. Traditional methods for this type of work commonly comprise of engaging building surveyors to undertake a condition assessment which involves a lengthy site inspection to produce a systematic recording of the physical condition of the building elements, including cost estimates of immediate and projected long-term costs of renewal, repair and maintenance of the building. Current asset condition assessment procedures are extensively time consuming, laborious, and expensive and pose health and safety threats to surveyors, particularly at height and roof levels which are difficult to access. We propose a method for automated detection and localisation of key building defects from images using deep learning and convolution neural networks. The proposed model is based on a pre-trained VGG-16 classifier with Class Activation Mapping (CAM) for object localisation. The model has proven to be robust and able to accurately detect and localise mould growth, stains, and paint deterioration defects arising from dampness in buildings. The approach is being developed with potentials to scale-up to support automated detection of defects and deterioration of buildings in real-time using mobile devices and drones.

In the extensive monitoring of maritime traffic, maritime management frequently encounters incomplete automatic identification system (AIS) data, particularly in critical static fields like vessel types. This shortfall presents substantial challenges in safety management, necessitating robust methods for data completion. Distinct from road traffic, where road width imposes constraints, maritime vessels of identical classifications often choose significantly divergent routes, deviating by several nautical miles. Sole dependence on trajectory clustering is insufficient. Addressing this issue, we segments the maritime area into spatiotemporal grids, transforming vessel paths into sequences of grid encodings. Utilizing natural language processing techniques, we integrate the Word2vec word embedding approach with our novel biLSTM self-attention chunk-max pooling net (biSAMNet) model, enhancing the classification of vessel trajectories. This method is crucial for determining static details like vessel types. Employing the Taiwan Strait as a case study and benchmarking against CNN, RNN, and methods based on the attention mechanism, our findings underscore our model’s superiority. The biSAMNet achieves an impressive vessel classification F1 score of 0.94, using only 5-dimensional word embeddings, highlighting the effectiveness of Word2vec pre-trained embedding layers. This research introduces a novel paradigm for processing vessel trajectory data, greatly enhancing the accuracy of filling in incomplete vessel type details.

For the generation of representative volume elements a statistical description of the relevant parameters is necessary. These parameters usually describe the geometric structure of a single grain. Commonly, parameters like area, aspect ratio and slope of the grain relative to the rolling direction are applied. However, usually simple distribution functions like log normal or gamma distribution are used. Yet, these do not take the interdependencies between the microstructural parameters into account. To fully describe any metallic microstructure though, these interdependencies between the singular parameters need to be accounted for. To accomplish this representation, a machine learning approach was applied in this study. By implementing a Wasserstein generative adversarial network, the distribution, as well as the interdependencies could accurately be described. A validation scheme was applied to verify the excellent match between microstructure input data and synthetically generated output data.

G protein-coupled receptors (GPCRs) play a key role in many cellular signaling mechanisms, and must select among multiple coupling possibilities in a ligand-specific manner in order to carry out a myriad of functions in diverse cellular contexts. Much has been learned about the molecular mechanisms of ligand-GPCR complexes from Molecular Dynamics (MD) simulations. However, to explore ligand-specific differences in the response of a GPCR to diverse ligands, as is required to understand ligand bias and functional selectivity, necessitates creating very large amounts of data from the needed large-scale simulations. This becomes a Big Data problem for the high dimensionality analysis of the accumulated trajectories. Here we describe a new machine learning (ML) approach to the problem that is based on transforming the analysis of GPCR function-related, ligand-specific differences encoded in the MD simulation trajectories into a representation recognizable by state-of-the-art deep learning object recognition technology. We illustrate this method by applying it to recognize the pharmacological classification of ligands bound to the 5-HT2A and D2 subtypes of class A GPCRs from the serotonin and dopamine families. The ML-based approach is shown to perform the classification task with high accuracy, and we identify the molecular determinants of the classifications in the context of GPCR structure and function. This study builds a framework for the efficient computational analysis of MD Big Data collected for the purpose of understanding ligand-specific GPCR activity.

In our increasingly urbanized world, precise and up-to-date maps of human settlements are essential for sustainable urban development policies. The availability of open-access Sentinel-2 data from the Copernicus program presents an opportunity to create a comprehensive global map of human settlements, offering a detailed view of built areas on a large scale. This study estimates large-scale built-up fractions using encoder-decoder deep learning architectures like U-net, Res-U-net, and Attention-U-net in the large and complex urban area of Delhi, India. Openly available datasets like Open Street Map (OSM) and Microsoft building footprint datasets are used to derive built-up fractions at 10×10m resolution cells for over 34,000 km2. Our results show that Attention-U-net with the Huber loss function performs the best in different built-up densities (i.e., urban, semi-urban or rural) with an R2 score of 0.631, while Res-U-net and U-net obtained an R2 score of 0.623 and 0.612, respectively. The investigated networks significantly improve the accuracy over the latest Global Human Settlement Layer product (GHSL-S2), which uses a deep-CNN and reaches an R2 of 0.387 in our case study area. The result of this study yields a valuable spatial layer for examining the spatial distribution of human settlements across the entire spectrum from rural to urban areas.

With recent focus on deep neural network architectures for development of algorithms for computer-aided diagnosis (CAD), we provide a review of studies within the last 3 years (2015-2017) reported in selected top journals and conferences. 29 studies that met our inclusion criteria were reviewed to identify trends in this field and to inform future development. Studies have focused mostly on cancer-related diseases within internal medicine while diseases within gender-/age-focused fields like gynaecology/pediatrics have not received much focus. All reviewed studies employed image datasets, mostly sourced from publicly available databases (55.2%) and few based on data from human subjects (31%) and non-medical datasets (13.8%), while CNN architecture was employed in most (70%) of the studies. Confirmation of the effect of data manipulation on quality of output and adoption of multi-class rather than binary classification also require more focus. Future studies should leverage collaborations with medical experts to aid future with actual clinical testing with reporting based on some generally applicable index to enable comparison. Our next steps on plans for CAD development for osteoarthritis (OA), with plans to consider multi-class classification and comparison across deep learning approaches and unsupervised architectures were also highlighted.

Background: In today’s competitive market, predicting customer churn with high accuracy is crucial for enterprises to maintain growth and profitability. Traditional predictive models often lack in accuracy due to the complexity of customer behavior.Objective: This research aims to improve the accuracy of predicting customer churn by utilizing the Particle Swarm Optimization (PSO) algorithm for optimizing the hyperparameters of a composite deep learning model. The performance of this enhanced model is evaluated against traditional models such as LSRM_GRU, LSTM, GRU, and CNN_LSTM to demonstrate the effectiveness of PSO in hyperparameter tuning.Methods: A composite deep learning approach was employed, integrating various neural network architectures to leverage their strengths in modeling complex customer interactions. The PSO algorithm was used to optimize the model’s hyperparameters. Customer transaction and interaction data from different business operations served as the dataset for testing and analyzing the model’s performance. Evaluation metrics including accuracy, precision, recall, F1 score, and ROC AUC were utilized for a detailed comparison with established models.Findings: The PSO-enhanced composite deep learning model showed superior performance across all metrics, significantly outperforming the LSRM_GRU, LSTM, GRU, and CNN_LSTM models. Notably, improvements in ROC AUC and F1 score highlight the robustness and balanced precision-recall trade-off of the proposed model, demonstrating its effectiveness in identifying potential churners.Conclusion: The integration of PSO for hyperparameter optimization in composite deep learning models for customer churn prediction has proven to significantly enhance predictive accuracy and performance metrics over conventional models. This underscores the potential of evolutionary algorithms in improving deep learning applications. Future research should explore the scalability of this approach across various sectors and further refine predictive accuracy with evolving customer data.

Semantic image segmentation has recently witnessed considerable progress by training deep convolutional neural networks (CNNs). The core issue of this technique is the limited capacity of CNNs to depict visual objects. Existing approaches tend to utilize approximate inference in a discrete domain or additional aides and do not have a global optimum guarantee. We propose the use of the multi-label manifold ranking (MR) method in solving the linear objective energy function in a continuous domain to delineate visual objects and solve these problems. We present a novel embedded single stream optimization method based on the manifold ranking (MR) model to avoid approximations without sacrificing expressive power. In addition, we propose a novel network, which we refer to as dual multi-scale manifold ranking (\textbf{DMSMR}) network, that combines the dilated, multi-scale strategies with the single stream MR optimization method in the deep learning architecture to further improve the performance. Experiments on high resolution images, including close-range and remote sensing datasets, demonstrate that the proposed approach can achieve competitive accuracy without additional aides in an end-to-end manner.

Financial markets require a great deal of decision making from the investors and market makers. One metric that can help ease the process of decision making is investment risk which can be measured in two parts; systematic risk and idiosyncratic risk. Clear understanding of the volatilities in each risk component can be a powerful signal in recognizing the right assets to maximize the investment returns. In this paper, we focus on the idiosyncratic volatility values and pre-calculate the idiosyncratic volatility values for 31,198 members of NYSE, Amex and Nasdaq markets for the trades occurring between January 1963 and December 2019. Utilizing a subset of dataset, limited to Nasdaq100 index, we consider the application of machine learning techniques in predicting the idiosyncratic volatility values using the raw trade data to explore a data extension option for the future market trade records that have not yet occurred. We offer a deep learning based regression model and compare it with traditional tree-based methods on a small subset of our per-calculated idiosyncratic volatility dataset. Our analytical results show that the performance of the deep learning techniques is much more robust in comparison to that of the traditional tree-based baselines.

Deep learning (DL), a branch of machine learning (ML) and artificial intelligence (AI) is nowadays considered as a core technology of today's Fourth Industrial Revolution (4IR or Industry 4.0). Due to its learning capabilities from data, DL technology originated from artificial neural network (ANN), has become a hot topic in the context of computing, and is widely applied in various application areas like healthcare, visual recognition, cybersecurity, and many more. However, building an appropriate DL model is a challenging task, due to the dynamic nature and variations in real-world problems and data. Moreover, the lack of core understanding turns DL methods into black-box machines that hamper development at the standard level. This article presents a structured and comprehensive view on DL techniques including a taxonomy considering various types of real-world tasks like supervised or unsupervised. In our taxonomy, we take into account deep networks for supervised or discriminative learning, unsupervised or generative learning as well as hybrid learning and relevant others. We also summarize real-world application areas where deep learning techniques can be used. Finally, we point out ten potential aspects for future generation DL modeling with research directions. Overall, this article aims to draw a big picture on DL modeling that can be used as a reference guide for both academia and industry professionals.

Deep learning (DL), which is originated from an artificial neural network (ANN), is one of the major technologies of today's smart cybersecurity systems or policies to function in an intelligent manner. Popular deep learning techniques, such as Multi-layer Perceptron (MLP), Convolutional Neural Network (CNN or ConvNet), Recurrent Neural Network (RNN) or Long Short-Term Memory (LSTM), Self-organizing Map (SOM), Auto-Encoder (AE), Restricted Boltzmann Machine (RBM), Deep Belief Networks (DBN), Generative Adversarial Network (GAN), Deep Transfer Learning (DTL or Deep TL), Deep Reinforcement Learning (DRL or Deep RL), or their ensembles and hybrid approaches can be used to intelligently tackle the diverse cybersecurity issues. In this paper, we aim to present a comprehensive overview from the perspective of these neural networks and deep learning techniques according to today's diverse needs. We also discuss the applicability of these techniques in various cybersecurity tasks such as intrusion detection, identification of malware or botnets, phishing, predicting cyber-attacks, e.g. denial of service (DoS), fraud detection or cyber-anomalies, etc. Finally, we highlight several research issues and future directions within the scope of our study in the field. Overall, the ultimate goal of this paper is to serve as a reference point and guidelines for the academia and professionals in the cyber industries, especially from the deep learning point of view.

Several studies suggest that COVID-19 may be accompanied by symptoms such as a dry cough, muscle aches, sore throat, and mild to moderate respiratory illness. The symptoms of this disease indicate the fact that COVID-19 causes noticeable negative effects on the lungs. Therefore, considering the health status of the lungs using X-rays and CT scans of the chest can significantly help diagnose COVID-19 infection. Due to the fact that most of the methods that have been proposed to COVID-19 diagnose deal with the lengthy testing time and also might give more false positive and false negative results, this paper aims to review and implement artificial intelligence (AI) image-based diagnosis methods in order to detect coronavirus infection with zero or near to zero false positives and false negatives rates. Besides the already existing AI image-based medical diagnosis method for the other well-known disease, this study aims on finding the most accurate COVID-19 detection method among AI methods such as machine learning (ML) and artificial neural network (ANN), ensemble learning (EL) methods.

Vibration measurement serves as the basis for various engineering practices such as natural frequency or resonant frequency estimation. As image acquisition devices become cheaper and faster, vibration measurement and frequency estimation through image sequence analysis continue to receive increasing attention. In the conventional photogrammetry and optical methods of frequency measurement, vibration signals are first extracted before implementing the vibration frequency analysis algorithm. In this work, we demonstrated that frequency prediction can be achieved using a single feed-forward convolutional neural network. The proposed method is verified using a vibration signal generator and excitation system, and the result obtained was compared with that of an industrial contact vibrometer in a real application. Our experimental results demonstrate that the proposed method can achieve acceptable prediction accuracy even in unfavorable field conditions.

The intricate relationship between Gray-Level Co-occurrence Matrix (GLCM) metrics and machine learning model performance underscores the need for rigorous dataset evaluation and selection protocols to ensure the reliability and generalizability of classification outcomes. This study involved a thorough examination of selected publicly available plant diseases datasets, with an emphasis on how well they performed as measured by GLCM metrics. After first classifying the datasets according to their GLCM metrics, dataset_2 (D2) and dataset_5 (D5) were found, respectively, to be the best-performing dataset in all GLCM analyses. The same datasets were then used to train deep learning models, and their classification performances were assessed. A noteworthy association was observed between the results of training deep learning models and the performance ratings derived from GLCM studies. More specifically, dataset_2 (D2) performed best in both GLCM analysis and deep learning model performance, indicating a strong correlation between the accuracy of classification and the textural qualities that GLCM captured. In the context of plant disease identification, in particular, these results highlight the significance of clearly defined dataset selection criteria in deep learning applications. Scholars can improve the accuracy and dependability of deep learning models for diagnosing plant diseases by giving preference to datasets with favorable GLCM metrics. The research also emphasizes the importance of texture features being taken into account in addition to conventional image features, highlighting the necessity of transparency and rigor in dataset selection procedures.

Ransomware attacks have rapidly proliferated, inflicting severe financial damages on businesses and individuals. Machine learning approaches to automate ransomware detection have shown promise but grapple with challenges like limited training data. This study introduces a novel deep learning model for few-shot ransomware classification. The model employs entropy features derived directly from malware binaries coupled with a twin neural network architecture utilizing transfer learning. Tests on over 1000 samples across 11 families demonstrate a weighted F1-score of 85.8\%, surpassing existing methods. The approach mitigates biases in limited training data and preserves intricacies lost in image-based features. It exhibits precise classification capabilities even with sparse samples of new ransomware variants. The research highlights the potential of entropy-driven deep learning to equip defenses against emerging zero-day ransomware strains.

Purpose: To simplify the application of optical coherence tomography (OCT) for examination of subsurface morphology in the oral cavity and reduce barriers towards adoption of OCT as a biopsy guidance device. The aim of this work was to develop automated software tools for simplified analysis of the large volume of data collected during OCT. Methods: Imaging and corresponding histopathology were acquired in-clinic using a wide-field endoscopic OCT system. An annotated dataset (n = 294 images) from 60 patients (34 male, 26 female) was assembled to train four unique neural networks. A deep learning pipeline was built using convolutional and modified u-net models to detect the imaging field of view (network 1), detect artifacts (network 2), identify the tissue surface (network 3), and identify the presence and location of the basement membrane (network 4). Results: The area under the curve of the image and artifact detection networks was 1.00 and 0.94 respectively. The Dice similarity score for the surface and basement membrane segmentation networks was 0.98 and 0.83 respectively. Conclusion: Deep Learning (DL) techniques can identify the location and variations in the epithelial surface and basement membrane in OCT images of the oral mucosa. Segmentation results can be synthesized into accessible en face maps to allow easier visualization of changes.

The increasing growth in knowledge about functioning of the nervous system of mammals and humans, as well as the significant neuromorphic technologies development in recent decades, have led to the emergence of a large number of brain-computer interfaces and neuroprosthetics for the regenerative medicine tasks. Neurotechnologies have traditionally been developed for therapeutic purposes to help or replace motor, sensory or cognitive abilities damaged by injury or disease. They also have significant potential for memory enhancement. However, there are still no fully developed neurotechnologies and neural interfaces capable of restoring or expanding cognitive functions, in particular memory, in mammals or humans. In this regard, the search for new technologies in the field of restoration of cognitive functions is an urgent task of modern neurophysiology, neurotechnology and artificial intelligence. Hippocampus is an important brain structure connected to memory and information processing in the brain. The aim of this paper is to propose an approach based on deep neural networks for prediction of hippocampal signals in CA1 region based on received biological input in CA3 region. We compare results of prediction for two widely used deep architectures: reservoir computing (RC) and long short-term memory (LSTM) networks. Proposed study can be viewed as a first step in the complex task of the development of a neurohybrid chip, which allows one to restore memory functions in the damaged rodent hippocampus.

The use of visual signals in horticulture has attracted significant attention and encompassed a wide range of data types such as 2D images, videos, hyperspectral images, and 3D point clouds. These visual signals have proven to be valuable in developing cutting-edge computer vision systems for various applications in horticulture, enabling plant growth monitoring, pest and disease detection, quality and yield estimation, and automated harvesting. However, unlike other sectors, developing deep learning computer vision systems for horticulture encounters unique challenges due to the limited availability of high-quality training and evaluation datasets necessary for deep learning models. This paper investigates the current status of vision systems and available data in order to identify the high-quality data requirements specific to horticultural applications. We analyse the impact of the quality of visual signals on the information content and features that can be extracted from these signals. To address the identified data quality requirements, we explore the usage of a deep learning-based super-resolution model for generative quality enhancement of visual signals. Furthermore, we discuss how these can be applied to meet the growing requirements around data quality for learning-based vision systems. We also present a detailed analysis of the competitive quality generated by the proposed solution compared to cost-intensive hardware-based alternatives. This work aims to guide the development of efficient computer vision models in horticulture by overcoming existing data challenges and paving a pathway forward for contemporary data acquisition.

The ‘inverse problem’ of mass spectrometric molecular identification (‘given a mass spectrum, calculate/predict the 2D structure of the molecule whence it came’) is largely unsolved, and is especially acute in metabolomics where many small molecules remain unidentified. This is largely because the number of experimentally available electrospray mass spectra of small molecules is quite limited. However, the forward problem (‘calculate a small molecule’s likely fragmentation and hence at least some of its mass spectrum from its structure alone’) is much more tractable, because the strengths of different chemical bonds are roughly known. This kind of molecular identification problem may be cast as a language translation problem in which the source language is a list of high-resolution mass spectral peaks and the ‘translation’ a representation (for instance in SMILES) of the molecule. It is thus suitable for attack using the deep neural networks known as transformers. We here present MassGenie, a method that uses a transformer-based deep neural network, trained on ~6 million chemical structures with augmented SMILES encoding and their paired molecular fragments as generated in silico, explicitly including the protonated molecular ion. This architecture (containing some 400 million elements) is used to predict the structure of a molecule from the various fragments that may be expected to be observed when some of its bonds are broken. Despite being given essentially no detailed nor explicit rules about molecular fragmentation methods, isotope patterns, rearrangements, neutral losses, and the like, MassGenie learns the effective properties of the mass spectral fragment and valency space, and can generate candidate molecular structures that are very close or identical to those of the ‘true’ molecules. We also use VAE-Sim, a previously published variational autoencoder, to generate candidate molecules that are ‘similar’ to the top hit. In addition to using the ‘top hits’ directly, we can produce a rank order of these by ‘round-tripping’ candidate molecules and comparing them with the true molecules, where known. As a proof of principle, we confine ourselves to positive electrospray mass spectra from molecules with a molecular mass of 500Da or lower, including those in the last CASMI challenge (for which the results are known), getting 49/93 (53%) precisely correct. The transformer method, applied here for the first time to mass spectral interpretation, works extremely effectively both for mass spectra generated in silico and on experimentally obtained mass spectra from pure compounds. It seems to act as a Las Vegas algorithm, in that it either gives the correct answer or simply states that it cannot find one. The ability to create and to ‘learn’ millions of fragmentation patterns in silico, and therefrom generate candidate structures (that do not have to be in existing libraries) directly, thus opens up entirely the field of de novo small molecule structure prediction from experimental mass spectra.

The question of molecular similarity is core in cheminformatics, and is usually assessed via a pairwise comparison based on vectors of properties or molecular fingerprints. We recently exploited variational autoencoders to embed 6M molecules in a chemical space, such that their (Euclidean) distance within the latent space so formed could be assessed within the framework of the entire molecular set. However, the standard objective function used did not seek to manipulate the latent space so as to cluster the molecules based on any perceived similarity. Using a set of some 160,000 molecules of biological relevance, we here bring together three modern elements of deep learning to create a novel and disentangled latent space, viz transformers, contrastive learning, and an embedded autoencoder. The effective dimensionality of the latent space was varied such that clear separation of individual types of molecules could be observed within individual dimensions of the latent space. The capacity of the network was such that many dimensions were not populated at all. As before, we assessed the utility of the representation by comparing clozapine with its near neighbours, and did the same for various antibiotics related to flucloxacillin. Transformers, especially when as here coupled with contrastive learning, effectively provide one-shot learning, and lead to a successful and disentangled representation of molecular latent spaces that at once uses the entire training set in their construction while allowing ‘similar’ molecules to cluster together in an effective and interpretable way.

Measuring Marginal Reflex Distance-1 (MRD-1) is a crucial clinical tool used to evaluate the position of the eyelid margin in relation to the cornea. Traditionally, this assessment has been conducted manually by plastic surgeons, ophthalmologists, or trained technicians. However, with the advancements in Artificial Intelligence (AI) technology, there is a growing interest in the development of automated systems capable of accurately measuring MRD-1.In this context, we introduce novel MRD-1 measurement methods based on deep learning algorithms that can simultaneously capture images and compute the results. This prospective observational study involved 154 eyes of 77 patients aged over 18 years who visited Chungnam National University Hospital between September 1, 2023, and July 29, 2023. We collected four different MRD1 da-tasets from patients using three distinct measurement methods, each tailored to the individual patient. The mean MRD1 values, measured through the manual method using a penlight, the deep learning method, ImageJ analysis from RGB eye images, and ImageJ analysis from IR eye images in 56 eyes of 28 patients, were 2.64 ± 1.04 mm, 2.85 ± 1.07 mm, 2.78 ± 1.08 mm, and 3.07 ± 0.95 mm, respectively. Notably, the strongest agreement was observed between MRD1_deep learning (DL) and MRD1_IR (0.822, p < 0.01). In the Bland-Altman plot, the smallest difference was observed between MRD1_DL and MRD1_IR ImageJ, with a mean difference of 0.0611 and △LOA (limits of agreement) of 2.5162, which is the smallest among the other groups. In conclusion, this novel MRD1 measurement method, based on an IR camera and deep learning, demon-strates statistical significance and can be readily applied in clinical settings.

Deep Learning is a recent and important addition to the computational toolbox available for image reconstruction in fluorescence microscopy. We review state-of-the-art applications such as image restoration, super-resolution, and light-field imaging, and discuss how the latest Deep Learning research can be applied to other image reconstruction tasks such as structured illumination, spectral deconvolution, and sample stabilisation. Despite its successes, Deep Learning also poses significant challenges, has often misunderstood capabilities, and overlooked limits. We will address key questions, such as: What are the challenges in obtaining training data? Can we discover structures not present in the training data? And, what is the danger of inferring unsubstantiated image details?

The accurate classification of landfill waste diversion plays a critical role in efficient waste management practices. Traditional approaches, such as visual inspection, weighing and volume measurement, and manual sorting, have been widely used but suffer from subjectivity, scalability, and labour requirements. In contrast, machine learning approaches, particularly Convolutional Neural Networks (CNN), have emerged as powerful deep learning models for waste detection and classification. This paper analyses VGG-16, InceptionResNetV2, DenseNet121, Inception V3, and MobileNetV2 models to classify real-life waste when trained on pristine and unadulterated materials, versus samples collected at a landfill site. When training on DiversionNet, the unadulterated material dataset with labels required for landfill modelling, classification accuracy was limited to 49.69% in the real environment. Using real-world samples in the newly formed RealWaste dataset showed that practical applications for deep learning in waste classification is possible, with Inception V3 reaching 89.19% classification accuracy on the full spectrum of labels required for accurate modelling.

One of the most important challenges in the Machine and Deep Learning areas today is to build good models using small datasets, because sometimes it is not possible to have large ones. Several techniques have been proposed in the literature to address this challenge. This paper aims at studying the different available Deep Learning techniques and performing a thorough experimentation to analyze which technique or combination thereof improves the performance and effectiveness of the models. A complete comparison with classical Machine Learning techniques was carried out, to contrast the results obtained using both techniques when working with small datasets. Thirteen algorithms were implemented and trained using three different small datasets (MNIST, Fashion MNIST, and CIFAR-10). Each experiment was evaluated using a well-established set of metrics (Accuracy, Precision, Recall, F1, and the Matthews correlation coefficient). The experimentation allowed concluding that it is possible to find a technique or combination of them to mitigate a lack of data, but this depends on the nature of the dataset, the amount of data, and the metrics used to evaluate them.

We present and benchmark FilterNet, a flexible deep learning architecture for time series classification tasks, such as activity recognition via multichannel sensor data. It adapts popular CNN and CNN-LSTM motifs which have excelled in activity recognition benchmarks, implementing them in a many-to-many architecture to markedly improve frame-by-frame accuracy, event segmentation accuracy, model size, and computational efficiency. We propose several model variants, evaluate them alongside other published models using the Opportunity benchmark dataset, demonstrate the effect of model ensembling and of altering key parameters, and quantify the quality of the models’ segmentation of discrete events. We also offer recommendations for use and suggest potential model extensions. FilterNet advances the state of the art in all measured accuracy and speed metrics on the benchmarked dataset, and it can be extensively customized for other applications.

Data-driven models (DDMs) are extensively used in environmental modeling but face challenges due to limited training data and potential results not adhering to physical laws. To address this challenge, this study developed a process-guided deep learning (PGDL) model, integrating a long short-term memory (LSTM) neural network and a process-based model (PBM), CE-QUAL-W2 (W2), to predict water temperature in a stratified reservoir. The PGDL included an energy constraint term from W2's thermal energy equilibrium into the cost function of the LSTM, besides the mean square error term. In PGDL, parameters were optimized by penalizing deviations from the energy law, ensuring adherence to physical constraints. Compared to LSTM, PGDL demonstrated enhanced satisfaction with the energy balance and superior performance in water temperature prediction. Even with less field data for training, PGDL outperformed both LSTM and calibrated W2 after pre-training with data generated using the uncalibrated W2. Therefore, integration of DDM with a PBM ensured physical consistency in water temperature prediction for complex stratified reservoirs with limited data. Moreover, pre-training the PGDL with PBM proved highly effective in mitigating bias and variance due to insufficient field measurement data.

Aluminum profile surface defects can greatly affect the performance, safety and reliability of products. Traditional human-based visual inspection is low accuracy and time consuming, and machine vision-based methods depend on hand-crafted features which need to be carefully designed and lack robustness. To recognize the multiple types of defects with various size on aluminum profiles, a multiscale defect detection network based on deep learning is proposed. Then, the network is trained and evaluated using aluminum profile surface defects images. Results show 84.6%, 48.5%, 96.9%, 97.9%, 96.9%, 42.5%, 47.2%, 100%, 100%, 43.3% average precision(AP) for the ten defect categories, respectively, with a mean AP of 75.8%, which illustrate the effectiveness of the network in aluminum profile surface defects detection. In addition, saliency maps also show the feasibility of the proposed network.

Accurate segmentation of the Left Ventricle (LV) in Echocardiogram (Echo) images is essential for cardiovascular analysis. Conventional techniques are labor-intensive and exhibit inter-observer variability. Deep learning has emerged as a powerful tool for automated medical image segmentation, offering advantages in speed and potentially superior accuracy. This study explores the efficacy of employing the YOLO (You Only Look Once) segmentation model for automated LV segmentation in Echo images. YOLO, a cutting-edge object detection model, achieves exceptional speed-accuracy balance through its well-designed architecture. It utilizes efficient dilated convolutional layers and bottleneck blocks for feature extraction, while incorporating innovations like path aggregation and spatial attention mechanisms. These attributes make YOLO a compelling candidate for adaptation to LV segmentation in Echo images. We posit that by fine-tuning a pre-trained YOLO based model on a well-annotated Echo image dataset, we can leverage the model's strengths in real-time processing and precise object localization to achieve robust LV segmentation. The proposed approach entails fine-tuning a pre-trained YOLO model on a rigorously labeled Echo image dataset. Model performance has been evaluated using established metrics such as mean Average Precision (mAP) at an Intersection over Union (IoU) threshold of 50% (mAP50) with 98.31%and across a range of IoU thresholds from 50% to 95% (mAP50:95) with 75.27%. Successful implementation of YOLO for LV segmentation has the potential to significantly expedite and standardize Echo image analysis. This advancement could translate to improved clinical decision-making and enhanced patient care.

Microgrids (MGs) have evolved as critical components of modern energy distribution networks, providing increased dependability, efficiency, and sustainability. Effective control strategies are essential for optimising MG operation and maintaining stability in the face of changing environmental and load conditions. Traditional rule-based control systems are extensively used due to their interpretability and simplicity. However, these strategies frequently lack the flexibility for complex and changing system dynamics. This paper provides a novel method called hybrid intelligent control for adaptive MG that integrates basic rule-based-control and deep learning techniques, including gated recurrent units (GRU), basic recurrent neural network (RNN) and long short-term memory (LSTM). The main target of this hybrid approach is to improve MG management performance by combining the strengths of basic rule-based systems and deep learning techniques. These deep learning techniques readily enhance and adapt control decisions based on historical data and domain-specific rules, leading to increasing system efficiency, stability, and resilience in adaptive MG. Our results show that the proposed method optimises MG operation, especially under demanding conditions such as variable renewable energy supply and unanticipated load fluctuations. This study investigates special RNN architectures and hyper-parameter optimization techniques with the aim of predicting power consumption and generation within the adaptive MG system. Our promising results show the highest-performing models indicating high accuracy and efficiency in power prediction. The finest-performing model accomplishes an R2 value close to 1, representing a strong correlation between predicted and actual power values.

Considering the significant advancements in autonomous vehicle technology, research in this field is of interest to researchers. To drive vehicles autonomously, controlling steer angle, gas hatch, and brakes need to be learned. The behavioral cloning method is used to imitate humans’ driving behavior. We created a dataset of driving in different routes and conditions and using the designed model, the output used for controlling the vehicle is obtained. In this paper, the Learning of Self-driving Vehicles Based on Real Driving Behavior Using Deep Neural Network Techniques (LSV-DNN) is proposed. We designed a convolutional network which uses the real driving data obtained through the vehicle’s camera and computer. The response of the driver is during driving is recorded in different situations and by converting the real driver’s driving video to images and transferring the data to an excel file, obstacle detection is carried out with the best accuracy and speed using the Yolo algorithm version 3. This way, the network learns the response of the driver to obstacles in different locations and the network is trained with the Yolo algorithm version 3 and the output of obstacle detection. Then, it outputs the steer angle and amount of brake, gas, and vehicle acceleration. The LSV-DNN is evaluated here via extensive simulations carried out in Python and TensorFlow environment. We evaluated the network error using the loss function. By comparing other methods which were conducted on the simulator’s data, we obtained good performance results for the designed network on the data from KITTI benchmark, the data collected using a private vehicle, and the data we collected.

Responsible, efficient, and environmentally conscious energy consumption practices are increasingly essential for ensuring the reliability of the modern electricity grid. This study focuses on leveraging time series analysis to improve forecasting accuracy, crucial for various application domains where real-world time series data often exhibit complex, non-linear patterns. Our approach advocates for utilizing Long Short Term Memory (LSTM) and Bidirectional LSTM (BiLSTM) models for precise time series forecasting. To ensure a fair evaluation, we compare the performance of our proposed approach with traditional neural networks, time-series forecasting methods, and conventional decline curves. Additionally, individual models based on LSTM, Bi-LSTM, and other machine learning methods are implemented for a comprehensive assessment. Experimental results consistently demonstrate that our proposed model outperforms all benchmarking methods in terms of mean absolute error (MAE) across most datasets. Addressing the imbalance between activations by consumer and prosumer groups, our predictions show superior performance compared to several traditional forecasting methods, such as the Autoregressive Integrated Moving Average model (ARIMA) and Seasonal Autoregressive Integrated Moving Average model (SARIMA). Specifically, the root mean square error (RMSE) of Bi-LSTM is 5.35%, 46.08%, and 50.6% lower than LSTM, ARIMA, and SARIMA, respectively, on the May test data.

Accurate cloud quantification is essential in climate change research. In this work, we construct an automated computer vision framework by synergistically incorporating deep neural networks and finite sector clustering to achieve robust whole sky image-based cloud classification, adaptive segmentation, and recognition under intricate illumination dynamics. A bespoke YOLOv8 architecture attains over 95% categorical precision across four archetypal cloud varieties curated from extensive annual observations (2020) at a Tibetan highland station. Tailor-made segmentation strategies adapted to distinct cloud configurations, allied with illumination-invariant image enhancement algorithms, effectively eliminate solar interference and substantially boost quantitative performance even in illumination-adverse analysis scenarios. Compared with the traditional threshold analysis method, the cloud quantification accuracy calculated within the framework of this paper is significantly improved. Collectively, the methodological innovations provide an advanced solution to markedly escalate cloud quantification precision levels imperative for climate change research, while offering a paradigm for cloud analytics transferable to various meteorological stations.

Accurate detection of forest restoration sites using deep learning is crucial for effective ecosystem management, optimizing reforestation efforts, and ensuring environmental sustainability. While various deep learning-based systems have been developed for this purpose, the influence of different neural network architectures on model performance during the identification of forest restoration sites remains underexplored. This study aimed to address this gap by exploring the optimal methodology for extracting and classifying candidate sites for forest restoration based on Convolutional Neural Networks (CNNs), specialized for image recognition. Four categories—arable land, road and barren, quarry, and forest—were defined as candidate sites for forest restoration. A dataset comprising 17,043 samples was divided into training (11,929) and validation (5,114) sets at a ratio of 7:3 for model learning. Model accuracy was evaluated using pixel accuracy (PA) and mean intersection over union (mean IoU). The Resnet152-U-Net model demonstrated a pixel accuracy of 95.2% and a mean IoU of 61.3%, indicating excellent performance in extracting candidate sites for forest restoration. This approach offers spatial and temporal advantages over conventional field surveys or aerial image-based assessments, potentially serving as valuable data for selecting future forest restoration sites.

With the rapid expansion of intelligent resource-constrained devices and high-speed communication technologies, Internet of Things (IoT) has earned a wide recognition as the primary standard for low-power lossy networks (LLNs). Nevertheless, IoT infrastructures are vulnerable to cyber-attacks due to the constraints in computation, storage, and communication capacity of the endpoint devices. From one side, the majority of newly developed cyber-attacks are formed by slightly mutating formerly established cyber-attacks to produce a new attack tending to be treated as a normal traffic through the IoT network. From the other side, the influence of coupling the deep learning techniques with cybersecurity field has become a recent inclination of many security applications due to their impressive performance. In this paper, we provide a comprehensive development of a new intelligent and autonomous deep learning-based detection and classification system for cyber-attacks in IoT communication networks leveraging the power of convolutional neural networks, abbreviated as (IoT-IDCS-CNN). The proposed IoT-IDCS-CNN makes use of the high-performance computing employing the robust CUDA based Nvidia GPUs and the parallel processing employing the high-speed I9-Cores based Intel CPUs. In particular, the proposed system is composed of three subsystems: Feature Engineering subsystem, Feature Learning subsystem and Traffic classification subsystem. All subsystems are developed, verified, integrated, and validated in this research. To evaluate the developed system, we employed the NSL-KDD dataset which includes all the key attacks in the IoT computing. The simulation results demonstrated more than 99.3% and 98.2% of cyber-attacks’ classification accuracy for the binary-class classifier (normal vs anomaly) and the multi-class classifier (five categories) respectively. The proposed system was validated using k-fold cross validation method and was evaluated using the confusion matrix parameters (i.e., TN, TP, FN, FP) along with other classification performance metrics including precision, recall, F1-score, and false alarm rate. The test and evaluation results of the IoT-IDCS-CNN system outperformed many recent machine-learning based IDCS systems in the same area of study.

Background and Objective: Segmentation of the femur in Dual Energy X-ray (DXA) images poses challenges due to reduced contrast, noise, bone shape variations, and inconsistent X-ray beam penetration. In this study, we investigate the relationship between noise and certain deep learning (DL) techniques for semantic segmentation of the femur to enhance segmentation and bone mineral density (BMD) accuracy by incorporating noise reduction methods into DL models. Methods: Convolutional neural network (CNN) based models were employed to segment femurs in DXA images and evaluate the effects of noise reduction filters on segmentation accuracy and its effect on BMD calculation. Various noise reduction techniques were integrated into DL-based models to enhance image quality before training. We assessed the performance of the fully convolutional neural network (FCNN) in comparison to noise reduction algorithms and manual segmentation methods. Results: Our study demonstrated that FCNN outperformed noise reduction algorithms in enhancing segmentation accuracy and enabling precise calculation of BMD. The FCNN-based segmentation approach achieved a segmentation accuracy of 98.84% and a correlation coefficient of 0.9928 for BMD measurements, indicating its effectiveness in the clinical diagnosis of osteoporosis. Conclusions: In conclusion, integrating noise reduction techniques into DL-based models significantly improves femur segmentation accuracy in DXA images. The FCNN model, in particular, shows promising results in enhancing BMD calculation and clinical diagnosis of osteoporosis. These findings highlight the potential of DL techniques in addressing segmentation challenges and improving diagnostic accuracy in medical imaging.

The quality prediction of quaternary structure models of a protein complex, in the absence of its true structure, is known as the Estimation of Model Accuracy (EMA). EMA is useful for ranking predicted protein complex structures and using them appropriately in biomedical research, such as protein-protein interaction studies, protein design, and drug discovery. With the advent of more accurate protein complex (multimer) prediction tools, such as AlphaFold2-Multimer and ESMFold, the estimation of the accuracy of protein complex structures has attracted increasing attention. Many deep learning methods have been developed to tackle this problem; however, there is a noticeable absence of a comprehensive overview of these methods to facilitate future development. Addressing this gap, we present a review of deep learning EMA methods for protein complex structures developed in the past several years, analyzing their methodologies and impacts. We also provide a prospective summary of some potential new developments for further improving the accuracy of the EMA methods.

Pig behavior is an integral part of health and welfare management, as pigs usually reflect their inner emotions through behavior change. The livestock environment plays a key role in pigs' health and wellbeing. A poor farm environment increases the toxic GHGs, which might deteriorate pigs' health and welfare. In this study a computer-vision-based automatic monitoring and tracking model was proposed to detect short-term pigs' physical activities in a compromised environment. The ventilators of the livestock barn were closed for an hour, three times in a day (07:00-08:00, 13:00-14:00, and 20:00-21:00) to create a compromised environment, which increases the GHGs level significantly. The corresponding pig activities were observed before, during, and after an hour of the treatment. Two widely used object detection models (YOLOv4 and Fast-er R-CNN) were trained and compared their performances in terms of pig localization and posture detection. The YOLOv4, which outperformed the Faster R-CNN model, coupled with a Deep-SORT tracking algorithm to detect and track the pig activities. The results showed that the pigs became more inactive with the increase in GHG concentration, reducing their standing and walking activities. Moreover, the pigs also shortened their sternal-lying posture increasing the lateral lying posture duration at higher GHG concentration. The high detection accuracy (mAP: 98.67%) and tracking accuracy (MOTA: 93.86% and MOTP: 82.41%) signify the models’ efficacy in monitoring and tracking pigs' physical activities non-invasively.

Aim: To determine whether an AI model and single sensor measuring acceleration and ECG could model cognitive and physical fatigue for a self-paced trail run. Methods: A field-based protocol of continuous fatigue repeated hourly induced physical (~45 minutes) and cognitive (~10 minutes) fatigue on one healthy participant. Physical load was a 3.8 km, 200 m vertical gain, trail run with acceleration and electrocardiogram (ECG) data collected using a single sensor. Cognitive load was a Multi Attribute Test Battery (MATB) and separate assessment battery including the Finger Tap Test (FTT), Stroop, Trail Making A and B, Spatial Memory, Paced Visual Serial Addition Test (PVSAT), and a vertical jump. A fatigue prediction model was implemented using a Convolutional Neural Network (CNN). Results: When the fatigue test battery results were compared for sensitivity to the protocol load, FTT right hand (R2 0.71) and Jump Height (R2 0.78) were the most sensitive while the other tests were less sensitive (R2 values Stroop 0.49, Trail Making A 0.29, Trail Making B 0.05, PVSAT 0.03, spatial memory 0.003). Best prediction results were achieved with a rolling average of 200 predictions (102.4 s), during set activity types, mean absolute error for ‘walk up’ (MAE200 12.5%) and range of absolute error for ‘run down‘ (RAE200 16.7%). Conclusion: We were able to measure cognitive and physical fatigue using a single wearable sensor during a practical field protocol including contextual factors in conjunction with a neural network model. This research has practical application to fatigue research in the field.

Interest in drone solutions in forestry applications is growing. Using drones, datasets can be captured flexibly and at high spatial and temporal resolutions when needed. In forestry applications, fundamental tasks include the detection of individual trees, tree species classification, bio-mass estimation, etc. Deep Neural Networks (DNN) have shown superior results when comparing with conventional machine learning methods such as Multi-Layer Perceptron (MLP) in cases of huge input data. The objective of this research was to investigate 3D convolutional neural networks (3D-CNN) to classify three major tree species in a boreal forest: pine, spruce, and birch. The proposed 3D-CNN models were employed to classify tree species in a test site in Finland. The classifiers were trained with a dataset of 3039 manually labelled trees. Then the accuracies were assessed by employing independent datasets of 803 records. To find the most efficient set of feature combination, we compare the performances of 3D-CNN models trained with hyperspectral (HS) channels, RGB channels, and canopy height model (CHM), separately and combined. It is demonstrated that the proposed 3D-CNN model with RGB and HS layers produces the highest classification accuracy. The producer accuracy of the best 3D-CNN classifier on the test dataset were 99.6%, 94.8%, and 97.4% for pines, spruces, and birches, respectively. The best 3D-CNN classifier produced ~5% better classification accuracy than the MLP with all layers. Our results suggest that the proposed method provides excellent classification results with acceptable performance metrics for HS datasets. Our results show that pine class was detectable in most layers. Spruce was most detectable in RGB data, while birch was most detectable in the HS layers. Furthermore, the RGB datasets provide acceptable results for many low-accuracy applications.

(1) Background: Evidence-based policymaking requires data about the local population's socioeconomic status (SES) at detailed geographical level, however, such information is often not available, or is too expensive to acquire. Researchers have proposed solutions to estimate SES indicators by analyzing Google Street View images, however, these methods are also resource-intensive, since they require large volumes of manually labeled training data. (2) Methods: We propose a methodology for automatically computing surrogate variables of SES indicators using street images of parked cars and deep multiple instance learning. Our approach does not require any manually created labels, apart from data already available by statistical authorities, while the entire pipeline for image acquisition, parked car detection, car classification, and surrogate variable computation is fully automated. The proposed surrogate variables are then used in linear regression models to estimate the target SES indicators. (3) Results: We implement and evaluate a model based on the proposed surrogate variable at 30 municipalities of varying SES in Greece. Our model has $R^2=0.76$ and a correlation coefficient of $0.874$ with the true unemployment rate, while it achieves a mean absolute percentage error of $0.089$ and mean absolute error of $1.87$ on a held-out test set. Similar results are also obtained for other socioeconomic indicators, related to education level and occupational prestige. (4) Conclusions: The proposed methodology can be used to estimate SES indicators at the local level automatically, using images of parked cars detected via Google Street View, without the need for any manual labeling effort.

Many problems in the fields of finance and actuarial science can be transformed into the problem of solving backward stochastic differential equations (BSDE) and partial differential equations (PDE) with jumps, which are often difficult to solve in high-dimensional cases. To solve this problem, this paper applies the deep learning algorithm to solve a class of high-dimensional nonlinear partial differential equations with jump terms and their corresponding backward stochastic differential equations (BSDE) with jump terms. Using the nonlinear Feynman-Kac formula, the problem of solving this kind of PDE is transformed into the problem of solving the corresponding backward stochastic differential equations with jump terms, and the numerical solution problem is turned into a stochastic control problem. At the same time, the gradient and jump process of the unknown solution are separately regarded as the strategy function and they are approximated respectively by using two multilayer neural networks as function approximators. Thus, deep learning-based method is used to overcome the “curse of dimensionality” caused by high-dimensional PDE with jump, and the numerical solution is obtained. In addition, this paper proposes a new optimization algorithm based on the existing neural network random optimization algorithm, and compares the results with the traditional optimization algorithm, and achieves good results. Finally, the proposed method is applied to three practical high-dimensional problems: Hamilton-Jacobi-Bellman equation, bond pricing under the jump Vasicek model and high-dimensional option pricing model with default risk. The proposed numerical method has obtained satisfactory accuracy and efficiency. The method has important application value and practical significance in investment decision-making, option pricing, insurance and other fields.

Unmanned Underwater Vehicles (UUVs) are pivotal in ocean exploration, research, and various industrial activities such as marine mining and offshore engineering. However, traditional methods of navigating these vehicles face significant challenges, mainly due to the inefficacy of electromagnetic waves in water, leading to signal loss. To address these limitations, researchers are increasingly turned learning, an artificial intelligence technique capable of learning from data-enhancing underwater navigation, mainly through visual Simultaneous Localization and Mapping (SLAM). This paper explores integrating deep learning methodologies and sensor technologies to revolutionize underwater navigation for UUVs. Proprioceptive sensors, along with exteroceptive sensors, are crucial in accurately measuring and comprehending the underwater environment. Additionally, the paper provides detailed insights into the processes of underwater SLAM, camera-based underwater positioning systems, sonar systems for underwater navigation, and the utilization of Lidar in underwater navigation. Furthermore, it delves into applying deep learning techniques in underwater SLAM, offering a comprehensive understanding of the innovative processes driving advancements in underwater vehicle navigation. By leveraging these advancements, this research aims to improve underwater navigation systems' precision, reliability, and adaptability, thereby unlocking new frontiers in ocean exploration and industrial applications for UUVs.

Pattern recognition techniques form the heart of most, if not all, incoherent linear shift-invariant systems. When an object is recorded using a camera, the object information gets sampled by the point spread function (PSF) of the system, replacing every object point with the PSF in the sensor. The PSF is a sharp Kronecker Delta-like function when the numerical aperture (NA) is large with no aberrations. When the NA is small, and the system has aberrations, the PSF appears blurred. In the above case, if the PSF is known, then the object information can be obtained by scanning the PSF over the recorded object intensity pattern and looking for pattern matching conditions through a mathematical process called correlation. In this study, a recently developed deconvolution method, the Lucy-Richardson-Rosen algorithm (LR2A), has been implemented to computationally refocus images recorded in the presence of spatio-spectral aberrations. The performance of LR2A was compared against the Lucy-Richardson algorithm and non-linear reconstruction. LR2A exhibits a superior deconvolution capability even in extreme cases of spatio-spectral aberrations and blur. Experimental results of deblurring a picture captured using high-resolution smartphone cameras are presented. LR2A was implemented to significantly improve the performances of the widely used deep convolutional neural networks for image classification.

We have developed deep learning models for automatic identification of the maternal heart rate (MHR) and, more generally, false signals (FSs) on fetal heart rate (FHR) recordings. The models can be used to preprocess FHR data prior to automated analysis or as a clinical alert system to assist the practitioner. Three models were developed and used to detect (i) FSs on the MHR channel (the FSMHR model), (ii) the MHR and FSs on the Doppler FHR sensor (the FSDop model), and (iii) FSs on the scalp ECG channel (the FSScalp model). The FSDop model was the most useful because FSs are far more frequent on the Doppler FHR channel. All three models were based on a multilayer, symmetric, GRU and were trained on data recorded during the first and second stages of delivery. The FSMHR and FSDop models were also trained on antepartum recordings. The training dataset contained 1030 expert-annotated periods (mean duration: 36 min) from 635 recordings. In an initial evaluation of routine clinical practice, 30 fully annotated recordings for each sensor type (mean duration: 5 h for MHR and Doppler sensors, and 3 h for the scalp ECG sensor) were analyzed. The sensitivity, positive predictive value (PPV) and accuracy were respectively 62.20%, 87.1% and 99.90% for the FSMHR model, 93.1%, 95.6% and 99.68% for the FSDop model, and 44.6%, 87.2% and 99.93% for the FSScalp model. We built a second test dataset with a more solid ground truth by selecting 45 periods (lasting 20 min, on average) on which the Doppler FHR and scalp ECG signals were recorded simultaneously. Using scalp ECG data, the experts estimated the true FHR value more reliably and thus annotated the Doppler FHR channel more precisely. The models achieved a sensitivity of 53.3%, a PPV of 62.4%, and an accuracy of 97.29%. In comparison, two experts (blinded to the scalp ECG data) respectively achieved a sensitivity of 15.7%, a PPV of 74.3%, and an accuracy of 96.91% and a sensitivity of 60.7%, a PPV of 83.5% and an accuracy of 98.24%. Hence, the models performed at expert level (better than one expert and worse than the other), although a well-trained expert with good knowledge of FSs could probably do better in some cases. The models and datasets have been included in the Fetal Heart Rate Morphological Analysis open-source MATLAB toolbox and can be used freely for research purposes.

Antimicrobial peptides (AMPs) are a promising alternative to antibiotics to combat drug resistance in pathogenic bacteria. However, the development of AMPs with high potency and specificity remains a challenge, and new tools to evaluate antimicrobial activity are needed to accelerate the discovery process. As a step toward direct prediction of the experimental minimum inhibitory concentration (MIC) of AMPs, we proposed MBC-Attention, a combination of a multi-branch CNN architecture and attention mechanism. Using a curated dataset of 3929 AMP against Escherichia coli, the optimal MBC-Attention model achieved an average Pearson correlation coefficient of 0.775 and an RMSE of 0.533 (log μM) in three independent tests of 393 sequences each. This results in a 5–12% improvement in PCC and 7–13% improvement in RMSE compared with RF and SVM models. Ablation studies confirmed that both attention mechanisms contributed to performance improvement.

Background: Periprosthetic joint infection (PJI) following total hip and knee arthroplasty remains an extremely challenging and relatively high complication. This study aims to develop, validate and evaluate the use of machine learning (ML) algorithm to predict PJI in patients undergoing revision arthroplasties. Methods: A comprehensive review of patients undergoing hip or knee revision arthroplasty from 1 January 2015 to 31 March 2021 was conducted. Clinical data coming from preoperative patients history, laboratory analysis and demographic characteristics of patients were screened. Final data have been used to train a Logistic Regression model with the aim of predicting PJI preoperatively. Results: 1360 patients were enrolled, 1141 in the aseptic cohort and 219 in the infected cohort were included. ML demonstrated good discriminatory performance in predicting PJI in the selected patients (area under the curve 0.770 ± 0.006 in the training set and 0.720 ± 0.057 in the test set), and identified 3 significant predictors of PJI. Conclusion: ML algorithm trained using preoperative clinical data accurately predicted PJI. The incorporation of ML models into preoperative assessment of patients undergoing prosthetic revision procedures are useful in providing specific risk assessment to aid individualised counselling, shared decision making and presurgical optimization.

Commercial buildings situated in hot and humid tropical climates rely significantly on cooling systems to maintain optimal occupant comfort. A well-accurate day-ahead load profile prediction plays a pivotal role in planning the energy requirements of cooling systems. Despite the pressing need for effective day-ahead cooling load predictions, current methodologies have not fully harnessed the potential of advanced deep-learning techniques. This research paper aims to address this gap by investigating the application of innovative deep learning models in day-ahead hourly cooling load prediction for commercial buildings situated in tropical climates. A range of deep learning techniques, including Deep Neural Network, Convolutional Neural Networks, Recurrent Neural Networks, and Long Short-Term Memory networks are employed to enhance prediction accuracy. Furthermore, these individual deep learning techniques are synergistically integrated to create hybrid models, such as CNN-LSTM and Sequence-to-Sequence models. An extensive comparative analysis is conducted to identify the most effective hybrid model. Sequence-to-sequence model provided better performance compared to the other single and hybrid models. Experiments are conducted to choose the time horizons from the past that can serve as input to the models. Further, the influence of various categories of input parameters on prediction performance has been assessed. Historical cooling load, calendar features and outdoor weather parameters are found in decreasing order of influence on prediction accuracy. This research focuses on buildings located in Singapore and presents a comprehensive case study to validate the proposed models and methodologies.

To minimise environmental impact, avoid regulatory penalties, and improve competitiveness, energy-intensive manufacturing firms require accurate forecasts of their energy consumption so that precautionary and mitigation measures can be taken. Deep learning is widely touted as a superior analytical technique to traditional artificial neural networks, machine learning, and other classical time series models due to its high dimensionality and problem solving capabilities. Despite this, research on its application in demand-side energy forecasting is limited. We compare two benchmarks (Autoregressive Integrated Moving Average (ARIMA), and an existing manual technique used at the case site) against three deep learning models (simple Recurrent Neural Networks (RNN), Long Short-Term Memory (LSTM), and Gated Recurrent Unit (GRU)) and three machine learning models (Support Vector Regression (SVM), Random Forest, and K-Nearest Neighbors (KNN)) for short term load forecasting (STLF) using data from a Brazilian thermoplastic resin manufacturing plant. We use the grid search method to identify the best configurations for each model, and then use Diebold-Mariano testing to confirm the results. Results suggests that the legacy approach used at the case site is the worst performing, and that the GRU model outperformed all other models tested.

: Cybersecurity has become a critical area in the digital field in recent years. The expansion of networks has revolutionised the way network structures are organised and managed. However, with increased connectivity and the growing complexity of modern networks, the threat of cyber-attacks has become more intense. As technology continues to advance, it brings both opportunities and challenges. One of the major challenges is the need to secure networks and sensitive data from various malicious activities. Traditional networks have evolved to include Software Defined Network (SDN), which offers a more flexible and programmable framework. Researchers should focus on detecting attacks in SDN because SDN networks are becoming more popular and attractive targets for clients due to their programmability and dynamic nature. The centralised controller, known as the backbone of an SDN, becomes a single point of failure and a potential target for attackers if not properly secured. Researchers need to emphasise the detection of attacks in SDN in order to mitigate these risks. By understanding the potential vulnerabilities and attack methods specific to SDN, researchers can develop effective detection approaches and propose countermeasures. This methodology helps protect the network from potential threats and minimises the impact of successful attacks. Therefore, this survey paper provides a review of Intrusion Detection Systems (IDSs) in Software-Defined Networks (SDN) to provide a thorough understanding of SDN security issues.

The health and stability of trees are essential information for the safety of people and property in urban greenery, parks or along roads. The stability of the trees is linked to root stability but essentially also to trunk decay. Currently used internal tree stem decay assessment methods, such as tomography or penetrometry, are reliable but usually time-consuming and unsuitable for large-scale surveys. Therefore, a new method based on close-range remote sensed data, specifically close-range photogrammetry and iPhone LiDAR, was tested to detect decayed standing tree trunks automatically. The proposed study used the PointNet deep learning algorithm for 3D data classification. It was verified in three different datasets consisting of pure coniferous trees, pure deciduous trees or mixed data to eliminate the influence of the detectable symptoms for each group and species itself. The mean achieved validation accuracies of the models were 65.5 % for Coniferous trees, 58.4 % for Deciduous trees and 57.7 % for Mixed data classification. The accuracies indicate promising data, which can be either used by practitioners for preliminary surveys or for other researchers to acquire more input data and create more robust classification models.

This study explores the Edaravone solubility space encompassing both neat and binary dissolution media. The efforts were made toward revealing the concentration limits inherently associated with common pure and mixed solvents. For this purpose, the published solubility data of the title drug were scrupulously inspected and cured. This enabled to make the dataset consistent and coherent. However, the lack of some important types of solvents in the collection called for an extension of the available pool of Edaravone solubility data. Hence, new measurements were performed for collecting Edaravone solubility in polar non-protic and polar di-protic media. Such an extended set of data was used for tuning the parameters of regressors models and formulating the ensemble for predicting new data. In both phases, namely the model training and ensemble formulation, close attention was paid not only to minimizing the deviation of computed values from the experimental ones but also to ensuring the high predictive power and computing solubility for new systems. Furthermore, the environmental friendliness characteristics determined based on the common green solvent selection criteria was included in the analysis. Our applied protocol led to the conclusion that the solubility space defined by ordinary solvents is limited, and it is unlikely to find solvents that are more suited for Edaravone dissolution than those depicted in this manuscript. The theoretical framework presented in this study provides a precise guide for conducting experiments, saving time and resources in the pursuit of new findings.

Urinary cytology is a useful, essential diagnostic method in routine urological clinical practice. Liquid-based cytology (LBC) for urothelial carcinoma screening is commonly used in the routine clinical cytodiagnosis because of its high cell collection rate. Since conventional screening processes by cytoscreeners and cytopathologists using microscopes is limited in terms of human resources, it is important to integrate new deep learning methods that can automatically and rapidly diagnose a large amount of specimens without delay. The goal of this study was to investigate the use of deep learning models for the classification of urine LBC whole-slide images (WSIs) into neoplastic and non-neoplastic (negative). We trained deep learning models using 786 WSIs by transfer learning, fully supervised, and weakly supervised learning approaches. We evaluated the trained models on two test sets (equal and clinical balance) with a combined total of 750 WSIs, achieving ROC-AUCs for WSI diagnosis in the range of 0.984-0.990 by the best model, demonstrating the promising potential use of our model for aiding urine cytodiagnostic processes.

Leader-follower autonomy (LFA) systems have so far only focused on vehicles following other vehicles. Though there have been several decades of research into this topic, there has not yet been any work on human-vehicle leader-follower systems. We present a system in which an autonomous vehicle — our ACTor 1 platform — can follow a human leader who controls the vehicle through gestures. We successfully developed a modular pipeline that uses artificial intelligence/deep learning to recognize hand-and-body gestures from a user in view of the vehicle’s camera and translate those gestures into physical action by the vehicle. We demonstrate our work using our ACTor 1 platform, a modified Polaris Gem 2 electric vehicle. This work has numerous applications, such as material transport in industrial contexts.

The advent of deep learning (DL) has revolutionized medical imaging, offering unprecedented avenues for accurate disease classification and diagnosis. DL models have shown remarkable promise for classifying brain tumor from Magnetic Resonance Imaging (MRI) scans. However, despite their impressive performance, the opaque nature of DL models poses challenges in understanding their decision-making processes, particularly crucial in medical contexts where interpretability is essential. This paper explores the intersection of medical image analysis and DL interpretability, aiming to elucidate the decision-making rationale of DL models in brain tumor classification. Leveraging state-of-the-art DL frameworks with transfer learning, we conduct a comprehensive evaluation encompassing both classification accuracy and interpretability. Using state-of-the-art DL frameworks with transfer learning, we conduct a thorough evaluation covering both classification accuracy and interpretability. We employ adaptive path-based techniques to understand the underlying decision-making mechanisms of these models. Grad-CAM and Grad-CAM++ highlight critical image regions where the tumors are located.

Accurate geometric modeling of blood vessels lumen given their 3D images is a basis for vessel quantification in support of cardiovascular diseases diagnosis, treatment and monitoring. Unlike other approaches, which assume circular or elliptical vessel cross-section, the proposed method employs parametric B-splines combined with the image formation system equations to accurately localize the highly curved lumen boundaries. No image segmentation is involved which might reduce the localization accuracy in effect of spatial discretization. It is shown that the model parameters can be reliably identified by a feedforward neural network which, driven by the cross-section images, predicts the parameter values many times faster than a reference least-squares (LS) model fitting algorithm. Two example applications: modeling the lower extremities artery-vein complexes visualized in steady-state contrast-enhanced magnetic resonance images (MRI) and the coronary arteries pictured in computed tomography angiograms (CTA), are presented and discussed to demonstrate the method potential. Besides application to medical diagnosis, blood-flow simulation and vessel-phantom design, it can serve as a tool for automated annotation of image datasets required to train machine-learning algorithms.

Industrial data scarcity is one of the largest factors holding back the widespread use of machine learning in manufacturing. To overcome this problem, the concept of transfer learning was developed and it achieved high attention in recent industrial research. Our paper focuses on the problem of time series segmentation and presents the first in-depth research about transfer learning for deep-learning based time series segmentation on the example of industrial end-of-line pump testing. In particularly, we investigate if the performance of deep learning models can be increased by pretraining the network with data from other domains. Three different scenarios are analyzed: source and target data being closely related, source and target data being distantly related, and source and target data being non-related. The results demonstrate that transfer learning can enhance the performance of time series segmentation models in respect to accuracy and training speed. The benefit is most clearly seen in scenarios where source and training data are closely related and the number of target training data samples is lowest. However, in the scenario of non-related datasets, cases of negative Transfer Learning were observed as well. Thus, the research emphasizes the potential, but also the challenges of industrial Transfer Learning.

Macula fovea detection is a crucial prerequisite towards screening and diagnosing macular diseases. Without early detection and proper treatment, any abnormality involving the macula may lead to blindness. However, with the ophthalmologist shortage and time-consuming artificial evaluation, neither accuracy nor effectiveness of the diagnose process could be guaranteed. In this project, we proposed a deep learning approach on ultra-widefield fundus (UWF) images for macula fovea detection. This study collected 2300 ultra-widefield fundus images from Shenzhen Aier Eye Hospital in China. Methods based on U-shape network (Unet) and Fully Convolutional Networks (FCN) are implemented on 1800 (before amplifying process) training fundus images, 400 (before amplifying process) validation images and 100 test images. Three professional ophthalmologists were invited to mark the fovea. A method from the anatomy perspective is investigated. This approach is derived from the spatial relationship between macula fovea and optic disc center in UWF. A set of parameters of this method is set based on the experience of ophthalmologists and verified to be effective. Results are measured by calculating the Euclidean distance between proposed approaches and the accurate grounded standard, which is detected by Ultra-widefield swept-source optical coherence tomograph (UWF-OCT) approach. Through a comparation of proposed methods, we conclude that, deep learning approach of Unet outperformed other methods on macula fovea detection tasks, by which outcomes obtained are comparable to grounded standard method.

Chest X-ray (CXR) is the first tool globally employed to detect cardiopulmonary pathologies. These acquisitions are highly affected by scattered photons due to the large field-of-view required. Scatter in CXRs introduces background in the images, which reduces their contrast. We developed three deep learning-based models to estimate and correct the scatter contribution to CXRs. We used a Monte Carlo (MC) ray-tracing model to simulate CXRs from human models obtained from CT scans using different configurations (depending on the availability of dual-energy acquisitions). The simulated CXRs contained the separated contribution of direct and scattered X-rays in the detector. These simulated datasets were then used as the reference for the supervised training of several NN. Three NN models (single and dual energy) were trained with the MultiResUNet architecture. The performance of the NN models was evaluated on CXRs obtained with the MC code from chest CTs of patients affected by COVID-19. The results showed that the NN models were able to estimate and correct the scatter contribution to CXRs with an error &lt;5%, being robust to variations in the simulation setup and improving the contrast in soft tissue. The single-energy model was tested with real CXRs, providing robust estimations of the scatter-corrected CXRs.

With metal oxide semiconductors being a promising candidate for accurate indoor air quality assessments, multiple drawbacks of the gas sensors prevent their widespread use. Examples include poor selectivity, instability over time, and sensor poisoning. Complex calibration methods and advanced operation modes can solve some of those drawbacks. However, this leads to long calibration times, which are unsuitable for mass production. In recent years, multiple attempts to solve calibration transfer have been made with the help of direct standardization, orthogonal signal correction, and many more methods. Besides those, a new promising approach is transfer learning from deep learning. This article will compare different calibration transfer methods, including direct standardization, piecewise direct standardization, transfer learning for deep learning models, and global model building. The machine learning methods to calibrate the initial models for calibration transfer are feature extraction, selection, and regression (established methods) and a custom convolutional neural network TCOCNN. It is shown that transfer learning can outperform the other calibration transfer methods regarding the root mean squared error, especially if the initial model is built with multiple sensors. It was possible to reduce the number of calibration samples by up to 99.3 % (from 10 days to approximately 2 hours) and still achieve an RMSE for acetone of around 18 ppb (15 ppb with extended individual calibration) if six different sensors were used for building the initial model. Furthermore, it was shown that the other calibration transfer methods (direct standardization and piecewise direct standardization) also work reasonably well for both machine learning approaches, primarily when multiple sensors are used for the initial model.

Deep learning has been applied in many areas that have had a significant impact on applications that improves real-life challenges. The success of deep learning in a wide range of areas is due in part to the use of activation functions, which are particularly effective at solving non-linear problems. Activation functions are a key focus for researchers in artificial intelligence who aim to improve the performance of neural networks. This article provides a comprehensive explanation and comparison of different activation functions, with a focus on the arc tangent and its variations specifically. The paper presents experimental results that show that variations of the arc tangent using irrational numbers such as pi, the golden ratio, and Euler’s number, as well as a self-arctan function, produce promising results. Since we experimented with promising activation functions on two different problems, and datasets, we reached a result that different irrationals work well for different problems. In other words, arctan ϕ gives the best results mostly for multiclass classification and arctan e gives the best results for time series prediction problems. The paper focuses on a multiclass classification problem applied to the Reuters Newswire dataset and a time-series prediction problem on Türkiye energy trade value to show the impacts of activation functions.

The work presented in this paper makes multiple scientific contributions related to the investigation of the global fear associated with COVID-19 by performing a comprehensive analysis of a dataset comprising survey responses of participants from 40 countries. First, the results of subjectivity analysis of responses where participants indicated their biggest concern related to COVID-19 showed that the average subjectivity in responses by the age group of 41-50 decreased from April 2020 to June 2020, the average subjectivity in responses by the age group of 71-80 drastically increased from May 2020, and the age group of 11-20 indicated the least level of subjectivity in their responses between June 2020 to August 2020. Second, subjectivity analysis also revealed the percentage of highly opinionated, neutral opinionated, and least opinionated responses per age-group where the analyzed age groups were 11-20, 21-30, 31-40, 41-50, 51-60, 61-70, 71-80, and 81-90. For instance, the percentage of highly opinionated, neutral opinionated, and least opinionated responses by the age group of 11-20 were 17.92%, 16.24%, and 65.84%, respectively. Third, data analysis of responses from different age groups showed that the highest percentage of responses indicating that they were very worried about COVID-19 came from individuals in the age group of 21-30. Fourth, data analysis of the survey responses also revealed that in the context of taking precautions to prevent contracting COVID-19, the percentage of individuals in the age group of 31-40 taking precautions was higher as compared to the percentages of individuals from the age groups of 41-50, 51-60, 61-70, 71-80, and 81-90. Finally, a deep learning model was developed to detect if the survey respondents were seeing or planning to see a psychologist or psychiatrist for any mental health issues related to COVID-19. The deep learning model used the responses to multiple questions in the context of fear, preparedness, and response related to COVID-19 from the dataset and achieved an overall performance accuracy of 91.62% after 500 epochs.

Due to the digitization and Internet of Things revolutions, the present electronic world has a wealth of cybersecurity data. Efficiently resolving cyber anomalies and attacks is becoming a growing concern in today's cyber security industry all over the world. Traditional security solutions are insufficient to address contemporary security issues due to the rapid proliferation of many sorts of cyber-attacks and threats. Utilizing artificial intelligence knowledge, especially machine learning technology, is essential to providing a dynamically enhanced, automated, and up-to-date security system through analyzing security data. In this paper, we provide an extensive view of machine learning algorithms, emphasizing how they can be employed for intelligent data analysis and automation in cybersecurity through their potential to extract valuable insights from cyber data. We also explore a number of potential real-world use cases where data-driven intelligence, automation, and decision-making enable next-generation cyber protection that is more proactive than traditional approaches. The future prospects of machine learning in cybersecurity are eventually emphasized based on our study, along with relevant research directions. Overall, our goal is to explore not only the current state of machine learning and relevant methodologies but also their applicability for future cybersecurity breakthroughs.

One of the most common and important destructive attacks on the victim system is Advanced Persistent Threats (APT)-attack. The APT attacker can achieve his hostile goals by obtaining information and gaining financial benefits regarding the infrastructure of a network. One of the solutions to detect a secret APT attack is using network traffic. Due to the nature of the APT attack in terms of being on the network for a long time and the fact that the network may crash because of high traffic, it is difficult to detect this type of attack. Hence, in this study, machine learning methods such as C5.0 decision tree, Bayesian network and deep neural network are used for timely detection and classification of APT-attacks on the NSL-KDD data set. Moreover, 10-fold cross validation method is used to experiment these models. As a result, the accuracy (ACC) of the C5.0 decision tree, Bayesian network and 6-layer deep learning models is obtained as 95.64%, 88.37% and 98.85%, respectively, and also, in terms of the important criterion of the false positive rate (FPR), the FPR value for the C5.0 decision tree, Bayesian network and 6-layer deep learning models is obtained as 2.56, 10.47 and 1.13, respectively. Other criterions such as sensitivity, specificity, accuracy, false negative rate and F-measure are also investigated for the models, and the experimental results show that the deep learning model with automatic multi-layered extraction of features has the best performance for timely detection of an APT-attack comparing to other classification models.

In the current natural environment, due to the complexity of the background and the high similarity of the color between immature green tomato and plant, the occlusion of the key organs (flower and fruit) by the leaves and stems will lead to low recognition rate and poor generalization of the detection model. Therefore, an improved tomato organ detection method based on convolutional neural network has been proposed in this paper. Based on the original Faster R-CNN algorithm, Resnet-50 with residual blocks was used to replace the traditional vgg16 feature extraction network, and K-means clustering method was used to adjust more appropriate anchor size than manual setting to improve detection accuracy. A variety of data augmentation techniques were used to train the network. The test results showed that compared with the traditional Faster R-CNN model, the mean average precision (mAP) of the optimal model was improved from 85.2% to 90.7%, the memory requirement decreased from 546.9MB to 115.9 MB, and the average detection time was shortened to 0.073S/sheet. As the performance greatly improved, the training model can be transplanted to the embedded system, which lays a theoretical foundation for the development of precise targeting pesticide application system and automatic picking device.

Change detection with heterogeneous remote sensing images (Hete-CD) plays an important role in practical applications, especially when homogenous remote sensing images are unavailable. However, bitemporal heterogeneous remote sensing images (HRSIs) cannot compare directly to measure change magnitude, and many deep learning methods require large amounts of samples to train the module. Moreover, labeling many samples for land cover change detection with HRSIs is time-consuming and labor-intensive. Therefore, acquiring satisfactory performance of Hete-CD remains a challenge for deep learning networks with very small training samples. In this study, we promote a novel deep-learning framework for Hete-CD to obtain satisfactory performance while the initial samples are very small. We initially design a multiscale network with select kernel-attention module to focus on capturing different change targets with various sizes and shapes. Then, a simple yet effective non-parametric sample-enhanced algorithm based on the Pearson correlation coeffi-cient is promoted to explore potential samples around each initial sample. Finally, the proposed network and sample-enhanced algorithm are fused into one iterative framework to improve the change detection performance with very small samples. Experimental results conducted on four pairs of actual HRSIs indicated that the proposed framework can achieve competitive accuracies with very small samples for initialization when compared with some state-of-the-art methods. For example, the improvement is approximately 3.38% and 1.99% when compared with the selected traditional methods and deep learning methods, respectively.

Recently, the number of vehicles on the road, especially in urban centers, has increased dramatically due to the increasing trend of individuals towards urbanization. As a result, manual detection and recognition of vehicles (i.e., license plates and vehicle manufacturer) become an arduous task and beyond human capabilities. In this paper, we have developed a system using transfer learning-based DL techniques for automatic identification of Jordanian vehicles. The YOLOv3 (You Only Look Once) model was re-trained using transfer learning to accomplish the license plate detection, character recognition, and vehicle logo detection. While VGG16 (Visual Geometry Group) model was retrained to accomplish the vehicle logo recognition. To train and test these models, four datasets have been collected. The first dataset consists of 7,035 Jordanian vehicle images, the second dataset consist of 7,176 Jordanian license plates, and the third dataset consists of 8,271 Jordanian vehicle images. These datasets have been used to train and test the YOLOv3 model for Jordanian license plate detection, character recognition, and vehicle logo detection, respectively. While the fourth dataset consists of 158,230 vehicle logo images used to train and test the VGG16 model for the vehicle logo recognition. Text measures were used to evaluate the performance of our developed system. Moreover, mean average precision (mAP) measure was used to evaluate the YOLOv3 model of the detection tasks (i.e., license plate detection and vehicle logo detection). For license plate detection, the precision, recall, F-measure, and mAP were 99.6%, 100%, 99.8%, and 99.9%, respectively. While for character recognition, the precision, recall, and F-measure were 100%, 99.9%, and 99.95%, respectively. The performance of license plate recognition stage was evaluated by evaluating these two sub-stages as a sequence, where the precision, recall, and F-measure were 99.8%, 99.8%, and 99.8%, respectively. Furthermore, for vehicle logo detection, the precision, recall, F-measure, and mAP were 99%, 99.6%, 99.3%, and 99.1%, respectively, while for vehicle logo recognition, the precision, recall, F-measure were 98%, 98%, and 98%, respectively. The performance of vehicle logo recognition stage was evaluated by evaluating these two sub-stages as a sequence, where the precision, recall, and F-measure were 95.3%, 99.5%, and 97.4%, respectively.

Digital Twin Cities (DTCs) play a fundamental role in city planning and management. They allow three-dimensional modeling and simulation of cities. 3D semantic segmentation is the foundation for automatically creating enriched DTCs, as well as their updates. Past studies indicate that prior level fusion approaches demonstrate more promising precisions in 3D semantic segmentation compared to point level fusion, features level fusion, and decision level fusion families. In order to improve point cloud enriched semantic segmentation outcomes, this article proposes a new approach for 3D point cloud semantic segmentation through developing and benchmarking three prior level fusion scenarios. A reference approach based on point clouds and aerial images was proposed to compare it with the different developed scenarios. In each scenario, we inject a specific prior knowledge (geometric features,classified images ,etc) and aerial images as attributes of point clouds into the neural network’s learning pipeline. The objective is to find the one that integrates the most significant prior knowledge and enhances neural network knowledge more profoundly, which we have named the "smart fusion approach". The advanced Deep Learning algorithm "RandLaNet" was adopted to implement the different proposed scenarios and the reference approach, due to its excellent performance demonstrated in the literature. The introduction of some significant features associated with the label classes facilitated the learning process and improved the semantic segmentation results that can be achievable with the same neural network alone. Overall, our contribution provides a promising solution for addressing some challenges, in particular more accurate extraction of semantically rich objects from the urban fabric. An assessment of the semantic segmentation results obtained by the different scenarios is performed based on metrics computation and visual investigations. Finally,the smart fusion approach was derived based on the obtained qualitative and quantitative results.

The new energy structure needs to balance energy security and dual carbon goals, which has brought major challenges to coal-fired power plants, the pollution reduction and carbon emissions reduction of coal-fired power plants will be a key task in the future. This article proposes an optimization operation of electrostatic precipitators(ESP), based on the working mechanism and historical data of ESP, voltage-current characteristic model and outlet dust concentration prediction model are constructed by deep learning algorithm, Particle Swarm Optimization(PSO) is used to achieve the optimal energy consumption while ensuring stable outlet dust concentration. By training with historical data collected on site, accurate prediction of secondary current and outlet dust concentration of ESP has been achieved, the Mean Absolute Percentage Error(MAPE) of secondary current is less than 1.5%, and the MAPE of outlet dust concentration of ESP on the test set is less than 5.2%. Finally, the optimization experiment is carried out in a 330MW coal-fired power plants, the results showed that the fluctuation of the outlet dust concentration is more stable, and the energy saving is about 43% after optimization, according to the annual operation of 300 days, the annual average carbon reduction is approximately 2621.34 tons. This method is effective and can be applied widely.

Deep learning has been applied in many areas that have had a significant impact on applications that improves real-life challenges. The success of deep learning in a wide range of areas is due in part to the use of activation functions, which are particularly effective at solving non-linear problems. Activation functions are a key focus for researchers in artificial intelligence who aim to improve the performance of neural networks. This article provides a comprehensive explanation and comparison of different activation functions, with a focus on the arc tangent and its variations specifically. The paper presents experimental results that show that variations of the arc tangent using irrational numbers such as pi, the golden ratio, and Euler’s number, as well as a self-arctan function, produce promising results. Since we experimented with promising activation functions on two different problems, and datasets, we reached a result that different irrationals work well for different problems. In other words, arctan ϕ gives the best results mostly for multiclass classification and arctan e gives the best results for time series prediction problems. The paper focuses on a multi-class classification problem applied to the Reuters Newswire dataset and a time-series prediction problem on Türkiye energy trade value to show the impacts of activation functions.

One of the critical problems in multiclass classification tasks is the imbalance of the dataset. This is especially true when using contemporary pre-trained neural networks, where, in fact, the last layers of the neural network are retrained. Therefore, the large datasets with highly unbalanced classes are not good for models’ training since the use of such a dataset leads to overfitting and, accordingly, poor metrics on test and validation datasets. In this paper the sensitivity to a dataset imbalance of Xception, ViT-384, ViT-224, VGG19, ResNet34, ResNet50, ResNet101, Inception_v3, DenseNet201, DenseNet161, DeIT was studied using a highly imbalanced dataset of 20,971 images sorted into 7 classes. It is shown that the best metrics were obtained when using a cropped dataset with augmentation of missing images in classes up to 15% of the initial number. So, the metrics can be increased by 2-6% compared to the metrics of the models on the initial unbalanced data set. Moreover, the metrics of the rare classes' classification also improved significantly – the TruePositive value can be increased by 0.3 and more. As result, the best approach to train considered networks on an initially unbalanced dataset was formulated.

This study aims to evaluate a new approach in modeling gully erosion susceptibility based on deep learning neural network (DLNN) model, ensemble Particle swarm optimization (PSO) algorithm with DLNN (PSO-DLNN) and comparing these approaches with common artificial neural network (ANN) and support vector machine (SVM) models in Shiran watershed, Iran. For this purpose, 13 independent variables affecting gully erosion susceptibility in the study area, including altitude, slope, aspect, plan curvature, profile curvature, drainage density, distance from river, land use, soil, lithology, rainfall, , stream power index (SPI), topographic wetness index (TWI), were prepared. Also, 132 gully erosion locations were identified during field visits. Data for modeling were divided into two categories of training (70%) and testing (30%). Receiver operating characteristic (ROC) parameters including sensitivity, specificity, negative predictive value (NPV), positive predictive value (PPV) and area under curve (AUC) were used to evaluate the performance of the models. The results showed that, the AUC values from ROC with considering testing datasets of PSO-DLNN is 0.89 and which is associated with superb accuracy. Rest of the models also associated with optimal accuracy and near about PSO-DLNN model; the AUC values from ROC of DLNN, SVM and ANN for testing datasets are 0.87, 0.85 and 0.84 respectively. The PSO algorithm has updated and optimized the weights of DLNN model, and as a result, the efficiency of this model in predicting gully erosion susceptibility has increased. Therefore, it can be concluded that the use of DLNN model and its ensemble with PSO algorithm can be used as a novel and practical method in predicting the susceptibility of gully erosion that helps planners and managers in managing and reducing the risk of this phenomenon.

Hybrid forecasting methods have emerged as a solution surpassing the limitations of both statistical and deep learning approaches. While the first emphasize the significance of variables, they often produce worse forecasting results when compared to newer techniques. In contrast, deep learning models remain enigmatic "black boxes" in terms of interpretability, although achieving better results in forecasting. This article introduces the Comprehensive Cross-Correlation and Lagged Linear Regression Deep Learning (CCLR-DL) framework, designed to harness the best of both approaches, enhancing forecasting accuracy while retaining model interpretability through a feature selection process. CCLR-DL blens cross-correlation analysis, lagged multiple linear regression and granger's causality procedures with deep learning architectures based on LSTM. In a practical demonstration, CCLR-DL was applied to a real database of clinical visits associated to diagnoses in Catalonia, Spain (tracking a population of 6.3 million patients during 10 years). Predicting visits enables the healthcare managers to be ready for future demand shifts. Results demonstrate a consistent and substantial improvement over standalone statistical and deep learning methods when predicting healthcare demand. This hybrid approach not only showcases its efficacy but also offers a promising solution to the challenge of balancing predictive accuracy with model explicability. In this context, this work aims to design and validate a method for feature selection and forecasting of multivariate high dimensional time series datasets not only to improve prediction accuracy but also to model transparency by identifying a subset of variables that improve predictions and G-cause the target variable.

Current stock market forecasting methods encompass fundamental, technical, emotional, and bargaining factors. Predominantly, price prediction hinges on order volume and price, although correlating these two within existing models proves challenging. This study employs Cycle Generative Adversarial Network (Cycle GAN) to unravel the intricate price-volume relationship, combining it with Bollinger Bands for trading signal analysis, overcoming hurdles in short-term forecasting prevalent in numerical analysis and AI. Focusing on TSMC (2330.TW) stock price, the research leverages Cycle GAN in deep learning to master the price-volume nexus, juxtaposed with LSTM and RNN. Historical TSMC closing prices and transaction counts are model inputs, scrutinizing their interconnectedness for predictions. This innovative approach aligns stock price, volume, market value, taxes, and prior changes via system engineering. By intertwining Bollinger Bands with stock price forecasts, trading signals are distilled, factoring in extended index %b for a comprehensive market picture.

This study employs the reflection symmetry decomposition (RSD) method to extract polarization scattering features from ground object images, aiming to determine the optimal data input scheme for deep learning networks in polarimetric synthetic aperture radar classification. Eight distinct polarizing feature combinations are designed, and the classification accuracy of various approaches is evaluated using the classic convolutional neural networks(CNNs) AlexNet and VGG16. The findings reveal that the commonly employed 6-parameter input scheme, favored by many researchers, lacks the comprehensive utilization of polarization information and warrants attention. Intriguingly, leveraging the complete 9-parameter input scheme based on the polarization coherence matrix results in improved classification accuracy. Furthermore, the input scheme incorporating all 21 parameters from the RSD and polarization coherence matrix notably enhances overall accuracy and the Kappa coefficient compared to the other 7 schemes. This comprehensive approach maximizes the utilization of polarization scattering information from ground objects, emerging as the most effective CNN input data scheme in this study. Additionally, the classification performance using the second and third component total power values (P2 and P3) from the RSD surpasses the approach utilizing surface scattering power value (PS) and secondary scattering power value (PD) from the same decomposition.

The introduction of non-conventional renewable energies (photovoltaic and wind, in the residential case) demand new proposals and opportunities to obtain a domestic energy management system (HEMS), which allows reducing the use of electrical energy. The HEMS incorporates artificial intelligence (AI) techniques to respond to energy demand (DR), which can control, switch, turn on and off, modifying the consumption profile, reducing monthly billing, which as a consequence brings improvement. of the quality of life at home. Based on what has been described, a model for identifying electrical appliances and determining consumption patterns for the home is proposed, using intrusive measurement/actuator equipment called Smart Socket. The Smart Socket measures the electrical variables of voltage and current in real time, calculating the powers and active-reactive-apparent energies of each household appliance (or the most relevant ones). The information provided by the Smart Socket is used as a basis to configure the HEMS, with the aim of optimizing energy use using artificial intelligence techniques and tools, thus reducing energy consumption, CO2 emissions and the user's monthly billing. residential.

Timely and accurate information on tree species is crucial for the sustainable management of natural resources, forest inventory, biodiversity detection, and carbon stock calculation. The advancement of remote sensing technology and artificial intelligence has facilitated the acquisition and analysis of remote sensing data, resulting in more precise and effective classification of tree species. Multimodal remote sensing data and deep learning seem to be the current tree species classification research mainstream, whether or not. The current review on the remote sensing data and deep learning tree species classification methods perspectives to analyze the unimodal and multimodal remote sensing data and classification methods in this realm is missing. To bridge the gap, we search for major trends in the remote sensing data and tree species classification methods, provide a detailed overview of classic deep learning-based methods for tree species classification, and discuss the limitations.

Objective: Radiomics and deep transfer learning are two popular technologies used to develop computer-aided detection and diadnosis (CAD) schemes of medical images. This study aims to investigate and compare advantages and potential limitations of applying these two technologies in developing CAD schemes. Methods: A relatively large and diverse retrospective dataset including 3,000 digital mammograms is assembled in which 1,496 images depict malignant lesions and 1,054 images depict benign lesions. Two CAD schemes are developed to classify breast lesions. The first scheme is developed using four steps namely, applying an adaptive multi-layer topographic region growing algorithm to segment lesions, computing initial radiomics features, applying a principal component algorithm to generate an optimal feature vector, and building a support vector machine classifier. The second CAD scheme is built based a pre-trained residual net architecture (ResNet50) as a transfer learning model to classify breast lesions. Both CAD schemes are trained and tested using a 10-fold cross-validation method. Several score fusion methods are also investigated to classify breast lesions. CAD classification performance is evaluated by the area under ROC curve (AUC). Results: ResNet50 model-based CAD scheme yields AUC = 0.85±0.02, which is significantly higher than radiomics feature-based CAD scheme with AUC = 0.77±0.02 (p < 0.01). Additionally, fusion of classification scores generated by two CAD schemes does not further improve classification performance. Conclusion: This study indicates that using deep transfer learning is more efficient to develop CAD schemes and enables to yield higher lesion classification performance than CAD schemes developed using radiomics-based technology.

Commercial buildings are a significant consumer of energy worldwide. Logistics facilities, and specifically warehouses, are a common building type yet under-researched in the demand-side energy forecasting literature. Warehouses have an idiosyncratic profile when compared to other commercial and industrial buildings with a significant reliance on a small number of energy systems. As such, warehouse owners and operators are increasingly entering in to energy performance contracts with energy service companies (ESCOs) to minimise environmental impact, reduce costs, and improve competitiveness. ESCOs and warehouse owners and operators require accurate forecasts of their energy consumption so that precautionary and mitigation measures can be taken. This paper explores the performance of three machine learning models (Support Vector Regression (SVR), Random Forest, and Extreme Gradient Boosting (XGBoost)), three deep learning models (Recurrent Neural Networks (RNN), Long Short-Term Memory (LSTM), and Gated Recurrent Unit (GRU)), and a classical time series model, Autoregressive Integrated Moving Average (ARIMA) for predicting daily energy consumption. The dataset comprises 8,040 records generated over an 11-month period from January to November 2020 from a non-refrigerated logistics facility located in Ireland. The grid search method was used to identify the best configurations for each model. The proposed XGBoost models outperform other models for both very short load forecasting (VSTLF) and short term load forecasting (STLF); the ARIMA model performed the worst.

Individual investors often struggle to predict stock prices due to the limitations imposed by the computational capacities of personal laptop Graphics Processing Units (GPUs) when running intensive deep learning models. This study proposes solving these GPU constraints by integrating deep learning models with technical analysis methods. This integration significantly reduces analysis time and equips individual investors with the ability to identify stocks that may yield potential gains or losses in an efficient manner. Thus, a comprehensive buy and sell algorithm, compatible with average laptop GPU performance, is introduced in the study. This algorithm offers a lightweight analysis method that emphasizes factors identified by technical analysis methods, thereby providing a more accessible and efficient approach for individual investors. To evaluate the efficacy of this approach, we analyzed the performance of eight deep learning models: LSTM (4 layers), CNN, BiLSTM (4 layers), CNN Attention, BiGRU CNN BiLSTM Attention, BiLSTM Attention CNN, CNN BiLSTM Attention, and CNN Attention BiLSTM. These models were used to predict stock prices for Samsung Electronics and Celltrion Healthcare. The CNN Attention BiLSTM model displayed superior performance among these models, with the lowest validation mean absolute error value. In addition, an experiment was conducted using WandB Sweep to determine the optimal hyperparameters for four individual hybrid models. These optimal parameters were then implemented in each model to validate their back-testing rate of return. The CNN Attention BiLSTM hybrid model emerged as the highest-performing model, achieving an approximate rate of return of 5 percent. Overall, this study offers valuable insights into the performance of various deep learning and hybrid models in predicting stock prices. These findings can assist individual investors in selecting appropriate models that align with their investment strategies, thereby increasing their likelihood of success in the stock market.

Protein phosphorylation and ADP-ribosylation (ADPr), as two types of post-translational modifications (PTM), are the process of adding phosphate group and ADP-ribose moieties to proteins, respectively. Although both PTM types can occur on many amino acid types, serine is the most common. Serine phosphorylation (pS), serine ADPr (SADPr), and their in situ crosstalks (pSADPr) play essential roles in biological processes. Although in silico classifiers have been developed for predicting pS and SADPr sites, the classifier for predicting pSADPr sites is unavailable. In this study, we developed classifiers to predict pSADPr sites. Specifically, we collected 3250 human pSADPr, 7520 SADPr, 151,227 pS and 80,096 unmodified serine sites. Based on them, we investigated the characteristics of pSADPr sites and constructed three classifiers to predict pSADPr sites from the pS dataset, the SADPr dataset and the protein sequences separately. We built and evaluated five deep-learning classifiers in ten-fold cross-validation and independent test datasets. Three of them (e.g. Convolutional Neural Network with the One-Hot encoding, dubbed CNNOH) performed better than the rest two. For instance, CNNOH had the AUC values of 0.700, 0.914 and 0.954 for recognizing pSADPr sites from the SADPr, pS and unmodified serine sites.Therefore, it is challenging to distinguish pSADPr sites from SADPr sites compared to the other two. It is consistent with our observation that pSADPr's characteristics are more similar to those of SADPr than the rest. Furthermore, we used the classifiers as base classifiers to develop a few stacking-based ensemble classifiers to improve performance. However, none of the ensemble classifiers showed better performances, suggesting that the base classifiers have good enough performances. Finally, we developed an online tool for extensively predicting human pSADPr sites based on the CNNOH classifier, dubbed EdeepSADPr. It is freely available through http://edeepsadpr.bioinfogo.org/.

Text classification is one of the most important task in natural language processing, In this research, we carried out several experimental research on three (3) of the most popular Text classification NLP classifier in Convolutional Neural Network (CNN), Multinomial Naive Bayes (MNB), and Support Vector Machine (SVN). In the presence of enough training data, Deep Learning CNN works best in all parameters for evaluation with 77% accuracy, followed by SVM with accuracy of 76%, and multinomial Bayes with least performance of 69% accuracy. CNN has the best performance in the presence of large enough training dataset because of the presence of filter/ kernels which help to indentify patterns in text data regardless of their position in the sentence. We repeated the training again with just one-third of our data, at this point SVM comes with the best performance, the performance of CNN noticeably drops but still better than Multinomial Naive Bayes, the reason why SVM works best when we reduce the training data was because of its ability to look for a hyper-plane that creates a boundary between different classes of data so as to properly classify them, so we believed that getting the hyper-plane was more efficient when we reduce the dataset, hence reason for the good performance. Multinomial Naive Bayes have the least performance which we attributed to its assumption of independency between the features which sometimes does not hold true. We concluded that availability of data should be an important factor when choosing classifier for Natural Language Processing Text Classification task. CNN should be use in the presence of enough dataset, and SVM should be use when data is not enough. Multinomial Naive Bayes must not be trusted with state of the art NLP task due to its assumption of independency between the features.

：An effective technology for predicting the end-point phosphorous content of basic oxygen furnace (BOF) can provide theoretical instruction to improve the quality of steel via controlling the hardness and toughness. Given the slightly inadequate prediction accuracy in the existing prediction model, a novel hybrid method was suggested to more accurately predict the end-point phosphorus content by integrating an enhanced sparrow search algorithm (ESSA) and a multi-strategy with a deep extreme learning machine (DELM) as ESSA-DELM in this study. To begin with, the input weights and hidden biases of DELM were randomly selected, resulting in that DELM inevitably had a set of non-optimal or unnecessary weights and biases. Therefore, the ESSA was used to optimize the DELM in this work. For the ESSA, the Trigonometric substitution mechanism and Cauchy mutation were introduced to avoid trapping in local optima and improve the global exploration capacity in SSA. Finally, to evaluate the prediction efficiency of ESSSA-DELM, the proposed model was tested on process data of the converter from the Baogang steel plant. The efficacy of ESSA-DELM was more superior to that of other DELM-based hybrid prediction models and conventional models. The result demonstrated that the hit rate of end-point phosphorus content within ±0.003%, ±0.002%, and ±0.001% was 91.67%, 83.33%, and 63.55%, respectively. The proposed ESSA-DELM model could possess better prediction accuracy compared with other models, which could guide field operations.

The ongoing revolution in Deep Learning is redefining the nature of computing that is driven by the increasing amount of pattern classification and cognitive tasks. Specialized digital hardware for deep learning still holds its predominance due to the flexibility offered by the software implementation and maturity of algorithms. However, it is being increasingly desired that cognitive computing occurs at the edge, i.e. on hand-held devices that are energy constrained, which is energy prohibitive when employing digital von Neumann architectures. Recent explorations in digital neuromorphic hardware have shown promise, but offer low neurosynaptic density needed for scaling to applications such as intelligent cognitive assistants (ICA). Large-scale integration of nanoscale emerging memory devices with Complementary Metal Oxide Semiconductor (CMOS) mixed-signal integrated circuits can herald a new generation of Neuromorphic computers that will transcend the von Neumann bottleneck for cognitive computing tasks. Such hybrid Neuromorphic System-on-a-chip (NeuSoC) architectures promise machine learning capability at chip-scale form factor, and several orders of magnitude improvement in energy efficiency. Practical demonstration of such architectures has been limited as performance of emerging memory devices falls short of the expected behavior from the idealized memristor-based analog synapses, or weights, and novel machine learning algorithms are needed to take advantage of the device behavior. In this work, we review the challenges involved and present a pathway to realize ultra-low-power mixed-signal NeuSoC, from device arrays and circuits to spike-based deep learning algorithms, with ‘brain-like’ energy-efficiency.

(1) Background: In the year of 2020 Covid-19 was declared epidemic by WHO. From that time millions of people were affected and died by this disease. The main detection process for this epidemic is RT-PCR test or reverse polymerase transcription chain reaction test. One of the reason of spreading of this disease so much is lack of efficiency in RT-PCR test. Sampling error and low viral load were two main reasons for what the testing process faced such problems. Lung infection is a very common symptom for covid-19 patients, so, CT scan and chest X-ray imaging technique can be applied to detect patient at a very early stage of infection. Which will be very effective and also can be a better option to RT-PCR test; (2) Methods: We searched data in Scopus for articles published between 2020 and 2023. The initial set of articles was 189, from which 21 articles will eventually be selected by exclusion criteria; (3) Results: A total of thirteen (61.90%) articles were found working on detecting Covid-19 by extracting the X-ray and CT scan data individually. Three (14.28%) of those articles focused on a hybrid model of CT scan and X-ray Image Data. Another four articles made a comparison between Covid-19, pneumonia and a normal person to identify a Covid-19 patient. Where others have worked on unsupervised learning methods or SVM to detect Covid-19.; (4) Conclusions: We have conducted a systematic review of the studies that have been published up to this time, with the purpose to present a summary of the evidence about the detection of COVID-19. In this article, we have summarized and critically reviewed literatures which worked on development or application or both of different AI or ML technique from chest CT and X-ray images to find a solution to detect covid-19.

Although commercially-available low-cost air quality sensors have low accuracy, the sensor system are being used to collect the data for the regulation of PM2.5 emission caused by industrial activities or to estimate the personal exposure for PM2.5. In this work, to solve the accuracy problem of low-cost PM sensor, we developed a new PM2.5 calibration model by combining the deep neural network (DNN) optimized in calibration problem and a LSTM optimized in time-dependent characteristics. First, two datasets were generated to test the accuracy performance and generalization performance of the PM2.5 calibration machine learning (ML) model. The PM2.5 concentrations, temperature and humidity by low-cost sensor and gravimetric-based PM2.5 measuring instrument were sampled for a sufficiently long time. The proposed model was compared with benchmark (multiple linear regression model) and low-cost sensor results. For root mean square error (RMSE) for PM2.5 concentrations, the proposed model reduced 41-60% of error compared to the raw data of low-cost sensor, and reduced 30-51% of error compared to the benchmark model. R2 of ML model, MLR and raw data were 93, 80 and 59 %. Also, the developed model still showed consistent calibration performance when calibrated with new sensors in different locations. Low-cost sensors combined with ML model not only can improve the calibration performance of benchmark, but also can be applied to the sensor monitoring systems for various epidemiologic investigations and regulatory decisions.

Physicians often label anatomical structure sets in Digital Imaging and Communications in Medicine (DICOM) images with nonstandard names. As these names vary widely, the standardization of the nonstandard names in the Organs at Risk (OARs), Planning Target Volumes (PTVs), and 'Other' organs inside the area of interest is a vital problem. Prior works considered traditional machine learning approaches on structure sets with moderate success. This paper presents integrated deep learning methods applied to structure sets by integrating the multimodal data compiled from the radiotherapy centers administered by the US Veterans Health Administration (VHA) and the Department of Radiation Oncology at Virginia Commonwealth University (VCU). The de-identified radiation oncology data collected from VHA and VCU radiotherapy centers have 16,290 prostate structures. Our method integrates the heterogeneous (textual and imaging) multimodal data with Convolutional Neural Network (CNN)-based deep learning approaches like CNN, Visual Geometry Group (VGG) network, and Residual Network (ResNet). Our model presents improved results in prostate (RT) structure name standardization. Evaluation of our methods with macro-averaged F1 Score shows that our deep learning model with single-modal textual data usually performs better than the previous studies. We also experimented with various combinations of multimodal data (masked images, masked dose) besides textual data. The models perform well on the textual data alone, while the addition of imaging data shows that deep neural networks can achieve improved performance using information present in the other modalities. Additionally, using masked images and masked doses along with text leads to an overall performance improvement with the various CNN-based architectures than using all the modalities together. Undersampling the majority class leads to further performance enhancement. The VGG network on the masked image-dose data combined with CNNs on the text data performs the best and establishes the state-of-the-art in this domain.

This study presents a novel semi-supervised deep learning method utilizing 3D TOF-MRA images for the detection of brain aneurysms, employing landmark-based techniques that mimic radiologist practices to enhance detection accuracy and efficiency.

Advancements in technology have revolutionized communication on social media platforms, where reviews and comments greatly influence product selection. Current opinion mining methods predominantly focus on textual content, overlooking the rich information within customer-posted images, termed here as Multus-Medium. This research introduces an innovative deep learning approach, Multus Medium Opinion Mining (MMOM), capitalizing on both text and image data for comprehensive product review analysis. MMOM employs an integrated model of Bidirectional Long Short-Term Memory (BiLSTM) and embedded Convolutional Neural Network (CNN), incorporating architectures of GoogleNet and VGGNet, thus enabling efficient extraction and fusion of textual and visual features. This synergistic approach enables data collection, preprocessing, feature extraction, fusion strategy-based feature vector generation, and subsequent product recommendation. Performance evaluation on two diverse real-world datasets name as “flicker8k” and “t4sa”. These datasets show substantial improvement over existing methods. MMOM outperforms standard benchmark models, achieving an accuracy, F1 score and ROC over fliker8k are 90.38%, 88.75% and 93.08%, whereas for twitter dataset are 88.54%, 86.34%, and 92.26% respectively, the accuracy of purposed model is 7.34% and 9.54% higher than the other two mentioned techniques. These statistics highlight the robustness and applicability of MMOM across various domains. The compelling results underscore the potential of MMOM, providing a more holistic and precise approach to opinion mining in the era of social media product reviews. This research, product recommendation helps the customer to make purchasing decision. Last but not the least, the purposed scheme can further be expanded in any other sentiment task like hospital recommendation system, crop farming recommendation and medical diagnostic system etc.

In this study, deep neural network (DNN) and transfer learning (TL) techniques were employed to predict the viscous resistance and wake distribution based on the positions of flow control fins (FCFs) applied to containerships of various sizes. Both methods utilized data collected through Computational Fluid Dynamics (CFD) analysis. The position of the flow control fin (FCF) and hull-form information were utilized as input data, and the output data included viscous resistance coefficients and components of propeller axial velocity. The base DNN model was trained and validated using a source dataset from a 1000 TEU containership. Grid search cross-validation technique was employed to optimize the hyperparameters of the base DNN model. Then, transfer learning was applied to predict the viscous resistance and wake distribution for containerships of varying sizes. To enhance the accuracy of feature prediction with a limited amount of dataset, learning rate optimization was conducted. Transfer learning involves retraining and reconfiguring the base DNN model, and the accuracy was verified based on the fine-tuning method of the learning model. The results of this study can provide hull designers for containerships with performance evaluation information by predicting wake distribution, without relying on CFD analysis.

The Internet of Things (IoT) is one of the most widely used technologies today, and it has a significant effect on our lives in a variety of ways, including social, commercial, and economic aspects. In terms of automation, productivity, and comfort for consumers across a wide range of application areas, from education to smart cities, the present and future IoT technologies hold great promise for improving the overall quality of human life. However, cyber-attacks and threats greatly affect smart applications in the environment of IoT. The traditional IoT security techniques are insufficient with the recent security challenges considering the advanced booming of different kinds of attacks and threats. Utilizing artificial intelligence (AI) expertise, especially machine and deep learning solutions, is the key to delivering a dynamically enhanced and up-to-date security system for the next-generation IoT system. Throughout this article, we present a comprehensive picture on IoT security intelligence, which is built on machine and deep learning technologies that extract insights from raw data to intelligently protect IoT devices against a variety of cyber-attacks. Finally, based on our study, we highlight the associated research issues and future directions within the scope of our study. Overall, this article aspires to serve as a reference point and guide, particularly from a technical standpoint, for cybersecurity experts and researchers working in the context of IoT.

In an industrial environment, the quality assurance of weld seams requires extensive efforts. The most commonly used methods for that are expensive and time-consuming destructive tests, since quality assurance procedures are difficult to integrate into production processes. Beyond that, available test methods allow only the assessment of a very limited set of characteristics. They are either suitable for determining selected geometric features or for locating and evaluating internal seam defects. The presented work describes an evaluation methodology based on microfocus X-ray computed tomography scans (µCT scans) which enable the 3D characterization of weld seams, including internal defects such as cracks and pores. A 3D representation of the weld contour, i.e., the complete geometry of the joint area in the component with all quality-relevant geometric criteria, is an unprecedented novelty. Both the dimensions of the weld seam and internal defects can be revealed, quantified with a resolution down to a few micrometers and precisely assigned to the welded component. On the basis of the methodology developed within the framework of this study, the results of the scans performed on the alloy AA 2219 can be transferred to other aluminum alloys. In this way, the data evaluation framework can be used to obtain extensive reference data for the calibration and validation of inline process monitoring systems employing Deep Learning-based data processing.