Deep Learning (DL) oriented document processing is widely used in different fields for extraction, recognition, and classification processes from raw corpora of data. The article examines the application of deep learning approaches, based on different neural network methods, including Gated Recurrent Unit (GRU), Long Short-Term Memory (LSTM), and Convolutional Neural Networks (CNN). The compared models were combined with two different word embedding techniques, namely: Bidirectional Encoder Representations from Transformers BERT and Gensim Word2Vec. The models are designed to evaluate the performance of architectures based on neural network techniques for the classification of CVs of Moroccan engineering students at ENSAK(National School of Applied Sciences of Kenitra, Ibn Tofail University). The used dataset included resumes collected from engineering students at ENSAK in 2023 for a project on the employability of Moroccan engineers in which new approaches were applied, especially machine learning, deep learning, and big data. Accordingly, 867 resumes were collected from five specialties of study (Electrical Engineering, Networks and Systems Telecommunications, Computer Engineering, Automotive Mechatronics Engineering, Industrial Engineering). The results revealed good performance of the proposed models based on the BERT embedding approach compared to models based on the Gensim Word2Vec embedding approach. Accordingly, the CNN-GRU/Bert model achieved slightly better accuracy with 0.9251 compared to other hybrid models.

Capability assessment plays a crucial role in the demonstration and construction of equipment. To improve the accuracy and stability of capability assessment, we study the neural network learning algorithms in the field of capability assessment and index sensitivity. Aiming at the problem of over-fitting and parameter optimization in neural network learning, the paper proposes an improved machine learning algorithm—the Ensemble Learning Based on Policy Optimization Neural Networks (ELPONN) algorithm with the policy optimization and ensemble learning. This algorithm presents optimized neural network learning algorithm through different strategies evolution, and builds an ensemble learning model of multi-intelligent algorithms to assessment the capability and analyze the sensitivity of the indexes. Through the assessment of capabilities, the algorithm effectively avoids parameter optimization from entering the minimum point in performance to improve the accuracy of equipment capability assessment, which is significantly better than previous neural network assessment methods. The experimental results show that the mean relative error is 4.10%, which is better than BP, GABP, and early stopping. The ELPONN algorithm has better accuracy and stability performance, and meets the requirements of capability assessment.

In the intricate domain of text analysis, syntactic dependency trees play a crucial role in unraveling the web of relations among entities embedded in the text. These trees provide a structural roadmap, guiding us through the complex syntax to pinpoint the interactions and connections between different entities. However, the challenge lies in sifting through this intricate structure to extract relevant information, a task that requires precision and discernment. Traditional approaches often rely on rule-based pruning methods to simplify these dependency structures, focusing on certain parts while discarding others. Yet, this approach has its pitfalls, as it can overlook critical nuances and connections that are vital for a comprehensive understanding of the text. Addressing this gap, our research introduces the Syntactic Dependency-Aware Neural Networks (SDANNs), a groundbreaking model designed to harness the full power of the entire dependency tree. This approach marks a significant departure from the conventional methods. Instead of the rigid rule-based pruning, SDANNs implement a more flexible and dynamic 'soft-pruning' technique. This method allows the model to adaptively focus on the sub-structures within the dependency tree that are most relevant for understanding the relationships between entities. By doing so, it ensures that no vital information is overlooked, and all potential connections are considered. The efficacy of SDANNs is not just theoretical but has been empirically validated through extensive testing and evaluations across a wide range of tasks. These tasks include the extraction of complex relations spanning multiple sentences, as well as detailed analyses at the sentence level. In each of these scenarios, SDANNs have demonstrated a remarkable ability to leverage the full structural complexity of dependency trees. This capability sets them apart from existing models, enabling a more nuanced and comprehensive analysis of textual relations. The results of these evaluations consistently show that SDANNs not only meet but significantly exceed the performance of prior models. This superiority is evident in the way SDANNs handle the multifaceted and often subtle interactions within the text, offering insights that were previously inaccessible with conventional methods. In summary, the Syntactic Dependency-Aware Neural Networks represent a significant advancement in the field of text analysis. By fully embracing the complexity of syntactic dependency trees and employing a sophisticated 'soft-pruning' approach, SDANNs open new avenues for exploring and understanding the intricate relationships that exist within written language. This model stands as a testament to the potential of combining advanced neural network architectures with a deep understanding of linguistic structures, paving the way for more accurate, nuanced, and comprehensive analyses of text.

Military medical service is an important component of the Armed Forces of any country [1]. In wartime, healthcare performs critical functions of medical support for military operations, providing effective and qualified care to the wounded and sick. Consequently, it is imperative to equip military hospitals with state-of-the-art materials and technical resources commensurate with the predefined requirements. In this study, we examine a series of enhancements applied to the TurtleBot 3 Waffle Pi-based medical assistant robot, tailored for deployment in military hospital settings.

Accurate and robust deep learning models for facial expression recognition are challenging to achieve, given the diversity of human faces and variations in images, including different facial poses and lighting conditions. In this work, we proposed a clock-driven convolutional Spiking Neural Network (SNN) and STDP fine-tune architecture, meticulously calibrated its hyperpa-rameters, and experimented with various optimization methods. The best model resulted was trained and evaluated on the Fer2013 and FER+ database, obtaining an accuracy of 61.87% and 79.97% without requiring auxiliary training data or face registration. To our best knowledge, the proposed SNN achieved comparable accuracy to CNNs of similar depth and possessed advantages of low energy consumption and high computational efficiency. The computational efficiency of the proposed SNN is approximately three times that of CNNs. Along with this, we introduced the very recent cumulative spike guided encoder visualization technique and revealed the strong encoding capability of the proposed SNN.

Emotion recognition defined as identifying human emotion and is directly related to different fields such as human-computer interfaces, human emotional processing, irrational analysis, medical diagnostics, data-driven animation, human-robot communi- cation and many more. The purpose of this study is to propose a new facial emotional recognition model using convolutional neural network. Our proposed model, “ConvNet”, detects seven specific emotions from image data including anger, disgust, fear, happiness, neutrality, sadness, and surprise. This research focuses on the model’s training accuracy in a short number of epoch which the authors can develop a real-time schema that can easily fit the model and sense emotions. Furthermore, this work focuses on the mental or emotional stuff of a man or woman using the behavioral aspects. To complete the training of the CNN network model, we use the FER2013 databases, and we test the system’s success by identifying facial expressions in the real-time. ConvNet consists of four layers of convolution together with two fully connected layers. The experimental results show that the ConvNet is able to achieve 96% training accuracy which is much better than current existing models. ConvNet also achieved validation accuracy of 65% to 70% (considering different datasets used for experiments), resulting in a higher classification accuracy compared to other existing models. We also made all the materials publicly accessible for the research community at: https://github.com/Tanoy004/Emotion-recognition-through-CNN.

Facial Emotion Recognition (FER) is crucial across psychology, neuroscience, computer vision, and machine learning due to the diversified and subjective nature of emotions, varying considerably across individuals, cultures, and contexts. This paper investigates the impact of histogram equalization, data augmentation and model optimization strategies on three well-known FER datasets using pre-trained VGG models. Additionally, this paper showcases the effectiveness of different regularization techniques, callbacks, and learning schedulers in enhancing model performance by conducting extensive experiments. The model evaluation is discussed in terms of the following metrics: Accuracy, Area Under the Receiver Operating Characteristic Curve (AUC-ROC), Area Under the Precision-Recall Curve (AUC-PRC), and Weighted F1 score. Notably, the fine-tuned VGG models exhibit state-of-the-art performance compared to more complex conventional transfer learning models resulting in accuracies of 100%, 95.92%, and 69.65% on the CK+, KDEF, and FER2013 datasets, respectively.

The Spiking Neural Network (SNN) has attracted more and more attention recently. It adopts binary spike signals to transmit information. Benefitting from the information passing paradigm of SNNs, the multiplications of activations and weights can be replaced by additions, which are more energy-efficient. However, its ``Hard Reset" mechanism for the firing activity would ignore the difference among membrane potentials when the membrane potential is above the firing threshold, causing information loss. Meanwhile, quantifying the membrane potential to 0/1 spikes at the firing instants will inevitably introduce the quantization error thus bringing about information loss too. To address these problems, we propose to use the ``Soft Reset" mechanism for the supervised training-based SNNs, which will drive the membrane potential to a dynamic reset potential according to its magnitude, and Membrane Potential Rectifier (MPR) to reduce the quantization error via redistributing the membrane potential to a range close to the spikes. Results show that the SNNs with the ``Soft Reset" mechanism and MPR outperform their vanilla counterparts on both static and dynamic datasets.

Neural networks require a large quantity of training spectra and detector responses in order to learn to solve the inverse problem of neutron spectrum unfolding. In addition, due to the under-determined nature of unfolding, non-physical spectra which would not be encountered in usage should not be included in the training set. While physically realistic training spectra are commonly determined experimentally or generated through Monte Carlo simulation, this can become prohibitively expensive when considering the quantity of spectra needed to effectively train an unfolding network. In this paper, we present three algorithms for the generation of large quantities of realistic and physically motivated neutron energy spectra. Using an IAEA compendium of 251 spectra, we compare the unfolding performance of neural networks trained on spectra from these algorithms, when unfolding real-world spectra, to two baselines. We also investigate general methods for evaluating the performance of and optimizing feature engineering algorithms.

The emergence of the Internet of Things (IoT) has led to the deployment of various types of sensors in many application fields, including environment monitoring, smart cities, health, industries, and others. The increasing number of connected devices has led to the creation of massive quantities of data that need to be analyzed. Typically, this data is ordered by time, as a time series. In this context, this paper presents a time series prediction model based on Recurrent Neural Networks in order to predict one step ahead. Result obtained through five Internet of Things monitoring datasets, showed that the Recurrent Neural Network obtained better performance that the prediction methods, ARIMA and SVM.

Although neural network architectures are critical for their performance, how the structural characteristics of a neural network affect its performance has still not been fully explored. We here map architectures of neural network to directed acyclic graphs, and find that incoherence, a structural characteristic to measure the order of directed acyclic graphs, is a good indicator for the performance of corresponding neural networks. Therefore we propose a deep isotropic neural network architecture by folding a chain of same blocks then connecting the blocks with skip connections at different distances. Our models, named FoldNet, have two distinguishing features compared with traditional residual neural netowrks. First, the distances between block pairs connected by skip connections increase from always equal to one to specially selected different values, which lead to more incoherent graphs and let the neural network explore larger receptive fields and thus enhance its multi-scale representation ability. Second, the number of direct paths increases from one to multiple, which leads to a larger proportion of shorter paths and thus improve the direct propagation of information throughout the entire network. Image classification results on CIFAR-10 and Tiny ImageNet benchmarks suggested that our new network architecture performs better than traditional residual neural networks.

Optical motion capture is a mature contemporary technique for the acquisition of motion data, alas it is non-error-free. Due to technical limitations and occlusions of markers, gaps might occur in such recordings. The article reviews various neural network architectures applied for gap filling problem in motion capture sequences within FBM framework providing the representation for body kinematic structure. The results are compared with interpolation and matrix completion methods. We found out, that for longer sequences simple linear feedforward neural networks can outperform the other, sophisticated architectures. We were also able to identify, that acceleration and monotonicity of input sequence are the parameters that have a notable impact on the obtained results.

We consider the problem of coordinating the charging of an entire fleet of electric vehicles (EV), using a model-free approach, i.e. purely data-driven reinforcement learning (RL). The objective of the RL-based control is to optimize charging actions, while fulfilling all EV charging constraints (e.g. timely completion of the charging). In particular, we focus on batch-mode learning and adopt fitted Q-iteration (FQI). A core component in FQI is approximating the Q-function using a regression technique, from which the policy is derived. Recently, a dueling neural networks architecture was proposed and shown to lead to better policy evaluation in the presence of many similar-valued actions, as applied in a computer game context. The main research contributions of the current paper are that (i)we develop a dueling neural networks approach for the setting of joint coordination of an entire EV fleet, and (ii)we evaluate its performance and compare it to an all-knowing benchmark and an FQI approach using EXTRA trees regression technique, a popular approach currently discussed in EV related works. We present a case study where RL agents are trained with an epsilon-greedy approach for different objectives, (a)cost minimization, and (b)maximization of self-consumption of local renewable energy sources. Our results indicate that RL agents achieve significant cost reductions (70--80%) compared to a business-as-usual scenario without smart charging. Comparing the dueling neural networks regression to EXTRA trees indicates that for our case study's EV fleet parameters and training scenario, the EXTRA trees-based agents achieve higher performance in terms of both lower costs (or higher self-consumption) and stronger robustness, i.e. less variation among trained agents. This suggests that adopting dueling neural networks in this EV setting is not particularly beneficial as opposed to the Atari game context from where this idea originated.

Condition monitoring of rotating shafts is essential for ensuring the reliability and optimal performance of machinery in diverse industries. In this context, as industrial systems become increasingly complex, the need for efficient data processing techniques is paramount. Deep learning has emerged as a dominant approach due to its capacity to capture intricate data patterns and relationships. However, a prevalent challenge lies in the black-box nature of many deep learning algorithms, which often operate without adhering to the underlying physical characteristics intrinsic to the studied phenomena. To address this limitation and enhance the fusion of data-driven methodologies with the fundamental physics of the system under study, this paper leverages physics-informed neural networks (PINNs). Specifically, a simple but realistic numerical case study of an extended Jeffcott rotor model, encompassing damping effects and anisotropic supports for a more comprehensive modelling, is considered. PINNs are used for the estimation of five parameters that characterize the health state of the system. These parameters encompass the radial and angular position of the static unbalance due to the disk installed on the shaft, the stiffness along the principal axes of elasticity, and the non-rotating damping coefficient. The estimation is conducted solely by exploiting the displacement signals from the centre of the disk and, to showcase the efficacy and precision provided by this novel methodology, various scenarios involving different constant rotational speeds are examined. Additionally, the impact of noisy input data is also taken into account within the analysis.

Retinopathy is a human eye disease that causes changes in retinal blood vessels that leads to bleed, leak fluid and vision impairment. Symptoms of retinopathy are blurred vision, changes in color perception, red spots, and eye pain. In this paper, a new methodology based on Convolutional Neural Networks (CNN) is developed and proposed to diagnose and give a decision about the presence of retinopathy. The CNN model is trained by different images of eyes that have retinopathy and those which do not have retinopathy. The performance of the proposed model is compared with the related methods of DREAM, KNN, GD-CNN and SVM. Experimental results show that the proposed CNN performs better.

Abstract Background and Objective: Medical Microwave Radiometry (MWR) is used to capture the thermal properties of internal tissues and has usages in breast cancer detection. Our goal in this paper is to improve classification performance and investigate automated neural architecture search methods. Methods: We investigate optimizing the weights of a weight agnostic neural network using bi-population covariance matrix adaptation evolution strategy (BIPOP-CMA-ES) once the topology is found. We compare it against a weight agnostic and cascade correlation neural network. Results: The experiments are conducted on a breast cancer dataset of 4912 patients. Our proposed weight agnostic BIPOP-CMA-ES model achieved the best performance. It obtained an F1-score of 0.9225, accuracy of 0.9219, precision of 0.9228, recall of 0.9217 and topology of 153 connections. Conclusions: The results are an indication of the potential of MWR utilizing a neural network-based diagnostic tool for cancer detection. By separating the tasks of topology search and weight training, we are able to improve the overall performance.

Artificial intelligence (AI) is a versatile term that is a conclusive remedy to solve the problem using past rational data after deep contemplation using these terms i-e basic statistics, carving data, familiarity with common AI algorithms. Seafood especially tiger prawn export as a busi-ness will provide enormous foreign exchange to any country if the farmers overcome the corre-lated vulnerabilities in prawn farming. This research is elucidating lacking in Tiger prawn (TP) farming like curbing of Oxygen, pH, water temperature, and nutrients, etc. Moreover, hatchery statistics in terms of juveniles will depict this study's clear picture of curbed aquaculture. For normative decisions, the Analytical Hierarchical Process (AHP) is used. The problem which has been faced by local prawn farmers that there is a stagnant TP growth in ponds, the reason is the predominant sensitivity factor in TP. For this reason, they need indemnification of thirteen fac-tors with natural resources to get the plausible results to get calmness in their lives. This study will solely focus on the TP growth model, and the monitoring effect will be established by the Artificial Intelligence algorithm. This study will employ the AHP, 0-1 scaling method, data cura-tion techniques, and ecological statistics. The life of Tiger Prawn (TP) depends upon these factors mainly, a) Physical and b) Chemical parameters. Physical parameters contain environment (E) provided to TP like season (S) and temperature (T) etc. whereas the quality of Ammonia NH3 (N) from fish waste, Oxygen level (O), and water quality hard & soft (W) lies in chemicals do-main. This research will Elucidate the factors which cause conceptual muddles in the aquamarine life of TP, for this reason, Statistical tools will assess the current result, forecast the gap. AHP will analyze the domain inputs, circumspect ramification which will depict visceral factors, later results depict which pond suits the TP. In curtail, these factors will be curbed to improve the growth of TP in a control conditioned environment.

Over the years, various models have been developed in the stages of the mining process that have allowed predicting and enhancing results, but it is the breakage the variable that connects all the activities of the mining process from the point of view of costs (drilling, blasting, loading, hauling, crushing and grinding). To improve this process, we come up with an idea to develop a breakage prediction model; on the basis of the main variables involved in the drilling and blasting process. For this purpose, we design a computer model based on an Artificial Neuronal Networks (ANN), built by using the most representative variables that come into play with drilling and blasting, such as: the properties of the explosives, the geomechanical parameters of the rock mass, and the design parameters of drilling-blasting. For its experimentation and validation, we have taken the data from a copper mine as reference located in the north of Chile, because of we have the dataset of that ore deposit, which is valid and reliable to evaluate prediction problems based on ANN applied to copper mines. The ANN architecture was of the supervised type, feedforward, with 3 layers and 13 neurons in the only hidden layer, trained with the input data using a dataset with the previously mentioned variables, which then were compared with the breakage results. The model was feed backed in its learning process until it becomes perfected, and is a prediction option that can be used in future blasting of ore deposits with similar characteristics using the same representative variables. Therefore, this is a valid alternative for predicting rock breakage, given that it has been experimentally validated, and has achieved moderately reliable results, providing higher correlation coefficients than traditional models, and with the additional advantage that an ANN model provides, due to its ability to learn and recognize compiled dataset patterns. In this way, using this computer model we can obtain satisfactory results that allow us to predict breakage, providing an alternative for evaluating the costs that this entails.

Optical artificial neural networks (ONNs) have significant potential for ultra-high computing speed and energy efficiency. We report a novel approach to ONNs that uses integrated Kerr optical micro-combs. This approach is programmable and scalable and is capable of reaching ultra-high speeds. We demonstrate the basic building block ONNs — a single neuron perceptron — by mapping synapses onto 49 wavelengths to achieve an operating speed of 11.9 x 10⁹ operations per second, or Giga-OPS, at 8 bits per operation, which equates to 95.2 gigabits/s (Gbps). We test the perceptron on handwritten-digit recognition and cancer-cell detection — achieving over 90% and 85% accuracy, respectively. By scaling the perceptron to a deep learning network using off-the-shelf telecom technology we can achieve high throughput operation for matrix multiplication for real-time massive data processing.

In the rapidly evolving field of real estate economics, the prediction of house prices continues to be a complex challenge, intricately tied to a multitude of socio-economic factors. However, traditional predictive models have often overlooked the spatial interdependencies that play a vital role in shaping housing prices. This study applies Graph Neural Networks (GNNs) on Open Statistics Knowledge Graphs to model spatial dependencies and predict house prices across Scotland’s 2011 data zones. To this end, integrated statistical indicators are retrieved from the official Scottish Open Government Data portal. The three representative GNN algorithms employed - ChebNet, GCN, and GraphSAGE - demonstrate higher prediction accuracy than traditional models, including the tabular-based XGBoost and a simple Multi-Layer Perceptron (MLP). In addition, local and global explainability are employed to increase transparency and trust in the predictions made by the most accurate GNN - GraphSAGE. The global feature importance is determined by a logistic regression surrogate model while the local, region-level understanding of the GNN predictions is achieved through the use of GNNExplainer. Explainaibility results are compared with those from a previous work that applied the XGBoost machine learning algorithm and the SHapley Additive exPlanations (SHAP) explainability framework on the same dataset. Interestingly, both global surrogate model and the SHAP approach underscored the Comparative Illness Factor, a health indicator, and the ratio of detached dwellings as the most crucial features in the global explainability. In the case of local explanations, while both methods showed similar results, the GNN approach provided a richer, more comprehensive understanding of the predictions for two specific data zones.

This article reviews the introduction of Alzheimer's Disease (AD), neural networks, training and learning of neural networks, applications of neural networks in early diagnosis of AD, applications of neural networks in AD drug discovery, other brain diseases, and challenges faced by AD. First, the paper introduces the background and characteristics of AD. AD is a degenerative neurological disorder characterized by impaired memory, decreased cognitive function, and loss of neurons. These characteristics place a huge burden on the lives and families of patients. Next, the basic principle and structure of neural network are discussed. A neural network is a computational model made up of multiple neurons that can perform tasks by learning and adapting to input data. In particular, the key concepts of neural network hierarchy, activation function and weight adjustment are discussed. Then, the training and learning methods of neural networks are discussed. Common techniques such as backpropagation algorithm and gradient descent optimizer are introduced in detail, as well as the importance of data preprocessing and model evaluation. Next, the paper focuses on the application of neural network in early diagnosis of AD. By extracting features from brain image data, neural networks can automatically identify differences between AD patients and healthy subjects, enabling early diagnosis and intervention. In addition, the application of neural networks in AD drug discovery is also discussed. By analyzing and predicting a database of known drugs, neural networks can help discover potential treatments for AD and speed up the drug discovery process. The paper further explores the application of neural networks in other brain diseases and highlights the challenges faced by AD, such as the lack of reliable biomarkers, complex pathological mechanisms, etc. In summary, this paper presents a systematic overview of AD, neural networks, training and learning of neural networks, applications of neural networks in early diagnosis of AD and drug discovery, and other brain diseases and challenges associated with AD.

The vast majority of autonomous driving systems are limited to applications on roads with clear lane markings and are implemented using commercial grade sensing systems coupled with specialized graphic accelerator hardware. This research reviews an alternative approach for autonomously steering vehicles that eliminates the dependency on road markings and specialized hardware. A combination of machine vision, machine learning and artificial intelligence based on popular pre-trained Convolutional Neural Networks (CNN) and Recurrent Neural Networks (RNN) was used to drive a vehicle along roads lacking lane markings (unmarked roads). The team developed and tested this approach on the ACTor (Autonomous Camus TranspORt) vehicle - an autonomous vehicle development and research platform coupled with a low-cost webcam based sensing system and minimal computational resources. The proposed solution was evaluated on real-world roads and varying environmental conditions. It was found that this solution may be used to successfully navigate unmarked roads autonomously with acceptable road following behavior.

State-of-the-art remote sensing scene classification methods employ different Convolutional Neural Network architectures for achieving very high classification performance. A trait shared by the majority of these methods is that the class associated with each example is ascertained by examining the activations of the last fully connected layer, and the networks are trained to minimize the cross-entropy between predictions extracted from this layer and ground-truth annotations. In this work, we extend this paradigm by introducing an additional output branch which maps the inputs to low dimensional representations, effectively extracting additional feature representations of the inputs. The proposed model imposes additional distance constrains on these representations with respect to identified class representatives, in addition to the traditional categorical cross-entropy between predictions and ground-truth. By extending the typical cross-entropy loss function with a distance learning function, our proposed approach achieves significant gains across a wide set of benchmark datasets in terms of classification, while providing additional evidence related to class membership and classification confidence.

This paper presents a study of an automated system for identifying planktic foraminifera at the species level. The system uses a combination of deep learning methods, specifically Convolutional Neural Networks (CNNs), to analyze digital images of foraminifera taken at different illumination angles. The dataset is composed of 1437 groups of sixteen grayscale images, one group for each foraminifer, that are then converted to RGB images with various processing methods. These RGB images are fed into a set of CNNs, organized in an Ensemble Learning (EL) environment. The ensemble is built by training different networks using different approaches for creating the RGB images. The study finds that an ensemble of CNN models trained on different RGB images improves the system's performance compared to other state-of-the-art approaches. The proposed system was also found to outperform human experts in classification accuracy.

Nowadays the dynamism induced by the constant change of strategic decisions on the markets produces an extra difficulty in the management of an organization. The strategic decisions made by managers can become obsolete in a short period of time. One of the major difficulties in managing a commercial organization is to predict, with some precision degree, the impact some strategic decisions have on the financial results. Business Intelligence (BI) is an area widely used to help managers to make strategic decisions. But the methods used, behind the scenes, to achieve the conclusions are kept secret by BI company-based services. Modelling the environment can be a good option to study the impact of an action, provided by a strategic decision, in a real environment. A good model should give an approximate result of an action applied to a previous state of the environment. Artificial Neural Networks (ANNs) are proven to be excellent in modeling environments with some level of data noise. The same strategic action can have different results in different organizations. A tool that allows an evaluation of a strategic action applied to an environment should have a major importance in the management scope. Modeling the environment should save time and money for the organization to improve the performance of the strategic plan. If one evaluates the state of the environment after a strategic action being applied, it can be possible to mitigate the risk of failure of the strategic plan. As we will be able to verify, it is possible to use ANNs to model strategic environments that should allow the prediction of sales and operating results by the used strategies with some precision, although the level of success will be directly related with the user’s necessity.

Background: Genome-wide association studies (GWAS) have shed light on various complex diseases and traits, by detecting more than 400,000 associated genetic loci. This number is expected to drastically increase because of the use of novel artificial intelligence methods offering new ways to study the effects of variants. Deep learning using artificial neural networks (ANN) is a sub-field of artificial intelligence, which simulates how the human brain learns. We aimed at assessing the potential of deep learning in human genetic studies of Alzheimer’s Disease (AD) and how these compare to the traditional statistical methods used in GWAS, by simultaneously testing the two approaches on the same dataset, while discovering new genetic loci associated to AD. Methods: To address this aim, phenotypic and genome-wide SNP data from the UK Biobank was analysed on a binary outcome, AD diagnosis, in two different data balance options, of one-to-one and one-to-two case-control datasets, using 2,764 cases vs 2,764 controls and 5,528 controls respectively matched on gender, age, ethnicity, PC1-20 and genotyping array. Genetic data handling and GWAS were performed using PLINK, whereas neural networks were trained using GenNet, a new ANN tool, with the same datasets, separated into training (60%), validation (20%) and test (20%) sets. Neural network layers were determined using biological knowledge, by annotating SNPs to genes and genes to AD related pathways, using ANNOVAR annotations followed by GeneSCF and KEGG. Results: Significant associations were detected between four SNPs linked to two different genes and AD for the 1 to 1 case-control study design and six SNPs linked to four different genes for the 1 to 2 case-control study design by using PLINK. All identified regions have been previously associated to AD. GenNet identified twelve SNPs on seven genes to be associated with AD, all with biological plausibility, achieving an AUC of 0.80 when using three biologically determined layers and 0.73 when using two layers at the neural networks. No common top SNPs were identified between the machine learning and GWAS models. Conclusion: This is one of the first studies attempting to compare the traditional GWAS to more sophisticated state-of-art methods for understanding the genetic architecture of complex phenotypes using the same dataset. More systematic comparisons with such approaches on real data are needed to enable best practises for machine learning in the analysis of genome-wide genetic data.

Formaldehyde (HCHO) is one of the most important carcinogenic air contaminants. However, the lack of global surface concentration of HCHO monitoring is currently hindering research on outdoor HCHO pollution. Traditional methods are either restricted to small areas or data- demanding for a global scale of research. To alleviate this issue, we adopted neural networks to estimate surface HCHO concentration with confidence intervals in 2019, where HCHO vertical column density data from TROPOMI, in-situ data from HAPs (harmful air pollutants) monitoring network and ATom mission are utilized. Our result shows that the global surface HCHO average concentration is 2.30 μg/m3. Furthermore, in terms of regions, the concentration in Amazon Basin, Northern China, South-east Asia, Bay of Bengal, Central and Western Africa are among the highest. The results from our study provides a first dataset of the global surface HCHO concentration. In addition, the derived confidence interval of surface HCHO concentration adds an extra layer for the confidence to our results. As a pioneer work in adopting confidence interval estimation into AI-driven atmospheric pollutant research and the first global HCHO surface distribution dataset, our paper will pave the way for the rigorous study on global ambient HCHO health risk and economic loss, thus providing a basis for pollutant controlling policies worldwide.

Leader-follower autonomy (LFA) systems have so far only focused on vehicles following other vehicles. Though there have been several decades of research into this topic, there has not yet been any work on human-vehicle leader-follower systems. We present a system in which an autonomous vehicle — our ACTor 1 platform — can follow a human leader who controls the vehicle through gestures. We successfully developed a modular pipeline that uses artificial intelligence/deep learning to recognize hand-and-body gestures from a user in view of the vehicle’s camera and translate those gestures into physical action by the vehicle. We demonstrate our work using our ACTor 1 platform, a modified Polaris Gem 2 electric vehicle. This work has numerous applications, such as material transport in industrial contexts.

Currently, skin cancer is the most commonly diagnosed form of cancer in humans and is one of the leading causes of death in patients with cancer. Biopsy methods are an invasive research method and are not always available for primary diagnosis. Imaging methods have low accuracy and depend on the experience of the dermatologist. Artificial intelligence technologies can match and surpass visual analysis methods in accuracy, but they have the risk of a false negative response when a malignant pigmented lesion can be recognized as benign. One possible way to improve accuracy and reduce the risk of false negatives is to analyze heterogeneous data, combine different preprocessing methods, and use modified loss functions to eliminate the negative impact of unbalanced dermatological data. The paper proposes a multimodal neural network system with a modified cross-entropy loss function that is sensitive to unbalanced heterogeneous dermatological data. The accuracy of recognition in 10 diagnostically significant categories for the proposed system was 85.19%. The novelty of the proposed system lies in the use of cross-entropy loss when training the modified function with the help of weight coefficients. The introduction of weighting factors has reduced the number of false negative forecasts, as well as improved accuracy by 1.02-4.03 percentage points compared to the original multimodal systems. The introduction of the proposed multimodal system as an auxiliary diagnostic tool can reduce the consumption of financial and labor resources involved in the medical industry, as well as increase the chance of early detection of skin cancer.

Alzheimer's disease (AD) is a progressive and evolving neurodegenerative disease with an insidious onset that can lead to memory loss and cognitive impairment. There is no effective treatment for this disease. However, early diagnosis plays an important role in treatment planning to slow down its progression, as treatment has the greatest impact in the early stages of the disease. Neurological images obtained through different imaging techniques provide powerful information and help diagnose the disease. With the wide application of deep learning techniques in disease diagnosis, especially the prominence of Convolutional Neural Networks (CNNs) in computer vision and image processing, more and more studies are proposing the use of this algorithm for the diagnosis of AD. In this paper, we first systematically introduce the impact of AD on people, detailing the biomarkers, early clinical symptoms, and risk factors of this disease. Secondly, it goes on to detail the development of CNNs, their form, and methods to help diagnose AD. It is proposed that CNNs can help diagnose AD by analyzing medical imaging data, particularly structural brain scans such as magnetic resonance imaging (MRI) and functional scans such as positron emission tomography (PET). Finally, it is concluded that CNNs are of great importance for the diagnosis of AD and that they are likely to play an increasingly important role in the early detection of the disease, the understanding of disease mechanisms, and ultimately, in the development of effective AD therapies and interventions. CNNs are playing an increasingly important role in the Their potential impact on healthcare emphasizes the importance of continued research and innovation in neural networks and medical imaging.

Singing melody extraction is the task that identifies the melody pitch contour of singing voice from polyphonic music. Most of the traditional melody extraction algorithms are based on calculating salient pitch candidates or separating the melody source from the mixture. Recently, classification-based approach based on deep learning has drawn much attentions. In this paper, we present a classification-based singing melody extraction model using deep convolutional neural networks. The proposed model consists of a singing pitch extractor (SPE) and a singing voice activity detector (SVAD). The SPE is trained to predict a high-resolution pitch label of singing voice from a short segment of spectrogram. This allows the model to predict highly continuous curves. The melody contour is smoothed further by post-processing the output of the melody extractor. The SVAD is trained to determine if a long segment of mel-spectrogram contains a singing voice. This often produces voice false alarm errors around the boundary of singing segments. We reduced them by exploiting the output of the SPE. Finally, we evaluate the proposed melody extraction model on several public datasets. The results show that the proposed model is comparable to state-of-the-art algorithms.

To drive safely, the driver must be aware of the surroundings, pay attention to the road traffic, and be ready to adapt to new circumstances. Most studies on driving safety focus on detecting anomalies in driver behavior and monitoring the cognitive capabilities of drivers. In our study, we proposed a classifier for basic activities in driving a car, based on a similar approach that could be applied to the recognition of basic activities in daily life, that is, using electrooculographic (EOG) signals and a one-dimensional convolutional neural network (1D CNN). Our classifier achieved an accuracy of 80% for the 16 primary and secondary activities. The accuracy related to primary activities in driving, including crossroad, parking, roundabout was 97.9%, 96.8%, 97.4%, and 99.5%, respectively. The F1 score for secondary driving actions (0.99) was higher than for primary driving activities (0.93–0.94). Furthermore, using the same algorithm, it was possible to distinguish four secondary activities related to activities of daily life and secondary when driving a car.

The Physics-Informed Neural Networks (PINNs) improve the efficiency of data utilization by combining physical principles with neural network algorithms and ensure that the predictions are consistent and stable with the physical laws. PINNs opens up a new approach to address inverse problems in fluid mechanics. Based on the single-relaxation-time lattice Boltzmann method (SRT-LBM) with the Bhatnagar-Gross-Krook (BGK) collision operator, the PINN-SRT-LBM model is proposed in this paper for solving the inverse problem in fluid mechanics. The PINN-SRT-LBM model consists of three components. The first component involves a deep neural network that predicts the equilibrium control equations in different discrete velocity directions within SRT-LBM. The second component employs another deep neural network to predict non-equilibrium control equations, enabling inference of the fluid's non-equilibrium characteristics. The third component, a physics informed function translates the outputs of the first two networks into physical infor-mation. By minimizing the residuals of the physical partial differential equations (PDEs), the physics informed function infers relevant macroscopic quantities of the flow. The model evolves two sub-models applicable to different dimensions, named PINN-SRT-LBM-I and PINN-SRT-LBM-II models according to the construction of the physical informed function. The innovation of this work is the introduction of SRT-LBM and discrete velocity models as physical drivers into the neural network through the interpretation function. Therefore, PINN-SRT-LBM allows the neural network to handle inverse problems of various dimensions and focus on problem-specific solving. Experimental results confirm the accurate prediction of flow infor-mation at different Reynolds numbers within the computational domain. Relying on the PINN-SRT-LBM models, inverse problems in fluid mechanics can be solved efficiently.

In Holter monitoring, the precise detection of standard heartbeats and Ventricular Premature Contractions (PVCs) is paramount for accurate cardiac rhythm assessment. This study introduces a novel application of the 1D U-Net neural network architecture aimed at enhancing PVC detection in Holter recordings. Training data comprised the Icentia11k, INCART DB, and our custom dataset. The model’s efficacy was subsequently validated against traditional Holter analysis methodologies across multiple databases, including AHA DB, MIT 11 DB, NST, and another custom dataset encompassing challenging real-world examples. The results underscored the 1D U-Net model’s prowess in QRS complex detection, achieving near-perfect balanced accuracy scores across all databases. PVC detection exhibited variability, with balanced accuracy scores ranging from 0.909 to 0.986. Despite some databases, like the AHA DB, showcasing lower sensitivity metrics, their robust balanced accuracy accentuates the model’s equitable performance in discerning both false positives and negatives. In conclusion, while the 1D U-Net architecture is a formidable tool for QRS detection, there’s a clear avenue for further refinement in its PVC detection capability, given the inherent complexities and noise challenges in real-world PVC occurrences.

This paper presents the use of Graph Neural Networks (GNNs) to predict the tensile strength of Fused Deposition Modeling (FDM) specimens. In the present work, there are four main input parameters i.e. Infill percentage, Layer height, Print speed and Extrusion temperature while the Tensile Strength is an output parameter were considered. This study includes use of central composite design based response surface methodology to finalize the experimental layout. 3D printed specimen were manufactured as per the ASTM E8 standard on FDM printer using Polylactic Acid (PLA) as filament. Micro-tensile test were performed on the printed specimen as per ASTM E8 standard. The GNN algorithm was trained on a dataset of FDM specimens, achieving a mean squared error (MSE) of 2.47, mean absolute error (MAE) of 1.14, and R-squared value of 0.78. An adjacency matrix, which shows the connections between nodes in a graph. The obtained plot for nodes and weights in a GNN provide valuable information about the model and its performance. The results show the potential of using GNNs in predicting the mechanical properties of additively manufactured specimens and provide a promising direction for further research in this field.

In urban Vehicular Ad Hoc Network (VANET) environments, buildings play a crucial role as they can act as obstacles that attenuate the transmission signal between vehicles. Quantifying the impact of buildings on the transmission quality is essential, especially in critical scenarios involving emergency vehicles, where reliable communication is of utmost importance. In this research, we propose a supervised learning approach based on artificial neural networks (ANNs) to develop a predictive model capable of estimating the level of signal degradation, represented by the bit error rate (BER), based on the obstacles perceived by moving emergency vehicles. By establishing a relationship between the level of signal degradation and the encountered obstacles, our proposed mechanism enables efficient routing decisions to be made prior to the transmission process. Consequently, data packets are routed through paths that exhibit the lowest BER. To gather the necessary training data, we employed SUMO and NS-3 simulations. The simulation results demonstrate that our developed model successfully learns and accurately estimates the BER for new data instances. Overall, our research contributes to enhancing the performance and reliability of communication in urban VANET environments, especially in critical scenarios involving emergency vehicles, by leveraging supervised learning and artificial neural networks to predict signal degradation levels and optimize routing decisions accordingly.

Over the last several decades, large wildfires are increasingly common across the United States causing disproportionate impact on forest health and function, human well-being, and economy. Here, we examine the severity of large wildfires across the Contiguous United States over the past decade (2011-2020) using a wide array of meteorological, vegetational, and topographical features in the Deep Neural Network model. A total of 4,538 wildfire incidents were used in the analysis covering 87,305 square miles of burned area. We observed the highest number of large wildfires in California, Texas, and Idaho, with lightning causing 43 % of these incidents. Importantly, results indicate that the severity of wildfire occurrences is highly correlated with the climatological forcings, land cover, location, and elevation of the ecosystem. Overall, results may serve useful guide in managing landscapes under changing climate and disturbance regimes.

Facial expression recognition is crucial for understanding human emotions and nonverbal communication. With the growing prevalence of facial recognition technology and its various applications, accurate and efficient facial expression recognition has become a significant research area. However, most previous methods have focused on designing unique deep-learning architectures while overlooking the loss function. This study presents a new loss function that allows simultaneous consideration of inter- and intra-class variations to be applied to CNN architecture for facial expression recognition. More concretely, this loss function reduces the intra-class variations by minimizing the distances between the deep features and their corresponding class centers. It also increases the inter-class variations by maximizing the distances between deep features and their non-corresponding class centers, and the distances between different class centers. Numerical results from several benchmark facial expression databases, such as Cohn-Kanade Plus, Oulu-Casia, MMI, and FER2013, are provided to prove the capability of the proposed loss function compared with existing ones.

Boiler efficiency is called to some extent of total thermal energy which can be recovered from the fuel. Boiler efficiency losses are due to four major factors: the dry gas flux, the latent heat of steam in the flue gas, the combustion loss or the loss of unburned fuel, radiation and convection losses. In this research, the thermal behavior of boilers in gas refinery facilities is studied and their efficiency and their losses are calculated. The main part of this research is comprised of analyzing the effect of various parameters on efficiency such as excess air, fuel moisture, air humidity, fuel and air temperature, the temperature of combustion gases, and thermal value of the fuel. Based on the obtained results, it is possible to analyze and make recommendations for optimizing boilers in the gas refinery complex using response-surface method (RSM).

This paper proposes a radiation pattern synthesis of the almost periodic antenna arrays including mutual coupling effects (that extracted by the Floquet analysis according to our previous work), which principally has a high directivity and large bandwidth. For modeling the given structures, the moment method combined with the Generalized Equivalent Circuit (MoM-GEC) is proposed. The artificial neural network (ANN) as a powerful computational model has been successfully applied to the antenna array pattern synthesis. The results showed that the multilayer feedforward neural networks are rugged and can successfully and efficiently resolve various distinctive complex almost periodic antenna patterns (with different source amplitudes) (in particular, both periodic and randomly aperiodic structures are taken into account). However, the artificial neural network (ANN) is capable of quickly producing the synthesis results using generalization with the early stopping (ES) method. A significant time gain and memory consumption are achieved by using this given method to improve the generalization (called early stopping). To justify this work, several examples are developed and discussed.

Recently, the number of vehicles on the road, especially in urban centers, has increased dramatically due to the increasing trend of individuals towards urbanization. As a result, manual detection and recognition of vehicles (i.e., license plates and vehicle manufacturer) become an arduous task and beyond human capabilities. In this paper, we have developed a system using transfer learning-based DL techniques for automatic identification of Jordanian vehicles. The YOLOv3 (You Only Look Once) model was re-trained using transfer learning to accomplish the license plate detection, character recognition, and vehicle logo detection. While VGG16 (Visual Geometry Group) model was retrained to accomplish the vehicle logo recognition. To train and test these models, four datasets have been collected. The first dataset consists of 7,035 Jordanian vehicle images, the second dataset consist of 7,176 Jordanian license plates, and the third dataset consists of 8,271 Jordanian vehicle images. These datasets have been used to train and test the YOLOv3 model for Jordanian license plate detection, character recognition, and vehicle logo detection, respectively. While the fourth dataset consists of 158,230 vehicle logo images used to train and test the VGG16 model for the vehicle logo recognition. Text measures were used to evaluate the performance of our developed system. Moreover, mean average precision (mAP) measure was used to evaluate the YOLOv3 model of the detection tasks (i.e., license plate detection and vehicle logo detection). For license plate detection, the precision, recall, F-measure, and mAP were 99.6%, 100%, 99.8%, and 99.9%, respectively. While for character recognition, the precision, recall, and F-measure were 100%, 99.9%, and 99.95%, respectively. The performance of license plate recognition stage was evaluated by evaluating these two sub-stages as a sequence, where the precision, recall, and F-measure were 99.8%, 99.8%, and 99.8%, respectively. Furthermore, for vehicle logo detection, the precision, recall, F-measure, and mAP were 99%, 99.6%, 99.3%, and 99.1%, respectively, while for vehicle logo recognition, the precision, recall, F-measure were 98%, 98%, and 98%, respectively. The performance of vehicle logo recognition stage was evaluated by evaluating these two sub-stages as a sequence, where the precision, recall, and F-measure were 95.3%, 99.5%, and 97.4%, respectively.

An automatic recognizing system of white blood cells can assist hematologists in the diagnosis of many diseases, where accuracy and efficiency are paramount for computer-based system. In this paper, we present a new image processing system to recognize the five types of white blood cells in peripheral blood with marked improvement in efficiency when juxtaposed against mainstream methods. The prevailing deep learning segmentation solutions often utilize millions of parameters to extract high-level image features and neglect the incorporation of prior domain knowledge, which consequently consume substantial computational resources and increase the risk of overfitting, especially when limited medical image samples are available for training. To address these challenges, we propose a novel memory-efficient strategy that exploits graph structures derived from the images. Specifically, we introduce a lightweight superpixel-based Graph Neural Network (GNN) and break new ground by introducing superpixel metric learning to segment nucles and cytoplasm. Remarkably, our proposed segmentation model (SMGNN) achieves state-of-the-art segmentation performance while utilizing at most 10000$\times$ less than the parameters compared to existing approaches. The subsequent segmentation-based cell type classification processes show satisfactory results that such automatic recognizing algorithms are accurate and efficient to execeute in hematological laboratories.

This study explores the use of Bayesian Neural Networks (BNNs) for estimating chlorophyll-a concentration ([CHL-a]) from remotely sensed data. The BNN model enables uncertainty quantification, offering additional layers of information compared to traditional ocean colour models. An extensive in situ bio-optical dataset is utilized, generated by merging 27 data sources across the world’s oceans. The BNN model demonstrates remarkable capability in capturing mesoscale features and ocean circulation patterns, providing comprehensive insights into spatial and temporal variations in [CHL-a] across diverse marine ecosystems. In comparison to established ocean colour algorithms, such as Ocean Colour 4 (OC4), the BNN shows comparable performance in terms of correlation coefficients, errors, and biases when compared with the in situ data. The BNN, however, further provides critical information about the distribution of [CHL-a], which can be used to assess uncertainties in the prediction. Moreover, the BNN model’s ability to provide more information for coastal waters, especially with higher spatial resolution imagery, offers valuable advantages for marine research and management. By quantifying uncertainty, the model builds confidence in [CHL-a] predictions, even in regions with limited data coverage. This is reflected by the spread in the model predictions, where the BNN can detect a range of uncertainty bounds that reflect confidence in the retrievals. Introducing BNNs as probabilistic ocean colour models represents a significant advancement, enabling more accurate and reliable predictions of [CHL-a] in the global oceans.

Survival models have become popular for credit risk estimation. Most current credit risk survival models use an underlying linear model. This is beneficial in terms of interpretability but is restrictive for real-life applications since it cannot discover hidden nonlinearities and interactions within the data. This study uses discrete time survival models with embedded neural networks as estimators of time to default. This provides flexibility to express nonlinearities and interactions between variables, and hence allows for models with better overall model fit. Additionally, the neural networks are used to estimate Age-Period-Cohort (APC) models so that default risk can be decomposed into time components for loan age (maturity), origination (vintage) and environment (e.g., economic, operational and social effects). These can be built as general models, or as local APC models for specific customer segments. The local APC models reveal special conditions for different customer groups. The corresponding APC identification problem is solved by a combination of regularization and fitting the decomposed environment time risk component to macroeconomic data, since the environmental risk is expected to have a strong relationship with macroeconomic conditions. Our approach is shown to be effective when tested on a large publicly available US mortgage data set. This novel framework can be adapted by practitioners in the financial industry to improve modelling, estimation and assessment of credit risk.

In the last decade, the genetic engineering world has been shaken up by a relatively new genetic editing tool based on RNA-guided Nucleases (RGNs): the CRISPR/Cas9 system. Since the first report in 1987 and its characterization in 2007 as a bacterial defense mechanism, the interest and research on this system have grown exponentially. CRISPR systems provide immunity to bacteria against invading genetic material; however, with specific modifications in sequence and structure, it becomes a precise editing system that makes it possible to genetically modify almost any organism. There are diverse approaches regarding the refinement of these modifications, such as constructing more accurate nucleases, understanding the cellular context and facing the epigenetic conditions, or re-designing guide RNAs (gRNAs). Considering the critical importance for the correct CRISPR/Cas9 systems performance, our scope will emphasize in the latter approach. Hence, we present an overview of the past and the most recent guide RNA web-based design tools, highlighting their computational architecture and gRNA characteristics evolution through the years. Our study concisely explains the computational approaches that use machine learning techniques, deep neural networks, and large datasets of gRNA/target interactions to make possible both predictions and classifications directed to design, optimize, and create promising gRNAs suitable for future gene therapies.

This paper inquires on the options pricing modeling using Artificial Neural Networks to price Apple(AAPL) European Call Options. Our model is based on the premise that Artificial Neural Networks can be used as functional approximators and can be used as an alternative to the numerical methods to some extent, for a faster and an efficient solution. This paper provides a neural network solution for two financial models, the BlackScholes-Merton model, and the calibrated-Heston Stochastic Volatility Model, we evaluate our predictions using the existing numerical solutions for the same, the analytic solution for the Black-Scholes equation, COS-Model for Heston’s Stochastic Volatility Model and Standard Heston-Quasi analytic formula. The aim of this study is to find a viable time-efficient alternative to existing quantitative models for option pricing.

Amid escalating urbanization, devising rational commercial space layouts is a critical challenge. Leveraging machine learning, this study uses a Back-propagation (BP) neural network to optimize commercial spaces in Weinan City's central urban area. The results indicate an increased number of commercial facilities with a trend of multi-centered agglomeration and outward expansion. Based on these findings, we propose a strategic framework for rational commercial space development emphasizing aggregation centers, development axes, and spatial guidelines. This strategy provides valuable insights for urban planners in small and medium-sized cities in the Yellow River Basin and metropolitan areas, ultimately showcasing the power of machine learning in enhancing urban planning.

In recent years, there has been a growing interest in the prediction of facial age due to its diverse applications in fields such as security, entertainment, and healthcare. The current scourge of underaged voting in Nigeria is a problem and this research delves into the realm of facial age prediction by employing four well-known convolutional neural network (CNN) architectures, namely VGG16, ResNet50, Mobile Net, and VGG19 to predict chronological ages from facial pictures of person. The objective is to achieve precise age estimation from facial images, utilizing two datasets: UTKFace and CASIA African Facial Datasets. The results of this investigation are noteworthy. Specifically, the VGG16 model which demonstrated remarkable performance, yielding a Mean Absolute Error (MAE) of 1.76 when applied to the UTK-Face dataset. Additionally, when utilizing Mobile-Net, an unprecedented MAE of 1.10 was achieved for the Casia Africa Face Dataset. Notably, this marks the first instance of employing the dataset for facial age detection with CNNs, and this approach outperformed previous works, yielding the lowest MAE among all the studies reviewed.

This paper analyzes the robust dissipativity of uncertain stochastic generalized neural networks (USGNNs) with Markovian jumping parameters and time-varying delays. In practical applications most of the systems refer to uncertainties, hence, the norm-bounded parameter uncertainties and stochastic disturbance are considered. Then, by constructing an appropriate Lyapunov-Krasovskii functional (LKF) and by employing integral inequalities LMI-based sufficient conditions of the considered systems are established. Numerical simulations are given to show the merit of the presented results.

The paper illustrates a general framework in which a neural network application can be easily integrated and proposes a traffic forecasting approach that uses neural networks based on graphs. The method minimizes the communication network (between vehicles and the database servers) load and represents a reasonable trade-off between communication network load and forecasting accuracy. Traffic prediction leads to the choice of less congested routes and therefore to the reduction of energy consumption. The traffic is forecasting using a LTSM neural network with a regression layer. The inputs of the neural network are sequences - obtained from graph that represent the road network - at specific moments of time that are read from traffic sensors or the outputs of neural network (forecasting sequences). The input sequences can be filtered to improve the forecasting accuracy. This general framework is based on Contiki IoT operating system that ensure support for wireless communication and efficient implementation of processes in a resource constrained system and it is particularized to implement a graph neural network. Two cases are studied: one case in which the traffic sensors are periodically read and the other case in which the traffic sensors are read when their values changes are detected. A comparison between the cases is made and the influence of filtering is evaluated. The obtained accuracy is very good, very close to the accuracy obtained in infinite precision simulation, and the computation time is low enough and the system can work in real time.

Next-generation cancer and oncology research needs to take full advantage of the multi-modal structured, or graph, information, with the graph datatypes ranging from molecular structures to spatially resolved imaging and digital pathology to biological networks to knowledge graphs. Graph Neural Networks (GNNs) efficiently combine the graph structure representations with the high predictive performance of deep learning, especially on the large multi-modal datasets. In this review article, we survey the landscape of recent (2020-present) GNN applications in the context of cancer and oncology research, and delineate six currently predominant research areas. Subsequently, we identify the most promising directions for future research. We compare GNNs with graphical models and "non-structured" deep learning, and devise the guidelines for cancer and oncology researchers or physician-scientists asking the question of whether they should adopt the GNN methodology in their research pipelines.

In South Korea, demolition waste (DW) management has become increasingly significant owing to the rising number of old buildings. Effective DW management requires an efficient approach that accurately quantifies and predicts the generation of DW (DWG) of various types, which necessitates access to the required information or technology capable of achieving this. Hence, we developed an artificial intelligence-based model that predicts the generation of ten DW types, specifically from buildings in redevelopment areas. We used an artificial neural network algorithm with < 10 neurons in the hidden layer to derive individual input variables and optimal hyperparameters for each DW type. All DWG prediction models achieved an average validation and test prediction performance (R²) of 0.970 and 0.952, respectively, with their ratios of percent deviation ≥ 2.5, ver-ifying them as excellent models. Moreover, a Shapley additive explanations analysis revealed that DWG was most impacted by the floor area for all the DW types, with a positive correlation with DWG. Conversely, other factors showed either a positive or negative correlation with DWG de-pending on the DW type. The study findings will enable demolition companies and local gov-ernments in making informed decisions for efficient DW management and resource allocation by accurately predicting the generation of various types of DW.

The continuous and precise mapping of glacier calving fronts is essential for monitoring and understanding rapid glacier changes in Antarctica and Greenland, which have the potential for significant sea level rise within the current century. This effort has been mostly restricted to the slow and painstaking manual digitalization of the calving front positions in thousands of satellite imagery products. Here, we have developed a machine learning toolkit to robustly and automatically detect glacier calving front margins in satellite imagery. The toolkit is based on semantic image segmentation using Convolutional Neural Networks (CNN) with a modified U-Net architecture to isolate the calving fronts from satellite images after having been trained with a dataset of images and their corresponding manually-determined calving fronts. As a case study we train our neural network on a varied set Landsat images with lowered resolutions from Jakobshavn, Sverdrup, and Kangerlussuaq glaciers, Greenland and test the results on novel images from Helheim glacier, Greenland to evaluate the performance of the approach. The neural network is able to identify the calving front in new images with a mean deviation of 96.3 m from the true fronts, equivalent to 1.97 pixels on average, while the corresponding error for manually-determined fronts on the same resolution images is 92.5 m. We find that the trained neural network significantly outperforms common edge detection techniques, and can be used to continuously map out calving-ice fronts with a variety of data products.

This paper reviews non-intrusive load monitoring (NILM) approaches that employ deep neural networks to disaggregate appliances from low frequency data, i.e. data with sampling rates lower than the AC base frequency. We first review the many degrees of freedom of these approaches, what has already been done in literature, and compile the main characteristics of the reviewed publications in an extensive overview table. The second part of the paper discusses selected aspects of the literature and corresponding research gaps. In particular, we do a performance comparison with respect to reported MAE and F$_1$-scores and observe different recurring elements in the best performing approaches, namely data sampling intervals below 10\,s, a large field of view, the usage of GAN losses, multi-task learning, and post-processing. Subsequently, multiple input features, multi-task learning and related research gaps are discussed, the need for comparative studies is highlighted, and finally, missing elements for a successful deployment of NILM approaches based on deep neural networks are pointed out. We conclude the review with an outlook on possible future scenarios.

With the exponential growth of machine learning and development of Artificial Neural Network (ANNs) in recent years, there is great opportunity to leverage this approach and accelarate bio-logical discoveries through applications on the analysis of bioinformatics data. Various types of datasets including for example protein or gene interaction networks, molecular structures and cellular signalling pathways, have already been used for machine learning by training ANNs for inference and pattern classification. However, unlike regular data structures that are commonly used in the computer science and engineering fields, bioinformatics datasets present challenges that require unique algorithmic approaches. The recent development of the geometric and deep learning approach within the machine learning field, is very promising towards accelerating analysis complex bioinformatics datasets. The principles of ANNs and their importance for bio-informatics machine learning is demonstrated herein, through presentation of the undelying mathematical and statistical foundations from group theory, symmetry, linear algebra. Further-more, the structure and functions of ANN algorithms that form the core principles of artificial intelligence are explained, in relation to the bioinformatics data domain. Overall, the manuscript provides guidance for researchers to understand the principles required for practicing machine learning and artificial intelligence, with the special considerations towards bioinformatics applications.

When extract building from high resolution remote sensing image with meter/sub-meter accuracy, the shade of trees and interference of roads are the main factors of reducing the extraction accuracy. Proposed a Bayesian Convolutional Neural Networks(BCNET) model base on standard fully convolutional networks(FCN) to solve these problems. First take building with no shade or artificial removal of shade as Sample-A, woodland as Sample-B, road as Sample-C. Set up 3 sample libraries. Learn these sample libraries respectively, get their own set of feature vector; Mixture Gauss model these feature vector set, evaluate the conditional probability density function of mixture of noise object and roofs; Improve the standard FCN from the 2 aspect:(1) Introduce atrous convolution. (2) Take conditional probability density function as the activation function of the last convolution. Carry out experiment using unmanned aerial vehicle(UVA) image, the results show that BCNET model can effectively eliminate the influence of trees and roads, the building extraction accuracy can reach 97%.

Direct measurement of strain within muscle is important for understanding muscle function in health and disease. Current technology (kinematics, dynamometry, electromyography) provides limited ability to measure strain within muscle. Regional fiber orientation and length are related with active/passive strain within muscle. Currently, ultrasound imaging provides the only non-invasive means of observing regional fiber orientation within muscle during dynamic tasks. Previous attempts to automatically estimate fiber orientation from ultrasound are not adequate, often requiring manual region selection, feature engineering, providing low-resolution estimations (one angle per muscle), and deep muscles are often not attempted. Here, we propose deconvolutional neural networks (DCNN) for estimating fiber orientation at the pixel-level. Dynamic ultrasound images sequences of the calf muscles were acquired (25 Hz) from 8 healthy volunteers (4 male, ages: 25–36, median 30). A combination of expert annotation and interpolation/extrapolation provided labels of regional fiber orientation for each image. We then trained DCNNs both with and without dropout using leave one out cross-validation. Our results demonstrated robust estimation of regional fiber orientation with approximately 3° error, which was an improvement on previous methods. The methods presented here provide new potential to study muscle in disease and health.

Red mud (RM), a bauxite residue, contains hazardous radioactive wastes and alkaline material and poses severe surface water and groundwater contamination risks, necessitating recycling. Pretreated RM can be used to make adsorbents for water treatment. However, its performance is affected by many factors, resulting in a nonlinear correlation and coupling relationship. This study aimed to identify the best formula for an RM adsorbent using a mathematical model that examines the relationship between 11 formulation types (e.g., pore-assisting agent, component modifier, and external binder) and nine properties (e.g., specific surface area, wetting angle, and Zeta potential, among others). This model was built using a back-propagation neural networks(BP) based on single-factor experimental data and orthogonal test data. The model trained and predicted the established network structure to obtain the optimal adsorbent formula. The RM particle adsorbents had a pH of 10.16, specific surface area(BET) of 48.92 m2·g−1, pore volume of 2.10 cm3·g−1, compressive strength(ST)) of 1.12 KPa, and 24-h immersion pulverization rate(η_m) of 3.72%. In the removal of total phosphorus in flotation tailings backwater, it exhibited a good adsorption capacity (Q)and total phosphorous removal rate(η) of 48.63 mg·g−1 and 95.13%, respectively.

The implementation of neural networks (NN) on edge devices enables local processing of wireless data but faces challenges such as high computational complexity and memory requirements when deep neural networks (DNN) are used. Shallow neural networks customized for specific problems are more efficient, requiring fewer resources, and resulting in a lower latency solution. An additional benefit of the smaller network size is that it is suitable for real-time processing on edge devices. The main concern with shallow neural networks is their accuracy performance compared to DNNs. In this paper, we demonstrate that a customized adaptive activation function (AAF) can meet the accuracy of a DNN. We designed an efficient FPGA implementation for a customized segmented spline curve neural network (SSCNN) structure to replace the traditional fixed activation function with an AAF. We compared our SSCNN with different neural network structures such as real-valued time delay neural network (RVTDNN), augmented real-valued time delay neural network (ARVTDNN), and deep neural networks with different parameters. Our proposed SSCNN implementation uses 40% fewer hardware resources and no Block RAMS compared to the DNN with similar accuracy. We experimentally validate this computationally efficient and memory-saving FPGA implementation of SSCNN for digital predistortion of RF power amplifiers using the AMD/Xilinx RFSoC ZCU111. The implemented solution uses less than 3% of the available resources. The solution also enables an increase of the clock frequency to 221.12 MHz, allowing the transmission of wide bandwidth signals.

In this research, we present a comprehensive methodology to categorize perfumes based on their fragrance profiles and subsequently aid in creating innovative odoriferous molecules using advanced neural networks. Drawing from data on Parfumo and the Good Scents Company webpage (Parfumo, 2008; The Good Scents Company, 2021), the study employs sophisticated web scraping techniques to gather diverse perfume attributes. Following this, a k-means algorithm is applied for perfume clustering, paving the way for recommending similar scents to consumers. The process then bridges customer preferences to molecular design by incorporating their feedback into generating new molecules via graph neural networks (GNNs). Through converting the Simple Molecular Input Line Entry System (SMILES) representation into graph structures, the GNN facilitates the creation of new molecular designs attuned to consumer desires. The proposed approach offers promising avenues for consumers to pinpoint similar perfume choices, incorporating feedback, and for manufacturers to conceptualize new fragrant molecules with a high likelihood of market resonance.

In this paper, a class of coupled competitive neural networks, which subject to disturbance and discontinuous activation functions, is concerned. To realize the fixed-time quasi-bipartite synchronization, an aperiodic intermittent controller is initially designed. Subsequently, by combining the fixed-time stability theory and nonsmooth analysis, several criteria are established to ensure the bipartite synchronization in fixed time. Moreover, synchronization error bounds and settling time estimates are provided. Finally, numerical simulations are presented to verify the main results.

In this paper, we present a groundbreaking methodology in the realm of aspect-level sentiment analysis, which capitalizes on the advanced capabilities of graph-based neural networks. Our approach, distinguished as the Aspect Correlation Graph Network (ACGN), represents a significant departure from conventional models. These traditional models often analyze aspects in isolation, failing to capture the intricate web of sentiment relationships that may exist within a single sentence. ACGN, however, is designed to address this gap by employing a sophisticated bidirectional attention mechanism, integrated with positional encoding. This unique combination not only enhances the model's ability to focus on relevant parts of the sentence but also aids in constructing detailed, aspect-focused representations. These representations are particularly crucial for understanding the nuanced interplay of sentiments associated with different aspects. Central to our model's architecture is the incorporation of a graph convolutional network. This network serves as a pivotal component in mapping and analyzing the complex network of sentiment correlations that can exist among various aspects within sentences. Through this integration, ACGN is able to unearth and interpret the subtle and often overlooked sentiment dynamics that traditional models might miss. Our comprehensive evaluations of the Aspect Correlation Graph Network, conducted using the SemEval 2014 datasets, have yielded promising results. These findings demonstrate a clear and significant advancement over the capabilities of existing models. Particularly, the results underscore the critical importance and utility of recognizing and understanding the connections between sentiments of different aspects in text analysis. This insight opens new avenues in the field of sentiment analysis, suggesting a broader application potential of ACGN in various contexts where understanding nuanced sentiment relationships is key. Overall, our study not only introduces a novel approach in aspect-level sentiment analysis but also sets a new standard for future research in this area. By highlighting the integral role of inter-aspect sentiment connections, ACGN paves the way for more sophisticated and accurate sentiment analysis tools, capable of handling the complexities of natural language with greater finesse and precision.

Objective A trained T1 class Convolutional Neural Network (CNN) model will be used to examine its ability to successfully identify motor imagery when fed pre-processed electroencephalography (EEG) data. In theory, and if the model has been trained accurately, it should be able to identify a class and label it accordingly. The CNN model will then be restored and used to try and identify the same class of motor imagery data using much smaller sampled data in an attempt to simulate live data. Approach PyCharm, a Python platform, will be used to house and process the CNN. The raw data used for the training of the CNN will be sourced from the PhysioBank website. The EEG signal data will then be pre-processed using Brainstorm software that is a toolbox used in conjunction with MATLAB. The sample data used to validate and test the trained CNN, will be also be extracted from Brainstorm but in a much smaller size compared to the training data which is comprised of thousands of images. The sample size would be comparable to a person wearing a Brain Computer Interface (BCI), offering approximately 20 seconds of motor imagery signal data. Results The raw EEG data was successfully extracted and pre-processed. The deep learning model was trained using the extracted image data along with their corresponding labels. After training, it was able to accurately identify the T1 class label at 100 percent. The python coding was then modified to restore the trained model and feed it test sample data in which it was found to recognise 6 out of 10 lines of T1 signal image data. The result suggested that the initial training of the model required a different, more varying approach, so that it would be able to detect varying sample signal image data. The outcome of which could mean that the model could be used in applications for multiple patients wearing the same BCI hardware to control a device or interface.

Hopfield Neural Networks (HNNs) are recurrent neural networks used to implement associative memory. Their main feature is their ability to pattern recognition, optimization, or image segmentation. However, sometimes it is not easy to provide the users with good explanations about the results obtained with them due to mainly the large number of changes in the state of neurons (and their weights) produced during a problem of machine learning. There are currently limited techniques to visualize, verbalize, or abstract HNNs. This paper outlines how we can construct automatic video generation systems to explain their execution. This work constitutes a novel approach to get explainable artificial intelligence systems in general and HNNs in particular building on the theory of data-to-text systems and software visualization approaches. We present a complete methodology to build these kinds of systems. Software architecture is also designed, implemented, and tested. Technical details about the implementation are also detailed and explained. Finally, we apply our approach for creating a complete explainer video about the execution of HNNs on a small recognition problem.

Against the backdrop of the current carbon peaking and carbon neutrality policies, higher requirements have been put forward for the upgrading and construction of smart grids. Non-intrusive load monitoring (NILM) technology is a key technology for advanced measurement systems at the end of the power grid. It obtains detailed power information of the load, without the need for traditional hardware deployment. The key step to solve this problem is load decomposition and identification. This paper first utilized Long Short Term Memory-Denoising Autoencoder(LSTM-DAE) to decompose the mixed current signal on the household busbar and obtain the current signals of multiple independent loads that constituted the mixed current. Then, the obtained independent current signals were combined with voltage signals to generate multi-cycle colored Voltage-Current(VI) trajectories, which were color-coded according to the background. These colored VI trajectories with background colors formed a feature library. When the CNN network was used for load recognition, considering the influence of hyperparameters on recognition results, the BOA algorithm was used for optimization, and the optimized CNN network was employed for VI trajectory recognition. Finally, the proposed method was validated using the PLAID dataset. Experimental results showed that the proposed method exhibited better performance in load decomposition and identification.

The small satellite market continues to grow year after year. A compound annual growth rate of 17% is estimated during the period between 2020 and 2025. Low-cost satellites can send a vast amount of images to be post-processed at the ground to improve the quality and extract detailed information. In this domain lies the resolution enhancement task, where a low-resolution image is converted to a higher resolution automatically. Deep learning approaches to Super-Resolution (SR) reached the state-of-the-art in multiple benchmarks; however, most of them were studied in a single-frame fashion. With satellite imagery, multi-frame images can be obtained at different conditions giving the possibility to add more information per image and improve the final analysis. In this context, we developed and applied to the PROBA-V dataset of multi-frame satellite images a model that recently topped the European Space Agency’s Multi-frame Super Resolution (MFSR) competition. The model is based on proven methods that worked on 2D images tweaked to work on 3D: the Wide Activation Super Resolution (WDSR) family. We show that with a simple 3D CNN residual architecture with WDSR blocks and a frame permutation technique as data augmentation better scores can be achieved than with more complex models. Moreover, the model requires few hardware resources, both for training and evaluation, so it can be applied directly from a personal laptop.

The COVID-19 pandemic situation has created even more difficulties in the quick identification and screening of the COVID-19 patients for the medical specialists. Therefore, a significant study is necessary for detecting COVID-19 cases using an automated diagnosis method, which can aid in controlling the spreading of the virus. In this paper, the study suggests a Deep Convolutional Neural Network-based multi-classification approach (COV-MCNet) using eight different pre-trained architectures such as VGG16, VGG19, ResNet50V2, DenseNet201, InceptionV3, MobileNet, InceptionResNetV2, Xception which are trained and tested on the X-ray images of COVID-19, Normal, Viral Pneumonia, and Bacterial Pneumonia. The results from 3-class (Normal vs. COVID-19 vs. Viral Pneumonia) showed that only the ResNet50V2 model provides the highest classification performance (accuracy: 95.83%, precision: 96.12%, recall: 96.11%, F1-score: 96.11%, specificity: 97.84%) compared to rest of the models. The results from 4-class (Normal vs. COVID-19 vs. Viral Pneumonia vs. Bacterial Pneumonia) demonstrated that the pre-trained model DenseNet201 provides the highest classification performance (accuracy: 92.54%, precision: 93.05%, recall: 92.81%, F1-score: 92.83%, specificity: 97.47%). Notably, the ResNet50V2 (3-class) and DenseNet201 (4-class) models in the proposed COV-MCNet framework showed higher accuracy compared to the rest six models. This indicates that the designed system can produce promising results to detect the COVID-19 cases on the availability of more data. The proposed multi-classification network (COV-MCNet) significantly speeds up the existing radiology-based method, which will be helpful to the medical community and clinical specialists for early diagnosis of the COVID-19 cases during this pandemic.

— In a smart healthcare system," Human Activity Recognition (HAR)" is considered as an efficient approach in pervasive computing from activity sensor readings. The "Ambient Assisted Living (AAL)" in the home or community helps the people to provide independent care and enhanced living quality. However, many AAL models are restricted to multiple factors that include both the computational cost and system complexity. Moreover, the HAR concept has more relevance because of its applications, such as content-based video search, sports play analysis, crowd behavior prediction systems, patient monitoring systems, and surveillance systems. This paper attempts to implement the HAR system using a popular deep learning algorithm, namely "Recurrent Neural Network (RNN)" with the activity data collected from smart activity sensors over time, and it is publicly available in the "UC Irvine Machine Learning Repository (UCI)". The proposed model involves three processes: (1) data collection, (b) optimal feature learning, and (c) activity recognition. The data gathered from the benchmark repository was initially subjected to optimal feature selection that helped to select the most significant features. The proposed optimal feature selection method is based on a new meta-heuristic algorithm called "Colliding Bodies Optimization (CBO)". An objective function derived from the recognition accuracy has been used for accomplishing the optimal feature selection. The proposed model on the concerned benchmark dataset outperformed the conventional models with enhanced performance.

Clients are increasingly looking for fast and effective means to quickly and frequently survey and communicate the condition of their buildings so that essential repairs and maintenance work can be done in a proactive and timely manner before it becomes too dangerous and expensive. Traditional methods for this type of work commonly comprise of engaging building surveyors to undertake a condition assessment which involves a lengthy site inspection to produce a systematic recording of the physical condition of the building elements, including cost estimates of immediate and projected long-term costs of renewal, repair and maintenance of the building. Current asset condition assessment procedures are extensively time consuming, laborious, and expensive and pose health and safety threats to surveyors, particularly at height and roof levels which are difficult to access. We propose a method for automated detection and localisation of key building defects from images using deep learning and convolution neural networks. The proposed model is based on a pre-trained VGG-16 classifier with Class Activation Mapping (CAM) for object localisation. The model has proven to be robust and able to accurately detect and localise mould growth, stains, and paint deterioration defects arising from dampness in buildings. The approach is being developed with potentials to scale-up to support automated detection of defects and deterioration of buildings in real-time using mobile devices and drones.

In the age of social media, images come from unreliable sources that may change their content to fit a narrative. Automated detection of forged images is a complex task, especially considering recent technological advances in image manipulation software. However, in the existing literature, it is possible to identify two general approaches adopted by detection methods, active and passive. Active techniques preemptively act on an image by inserting structures before any manipulation is made that can verify its authenticity. In contrast, passive methods analyze the content of an image in search of traces of manipulation. In this way, this research proposes a novel approach to image manipulation detection, combining two passive methods through neural networks, creating a generalist approach capable of detecting with greater accuracy than the methods that compose it. Furthermore, this work used a combination of four datasets available in the literature for training and evaluation. After training, the merged approach obtained an accuracy of 89.59% in the set of validation images, significantly higher than the model trained with only unaltered images, which obtained 78.64%, and the two other models trained using images with a feature selection method applied to enhance inconsistencies that obtained 68.02% for Error-Level Analysis images and 50.70% for the method using Discrete Wavelet Transform. In addition to the performance improvement, the proposed approach’s accuracy variation was lower than the other models.

The total tree height (h) and diameter at breast height (dbh) relationship is an essential tool in forest management and planning. The height—diameter (h-dbh) relationship had been studied with several approaches and for several species worldwide. The nonlinear mixed effect modeling (NLMEM) has been extensively used and lately the resilient backpropagation artificial neural network (RBPANN) approach has been a trend topic for modeling this relationship. The artificial neural network (ANN) is a computing system based in artificial intelligence and inspired in biological neural network for supervised learning. In this study the NLMEN and RBPANN approaches were used for modeling the h—dbh relationship for Durango pine species (Pinus durangensis Martínez) in mixed-species forest from Mexico. The total dataset considered 1,000 (11,472 measured trees) randomly selected from 14,390 temporary forest inventory plots and the dataset was randomly divided into two parts; 50% for training and 50% for testing. An unsupervised clustering analysis was used to grouped the dataset into 10 clusters based on k-means clustering method and plot-variables like density, basal area, mean dbh, mean h, quadratic mean diameter, altitude and aspect. The RBPANN was performed for tangent hyperbolicus (RBPANN-tanh), softplus (RBPANN-softplus), and logistic (RBPANN-logistic) activation functions for functions in cross product of the covariate or neurons and the weights for the ANN analysis. For both training and testing, 10 classical statistics (e.g., RMSE, R2, AIC, BIC, logLik) were computed for the residual values and assess the approaches for h—dbh relationship. For training and testing, the ANNs approach outperformed the NLMEM approach, and the RBPANN-tanh has the best performance in both training and testing phases.

Polar mesocyclones (MCs) are small marine atmospheric vortices. The class of intense MCs, called polar lows, are accompanied by extremely strong surface winds and heat fluxes and thus largely influencing deep ocean water formation in the polar regions. Accurate detection of polar mesocyclones in high-resolution satellite data, while challenging, is a time-consuming task, when performed manually. Existing algorithms for the automatic detection of polar mesocyclones are based on the conventional analysis of patterns of cloudiness and involve different empirically defined thresholds of geophysical variables. As a result, various detection methods typically reveal very different results when applied to a single dataset. We develop a conceptually novel approach for the detection of MCs based on the use of deep convolutional neural networks (DCNNs). As a first step, we demonstrate that DCNN model is capable of performing binary classification of 500x500km patches of satellite images regarding MC patterns presence in it. The training dataset is based on the reference database of MCs manually tracked in the Southern Hemisphere from satellite mosaics. We use a subset of this database with MC diameters falling in the range of 200-400 km. This dataset is further used for testing several different DCNN setups, specifically, DCNN built “from scratch”, DCNN based on VGG16 pre-trained weights also engaging the Transfer Learning technique, and DCNN based on VGG16 with Fine Tuning technique. Each of these networks is further applied to both infrared (IR) and a combination of infrared and water vapor (IR+WV) satellite imagery. The best skills (97% in terms of the binary classification accuracy score) is achieved with the model that averages the estimates of the ensemble of different DCNNs. The algorithm can be further extended to the automatic identification and tracking numerical scheme and applied to other atmospheric phenomena characterized by a distinct signature in satellite imagery.

In this article, the evolution in space and in time of the coronavirus pandemic is studied by utilizing a neural network with a self-organizing nature for the spatial analysis of data, and a fuzzy fractal method for capturing the temporal trends of the time series of the countries. Self-organizing neural networks possess the capability for clustering countries in the space domain based on their similar characteristics with respect to their coronavirus cases. In this form enabling finding the countries that are having similar behavior and thus can benefit from utilizing the same methods in fighting the virus propagation. To validate the approach, publicly available datasets of coronavirus cases worldwide have been used. In addition, a fuzzy fractal approach is utilized for the temporal analysis of time series of the countries. Then, a hybrid combination of both the self-organizing maps and the fuzzy fractal approach is proposed for efficient COVID-19 forecasting of the countries. Relevant conclusions have emerged from this study, that may be of great help in putting forward the best possible strategies in fighting the virus pandemic. A lot of the existing works concerned with the Coronavirus have look at the problem mostly from the temporal viewpoint that is of course relevant, but we strongly believe that the combination of both aspects of the problem is relevant to improve the forecasting ability. The most relevant contribution of this article is the proposal of combining neural networks with a self-organizing nature for clustering countries with high similarity and the fuzzy fractal approach for being able to forecast the times series and help in planning control actions for the Coronavirus pandemic.

Deep neural networks (DNN) have successfully delivered a cutting-edge performance in several fields. With the broader deployment of DNN models on critical applications, the security of DNNs becomes an active and yet nascent area. Attacks against DNNs can have catastrophic results, according to recent studies. Poisoning attacks, including backdoor and Trojan attacks, are one of the growing threats against DNNs. Having a wide-angle view of these evolving threats is essential to better understand the security issues. In this regard, creating a semantic model and a knowledge graph for poisoning attacks can reveal the relationships between attacks across intricate data to enhance the security knowledge landscape. In this paper, we propose a DNN Poisoning Attacks Ontology (DNNPAO) that would enhance knowledge sharing and enable further advancements in the field. To do so, we have performed a systematic review of the relevant literature to identify the current state. We collected 28,469 papers from IEEE, ScienceDirect, Web of Science, and Scopus databases, and from these papers, 712 research papers were screened in a rigorous process, and 55 poisoning attacks in DNNs were identified and classified. We extracted a taxonomy of the poisoning attacks as a scheme to develop DNNPAO. Subsequently, we used DNNPAO as a framework to create a knowledge base. Our findings open new lines of research within the field of AI security.

Training data for a deep learning (DL) neural network (NN) controller are obtained from the input and output signals of a conventional digital controller that is designed to provide the suitable control signal to a specified plant within a feedback digital control system. It is found that if the DL controller is sufficiently deep (four hidden layers), it can outperform the conventional controller in terms of settling time of the system output transient response to a unit-step reference signal. That is, the DL controller introduces a damping effect. Moreover, it does not need to be retrained to operate with a reference signal of different magnitude, or under system parameter change. Such properties make the DL control more attractive for applications that may undergo parameter variation, like sensor networks.

Convolutional Neural Networks combined with autonomous drones are increasingly seen as enablers of partially automating the aircraft maintenance visual inspection process. Such an innovative concept can have a significant impact on aircraft operations. Through supporting aircraft maintenance engineers detect and classify a wide range of defects, the time spent on inspection can significantly be reduced. Examples of defects that can be automatically detected include aircraft dents, paint defects, cracks and holes, and lightning strike damage. Additionally, this concept could also increase the accuracy of damage detection and reduce the number of aircraft inspection incidents related to human factors like fatigue and time pressure. In our previous work, we have applied a recent Convolutional Neural Network architecture known by MASK R-CNN to detect aircraft dents. MASK-RCNN was chosen because it enables the detection of multiple objects in an image while simultaneously generating a segmentation mask for each instance. The previously obtained F1 and F2 scores were 62.67% and 59.35% respectively. This paper extends the previous work by applying different techniques to improve and evaluate prediction performance experimentally. The approaches uses include (1) Balancing the original dataset by adding images without dents; (2) Increasing data homogeneity by focusing on wing images only; (3) Exploring the potential of three augmentation techniques in improving model performance namely flipping, rotating, and blurring; and (4) using a pre-classifier in combination with MASK R-CNN. The results show that a hybrid approache combining MASK R-CNN and augmentation techniques leads to an improved performance with an F1 score of (67.50%) and F2 score of (66.37%)

Applications for Wireless Sensor Networks (WSNs) range from monitoring the environment to automating factories. However, sustained and effective functioning is made more difficult by Sensor Nodes (SNs) limited energy supplies in which optimization is the main issue. So with the aim of increasing the lifespan by decreasing the energy consumption of WSN, Low-Energy Adaptive Clustering Hierarchy (LEACH) protocol with Deep Learning (DL) algorithm is analyzed in this paper. LEACH is a hierarchical mechanism that elects Cluster Heads (CHs) and regularly rotates their positions in order to allocate energy use effectively by using the same amount of energy. However, Deep Learning (DL) method is used to further improve energy optimization. In many applications, the types of Deep Learning methods like Artificial Neural Networks (ANNs) have shown to be very useful. Using this method, WSNs may make more efficient decisions that reduce energy consumption. Data aggregation, duty cycling, and transmission protocols may all be optimized by Deep Learning model's ability to recognize patterns and forecast network behavior. This results in lower energy consumption, a longer lifespan for the network, and better overall performance.

Many engineered approaches have been proposed over the years for solving the hard problem of performing indoor localisation using smartphone sensors. However, specialising these solutions for difficult edge cases remains challenging. Here we propose an end-to-end hybrid multimodal deep neural network localisation system, MM-Loc, relying on zero hand-engineered features, learning them automatically from data instead. This is achieved by using modality-specific neural networks to extract preliminary features from each sensing modality, which are then combined by cross-modality neural structures. We show that our choice of modality-specific neural architectures is capable of estimating the location with good accuracy independently. But for better accuracy, a multimodal neural network fusing the features of early modality-specific representations is a better proposition. Our proposed MM-Loc solution is tested on cross-modality samples characterised by different sampling rates and data representation (inertial sensors, magnetic and WiFi signals), outperforming traditional approaches for location estimation. MM-Loc elegantly trains directly from data unlike conventional indoor positioning systems, which rely on human intuition.

Emotions play a vital role in understanding human behavior and interpersonal relationships. The ability to recognize emotions through Electroencephalogram (EEG) signals offers an alternative to traditional methods, such as questionnaires, enabling the identification of emotional states in a non-intrusive manner. Automatic emotion recognition holds great potential, eliminating the need for clinical examinations or physical visits, thereby contributing significantly to the advancement of Brain-Computer Interface (BCI) technology. However, one of the key challenges lies in effectively selecting and extracting relevant features from the EEG signal to establish meaningful distinctions between different emotional states. The process of feature selection is often time-consuming and demanding. In this research, we propose a groundbreaking approach for automatically identifying three emotional states (positive, negative, and neutral) by leveraging auditory stimulation of EEG signals. Our novel method directly applies the raw EEG signal to a Convolutional Neural Network-Long Short Term Memory (CNN-LSTM) architecture, bypassing the conventional feature extraction and selection steps. This unconventional approach offers a significant departure from existing literature. Our proposed network architecture comprises ten convolutional layers, followed by three LSTM layers and two fully connected layers. Through extensive simulations and evaluations on 12 active channels, our algorithm demonstrates exceptional performance, achieving an accuracy of 97.42\% and 95.23\% for the binary classification of negative and positive emotions, as well as a Cohen's Kappa coefficient of 0.96 and 0.93 for the three-class classification (negative, neutral, and positive), respectively. These promising results highlight the efficacy of our novel methodology and its potential implications in advancing emotion recognition using EEG signals.

Timely diagnosis of Alzheimer's diseases(AD) is crucial to obtain more practical treatments. In this paper, a novel approach Based on Multi-Level Fuzzy Neural Networks (MLFNN) for early detection of AD is proposed. The focus of study was on the problem of diagnosing AD and MCI patients from healthy people using MLFNN and selecting the best feature(s) and most compatible classification algorithm. In this way, we achieve an excellent performance using only a single feature i.e. MMSE score, by fitting the optimum algorithm to the best area using optimum possible feature(s) namely one feature for a real life problem. It can be said, the proposed method is a discovery that help patients and healthy people get rid of painful and time consuming experiments. Experiments shows the effectiveness of proposed method in current research for diagnosis of AD with one of the highest performance (accuracy rates of 96.6%), ever reported in the literature.

In considering knowledge graphs in a diverse range of domains of interest, graph neural networks have demonstrated significant improvements in node classification and prediction when applied to graph representation with learning node embedding to effectively represent hierarchical properties of graphs. DiffPool is a deep-learning approach using a differentiable graph pooling technique that generates hierarchical representations of graphs. In operation DiffPool is a differentiable graph pooling technique that generates hierarchical representations of graphs. DiffPool learns a differentiable soft cluster assignment for nodes at each layer of a deep graph neural network with nodes mapped sets of clusters. However, control of the learning process is difficult given the complexity and large number of parameters on an `end-to-end’ model. To address this difficulty we propose an novel approach termed FPool which is predicated on the basic approach adopted in DiffPool (where pooling is applied directly to node representations). Methods designed to enhance data classification have been developed and evaluated using a number popular and publicly available sensor data sets. Experimental results for FPool demonstrate improved classification and prediction performance when compared to alternative methods. Moreover, FPool shows an important reduction in the training time over the basic DiffPool framework.

Audio recording in classrooms is a common practice in educational research, with applications ranging from detecting classroom activities to analysing student behaviour. Previous research has employed neural networks for classroom activity detection and speaker role identification. However, these recordings are often affected by background noise that can hinder further analysis, and the literature has only sought to identify noise with general filters and not specifically designed for classrooms. Although the use of high-end microphones and environmental monitoring can mitigate this problem, these solutions can be costly and potentially disruptive to the natural classroom environment. In this context, we propose the development of a neural network model that can specifically detect and filter out background noise in classroom recordings. This model would allow the use of lower quality recordings without compromising analysis capability, thus facilitating data collection in natural educational environments and reducing the costs associated with high-end recording equipment.

This paper studies the finite-time synchronization problem of fractional-order stochastic memristive bidirectional associative memory neural networks (MBAMNNs) with discontinuous jumps. A novel criterion for finite-time synchronization is obtained by utilizing the properties of quadratic fractional-order Gronwall inequality with time delay and the comparison principle. This criterion provides a new approach to analyze the finite-time synchronization problem of neural networks with stochasticity. Finally, numerical simulations are provided to demonstrate the effectiveness and superiority of the obtained results.

Electrical power quality is one of the main elements in power generation systems. At the same time, it is one of the most significant challenges regarding stability and reliability. Due to different switching devices in this type of architecture, different kinds of power generators, and non-linear loads are used for different industrial processes. As a result of this, the need to classify and analyze Power quality disturbance (PQD) to prevent and analyze the degradation of the system reliability affected by the non-linear and non-stationary oscillatory nature. This paper presents A Novel Mul-titasking Deep Neural Network (MDL) for the Classification and Analysis of Multiple Electrical Disturbances. The characteristics are extracted with a specialized and adaptive methodology for non-stationary signals, Empirical Mode Decomposition (EMD). The methodology’s design, devel-opment, and various performance tests are carried out with 28 different difficulty levels, such as severity, disturbance duration time, and noise in the 20 dB to 60 dB signal range. MDL was devel-oped with a diverse data set in difficulty and noise, with a quantity of 4500 records of different samples of multiple electrical disturbances. The analysis and classification methodology has an average accuracy percentage of 95% with multiple disturbances. In addition, an average accuracy percentage of 90% in analyzing important signal aspects for studying electrical power quality such as crest factor, Per Unit voltage analysis, Short Term Flicker Perceptibility (Pst), and Total Harmonic Distortion (THD), among others.

This study focuses on the impact of weak partial discharges (PDs) on defect recognition accuracy in epoxy resin through phase-resolved partial discharge (PRPD) analysis. Two measurement conditions are compared until PRPD pattern saturation: one-minute and one-hour durations. The PD data specifically target three void types located at different positions within the epoxy material. The aim is to evaluate how the presence of weak PDs at the PD extinction voltage (PDEV) influences defect recognition accuracy. This research sheds light on the potential implications of neglecting the significance of weak PD signals in defect detection. A convolutional neural network (CNN) model is trained using PRPD data recorded at the PD inception voltage (PDIV) and tested using the new PRPD data from both conditions recorded from a lower PDIV to a PDEV. The trained CNN model achieves a defect recognition accuracy of 100% for a one-hour duration, highlighting the importance of not neglecting weak PD signals. This emphasizes the significance of extended measurement duration and pattern saturation in capturing and analyzing weak PD signals for improved defect recognition. This study contributes to the advancement of practical applications by understanding the behavior of the epoxy material and enhancing defect detection techniques.

The objective of this article was to forecast the ultimate failure load laminate stacking sequence combination on bonding joints which are exposed to tensile strength by using artificial neural networks. We have glass fiber composite materials with three different sequence combinations ([0°/90°], [±45°], [0°/90°/±45°]). Various adherend thicknesses and also ductile type adhesive was used in the experiment. The bonding geometry is a single lap and has four types of overlap angles 30°, 45°, 60°, 75° respectively. The experimental results demonstrate that composite laminate stacking sequence profoundly affects the bonding joints of failure load. Taking experimental results into account, Levenberg–Marquardt learning algorithm model was used by preferring a three layer forward on ANN so as to discipline network. In order to procure a precise ANN tool, an integrate methodology of experimental method has been used. The outcomes are used to ensure the experimental data’s to the ANN. The method of ANN permits surveying much adequately the probabilities of composite laminate stacking sequence combination using the prevalent ones which are [0°/90°], [±45°] and [0°/90°/±45°]. Testing data and training results were quite well 0.998, 0.997 and 0.998 in turn. Consequences acquired can be used by engineers who are interested in the composite material design to enhance failure load.

Predicting trip purpose from comprehensive and continuous smart card data is beneficial for transport and city planners in investigating travel behaviours and urban mobility. Here we propose a framework, ActivityNET, using machine learning (ML) algorithms to predict passengers’ trip purpose from smart card data and Points-of-Interest (POIs) data. The feasibility of the framework is demonstrated in two phases. Phase I focuses on extracting activities from individuals’ daily travel patterns from smart card data and combining them with POIs using the proposed ‘activity-POIs consolidation algorithm’. Phase II feeds the extracted features into an artificial neural network (ANN) with multiple scenarios and predicts trip purpose under primary activities (home and work) and secondary activities (entertainment, eating, shopping, child drop-offs/pick-ups and part-time work) with high accuracy. As a case study, the proposed ActivityNET framework is applied in Greater London and illustrates a robust competence to predict trip purpose. The promising outcomes demonstrate that the cost-effective framework offers high predictive accuracy and valuable insights into transport planning.

In this paper, the fixed-time stabilization problem for a class of uncertain chained system is addressed by using a novel nonsingular recursive terminal sliding mode control approach. A fixed-time controller and an adaptive law are designed to guarantee the uncertain chained form system both Lyapunov stable and fixed-time convergent within the settling time. The advantage of the controller based on the sliding mode is that the settling time does not depend on the system initial state. Furthermore, we use RBF neural network to estimate the uncertainty of the system. Finally, the simulation results demonstrate the performance of the control laws.

Automated lip reading is a research problem that has developed considerably in recent years. Lip reading is evaluated both visually and audibly in some cases. The lip reading model is a field of use for detecting specific words using images from security cameras, but it is not possible to use audio-visual databases in this situation. It is not possible to obtain the sound input of the pronounced word in all cases. We collected a new Turkish dataset with only the image in this study. The new dataset is produced using Youtube videos, which is an uncontrolled environment. For this reason, images have difficult parameters in terms of environmental factors such as light, angle, color, and personal characteristics of the face. Despite the different features on the human face such as mustache, beard, and make-up, the visual speech recognition problem was developed on 10 classes including single words and two-word phrases using Convolutional Neural Networks (CNN) without any intervention on the data. The proposed study using only-visual data obtained a model which is automated visual speech recognition with a deep learning approach. In addition, since this study uses only-visual data, the computational cost and resource usage is less than in multi-modal studies. It is also the first known study to address the lip reading problem with a deep learning algorithm using a new dataset belonging to the Ural-Altaic languages.

Sparse training (ST) aims to ameliorate deep learning by replacing fully connected artificial neural networks (ANNs) with sparse or ultra-sparse ones, such as brain networks are, therefore it might benefit to borrow brain-inspired learning paradigms from complex network intelligence theory. Here, we launch the ultrasparse advantage challenge, whose goal is to offer evidence on the extent to which ultra-sparse (around 1% connection retained) topologies can achieve any leaning advantage on fully connected. Epitopological learning is a field of network science and complex network intelligence that studies how to implement learning on complex networks by changing the shape of their connectivity structure (epitopological plasticity). One way to implement Epitopological (epi- means new) Learning is via link prediction: predicting the likelihood of nonobserved links to appear in the network. Cannistraci-Hebb learning theory inspired the CH3-L3 network automata rule for link prediction which is effective for general-purpose link prediction. Here, starting from CH3-L3 we propose Epitopological Sparse Meta-deep Learning (ESML) to apply Epitopological Learning to sparse training. In empirical experiments, we find that ESML learns ANNs with ultra-sparse hyperbolic (epi-)topology in which emerges a community layer organization that is meta-deep (meaning that each layer also has an internal depth due to powerlaw node hierarchy). Furthermore, we discover that ESML can in many cases automatically sparse the neurons during training (arriving even to 30% neurons left in hidden layers), this process of node dynamic removal is called percolation. Starting from this network science evidence, we design Cannistraci-Hebb training (CHT), a 4-step training methodology that put ESML at its heart. We conduct experiments on 6 datasets and 3 network structures (MLPs, VGG16, ResNet50) comparing CHT to dynamic sparse training SOTA algorithms and fully connected network. The results indicate that, with a mere 1% of links retained during training, CHT surpasses fully connected networks on VGG16 and ResNet50. This key finding is an evidence for ultra-sparse advantage and signs a milestone in deep learning. CHT acts akin to a gradient-free oracle which adopts CH3-L3 based epitopological learning to guide the placement of new links in the ultra-sparse network topology to facilitate sparse-weight gradient learning, and this in turn reduces the convergence time of ultra-sparse training. Finally, CHT offers first examples of parsimony dynamic sparse training because, in many datasets, it canretain network performance by percolating and significantly reducing the node network size.

Memristive spiking neural networks are a promising emerging technology in the field of deep learning. These models can be implemented using neuromorphic hardware and therefore offer low power consumption and low latency, crucial to edge-computing applications. These systems, however, pose several challenges, including on-chip training and connectivity reduction. The latter is essential to ease the manufacturing complexity of the system. On-chip training can be realized using synaptic plasticity enabled by memristors. In this work, we compare two methods of connectivity reduction applicable to memristive spiking networks: an ensemble-based approach and a probabilistic sparse connectivity approach. We evaluate both of these methods in conjunction with a three-layer spiking neural network on the handwritten and spoken digits classification tasks using two memristive plasticity models and a classical time-dependent plasticity rule. On the handwritten digits recognition task, both methods achieve the F1-score of 0.89–0.93, and yield the 0.80–0.96 F1-score on the spoken digits recognition task.

Gradient based methods are popularly used in training neural networks and can be broadly categorized into first and second order methods. Second order methods have shown to have better convergence compared to first order methods, especially in solving highly nonlinear problems. The BFGS quasi-Newton method is the most commonly studied second order method for neural network training. Recent methods have shown to speed up the convergence of the BFGS method using the Nesterov’s acclerated gradient and momentum terms. The SR1 quasi-Newton method though less commonly used in training neural networks, are known to have interesting properties and provide good Hessian approximations when used with a trust-region approach. Thus, this paper aims to investigate accelerating the Symmetric Rank-1 (SR1) quasi-Newton method with the Nesterov’s gradient for training neural networks and briefly discuss its convergence. The performance of the proposed method is evaluated on a function approximation and image classification problem.

As a relatively high percentage of adenoma polyps are missed, a computer-aided diagnosis (CAD) tool based on deep learning can aid the endoscopist in diagnosing colorectal polyps or colorectal cancer in order to decrease polyps missing rate and prevent colorectal cancer mortality. Convolutional Neural Network (CNN) is a deep learning method and has achieved better results in detecting and segmenting specific objects in images in the last decade than conventional models such as regression, support vector machines or artificial neural networks. In recent years, based on the studies in medical imaging criteria, CNN models have acquired promising results in detecting masses and lesions in various body organs, including colorectal polyps. In this review, the structure and architecture of CNN models and how colonoscopy images are processed as input and converted to the output are explained in detail. In most primary studies conducted in the colorectal polyp detection and classification field, the CNN model has been regarded as a black box since the calculations performed at different layers in the model training process have not been clarified precisely. Furthermore, I discuss the differences between the CNN and conventional models, inspect how to train the CNN model for diagnosing colorectal polyps or cancer, and evaluate model performance after the training process.

Although automated Acute Lymphoblastic Leukemia (ALL) detection is essential, it is challenging due to the morphological correlation between malignant and normal cells. The traditional ALL classification strategy is arduous, time-consuming, often suffers inter-observer variations, and necessitates experienced pathologists. This article has automated the ALL detection task, employing deep Convolutional Neural Networks (CNNs). We explore the weighted ensemble of deep CNNs to recommend a better ALL cell classifier. The weights are estimated from ensemble candidates' corresponding metrics, such as accuracy, F1-score, AUC, and kappa values. Various data augmentations and pre-processing are incorporated for achieving a better generalization of the network. We train and evaluate the proposed model utilizing the publicly available C-NMC-2019 ALL dataset. Our proposed weighted ensemble model has outputted a weighted F1-score of 88.6%, a balanced accuracy of 86.2%, and an AUC of 0.941 in the preliminary test set. The qualitative results displaying the gradient class activation maps confirm that the introduced model has a concentrated learned region. In contrast, the ensemble candidate models, such as Xception, VGG-16, DenseNet-121, MobileNet, and InceptionResNet-V2, separately produce coarse and scatter learned areas for most example cases. Since the proposed ensemble yields a better result for the aimed task, it can experiment in other domains of medical diagnostic applications.

This work is part of an ongoing research project to develop an unmanned flying social robot to monitor dependants at home in order to detect the person’s state and bring the necessary assistance. In this sense, this paper focuses on the description of a virtual reality (VR) simulation platform for the monitoring process of an avatar in a virtual home by a rotatory-wing autonomous unmanned aerial vehicle (UAV). This platform is based on a distributed architecture composed of three modules communicated through the Message Queue Telemetry Transport (MQTT) protocol: the UAV Simulator implemented in MATLAB/Simulink, the VR Visualiser developed in Unity, and the new emotion recognition (ER) System developed in Python. Using a face detection algorithm and a convolutional neural network (CNN), the ER System is able to detect the person’s face in the image captured by the UAV’s on-board camera and classify the emotion among seven possible ones (surprise, fear, happiness, sadness, disgust, anger or neutral expression). The experimental results demonstrate the correct integration of this new computer vision module within the VR platform, as well as the good performance of the designed CNN, with around 85% in the F1-score, a mean of the precision and recall of the model. The developed emotion detection system can be used in the future implementation of the assistance UAV that monitors dependent people in a real environment, since the methodology used is valid for images of real people.

The power system has rapidly grown and expanded over the past decades and has been experiencing major changes and challenges. The increase in energy demand and the modern advancements in the smart grid, such as solar and wind energies and electric vehicles, have led to complexity and complications for utilities. A further layer of complexity and difficulty was added by the rapid expansion of behind-the-meter (BTM) photovoltaic (PV) systems with various designs and characteristic features. The rapid increase and the invisible solar power (BTM) have led to fluctuations in power grid stability and reliability and to inefficiency. Accurate forecasting of load generation will help to assure optimal planning, minimize the negative effects of the PV systems, and minimize the operational and maintenance costs. The authors propose a solution that uses combinations of K-means clustering with neural network machine learning models, AMI real-world PV load generation, and weather data to forecast the generation load at customer locations to achieve a 2.49% error between actual and predicted generation load.

In recent years, the unmanned aerial vehicle (UAV) synthetic aperture radar (SAR) has become a highly sought-after topic for its wide applications in the field of target recognition, detection, and tracking. However, SAR automatic target recognition (ATR) models based on deep neural networks (DNN) are suffering from adversarial examples. Generally, non-cooperators rarely disclose any information about SAR-ATR models, making adversarial attacks challenging. In this situation, we propose Transferable Adversarial Network (TAN) to attack these models with highly transferable adversarial examples. The proposed method improves the transferability via a two-player game, in which we simultaneously train two encoder-decoder models: a generator that crafts malicious samples through a one-step forward mapping from original data, and an attenuator that weakens the effectiveness of malicious samples by capturing the most harmful deformations. In particular, compared to traditional iterative methods, our approach is able to one-step map original samples to adversarial examples, thus enabling real-time attacks. Experimental results indicate that the proposed approach achieves state-of-the-art transferability with acceptable adversarial perturbations and minimum time costs compared to existing attack methods, i.e., it excellently realizes real-time transferable adversarial attacks.

This paper proposes training improvement methods of artificial neural networks (ANN) trajectory predictors. First, a dynamic power system time-series trajectory is split into several different segments to simplify the original ANN training problem. Moreover, the time-derivative of the trajectory is included to obtain an augmented loss function. Compared to previous studies which mainly focused on increasing the prediction accuracy, the aim of these novel techniques is to reduce the computational burden where the ANN output performance is still acceptable. The effectiveness of the developed methods is validated based on the WSCC three-machine nine-bus and IEEE 39-bus system models. The mean absolute error (MAE) and trajectory prediction results are analyzed, in which the numbers of neurons, hidden layers, and training epochs are constrained during the ANN training process. Rotor-angle difference between generators and the system frequency are investigated as the dynamic trajectories of the power system models. It is revealed that when the ANN architecture and epochs are constrained, the MAE results can be reduced, and the ANN training results can better reflect the original trajectory using the improvement approaches.

Non-invasive, electroencephalography (EEG)-based brain-computer interfaces (BCIs) on motor imagery movements translate the subject’s motor intention into control signals through classifying the EEG patterns caused by different imagination tasks, e.g. hand movements. This type of BCI has been widely studied and used as an alternative mode of communication and environmental control for disabled patients, such as those suffering from a brainstem stroke or a spinal cord injury (SCI). Notwithstanding the success of traditional machine learning methods in classifying EEG signals, these methods still rely on hand-crafted features. The extraction of such features is a difficult task due to the high non-stationarity of EEG signals, which is a major cause for the stagnating progress in classification performance. Remarkable advances in deep learning methods allow end-to-end learning without any feature engineering, which could benefit BCI motor imagery applications. We developed three deep learning models: 1) a long short-term memory (LSTM); 2) a proposed spectrogram-based convolutional neural network model (pCNN); and 3) a recurrent convolutional neural network (RCNN), for decoding motor imagery movements directly from raw EEG signals without (manual) feature engineering. Results were evaluated on our own, publicly available, EEG data collected from 20 subjects and on an existing dataset known as 2b EEG dataset from "BCI Competition IV". Overall, better classification performance was achieved with deep learning models compared to state-of-the art machine learning techniques, which could chart a route ahead for developing new robust techniques for EEG signal decoding. We underpin this point by demonstrating the successful real-time control of a robotic arm using our CNN based BCI.

With the rapid expansion of intelligent resource-constrained devices and high-speed communication technologies, Internet of Things (IoT) has earned a wide recognition as the primary standard for low-power lossy networks (LLNs). Nevertheless, IoT infrastructures are vulnerable to cyber-attacks due to the constraints in computation, storage, and communication capacity of the endpoint devices. From one side, the majority of newly developed cyber-attacks are formed by slightly mutating formerly established cyber-attacks to produce a new attack tending to be treated as a normal traffic through the IoT network. From the other side, the influence of coupling the deep learning techniques with cybersecurity field has become a recent inclination of many security applications due to their impressive performance. In this paper, we provide a comprehensive development of a new intelligent and autonomous deep learning-based detection and classification system for cyber-attacks in IoT communication networks leveraging the power of convolutional neural networks, abbreviated as (IoT-IDCS-CNN). The proposed IoT-IDCS-CNN makes use of the high-performance computing employing the robust CUDA based Nvidia GPUs and the parallel processing employing the high-speed I9-Cores based Intel CPUs. In particular, the proposed system is composed of three subsystems: Feature Engineering subsystem, Feature Learning subsystem and Traffic classification subsystem. All subsystems are developed, verified, integrated, and validated in this research. To evaluate the developed system, we employed the NSL-KDD dataset which includes all the key attacks in the IoT computing. The simulation results demonstrated more than 99.3% and 98.2% of cyber-attacks’ classification accuracy for the binary-class classifier (normal vs anomaly) and the multi-class classifier (five categories) respectively. The proposed system was validated using k-fold cross validation method and was evaluated using the confusion matrix parameters (i.e., TN, TP, FN, FP) along with other classification performance metrics including precision, recall, F1-score, and false alarm rate. The test and evaluation results of the IoT-IDCS-CNN system outperformed many recent machine-learning based IDCS systems in the same area of study.