The Ocean Color - Simultaneous Marine and Aerosol Retrieval Tool (OC-SMART) is a robust data processing platform that supports a large array of multi-spectral and hyper-spectral sensors. It provides accurate aerosol optical depths and remote sensing reflectances (Rrs estimates) that can be used to generate products such as absorption coefficients due to phytoplankton and detritus/Gelbstoff as well as backscattering coefficients due to particulate matter. The OC-SMART platform yields improved performance in complex environments by utilizing scientific machine learning (SciML) in conjunction with comprehensive radiative transfer computations. This paper expands the capability of OC-SMART by quantifying uncertainties in ocean color retrievals. Bayesian inversion is used to relate measured top of atmosphere radiances and a priori data to estimate posterior probability density functions and associated uncertainties. A framework of the methodology and implementation strategy is presented and uncertainty estimates for Rrs retrievals are provided to demonstrate the approach by applying it to MODIS, OLCI Sentinel-3, and VIIRS sensor data.

Network Embedding (NE) is an important method to learn the representations of network via a low-dimensional space. Conventional NE models focus on capturing the structure information and semantic information of vertices while neglecting such information for edges. In this work, we propose a novel NE model named BimoNet to capture both the structure and semantic information of edges. BimoNet is composed of two parts, i.e., the bi-mode embedding part and the deep neural network part. For bi-mode embedding part, the first mode named add-mode is used to express the entity-shared features of edges and the second mode named subtract-mode is employed to represent the entity-specific features of edges. These features actually reflect the semantic information. For deep neural network part, we firstly regard the edges in a network as nodes, and the vertices as links, which will not change the overall structure of the whole network. Then we take the nodes' adjacent matrix as the input of the deep neural network as it can obtain similar representations for nodes with similar structure. Afterwards, by jointly optimizing the objective function of these two parts, BimoNet could preserve both the semantic and structure information of edges. In experiments, we evaluate BimoNet on three real-world datasets and task of relation extraction, and BimoNet is demonstrated to outperform state-of-the-art baseline models consistently and significantly.

Equality and incomparability multi-label ranking have not been introduced to learning before. This paper proposes new native ranker neural network to address the problem of multi-label ranking including incomparable preference orders using a new activation and error functions and new architecture. Preference Neural Network PNN solves the multi-label ranking problem, where labels may have indifference preference orders or subgroups which are equally ranked. PNN is a nondeep, multiple-value neuron, single middle layer and one or more output layers network. PNN uses a novel positive smooth staircase (PSS) or smooth staircase (SS) activation function and represents preference orders and Spearman ranking correlation as objective functions. It is introduced in two types, Type A is traditional NN architecture and Type B uses expanding architecture by introducing new type of hidden neuron has multiple activation function in middle layer and duplicated output layers to reinforce the ranking by increasing the number of weights. PNN accepts single data instance as inputs and output neurons represent the number of labels and output value represents the preference value. PNN is evaluated using a new preference mining data set that contains repeated label values which have not experimented on before. SS and PS speed-up the learning and PNN outperforms five previously proposed methods for strict label ranking in terms of accurate results with high computational efficiency.

Cognitive neuroscience is the study of how the human brain functions on tasks like decision making, language, perception and reasoning. Deep learning is a class of machine learning algorithms that use neural networks. They are designed to model the responses of neurons in the human brain. Learning can be supervised or unsupervised. Ngram token models are used extensively in language prediction. Ngrams are probabilistic models that are used in predicting the next word or token. They are a statistical model of word sequences or tokens and are called Language Models or Lms. Ngrams are essential in creating language prediction models. We are exploring a broader sandbox ecosystems enabling for AI. Specifically, around Deep learning applications on unstructured content form on the web.

Computer network attacks are evolving in parallel with the evolution of hardware and neural network architecture. Despite major advancements in Network Intrusion Detection System (NIDS) technology, most implementations still depend on signature-based intrusion detection systems, which can’t identify unknown attacks. Deep learning can help NIDS to detect novel threats since it has a strong generalization ability. The deep neural network’s architecture has a significant impact on the model’s results. We propose a genetic algorithm based model to find the optimal number of hidden layers and the number of neurons in each layer of the deep neural network (DNN) architecture for the network intrusion detection binary classification problem. Experimental results demonstrate that the proposed DNN architecture shows better performance than classical machine learning algorithms at a lower computational cost.

Computer-aided image diagnostics (CAD) have been used in many fields of diagnostic medicine. It relies heavily on classical computer vision and artificial intelligence. Quantum neural network (QNN) has been introduced by many researchers around the world and presented recently by research corporations such as Microsoft, Google, and IBM. In this paper, the investigation of the validity of using the QNN algorithm for machine-based breast cancer detection was performed. To validate the learnability of the QNN, a series of learnability tests were performed alongside with classical convolutional neural network (CCNN). QNN is built using the Cirq library to perform the assimilation of quantum computation on classical computers. Series of investigations were performed to study the learnability characteristics of QNN and CCNN under the same computational conditions. The comparison was performed for real Mammogram data sets. The investigations showed success in terms of recognizing the data and training. Our work shows better performance of QNN in terms of successfully training and producing a valid model for smaller data set compared to CCNN.

Convolutional neural networks (CNN) have attracted much attention as a commonly used method for hyperspectral image (HSI) classification in recent years, however, CNNs can only be applied to Euclidean data and have limitations in dealing with relationships due to the limitations of local feature extraction. However, each pixel of a hyperspectral image contains a set of spectral bands that are correlated and interact with each other, and the methods used to process Euclidean data cannot effectively obtain these correlations. In contrast, the graph convolutional network (GCN) can be used in non-Euclidean data, but usually leads to oversmoothing and ignoring local detail features due to the need for superpixel segmentation processing to reduce computational effort. To overcome the above problems, we constructed a network a fusion network based on GCN and CNN, which contains two branches: a graph convolutional network based on superpixel segmentation and a convolutional network with added attention mechanism. The graph convolu-tional branch can extract the structural features and capture the relationships between the nodes, and the convolutional branch can extract the detailed features in the local fine region. Owing to the fact that the features extracted from the two branches are different, the classification performance can be improved by fusing the complementary features extracted from the two branches. To vali-date the proposed algorithm, experiments were conducted on three widely used datasets, namely Indian Pines, Pavia University, and Salinas, and the overall accuracy of 98.78% was obtained in the Indian Pines dataset, and the overall accuracy of 98.99% and 98.69% was obtained in the other two datasets. The results showed that the proposed fusion network can obtain richer features and achieve high classification accuracy.

Nowadays smartphone utilization for disease diagnosis and remote health care applications has become promising due to their ubiquity. Here, a novel convolutional neural network method for detecting keratoconus that is wholly implemented on a smartphone is proposed. The proposed method provides accurate detection of over 72.9% for all stages of keratoconus. Preliminary results indicate 90%, 83%, 64% and 52% detection rate for severe, advanced, moderate and mild stages of disease, respectively.

The Station logo is a way for a TV station to claim copyright, which can realize the analysis and understanding of the video by the identification of the station logo, so as to ensure that the broadcasted TV signal will not be illegally interfered. In this paper, we design a station logo detection method based on Convolutional Neural Network by the characteristics of the station, such as small scale-to-height ratio change and relatively fixed position. Firstly, in order to realize the preprocessing and feature extraction of the station data, the video samples are collected, filtered, framed, labeled and processed. Then, the training sample data and the test sample data are divided proportionally to train the station detection model. Finally, the sample is tested to evaluate the effect of the training model in practice. The simulation experiments prove its validity.

The many-body problem in quantum physics originates from the difficulty of describing the non-trivial correlations encoded in the exponential complexity of the many-body wave function. Motivated by the Giuseppe Carleo's work titled solving the quantum many-body problem with artificial neural networks [Science, 2017, 355: 602], we focus on finding the NNQS approximation of the unknown ground state of a given Hamiltonian $H$ in terms of the best relative error and explore the influences of sum, tensor product, local unitary of Hamiltonians on the best relative error. Besides, we illustrate our method with some examples.

Layout estimation is a challenge of segmenting a cluttered room image into floor, walls and ceiling. We applied Double refinement network proved to be efficient in the depth estimation to generate heat maps for room key points and edges. Our method is the first not using encoder-decoder architecture for the room layout estimation. ResNet50 was utilized as a backbone for the network instead of VGG16 commonly used for the task, allowing the network to be more compact and faster. We designed a special layout score function and layout ranking algorithm for key points and edges output. Our method achieved the lowest pixel and corner errors on the LSUN data set. The input image resolution is 224*224.

Geospatial Information Systems (GIS) can provide a great environment for using machine learning algorithm for spatial data such as satellite images. Integrating this functionality with artificial intelligence algorithms for analyzing spatial data enables us to predict challenging disasters such as deforestation. Deforestation as an environmental problems has been recorded the most serious threat to environmental diversity and one of the main components of land-use change. In this paper, we investigate spatial distribution of deforestation using artificial neural networks and satellite imagery. We modeled deforestation process using various factors in determining the relationship between deforestation and environmental and socioeconomic factors. Hence, for this purpose, the proximity to roads and habitats, fragmentation of the forest, height from sea level, slope, and soil type are considered in the model. In this research, we modeled land cover changes (forests) to predict deforestation using an artificial neural network due to its significant potential for the development of nonlinear complex models. The procedure involves image registration and error correction, image classification, preparing deforestation maps, determining layers, and designing a multi-layer neural network to predict deforestation. The satellite images for this study are of a region in Hong Kong which are captured from 2012 to 2016. The results of the study demonstrate that neural networks approach for predicting deforestation can be utilized and its outcomes show the areas that destroyed during the research period.

Deep learning models are part of the family of artificial neural networks and, as such, it suffers of catastrophic interference when they learn sequentially. In addition, most of these models have a rigid architecture which prevents the incremental learning of new classes. To overcome these drawbacks, in this article we propose the Self-Improving Generative Artificial Neural Network (SIGANN), a type of end-to-end Deep Neural Network system which is able to ease the catastrophic forgetting problem when leaning new classes. In this method, we introduce a novelty detection model to automatically detect samples of new classes, moreover an adversarial auto-encoder is used to produce samples of previous classes. This system consists of three main modules: a classifier module implemented using a Deep Convolutional Neural Network, a generator module based on an adversarial autoencoder; and a novelty detection module, implemented using an OpenMax activation function. Using the EMNIST data set, the model was trained incrementally, starting with a small set of classes. The results of the simulation show that SIGANN is able to retain previous knowledge with a gradual forgetfulness for each learning sequence. Moreover, SIGANN can detect new classes that are hidden in the data and, therefore, proceed with incremental class learning.

While modeling data in reality, uncertainties in both data (aleatoric) and model (epistemic) are not necessarily Gaussian. The uncertainties could appear with multimodality or any other particular forms which need to be captured. Gaussian mixture models (GMMs) are powerful tools that allow us to capture those features from a unknown density with a peculiar shape. Inspired by Fourier expansion, we propose a GMM model structure to decompose arbitrary unknown densities and prove the property of convergence. A simple learning method is introduced to learn GMMs through sampled datasets. We applied GMMs as well as our learning method to two classic neural network applications. The first one is learning encoder output of autoencoder as GMMs for handwritten digits images generation. The another one is learning gram matrices as GMMs for style-transfer algorithm. Comparing with the classic Expectation maximization (EM) algorithm, our method does not involve complex formulations and is capable of achieving applicable accuracy.

There is a considerable amount of research in online social networks, most of which focuses on the structural analysis of social graphs. The interpersonal relationships of social networks, especially friend circle, can solve the cold start and sparsity problems, and through the relationship between social networks can effectively recommend users' favorite items (items), such as music , videos, brands/products, preferred tags, location, services, etc. User relationships in social networks are diverse and there are many different perspectives on different social networks. Associations among users can form multi-layered composite networks, and multi-layered social networks present new challenges and opportunities. Different relationships can influence users' preferences to different degrees, which in turn affects their behavior. Therefore, fusing multiple social networks is an effective way to improve recommendation. Although some studies have started to address multiple social network recommendations, simple linear superposition cannot reflect the coupling and nonlinear association between multiple social networks. In this paper, we propose a graph neural network recommendation model under social relationships based on this background. We first propose to compute the 2nd order collaborative signals and their intensities directly from the neighboring matrix for updating the node embedding of the graph convolution layer. Secondly by embedding historical evaluations, various social networks constituting different dimensions, the attention integration of user preferences by different social networks is achieved, and its effectiveness and scalability are demonstrated in theoretical derivation and experimental validation. The theoretical derivation and experimental validation demonstrate its effectiveness and scalability.

We study latching dynamics in the adaptive Potts model network, through numerical simulations with randomly and also weakly correlated patterns, and we focus on comparing its slowly and fast adapting regimes. A measure, Q, is used to quantify the quality of latching in the phase space spanned by the number of Potts states S, the number of connections per Potts unit C and the number of stored memory patterns p. We find narrow regions, or bands in phase space, where distinct pattern retrieval and duration of latching combine to yield the highest values of Q. The bands are confined by the storage capacity curve, for large p, and by the onset of finite latching, for low p. Inside the band, in the slowly adapting regime, we observe complex structured dynamics, with transitions at high crossover between correlated memory patterns; while away from the band latching transitions lose complexity in different ways: below, they are clear-cut but last so few steps as to span a transition matrix between states with few asymmetrical entries and limited entropy; while above, they tend to become random, with large entropy and bi-directional transition frequencies, but indistinguishable from noise. Extrapolating from the simulations, the band appears to scale almost quadratically in the p - S plane, and sublinearly in p - C. In the fast adapting regime the band scales similarly, and it can be made even wider and more robust, but transitions between anti-correlated patterns dominate latching dynamics. This suggest that slow and fast adaptation have to be integrated in a scenario for viable latching in a cortical system. The results for the slowly adapting regime, obtained with randomly correlated patterns, remain valid also for the case with correlated patterns, with just a simple shift in phase space.

Many engineered approaches have been proposed over the years for solving the hard problem of performing indoor localisation using smartphone sensors. However, specialising these solutions for difficult edge cases remains challenging. Here we propose an end-to-end hybrid multimodal deep neural network localisation system, MM-Loc, relying on zero hand-engineered features, learning them automatically from data instead. This is achieved by using modality-specific neural networks to extract preliminary features from each sensing modality, which are then combined by cross-modality neural structures. We show that our choice of modality-specific neural architectures is capable of estimating the location with good accuracy independently. But for better accuracy, a multimodal neural network fusing the features of early modality-specific representations is a better proposition. Our proposed MM-Loc solution is tested on cross-modality samples characterised by different sampling rates and data representation (inertial sensors, magnetic and WiFi signals), outperforming traditional approaches for location estimation. MM-Loc elegantly trains directly from data unlike conventional indoor positioning systems, which rely on human intuition.

Neural networks present the characteristics that the results are strongly dependent on the training data, the weight initialisation, and the hyper-parameters chosen. The determination of the distribution of a statistical estimator, as the Mean Squared Error (MSE) or the accuracy, is fundamental to evaluate the performance of a neural network model (NNM). For many machine learning models, as linear regression, it is possible to analytically obtain information as variance or confidence intervals on the results. Neural networks present the difficulty of being not analytically tractable due to their complexity. Therefore, it is impossible to easily estimate distributions of statistical estimators. When estimating the global performance of an NNM by estimating the MSE in a regression problem, for example, it is important to know the variance of the MSE. Bootstrap is one of the most important resampling techniques to estimate averages and variances, between other properties, of statistical estimators. In this tutorial, the application of two resampling (including bootstrap) techniques to the evaluation of neural networks’ performance is explained from both a theoretical and practical point of view. Pseudo-code of the algorithms is provided to facilitate their implementation. Computational aspects, as the training time, are discussed since resampling techniques always require to run simulations many thousands of times and, therefore, are computationally intensive. A specific version of the bootstrap algorithm is presented that allows the estimation of the distribution of a statistical estimator when dealing with an NNM in a computationally effective way. Finally, algorithms are compared on synthetically generated data to demonstrate their performance.

The recent worldwide outbreak of the novel corona-virus (COVID-19) opened up new challenges to the research community. Artificial intelligence (AI) driven methods can be useful to predict the parameters, risks, and effects of such an epidemic. Such predictions can be helpful to control and prevent the spread of such diseases. The main challenges of applying AI is the small volume of data and the uncertain nature. Here, we propose a shallow Long short-term memory (LSTM) based neural network to predict the risk category of a country. We have used a Bayesian optimization framework to optimized and automatically design country-specific networks. We have combined the trend data and weather data together for the prediction. The results show that the proposed pipeline outperforms against state-of-the-art methods for 170 countries data and can be a useful tool for such risk categorization. The tool can be used to predict long-duration outbreak of such an epidemic such that we can take preventive steps earlier.

Based on the satellite attitude control method, this paper proposes an attitude control method based on neural network disturbance compensation. The paper firstly analyzes the neural network algorithm and proposes an orthogonal least squares algorithm to implement network learning. In this paper, a set of high-precision directional neural network compensation controllers is designed for the attitude control of acupuncture small satellites. The feasibility of the improved orthogonal least-squared algorithm combined with the neural network supplementary control method in satellite attitude control is verified by experiments.

Convolutional neural network (CNN), a class of artificial neural network (ANN) is attracting interests of researchers in all research domain. CNN was invented for computer vision. They have also shown to be useful for semantic parsing, sentence modeling and other natural language processing related tasks. Here in this paper we discuss the basics of CNN models and their scope to provide a reference/baseline to the researchers interested in using CNN models in their research.

With the increase of the Artificial Neural Network (ANN), machine learning has taken a forceful twist in recent times. One of the most spectacular kinds of ANN design is the Convolutional Neural Network (CNN). The Convolutional Neural Network (CNN) is a technology that mixes artificial neural networks and up to date deep learning strategies. In deep learning, Convolutional Neural Network is at the center of spectacular advances. This artificial neural network has been applied to several image recognition tasks for decades and attracted the eye of the researchers of the many countries in recent years as the CNN has shown promising performances in several computer vision and machine learning tasks. This paper describes the underlying architecture and various applications of Convolutional Neural Network.

It is important to reduce the computation complexity while maintaining the accuracy of convolution neural networks. We deem it is possible to further reduce the network complexity while ensuring the accuracy. In this paper, we propose a novel feature extraction network called DSRNet which is lightweight but effective. DSRNet follows the basic ideas of stacking modules and short connection, introduces Depthwise Separable convolution and utilizes the Dilated convolution. The proposed network has fewer parameters and achieves outstanding speed. We conducted comprehensive experiments on CIFAR10, CIFAR100 and STL10 datasets, and the results showed the DSRNet has great performance improvement in terms of accuracy and speed.

An appearance model plays a crucial rule in multi-target tracking. In traditional approaches, the two steps of appearance modeling i.e visual representation and statistically similarity measure are modeled separately. Visual representation is achieved either through hand-crafted features or deep features and statically similarity is measure through a cross entropy loss function. A loss function based on cross-entropy (KL-divergence, mutual information) find closely related probability distribution for the targets. However, if the targets have similar visual representation, it ends up mixing the targets. To tackle this problem, we come up with a synergetic appearance model named Single Shot Appearance Model based on Siamese neural network. The network is trained with a contrastive loss function for finding the similarity between different targets in a single shot. The input to the network is two target patches and based on their similarity, a contrastive score is output by the network. The proposed model is evaluated on accumulative dissimilarity metric on three datasets. Quantitatively, promising results are achieved against three baseline methods.

In recent years, artificial neural networks (ANNs) have won numerous contests in pattern recognition, machine learning, and artificial intelligence. The basic unit of an ANN is to mimic neurons in the brain. Neuron in ANNs is expressed as f (wx+b) or f (wx). This structure does not consider the information processing capabilities of dendrites. However, recently, studies shown that dendrites participate in pre-calculation in the brain. Concretely, biological dendrites play a role in the pre-processing to the interaction information of input data. Therefore, it is time to perfect the neuron of the neural network. This paper, add dendrite processing section, presents a novel artificial neuron, according to our previous studies (CR-PNN or Gang transform). The dendrite processing section can be expressed as . Because I perfected the basic unit of ANNs— neuron, there are so many networks to try. This article gives the basic architecture for reference in future research.

Effectively avoiding gas accident is vital to the security of mineral manufacture, and the coal mine gas accident is often caused by gas concentration overrun. The prediction accuracy of gas emission quantity in coal mine is the key to solve this problem. To maintain concentration of gas in a secure range，grey theory and neural network model increasingly diffusely used in forecasting gas emission quantity in coal mine critically. Nevertheless, the limitation of the grey neural network model is that researchers merely bonded the conventional neural network and grey theory. To enhance accuracy of prediction, a modified grey GM(1,1) and RBF neural network model is proposed combined amended grey GM(1,1) model and RBF neural network model. Then the proposed model was put into simulation experiment which is built based on Matlab software. Ultimately, conclusion of the simulation experiment verified that the modified grey GM(1,1) and RBF neural network model not only boosts the precision of prediction, but also restricts relative error in a minimum range. This showed that the modified grey GM(1,1) and RBF neural network model achieves effectiveness in precision of prediction much better than grey GM(1,1) model and RBF neural network model.

Motivated by the way animals react in nature, we introduce a fully-connected neural-network model, which incorporates the external pattern presented to the system as a fundamental tool of the recognition process. In this new scenario, in the absence of this external field, memories are not attractors inside basins of attraction, as in usual attractor neural networks, although basins may be created according to the external pattern, thus allowing storing a much larger number of memories. The key point consists in calibrating the influence of the external pattern such as to cancel the noise generated by those memories not correlated with the external pattern.We illustrate how this proposal works by including this new contribution in the standard Hopfield model, showing a significant increase in its recognition capacity (typically by a factor $10^{2}$). As an additional feature of this model, we show its ability to react promptly to changes in the external environment, a crucial attribute of living beings. This procedure can be applied to a wide variety of neural-network models to increase considerably their recognition and reaction capabilities.

Significant progress has made in pattern recognition technology. However, one obstacle that has not yet overcome is the recognition of words in the Brahmi script, specifically the identification of characters, compound characters, and word. This study proposes the use of the deep convolutional neural network with dropout to recognize the Brahmi words. This study also proposed a DCNN for Brahmi word recognition and a series of experiments are performed on standard Brahmi dataset. The practical operation of this method was systematically tested on accessible Brahmi image database, achieving 92.47% recognition rate by CNN with dropout respectively which is among the best while comparing with the ones reported in the literature for the same task.

In this study, we explore the potential of using a straightforward neural network inspired by the retina model to efficiently restore low-light images. The retina model imitates the neurophysiological principles and dynamics of various optical neurons. Our proposed neural network model reduces the computational overhead compared to traditional signal-processing models while achieving results similar to complex deep learning models from a subjective perceptual perspective. By directly simulating retinal neuron functionalities with neural networks, we not only avoid manual parameter optimization but also lay the groundwork for constructing artificial versions of specific neurobiological organizations.

In videos, the human's actions are of three-dimensional (3D) signals. These videos investigate the spatiotemporal knowledge of human behavior. The promising ability is investigated using 3D convolution neural networks (CNNs). The 3D CNNs have not yet achieved high output for their well-established two-dimensional (2D) equivalents in still photographs. Board 3D Convolutional Memory and Spatiotemporal fusion face training difficulty preventing 3D CNN from accomplishing remarkable evaluation. In this paper, we implement Hybrid Deep Learning Architecture that combines STIP and 3D CNN features to enhance the performance of 3D videos effectively. After implementation, the more detailed and deeper charting for training in each circle of space-time fusion. The training model further enhances the results after handling complicated evaluations of models. The video classification model is used in this implemented model. Intelligent 3D Network Protocol for Multimedia Data Classification using Deep Learning is introduced to further understand space-time association in human endeavors. In the implementation of the result, the well-known dataset, i.e., UCF101 to, evaluates the performance of the proposed hybrid technique. The results beat the proposed hybrid technique that substantially beats the initial 3D CNNs. The results are compared with state-of-the-art frameworks from literature for action recognition on UCF101 with an accuracy of 95%.

The diffusion of molecules in aqueous solutions in the domain of membrane technology is very critical in the efficiency of chemical engineering and purification processes. In this study, the diffusion in high and low concentration regions is simulated with finite difference method (FDM), and then the results of numerical computations are coupled with adaptive network-based fuzzy inference system (ANFIS) and bilayered neural network method (BNNM). Machine learning approach can individually predict diffusion phenomena across the domain based on understanding of the machine instead of the discretization of an ordinary differential equation (ODE). The findings of the machine learning method are in good agreement with those of FDM at different times of the simulation. In addition to numerical computation, the error of the system is computed for different iterations. The results show that by increasing the number of iterations and training datasets, all errors reduce significantly for both training and testing. BNN method is also used to train the prediction process of diffusion for a more accurate comparison. This technique is similar to ANFIS method in terms of prediction capability. According to the findings, ANFIS approach predicts diffusion slightly better than BNN method. In this regard, ANFIS technique produces R>0.99 while BNN method produces R around 0.98. Both machine learning methods are accurate enough to predict diffusion throughout the domain for different time steps. The computational time for both algorithms is less than that of FDM method to predict low and high concentrations in the domain. Besides, based on the results, artificial intelligence (AI) can find the relationship between inputs and outputs and determine which input has the main influence on the output in this study to optimize the process. As such, future studies can be focused on AI and other methods for faster prediction and optimization processes.

Accurately mapping individual tree species in densely forested environments is crucial to forest inventory. When considering only RGB images, this is a challenging task for many automatic photogrammetry processes. The main reason for that is the spectral similarity between species in RGB scenes, which can be a hindrance for most automatic methods. State-of-the-art deep learning methods could be capable of identifying tree species with an attractive cost, accuracy, and computational load in RGB images. This paper presents a deep learning-based approach to detect an important multi-use species of palm trees (Mauritia flexuosa; i.e., Buriti) on aerial RGB imagery. In South-America, this palm tree is essential for many indigenous and local communities because of its characteristics. The species is also a valuable indicator of water resources, which comes as a benefit for mapping its location. The method is based on a Convolutional Neural Network (CNN) to identify and geolocate singular tree species in a high-complexity forest environment, and considers the likelihood of every pixel in the image to be recognized as a possible tree by implementing a confidence map feature extraction. This study compares the performance of the proposed method against state-of-the-art object detection networks. For this, images from a dataset composed of 1,394 airborne scenes, where 5,334 palm-trees were manually labeled, were used. The results returned a mean absolute error (MAE) of 0.75 trees and an F1-measure of 86.9%. These results are better than both Faster R-CNN and RetinaNet considering equal experiment conditions. The proposed network provided fast solutions to detect the palm trees, with a delivered image detection of 0.073 seconds and a standard deviation of 0.002 using the GPU. In conclusion, the method presented is efficient to deal with a high-density forest scenario and can accurately map the location of single species like the M flexuosa palm tree and may be useful for future frameworks.

One of the most dangerous diseases that threaten people is Cancer. Cancer if diagnosed in earlier stages can be eradicated with its life threatening consequences. In addition, accuracy in prediction plays a major role. Hence, developing a reliable model that contributes much towards the medical community in early diagnosis of Biopsy images with perfect accuracy come to the scenario. The article aims towards development of better predictive models using multi-variate data and high-resolution diagnostic tools in clinical cancer research. This paper proposes the social spider optimization (SSO) algorithm tuned neural network to classify microscopic biopsy images of cancer. The significance of the proposed model relies on the effective tuning of the weights of the NN classifier by the SSO algorithm. The performance of the proposed strategy is analysed with the performance metrics, such as accuracy, sensitivity, specificity, and MCC measures, and are attained to be 95.9181%, 94.2515%, 97.125%, and 97.68% respectively, which shows the effectiveness of the proposed method in effective cancer disease diagnosis.

Activated sludge process is a well known method to treat municipal and industrial waste water. In this complex process, the oxygen concentration in the reactors plays a critical role in the plant efficiency. This paper proposes the use of a Long Short-Term Memory (LSTM) network to identify an input-output model suitable for the design of an oxygen concentration controller. The model is identified from easy-accessible measures collected from a real plant. This dataset covers almost a month. The performances achieved with the proposed LSTM model are compared with the ones obtained with a standard AutoRegressive model with eXogenous input (ARX). Both models catch the oscillation and the overall behaviour (ARX ρ=0.833 , LSTM ρ=0.921), but, while the ARX model fails in reaching the correct amplitude (FIT=41.20%), the LSTM presents satisfactory performance (FIT=60.56%).

The diagnosis of papillary thyroid carcinoma has always been a concerned and challenging issue and it is very important and meaningful to have a definite diagnosis before the operation. In this study, we tried to use an artificial intelligence algorithm instead of medical statistics to analyze the genetic fingerprint from gene chip results to identify papillary thyroid carcinoma. We trained 20 artificial neural network models with differential genes and other important genes related to the cell metabolic cycle as the list of input features, and apply them to the diagnosis of papillary thyroid cancer in the independent validation data set. The results showed that when we used the DEGs and all genes lists as input features the models got the best diagnostic performance with AUC=98.97% and 99.37% and the accuracy were both 96%. This study revealed that the proposed artificial neural network models constructed with genetic fingerprints could achieve a prediction of papillary thyroid carcinoma. Such models can support clinicians to make more accurate clinical diagnoses. At the same time, it provides a novel idea for the application of artificial intelligence in clinical medicine.

Recent neuroscience research results show that the nerve information in the brain is not only encoded by the spatial information. Spiking neural network based on pulse frequency coding plays a very important role in dealing with the problem of brain signal, especially complicated space-time information. In this paper, an unsupervised learning algorithm for bilayer feedforward spiking neural networks based on spike-timing dependent plasticity (STDP) competitiveness is proposed and applied to SAR image classification on MSTAR for the first time. The SNN learns autonomously from the input value without any labeled signal and the overall classification accuracy of SAR targets reached 80.8%. The experimental results show that the algorithm adopts the synaptic neurons and network structure with stronger biological rationality, and has the ability to classify targets on SAR image. Meanwhile, the feature map extraction ability of neurons is visualized by the generative property of SNN, which is a beneficial attempt to apply the brain-like neural network into SAR image interpretation.

Online transactions are becoming more popular in present situation where the globe is facing an unknown disease COVID-19. Now authorities of Countries requested peoples to use cashless transaction as far as possible. Practically it is not always possible to use it in all transactions. Since number of such cashless transactions have been increasing during lockdown period due to COVID-19, fraudulent transactions are also increasing in a rapid way. Fraud can be analysed by viewing a series of customer transactions data that was done in his/her previous transactions. Normally banks or other transaction authorities warned their customers about the transaction If any deviation is noticed by them from available patterns. These authorities think that it is possibly of fraudulent transaction. For detection of fraud during COVID-19, banks and credit card companies are applying various methods such as data mining , decision tree, rule based mining, neural network, fuzzy clustering approach and machine learning methods. These approaches is try to find out normal usage pattern of customers based on their past activities. The objective of this paper is to find out such fraud transactions during such unmanageable situation.Digital payment schemes are often threatened by fraudulent activities. Detecting fraud transaction in during money transfer may save customers from financial loss. Mobile based money transactions are focused in this paper for fraud detection. A Deep Learning (DL) framework is suggested in this paper that monitors and detects fraudulent activities. Implementing and applying recurrent neural network on PaySim generated synthetic financial dataset, deceptive transactions are identified. The proposed method is capable to detect deceptive transactions with an accuracy of 99.87%, F1-Score of 0.99 and MSE of 0.01.

Myocarditis is the form of an inflammation of the middle layer of the heart wall which is caused by a viral infection and can affect the heart muscle and its electrical system. It has remained as one of the most challenging diagnoses in cardiology. Myocardial is the prime cause of unexpected death in approximately 20% of adults less than 40 years of age. Cardiac MRI (CMR) has been considered as a noninvasive and golden standard diagnostic tool for suspected myocarditis and plays an indispensable role in diagnosing various cardiac diseases. However, the performance of CMR is heavily dependent on the clinical presentation and non-specific features such as chest pain, arrhythmia, and heart failure. Besides, other imaging factors like artifacts, technical errors, pulse sequence, acquisition parameters, contrast agent dose, and more importantly qualitatively visual interpretation can affect the result of the diagnosis. This paper introduces a new deep learning-based model called Convolutional Neural Network-Clustering (CNN-KCL) to diagnose the Myocarditis. The hybrid CNN-KCL method performs the early and accurate diagnosis of Myocarditis. To the best-of-our-knowledge, a Convolutional neural network has never been used before for the diagnosis of Myocarditis. In this study, we used 47 subjects to diagnose myocarditis patients from Tehran's Omid Hospital. The total number of data examined is 10425. Our results demonstrate that CNN-KCL achieves 92.3% in terms of diagnosis myocarditis prediction accuracy which is significantly better than those reported in previous studies.

This paper focuses on quantifying the uncertainty in the specific absorption rate values of brain induced by the uncertain positions of the electroencephalography electrodes placed on patient's scalp. To avoid running a large number of simulations, an artificial neural network architecture for uncertainty quantification involving high-dimensional data is proposed in this paper. The proposed method is demonstrated to be an attractive alternative to conventional uncertainty quantification methods because of its considerable advantage in the computational expense and speed.

Link prediction is to complete the missing links in the network or to predict the generation of new links according to the current network structure information, which is very important for mining and analyzing the evolution of the network such for construction and analysis of logical architecture in 6G network. Link prediction algorithms based on node similarity need predefined similarity functions, which is highly hypothetical and only applies to specific network structures without generality. To solve this problem, this paper proposes a link prediction algorithm based on the subgraph of the target node pair. In order to automatically learn the graph structure characteristics, the algorithm firstly extracts the h-hop subgraph of the target node pair, and then predicts whether the target node pair will be linked according to the subgraph. Experiments on seven real data sets show that the link prediction algorithm based on target node pair subgraph is suitable for various network structures and superior to other link prediction algorithms.

With the development of deep learning technology, various structures and research methods for super-resolution restoration of natural images and document images have been introduced. In particular, a number of recent studies have been conducted and developed in image restoration using generative adversarial network. Super-resolution restoration is ill-posed problem because of some complex restraints such as a lot of high-resolution images being restored for the same low-resolution image and also difficulty in restoring noises like edges, light smudging, and blurring. In this study, we utilized the spatially adaptive de-normalization (SPADE) structure for document image restoration to solve previous problems such as edge unclearness, hardness to catch features of texts, and the image color transition. Consequently, it can be confirmed that the edge of the character and the ambiguous stroke are restored more clearly when contrasting with the other previously suggested methods. Also, the proposed method’s PSNR and SSIM scores are geting 8% and 15% higher, respectively, compared to the previous methods.

In this work we present the results obtained with an artificial neural network (ANN) which we trained to predict the expected output of high-order harmonic generation (HHG) process, while exploring a multi-dimensional parameter space. We argue on the utility and efficiency of the ANN model and demonstrate its ability to predict the outcome of HHG simulations. In this case study we present the results for a loose focusing HHG beamline, where the changing parameters are: the laser pulse energy, gas pressure, gas cell position relative to focus and gas cell length. The physical quantity which we predict here using ANN is directly related to the total harmonic yield in a specified spectral domain (20-40 eV). We discuss the versatility and adaptability of the presented method.

Recent advancement of deep learning has been elevated the multifaceted nature in various applications of this field. Artificial neural networks are now turning into a genuinely old procedure in the vast area of computer science; the principal thoughts and models are more than fifty years of age. However, in this modern computing era, 3rd generation intelligent models are introduced by scientists. In the biological neuron, actual film channels control the progression of particles over the layer by opening and shutting in light of voltage changes because of inborn current flows and remotely led to signals. A comprehensive 3rd generation, Spiking Neural Network (SNN) is diminishing the distance between deep learning, machine learning, and neuroscience in a biologically-inspired manner. It also connects neuroscience and machine learning to establish high-level efficient computing. Spiking Neural Networks initiate utilizing spikes, which are discrete functions that happen at focuses as expected, as opposed to constant values. This paper is a review of the biological-inspired spiking neural network and its applications in different areas. The author aims to present a brief introduction to SNN, which incorporates the mathematical structure, applications, and implementation of SNN. This paper also represents an overview of machine learning, deep learning, and reinforcement learning. This review paper can help advanced artificial intelligence researchers to get a compact brief intuition of spiking neural networks.

Lung cancer is known as lung carcinoma. It is a disease which is malignant tumor leading to the uncontrolled cell growth in the lung tissue. Lung Cancer disease is one of the most prominent cause of death in all over world. Early detection of this disease can assist medical care unit as well as physicians to provide counter measures to the patients. The objective of this paper is to approach an automated tool that takes influential causes of lung cancer as input and detect patients with higher probabilities of being affected by this disease. A neural network classifier accompanied by cross-validation technique is proposed in this paper as a predictive tool. Later, this proposed method is compared with another baseline classifier Gradient Boosting Classifier in order to justify the prediction performance.

An application of deep convolutional neural network and recurrence plot for financial market movement prediction is presented. Though it is challenging and subjective to interpret its information, the pattern formed by a recurrence plot provide a useful insight into the dy- namical system. We used a recurrence plot of seven financial time series to train a deep neural network for financial market movement predic- tion. Our approach is tested on our dataset and achieved an average of 53.25% classification accuracy. The result suggests that a well trained deep convolutional neural network can learn a recurrence plot and pre- dict a financial market direction.

The basic challenges of this work are twofold: demonstrating the dependence between the functional and topological qualities of partition networks and finding the simplest—with respect to algorithmic complexity—network elements. The study of these problems is based on finding the solution to an appropriate antagonistic vertex game. The results of the numerical simulations of antagonistic partition games demonstrate that the winner’s graphs are “almost always” dense and hyperenergetic compared to the loser’s graphs. These observations reveal that successful evolutionary mechanisms can be realized, in principle, by the simplest objects (such as viruses).

This work aims to contribute to better understanding the use of public street spaces. (1) Background: In this sense, with a multidisciplinary approach, the objective of this work is to propose an experimental and reproducible method on a large scale. (2) Study area: The applied methodology uses artificial intelligence to analyze Google Street View (GSV) images at street level. (3) Method: The purpose is to validate a methodology that allows to characterize and quantify the use (pedestrians and cars) of some squares in Rome belonging to different historical periods. (4) Results: Through the use of machine vision techniques, typical of artificial intelligence and which use convolutional neural networks, a historical reading of some selected squares is proposed with the aim of interpreting the dynamics of use and identifying some critical issues in progress. (5) Conclusions: This work validated the usefulness of a method applied to the use of artificial intelligence for the analysis of GSV images at street level.

Recently, much attention has been devoted to finding highly efficient and powerful activation functions for CNN layers. Because activation functions inject different nonlinearities between layers that affect performance, varying them is one method for building robust ensembles of CNNs. The objective of this study is to examine the performance of CNN ensembles made with different activation functions, including six new ones presented here: 2D Mexican ReLU, TanELU, MeLU+GaLU, Symmetric MeLU, Symmetric GaLU, and Flexible MeLU. The highest performing ensemble was built with CNNs having different activation layers that randomly replaced the standard ReLU. A comprehensive evaluation of the proposed approach was conducted across fifteen biomedical data sets representing various classification tasks. The proposed method was tested on two basic CNN architectures: Vgg16 and ResNet50. Results demonstrate the superiority in performance of this approach. The MATLAB source code for this study will be available at https://github.com/LorisNanni.

In this effort we propose a data-driven learning framework for reduced order modeling of fluid dynamics. Designing accurate and efficient reduced order models for nonlinear fluid dynamic problems is challenging for many practical engineering applications. Classical projection-based model reduction methods generate reduced systems by projecting full-order differential operators into low-dimensional subspaces. However, these techniques usually lead to severe instabilities in the presence of highly nonlinear dynamics, which dramatically deteriorates the accuracy of the reduced-order models. In contrast, our new framework exploits linear multistep networks, based on implicit Adams-Moulton schemes, to construct the reduced system. The advantage is that the method optimally approximates the full order model in the low-dimensional space with a given supervised learning task. Moreover, our approach is non-intrusive, such that it can be applied to other complex nonlinear dynamical systems with sophisticated legacy codes. We demonstrate the performance of our method through the numerical simulation of a two-dimensional flow past a circular cylinder with Reynolds number Re = 100. The results reveal that the new data-driven model is significantly more accurate than standard projection-based approaches

Intelligent systems are transforming the world, as well as our healthcare system. We propose a deep learning-based cough sound classification model that can distinguish between children with healthy versus pathological coughs such as asthma, upper respiratory tract infection (URTI), and lower respiratory tract infection (LRTI). In order to train a deep neural network model, we collected a new dataset of cough sounds, labelled with clinician's diagnosis. The chosen model is a bidirectional long-short term memory network (BiLSTM) based on Mel Frequency Cepstral Coefficients (MFCCs) features. The resulting trained model when trained for classifying two classes of coughs -- healthy or pathology (in general or belonging to a specific respiratory pathology), reaches accuracy exceeding 84\% when classifying cough to the label provided by the physicians' diagnosis. In order to classify subject's respiratory pathology condition, results of multiple cough epochs per subject were combined. The resulting prediction accuracy exceeds 91\% for all three respiratory pathologies. However, when the model is trained to classify and discriminate among the four classes of coughs, overall accuracy dropped: one class of pathological coughs are often misclassified as other. However, if one consider the healthy cough classified as healthy and pathological cough classified to have some kind of pathologies, then the overall accuracy of four class model is above 84\%. A longitudinal study of MFCC feature space when comparing pathologicial and recovered coughs collected from the same subjects revealed the fact that pathological cough irrespective of the underlying conditions occupy the same feature space making it harder to differentiate only using MFCC features.

Epilepsy is a neurological disorder that affects approximately 1% of the world's population. To diagnose and estimate the occurrence of epilepsy, the analysis of recorded brain activity is performed by a neurologist, which is not only time-consuming and tedious but also occasionally accompanied by human error. Therefore, in recent decades, researchers have aimed to unravel an approach for designing and building an automated method for diagnosing and estimating the occurrence of epilepsy. Accordingly, the present study proposed two new-fangled ways based on brain signals and a convolutional neural network (CNN). Moreover, this research implements a CNN with a sequential three-layer structure. Numerous experiments were performed, and the accuracy of estimating epilepsy using the developed methods was achieved at 95% without feedback and 97% with feedback. The proposed methods were proven to be more accurate than the previous techniques and can be employed as a physician's assistant once entering the field of operation.

Deep learning has achieved lots of successes in many fields, but when trainable sample are extremely limited, deep learning often under or overfitting to few samples. Meta-learning was proposed to solve difficulties in few-shot learning and fast adaptive areas. Meta-learner learns to remember some common knowledge by training on large scale tasks sampled from a certain data distribution to equip generalization when facing unseen new tasks. Due to the limitation of samples, most approaches only use shallow neural network to avoid overfitting and reduce the difficulty of training process, that causes the waste of many extra information when adapting to unseen tasks. Euclidean space-based gradient descent also make meta-learner's update inaccurate. These issues cause many meta-learning model hard to extract feature from samples and update network parameters. In this paper, we propose a novel method by using multi-stage joint training approach to post the bottleneck during adapting process. To accelerate adapt procedure, we also constraint network to Stiefel manifold, thus meta-learner could perform more stable gradient descent in limited steps. Experiment on mini-ImageNet shows that our method reaches better accuracy under 5-way 1-shot and 5-way 5-shot conditions.

In recent years, data providers are generating and streaming a large number of images. More particularly, processing images that contain faces have received great attention due to its numerous applications, such as entertainment and social media apps. The enormous amount of images shared on these applications presents serious challenges and requires massive computing resources to ensure efficient data processing. However, images are subject to a wide range of distortions in real application scenarios during the processing, transmission, sharing, or combination of many factors. So, there is a need to guarantee acceptable delivery content, even though some distorted images do not have access to their original version. In this paper, we present a framework developed to estimate the images' quality while processing a large number of images in real-time. Our quality evaluation is measured using an integration of a deep network with random forests. In addition, a face alignment metric is used to assess the facial features. Experimental results have been conducted on two artificially distorted benchmark datasets, LIVE and TID2013. We show that our proposed approach outperforms the state-of-art methods, having a Pearson Correlation Coefficient (PCC) and Spearman Rank Order Correlation Correlation Coefficient (SROCC) with subjective human scores of almost 0.942 and 0.931 while minimizing the processing time from 4.8ms to 1.8ms.

Fusarium head blight (FHB) disease is extensively distributed worldwide. This disease damages grain quality and reduces yield. The detection of this disease in a high throughput way is crucial to planters and breeders. Our study focused on developing a method for processing wheat color images and accurately detecting disease areas using deep learning and image processing techniques. The color images of wheat at the milky stage were collected and processed to construct datasets, which were used to retrain a deep convolutional neural network model using transfer learning. Testing results showed that the model can detect spikes, and the coefficient of determination of the number of spikes between the manual count and the detection was 0.80. The model was assessed, and the mean average precision for the testing dataset was 0.9201. On the basis of the results of spike detection, a new color feature was applied to obtain the gray image of each spike. Then, a modified region growing algorithm was implemented to segment and detect the diseased areas of each spike. Results show that the region growing algorithm performs better than K-means and Otsu’s method in segmenting the FHB disease. Overall, this study demonstrates that deep learning techniques enable the accurate detection of FHB in wheat using color images, and the proposed method can effectively detect spikes and diseased areas, thereby improving the efficiency of FHB detection.

In this paper, the fixed-time stabilization problem for a class of uncertain chained system is addressed by using a novel nonsingular recursive terminal sliding mode control approach. A fixed-time controller and an adaptive law are designed to guarantee the uncertain chained form system both Lyapunov stable and fixed-time convergent within the settling time. The advantage of the controller based on the sliding mode is that the settling time does not depend on the system initial state. Furthermore, we use RBF neural network to estimate the uncertainty of the system. Finally, the simulation results demonstrate the performance of the control laws.

The fault diagnosis of safety critical systems such as wind turbine installations includes extremely challenging aspects that motivate the research issues considered in this paper. In fact, the prompt detection and the reliable diagnosis of faults in their earlier occurrence represent the key point especially for offshore installations. For these plants, operation and maintenance procedures in harsh environments would inevitably increase the cost of the energy production. Therefore, this work investigates fault diagnosis solutions that are considered in a viable way and used as advanced techniques for condition monitoring of dynamic processes. To this end, the work proposes the design of fault diagnosis strategies that exploit the estimation of the fault by means of data--driven approaches. This solution leads to the development of effective methods allowing the management of partially unknown information of the system dynamics, while coping with measurement errors, the model--reality mismatch and other disturbance effects. In mode detail, the proposed data--driven methodologies exploit fuzzy systems and neural networks in order to estimate the nonlinear dynamic relations between the input and output measurements of the considered process and the faults. To this end, the fuzzy and neural network structures are integrated with auto--regressive with exogenous input descriptions, thus making them able to approximate unknown nonlinear dynamic functions with arbitrary degree of accuracy. Once these models are estimated from the input and output data measurement acquired from the considered dynamic process, the capabilities of their fault diagnosis capabilities are validated by using a high--fidelity benchmark that simulates the healthy and the faulty behaviour of a wind turbine system. Moreover, at this stage the benchmark is also useful to analyse the robustness and the reliability characteristics of the developed tools in the presence of model--reality mismatch and modelling error effects featured by the wind turbine simulator. On the other hand, a hardware--in--the--loop tool is finally implemented for testing and comparing the performance of the developed fault diagnosis strategies in a more realistic environment and with respect to different fault diagnosis approaches.

Multi-floor environments are usually ignored while designing an autonomous robot for indoor cleaning applications. However, for efficient operation in such environments, the ability of a robotic platform to traverse staircases is crucial. Staircase detection and localization is highly important for planning the traversal on staircases. This paper describes a deep learning approach using Convolutional Neural Networks (CNNs) based Robot Operation System (ROS) to staircase detection and localization. We use an object detection network to detect staircases in images. We also localize these staircases using a contour detection algorithm to detect the target point, a point close to the center of the first step, and the angle of approach to the target point. Experiments are performed with data obtained from images captured on different types of staircases at different view points/angles. Results show that the approach is very accurate in identifying the presence of the staircase in the working environment and is also able to locate the target point with good accuracy.

Objectives: In order to solve the problem that most of the existing research focuses on the binary tactile attributes of objects,which ignores the tactile attribute strength and category recognition,an attribute strength and category recognition method based on convolutional neural network matrix-label is proposed. Methods:Firstly,in the data preparation stage,we preprocess the raw data and determine the matrix labels to build the haptic dataset.Secondly,in the feature extraction stage,we fuse the haptic data of two fingers and use the convolutional neural network to extract the attribute strength features.Finally,in the attribute strength and category recognition stage,all channel haptic data is fused to predict the attribute strength and category.Results:We compared with the multi-label convolutional neural network method in terms of elastic strength,hardness strength and category,and compared the attribute strength recognition capabilities of the two methods using novel objects outside the haptic dataset.The results show that the accuracy of the last 20 iterations of the matrix-label method has an average elastic strength of 96.73%,hardness strength of 97.34%,and category of 96.67%.The performance is better.When the Euclidean distance between the prediction of the novel object and the real label is less than 1,the accuracy of the elastic strength is best to reach 100%,and the hardness strength is best to reach 100%.The performance is better. Conclusions:The effectiveness of the method has been verified.Comparing with the convolutional neural network method,our method can effectively recognize the attribute strength and category of objects.

Abstract: The Hanqu Joint Station of the Dingbian Oil Production Plant of Yanchang Oilfield Co., Ltd is located at the edge of the desert in northern China. the bad field conditions and strong sandstorm, the hardware of computer monitoring system of Joint Station is often damaged. At the same time, the core equipment of the joint station three-phase separator oil chamber liquid level is hard to achieve high precision constant value control, the general control algorithm is difficult to meet the control requirements.This paper proposed a design scheme of a oilfield joint station computer monitoring system based on the Siemens S7-300 PLC, the hardware of the monitoring system adopts the redundancy scheme of dual monitoring computers, dual programmable logic controllers (PLCs), and dual industrial Ethernet. The BP neural network PID control algorithm was used to realize constant value control of the oil chamber liquid level of the three-phase separator of the core equipment of the joint station，and realized high control precision. The monitoring system could well adapt to the harsh environment of the scene, and showed high reliability and efficiency.

The great importance of estimating dissolved oxygen (DO) dictates utilizing proper evaluative models. In this work, a multi-layer perceptron (MLP) network is trained by three capable metaheuristic algorithms, namely multi-verse optimizer (MVO), black hole algorithm (BHA), and shuffled complex evolution (SCE) for predicting the DO using the data of the Klamath River Station, Oregon, US. The records (DO, water temperature, pH, and specific conductance) belonging to the water years 2015 - 2018 (USGS) are used for pattern analysis. The results of this process showed that all three hybrid models could properly infer the DO behavior. However, the BHA and SCE accomplished this task by simpler configurations. Next, the generalization ability of the developed patterns is tested using the data of the 2019 water year. Referring to the calculated mean absolute errors of 1.0161, 1.1997, and 1.0122, as well as Pearson correlation coefficients of 0.8741, 0.8453, and 0.8775, the MLPs trained by the MVO and SCE perform better than the BHA. Therefore, these two hybrids (i.e., the MLP-MVO and MLP-SCE) can be satisfactorily used for future applications.

Deep neural networks (DNNs) have made a huge impact in the field of machine learning by providing unbeatable humanlike performance to solve real-world problems such as image processing and natural language processing (NLP). Convolutional neural network (CNN) and recurrent neural network (RNN) are two typical architectures that are widely used to solve such problems. Time sequence-dependent problems are generally very challenging, and RNN architectures have made an enormous improvement in a wide range of machine learning problems with sequential input involved. In this paper, different types of RNN architectures are compared. Special focus is put on two well-known gated-RNN’s Long Term Short Memory (LSTM) and Gated Recurrent Unit (GRU). We evaluated these models on the task of force estimation system in pouring. In this study, four different models including multi-layers LSTM, multi-layers GRU, single-layer LSTM and single-layer GRU) were created and trained. The result suggests that multi-layer GRU outperformed other three models.

Computational simulation using mathematical models is a useful method for understanding the complex behavior of a living system. The majority of studies using mathematical models to reveal biological mechanisms uses one of the two main approaches: the bottom-up or the top-down approach. When we aim to analyze a large-scale network, such as a comprehensive knowledge-integrated model of a target phenomenon, for example a whole-cell model, the variation of analyses is limited to particular kind of analysis because of the size and complexity of the model. To analyze a large-scale regulatory network of neural differentiation, we developed a hybrid method that combines both approaches. To construct a mathematical model, we extracted network motifs, subgraph structures that recur more often in a metabolic network or gene regulatory network than in a random network, from a large-scale regulatory network, detected regulatory motifs among them, and combined these motifs. We confirmed that the model reproduced the known dynamics of HES1 and ASCL1 before and after differentiation, including oscillation and equilibrium of their concentrations. The model also reproduced the effects of overexpression and knockdown of the Id2 gene. Our model suggests that the characteristic change in HES1 and ASCL1 expression in the large-scale regulatory network is controlled by a combination of four feedback loops, including a large loop which has not been focused on. The model extracted by our hybrid method has the potential to reveal the critical mechanisms of neural differentiation. The hybrid method is applicable to other biological events.

Nowadays industrial dryers are used instead of traditional methods for drying. In designing dryers suitable for controlling the process of drying and reaching a high quality product, it is necessary to predict the instantaneous moisture loss during drying. For this purpose, ten mathematical-experimental models with a neural network model based on the kinetic data of pistachio drying are studied. The data obtained from the cabinet dryer will be evaluated at four temperatures of inlet air and different air velocities. The pistachio seeds will be placed in a thin layer on an aluminum sheet on a drying tray and weighed by a scale attached to the computer at different times. In the neural network, data are divided into three parts: educational (60%), validation (20%) and test (20%). Finally, the best mathematical-experimental model using genetic algorithm and the best neural network structure for predicting instantaneous moisture are selected based on the least squared error and the highest correlation coefficient.

The core of eLoran ground-based timing navigation system is the accurate measurement of groundwave propagation delay. However, meteorological changes will disturb the conductive characteristic factors along the groundwave propagation path, especially for complex terrestrial propagation environment, and may even lead to microsecond-level propagation delay fluctuation, seriously affecting the timing accuracy of the system. Aiming at this problem, this paper proposes a propagation delay prediction model based on Back-Propagation neural network (BPNN) for complex meteorological environment, which realizes the function of directly mapping propaga-tion delay fluctuation through meteorological factors. Firstly, the theoretical influence of meteoro-logical factors on each component of propagation delay is analyzed based on calculation parame-ters. Then, through the correlation analysis of the measured data, the complex relationship be-tween the seven main meteorological factors and the propagation delay, as well as their regional differences are demonstrated. Finally, a BPNN prediction model considering regional changes of multiple meteorological factors is proposed, and the validity of the model is verified by long-term collected data. Experimental results show that the proposed model can effectively predict the propagation delay fluctuation in the next few days, and its overall performance is significantly improved compared with the existing linear model and simple neural network model. eLoran; meteorological factor; propagation delay prediction model; Back-Propagation neural network;

Working memory (WM) is a brain system for short-term storage and manipulation of information and plays an important role in complex cognitive tasks. In the synaptic theory of WM memorized elements are stored in the form of short-term potentiated connections in a sample population of neurons. In this paper, we show that such populations can be formed due to the mechanisms of spike-timing-dependent plasticity (STDP) – the phase dependence associated with the ratio of the pulse times of the interacting neurons. We propose a WM model considering two types of plasticity: short-term plasticity and STDP. We have shown formation of neuronal clusters encoding items in the WM model, that can be formed by external stimulation of a group of neurons due to the mechanisms of STDP and hold and reactivated by short-term plasticity mechanisms. The dynamic formation of neuronal clusters instead of pre-formed clusters gives additional flexibility to the model.

Data-driven models based on artificial intelligence are efficiently used to solve complex problems. The quality of groundwater is of utmost importance, as it directly impacts human health and the environment. In major parts of the world groundwater is the main source of drinking water, it is essential to periodically monitor water quality. Conventional water quality monitoring techniques involve periodical collection of water samples and analysis in the laboratory. This process is expensive, time consuming and involves lot of manual labor. The aim of our study is to build an ant colony optimized neural network for predicting groundwater quality parameters. We have proposed artificial neural network comprising of six hidden layers. The approach was validated using our groundwater quality dataset of a hard rock region in the northern part of Karnataka, India. Groundwater samples were collected by us periodically from March 2014 to October 2020 from 50 wells in this region. These samples where analysed for measuring the pH, Electrical Conductivity, Na+, Ca+, Na+, K+, Mg2+, F-, Cl- and U+. The temporal dataset was split for training, testing and validation of our model. Metrics such as R2 (Coefficient of Determination), RMSE (Root Mean Squared Error), NSE (Nash–Sutcliffe efficiencies) and MAE (Mean Absolute Error) were used to evaluate the prediction error and model performance. These performance evaluation metrics indicated the efficiency of our model in predicting the temporal variation in groundwater quality parameters. The method proposed by us can be used for prediction and the temporal frequency of sample collection can be reduced to save time and cost. The results also confirm that the combination of artificial neural network with ACO is a promising tool to optimize weights while training the network, and hence will help in reasonable prediction of groundwater quality parameters.

Lignocellulosic fibers are widely applied as composite reinforcement due to their properties. The thermal degradation behavior determines the maximum temperature in which the fiber can be applied without significant mass loss. It is possible to determine these temperatures using Thermogravimetric Analysis (TG). In particular, when curves are obtained at different heating rates, kinetic parameters can be determined and more detailed characteristics of the material are obtained. However, every curve obtained at a distinct heating rate demands material, cost, and time. Methods to predict thermogravimetric curves can be very useful in the materials science field and in this sense mathematical approaches are powerful tools if well employed. For this reason, in the present study, curaua TG curves were obtained at three different heating rates (5, 10, 20, and 40 °C.min-1) and Vyazovkin kinetic parameters were obtained. After, the experimental curves were fitted using an artificial neural network (ANN) approach followed by a Surface Response Methodology (SRM). Curves at any heating rate between the minimum and maximum experimental heating rates were obtained with high reliability. Finally, Vyazovkin kinetic parameters were tested again with the new curves showing similar kinetic parameters from the experimental ones. In conclusion, due to the capability to learn from the own data, ANN combined with SRM seems to be an excellent alternative to predict TG curves that do not test experimentally, opening the range of applications.

Increase in the emission of Greenhouse Gases (GHS) is among the significant concerns of government, societies, and policymakers. Due to the highest share of carbon dioxide in the produced GHGs, it is necessary to assess the factors that influence its emission. Energy systems and economic activities noticeably influence the amount of carbon dioxide production of countries. In this article, Artificial Neural Network (ANN) in addition to a linear correlation used to predict carbon dioxide emission of four CIS countries, including Turkmenistan, Uzbekistan, Kazakhstan, and Azerbaijan based the consumption of various energy sources and GDP, as the economic indicator. According to the obtained data by the proposed models, carbon dioxide emission can be accurately estimated by utilizing the mentioned input data. Models’ R-squared value are 0.9997 and 0.9999 in the cases of applying the correlation and ANN-based model. Moreover, the average absolute relative deviations by utilizing the correlation and GMDH ANN are approximately 1.05% and 0.61%, respectively. These statistical values demonstrate more proper performance of the ANN-based model compared with the applied linear correlation.

Bearings are widely used in various types of electrical machinery and equipment. As their core components, failures will often cause serious consequences . At present, most methods of parameter adjustment are still manual adjustment of parameters. This adjustment method is susceptible to prior knowledge and easy to fall into the local optimal solution, failing to obtain the global optimal solution and requires a lot of resources.Therefore, this paper proposes a new method of bearing fault diagnosis based on wavelet packet transform and convolutional neural network optimized by simulated annealing algorithm.The experimental results show that the method proposed in this paper has a more accurate effect in feature extraction and fault classification compared with traditional bearing fault diagnosis methods. At the same time, compared with the traditional artificial neural network parameter adjustment, this paper introduces the simulated annealing algorithm to automatically adjust the parameters of the neural network, thereby obtaining an adaptive bearing fault diagnosis method. To verify the effectiveness of the method, the Case Western Reserve University bearing database was used for testing, and the traditional intelligent bearing fault diagnosis method was compared. The results show that the method proposed in this paper has good results in bearing fault diagnosis. Provides a new way of thinking in the field of bearing fault diagnosis in parameter adjustment and fault classification algorithms

In this paper, we presented a method of retrieving sea surface wind speed from Sentinel-1 synthetic aperture radar (SAR) horizontal-horizontal (HH) polarization data in extra-wide mode, which have been extensively acquired over the Arctic for sea ice monitoring. In contrast to the conventional algorithm, i.e., using a geophysical model function (GMF) to retrieve sea surface wind by spaceborne SAR, we introduced an alternative method based on physical model guided neural network. Parameters of SAR normalized radar cross section, incidence angle, and wind direction are used as the inputs of the backward propagation (BP) neural network, and the output is the sea surface wind speed. The network is developed based on more than 11,000 HH-polarized EW images acquired in the marginal ice zone (MIZ) of the Arctic and their collocations with scatterometer measurements. Verification of the neural network based on the testing dataset yields a bias of 0.23 m/s and a root mean square error (RMSE) of 1.25 m/s compared to the scatterometer wind speed. Further comparison of the SAR retrieved sea surface wind speed with independent buoy measurements shows a bias and RMSE of 0.12 m/s and 1.42 m/s, respectively. We also analyzed the uncertainty of retrieval when the wind direction data of a reanalysis model are used as inputs to the neural network. By combining the detected sea ice cover information based on the EW data, one can expect to derive simultaneously sea ice and marine-meteorological parameters by spaceborne SAR in a high spatial resolution in the Arctic.

This paper presents a novel approach for semantic segmentation of building roof in dense urban environment with Deep Convolution Neural Network (DCNN) using imagery acquired by a Chinese Very High Resolution (VHR) satellite mission, i.e. GaoFen-2 (GF-2). To provide an operational end-to-end work flow for accurate build roof mapping with feature extraction as well as image segmentation, a fully convolutional DCNN with both convolutional and deconvolutional layers is designed to perform the VHR image analysis for labeling pixels. Since the diverse urban patterns and building styles in large areas, sample image data sets of building roof and non-building roof are collected over different metropolitan regions in China. We selected typical cities with dense urban environment in each metropolitan region as study areas for collecting training and test samples. High performance cluster with GPU-mounted workstations is employed to perform the model training and optimization. With the building roof samples collected over different cities, the predictive model with multiple NN layers is developed for building roof labeling. The validation of the building roof map shows that the overall accuracy(OA) and the mean Intersection Over Union( mIOU) of DCNN based segmentation are 94.67%, 0.85 respectively, while CRF-refined segmentation achieved OA of 94.69% and mIOU of 0.83. The results suggest that the proposed approach is a promising solution for building roof mapping with VHR images over large areas across different urban and building patterns. With the operational acquisition of GF2 VHR imagery, it is expected to develop an automated pipeline for operational built-up area monitoring and timely update of building roof map over large areas.

The detection of seismic events at regional and teleseismic distances is critical to Nuclear Treaty Monitoring. Traditionally, detecting regional and teleseismic events has required the use of an expensive multi-instrument seismic array; however in this work, we present DeepPick, a novel seismic detection algorithm capable of array-like performance from a single trace. We achieve this directly, by training our single-trace detector against labeled events from an array catalog, and by utilizing a deep temporal convolutional neural network. The training data consists of all arrivals in the International Seismological Centre Catalog for seven seismic arrays over a five year window from 1 Jan 2010 to 1 Jan 2015, yielding a total training set of 608,362 detections. The test set consists of the same seven arrays over a one year window from 1 Jan 2015 to 1 Jan 2016. We report our results by training the algorithm on six of the arrays and testing it on the seventh, so as to demonstrate the transportability and generalization of the technique to new stations. Detection performance against this test set is outstanding. Fixing a type-I error rate of 1%, the algorithm achieves an overall recall rate of 73% on the 141,095 array beam picks in the test set, yielding 102,394 correct detections. This is more than 4 times the 23,259 detections found in the analyst-reviewed single-trace catalogs over the same period, and represents an 8dB improvement in detector sensitivity over current methods. These results demonstrate the potential of our algorithm to significantly enhance the effectiveness of the global treaty monitoring network.

For enhancing the maximized profit from bank as well as customer perspective, term deposit can accelerate finance fields. This paper focuses on likelihood of term deposit subscription taken by the customers. Bank campaign efforts and customer details are influential while considering possibilities of taking term deposit subscription. An automated system is provided in this paper that approaches towards prediction of term deposit investment possibilities in advance. Neural network(NN) along with stratified 10-fold cross-validation methodology is proposed as predictive model which is later compared with other benchmark classifiers such as k-Nearest Neighbor (k-NN), Decision tree classifier (DT), and Multi-layer perceptron classifier (MLP). Experimental study concluded that proposed model provides significant prediction results over other baseline models with an accuracy of 88.32% and Mean Squared Error (MSE) of 0.1168.

Greenhouse Gases (GHGs) emission has considerable impact on global warming and climate change. Since energy systems and their features noticeably influence on the amount of GHGs emission, it can be modeled based on the specifications of energy sources utilized by the countries. In addition, economic activity is another factor which should be considered in GHG emission modeling. In this work, Artificial Neural Network (ANN) is used for estimating carbon dioxide emission, as one of the most abundant GHGs, on the basis of shares of various energy sources used as primary energy supply and GDP as an indicator for economic activities. Five countries including Iran, Kuwait, Qatar, Saudi Arabia and United Arab Emirates (UAE) are considered as case studies. Comparing between the estimated data by the achieved model and actual quantities showed acceptable precision of the ANN model for prediction of carbon dioxide emission. The average absolute relative error and the R-squared values of the GMDH model are approximately 2.28% and 0.9998, respectively. The obtained values for the mentioned statistical criteria show the precision of the model in forecasting the emission of Co2.

Precipitation downscaling is widely employed for enhancing the resolution and accuracy of precipitation products from general circulation models (GCMs). In this study, we propose a novel statistical downscaling method to foster GCMs’ precipitation prediction resolution and accuracy for monsoon region. We develop a deep neural network composed of convolution and Long Short Term Memory (LSTM) recurrent module to estimate precipitation based on well-resolved atmospheric dynamical fields. The proposed model is compared against GCM precipitation product and classical downscaling methods in the Xiangjiang River Basin in South China. Results show considerable improvement compared to the ECMWF-Interim reanalysis precipitation. Also, the model outperforms benchmark downscaling approaches, including 1) quantile mapping, 2) support vector machine, and 3) convolutional neural network. To test the robustness of the model and its applicability in practical forecast, we apply the trained network for precipitation prediction forced by retrospective forecasts from ECMWF model. Compared to ECMWF precipitation forecast, our model makes better use of the resolved dynamical field for more accurate precipitation prediction at lead time from 1 day up to 2 weeks. This superiority decreases along forecast lead time, as GCM’s skill in predicting atmospheric dynamics being diminished by the chaotic effect. At last, we build a distributed hydrological model and force it with different sources of precipitation inputs. Hydrological simulation forced with the neural network precipitation estimation shows significant advantage over simulation forced with the original ERA-Interim precipitation (with NSE value increases from 0.06 to 0.64), and the performance is just slightly worse than the observed precipitation forced simulation (NSE=0.82). This further proves the value of the proposed downscaling method, and suggests its potential for hydrological forecasts.

The paper presents the identification and classification of steel cord failures in the conveyor belt core based on an analysis of a two-dimensional image of magnetic field changes recorded using the Diagbelt system around scanned failures in the test belt. The obtained set of identified changes in images obtained for numerous devices parameters settings were the base for statistical analysis. It makes it possible to determine the Pearson’s linear correlation coefficient between the parameters being changed and the image of the failures. In the second stage of the research, artificial intelligence methods were applied to construct a multilayer neural network (MLP) and to teach its appropriate identification of damage. In both methods were used the same data sets, which made it possible to compare methods.

Respiratory rate is an important biomarker that indicates changes in the clinical condition of critically ill patients, so a surveillance tool that can accurately monitor the changing respiratory rate in real time is needed. Through investigating various pairs of machine learning models, we proposed new machine learning model for real-time respiratory rate estimation using photoplethysmogram. New photoplethysmogram-driven respiratory rate dataset(StMary) was collected from surgical intensive care unit of a tertiary referral hospital, using photoplethysmogram signal collector. For 50patients and 50healthy volunteers, 2-minute photoplethysmogram was collected for each subject twice. To evaluate the respiratory rate of subject, it was inputted into the deep neural network model we built, and dataset was splitted into training, validation, testing dataset, then 4-fold cross validation was exploited. Our deep neural network model trained with StMary and two public datasets(BIDMC and CapnoBase) individually, or selectively merged dataset had shown a low error rate in respiration rate measurements. Our model trained with StMary showed low mean absolute error score(1.0273±0.8965), and trained with 3 datasets(CapnoBase, BIDMC and StMary) showed a lower error rate(1.7359±1.6724) than the model trained with CapnoBase and BIDMC(1.9480±1.6751). We could verify the performance of model evaluating respiratory rate from photoplethysmogram, and our dataset could contribute as the clinical research data that supports artificial intelligence models evaluating respiratory rate and surveillance tools to test whether their monitoring function works properly.

The dynamics of suspended sediment involves inherent non-linearity and complexity as a result of the presence of both spatial variability of the basin characteristics and temporal climatic patterns. As a result of this complexity, the conventional sediment rating curve (SRC) and other empirical methods produce inaccurate predictions. Deep neural networks (DNNs) have emerged as one of the advanced modeling techniques capable of addressing inherent non-linearity in hydrological processes over the last few decades. DNN algorithms are used to perform predictive analysis and investigate the interdependencies among the most pivotal water quantity and quality parameters i.e., discharge, suspended sediment concentration (SSC), and turbidity. In this study, the Long short-term memory (LSTM) algorithm of DNNs is used to model the discharge-suspended sediment relationship for the Stony Clove Creek. The simulations were run using primary data on discharge, SSC and turbidity. For the development of the DNN models and examining the effects of input vectors, combinations of different input vectors (namely discharge, and SSC) for the current and previous days are considered. Furthermore, a suitable modelling approach with an appropriate model input structure is suggested based on model performance indices for the training and testing phases. The performance of developed models is assessed using statistical indices such as root mean square error (RMSE), mean absolute error (MAE), and coefficient of determination (R2). Statistically, the performance of DNN-based models in simulating the daily SSC performed well with observed sediment concentration series data. The study demonstrates the suitability of the DNN approach for simulation and estimation of daily SSC, opening up new research avenues for applying hybrid soft computing models in hydrology.

Deep learning has shown remarkable results in every field, especially in the biomedical field, due to its ability to exploit large-scale datasets. A convolutional neural network (CNN) is a widely used deep learning approach to solve medical imaging problems. Over the past few years, many studies have focused on CNN-based techniques for brain tumor diagnosis. There are, however, still some critical challenges that CNNs face towards clinic application. This study presents a comprehensive review of current literature that involves CNN architectures for brain tumor classification. We compare the key achievements in the performance evaluation metrics of the applied classification algorithms. In addition, this review assesses the clinical effectiveness of the included studies to elaborate on the limitations and directions of this area for future work. No review focusing on the clinical effectiveness of previous works in this field has been published. We believe that this study has the potential to elevate the application of CNN-based deep learning methods in clinical practice and also can be a quick reference for biomedical researchers who are interested in this field.

Deepfakes are manipulated or altered images, or video, that are created using deep learning models with high levels of photorealism. The two popular methods of producing a deepfake are based on either convolutional neural networks (CNN), or autoencoders. Deepfakes created using CNN comparatively show higher qualities of realism, yet oftentimes leave artifacts and distortions in the generated media that can be detected using machine learning and deep learning algorithms. In recent years, there has been an influx of periocular image and video data because of the increase usage of face masks. By wearing masks, much of what is used for facial recognition is hidden, leaving only the periocular region visible to an observer. This loss of vital information leads to easier misidentification of media, allowing deepfakes to less likely be identified as fake. In this work, feature extraction methods, such as Scale-Invariant Feature Transform (SIFT), Histogram of Oriented Gradients (HOG), and CNN, are used to train an ensemble deep learning model to detect deepfakes in videos on a frame-by-frame level based on the periocular region. Our proposed model is able to distinguish original and manipulated images with accuracies around 98.9 percent, which is an improvement to previous works by combining SIFT and HOG for deepfake detection in convolutional neural networks.

The environmental issues we are currently facing require long-term prospective efforts for sustainable growth. Renewable energy sources seem to be one of the most practical and efficient alternatives in this regard. Understanding a nation's pattern of energy use and renewable energy production is crucial for developing strategic plans. No previous study has been performed to explore the dynamics of power consumption with the change in renewable energy production on a country-wide scale. In contrast, a number of deep learning algorithms demonstrated acceptable performance while handling sequential data in the era of data-driven predictions. In this study, we developed a scheme to investigate and predict total power consumption and renewable energy production time series for eleven years of data using a Recurrent Neural Network (RNN). The dynamics of the interaction between the total annual power consumption and renewable energy production are investigated through extensive Exploratory Data Analysis (EDA) and a feature engineering framework. The performance of the model is found satisfactory through the comparison of the predicted data with the observed data, visualization of the distribution of the errors and Root Mean Squared Error (RMSE) value of 0.084. Higher performance is achieved through the increase in the number of epochs and hyperparameter tuning. The proposed framework can be used and transferred to investigate the trend of renewable energy production and power consumption and predict the future scenarios for different communities. Incorporation of the cloud-based platform into the proposed pipeline may lead to real-time forecasting.

The agricultural sector in Palestine has a significant role in its economy. However, the production of this sector is affected by different kinds of plant diseases, specifically leaf diseases. Automatic agricultural leaf disease detection is essential for the early diagnosis and controls the overall health of fields. Image segmentation techniques, clustering, and deep learning are often used to detect diseased leaves. This study proposes a novel hybrid approach based on image classification. The hybrid approach combines the k-means clustering algorithm with Convolutional Neural Network (CNN), where k-means is used to detect the leaf’s infected area, then CNN is used for specifying the disease. We used the PlantVillage dataset for experimental verification as it contains several crops with different kinds of challenging diseases. We also examined the selection of optimal k-value using the Silhouette coefficient, Elbow method, and Kneedle Algorithm. The Silhouette technique was analyzed using three distance metrics; Euclidean, Manhattan, and Cosine. Its scores for the three-distance metrics were low, near-zero, and failed to produce the optimum k value. Besides, the Elbow method was complicated to use in image segmentation in terms of executing and visualizing the k value in its graph plot. Based on verification results, the Kneedle Algorithm produced better results in the consistency of choosing the optimal k value and showed superiority over other approaches. Therefore, the processed images were segmented with the k-means clustering algorithm with a Kneedle algorithm-based k value. Finally, a Convolutional Neural Network (CNN) is trained to classify the type of disease based on analyzing and testing leaf images. The hybrid model achieved high accuracy of 93.79% in disease identification, confirming the proposed model’s robustness.

Deep learning (DL), which is originated from an artificial neural network (ANN), is one of the major technologies of today's smart cybersecurity systems or policies to function in an intelligent manner. Popular deep learning techniques, such as Multi-layer Perceptron (MLP), Convolutional Neural Network (CNN or ConvNet), Recurrent Neural Network (RNN) or Long Short-Term Memory (LSTM), Self-organizing Map (SOM), Auto-Encoder (AE), Restricted Boltzmann Machine (RBM), Deep Belief Networks (DBN), Generative Adversarial Network (GAN), Deep Transfer Learning (DTL or Deep TL), Deep Reinforcement Learning (DRL or Deep RL), or their ensembles and hybrid approaches can be used to intelligently tackle the diverse cybersecurity issues. In this paper, we aim to present a comprehensive overview from the perspective of these neural networks and deep learning techniques according to today's diverse needs. We also discuss the applicability of these techniques in various cybersecurity tasks such as intrusion detection, identification of malware or botnets, phishing, predicting cyber-attacks, e.g. denial of service (DoS), fraud detection or cyber-anomalies, etc. Finally, we highlight several research issues and future directions within the scope of our study in the field. Overall, the ultimate goal of this paper is to serve as a reference point and guidelines for the academia and professionals in the cyber industries, especially from the deep learning point of view.

Vibration measurement serves as the basis for various engineering practices such as natural frequency or resonant frequency estimation. As image acquisition devices become cheaper and faster, vibration measurement and frequency estimation through image sequence analysis continue to receive increasing attention. In the conventional photogrammetry and optical methods of frequency measurement, vibration signals are first extracted before implementing the vibration frequency analysis algorithm. In this work, we demonstrated that frequency prediction can be achieved using a single feed-forward convolutional neural network. The proposed method is verified using a vibration signal generator and excitation system, and the result obtained was compared with that of an industrial contact vibrometer in a real application. Our experimental results demonstrate that the proposed method can achieve acceptable prediction accuracy even in unfavorable field conditions.

Grapes are a globally popular fruit, with grape cultivation worldwide being second only to citrus. This article focuses on the low efficiency and accuracy of traditional manual grading of red grape external appearance and proposes a small-sample red grape external appearance grading model based on transfer learning with convolutional neural networks (CNNs). Initially, the CNN transfer learning method was used to transfer the pre-trained AlexNet, VGG16, GoogleNet, InceptionV3, and ResNet50 network models on the ImageNet image dataset to the red grape image grading task. By comparing the classification performance of the CNN models of these five different network depths with fine-tuning, ResNet50 with a learning rate of 0.001 and a loop number of 10 was determined to be the best feature extractor for red grape images. Moreover, given the small number of red grape image samples in this study, different convolutional layer features output by the ResNet50 feature extractor were analyzed layer by layer to determine the effect of deep features extracted by each convolutional layer on SVM classification performance. This analysis helped to obtain a ResNet50+SVM red grape external appearance grading model based on the optimal ResNet50 feature extraction strategy. Experimental data showed that the classification model constructed using the feature parameters extracted from the 10th node of the ResNet50 network achieved an accuracy rate of 95.08% for red grape grading. These research results provide a reference for the online grading of red grape clusters based on external appearance quality and have certain guiding significance for the quality and efficiency of grape industry circulation and production.

The fatigue life of concrete is affected by many interwoven factors whose effect is nonlinear. Be-cause of its unique self-learning ability and strong generalization capability, the Bayesian regu-larized backpropagation neural network (BR-BPNN) is proposed to predict concrete behavior in tensile fatigue. The optimal model was determined through various combinations of network parameters. The average relative impact value (ARIV) was constructed to evaluate the correla-tion between fatigue life and its influencing parameters (maximum stress level Smax, stress ratio R, static strength f, failure probability P). ARIV results were also compared with other factor as-sessment methods (weight equation and multiple linear regression analyses). Using BR-BPNN, S-N curves were then obtained for the combinations of R=0.1, 0.2, 0.5; f=5, 6, 7MPa; P=5%, 50%, 95%. The tensile fatigue results under different testing conditions were finally compared for compatibility. It was concluded that Smax has the most significant negative effect on fatigue life; the degree of influence of R, P, and f, which positively correlate with fatigue life, decreases suc-cessively. ARIV is confirmed as a feasible way to analyze the importance of parameters and could be recommended for future applications. The tensile fatigue performance of plain concrete under different stress states (flexural tension, axial tension, splitting tension) does not differ sig-nificantly. Besides utilizing the valuable fatigue test data scattered in the literature, insights gained from this work could provide a reference for subsequent fatigue test program design and fatigue evaluation.

Accurate and efficient traffic incident detection methods can effectively alleviate traffic congestion caused by traffic incidents, prevent secondary accidents and improve the safety of urban road traffic.Aiming at the problems that the traditional machine learning event detection method cannot fully extract the parameter characteristics of traffic flow and is not suitable formulti-dimensional and non-linear massive data, we propose a new traffic event detection method(CNN-XGBoost).This method combines the respective advantages of Convolution Neural Network(CNN) and Extreme Gradation Boosting (XGBoost). Firstly, we preprocessed the original freeway traffic incident detection data set by constructing initial variable set, data normalization, data balance processing and dimension reorganization. Secondly,we use CNN network to automatically extract the deep features of event detection data, and use XGBoost as a classifier to classify the extracted features for expressway traffic event detection.Finally, we use the data set of Hangzhou expressway microwave detector in China to carry out simulation experiments on CNN-XGBoost. The experimental results show that compared with XGBoost, CNN, Support Vector Machine (SVM) and Gradient Boosting Decision Tree (GBDT) and other methods, CNN-XGBoost method can effectively improve the accuracy of expressway traffic event detection and has better generalization ability.

Blind deconvolution (BD) is one of the effective methods that help pre-process vibration signals and assist in bearing fault diagnosis. Currently, most BD methods design an optimization criterion and use frequency or time domain information independently to optimize a deconvolution filter. It recovers weak periodic impulses related to incipient faults. However, the random noise interference may cause the optimizer to overfit. The time-domain-based BD methods tend to extract fault-unrelated single peak impulse, and the frequency-domain-based BD methods tend to retain the maximum energy frequency component, which will lose the fault-related harmonics frequency components. To solve the above issue, we propose a hybrid criterion that combines the kurtosis for time domain optimization and the $G-l_1/l_2$ norm for the frequency domain. These two criteria are monotonically increasing and decreasing, so they mutually constrain to avoid overfitting. After that, we design a multi-task one-dimensional convolutional neural network with time and frequency branches to achieve an optimal solution for this hybrid criterion. The multi-task neural network realizes the simultaneous optimization of two domains. Experimental results show that our proposed method outperforms other state-of-the-art methods.

The computer network world is changing and the NetDevOps approach has brought the dynamics of applications and systems into the field of communication infrastructure. Businesses are changing and businesses are faced with difficulties related to the diversity of hardware and software that make up those infrastructures. The "Intent-Based Networking - Concepts and Definitions" document describes the different parts of the ecosystem that could be involved in NetDevOps. The recognize, generate intent, translate and refine features need a new way to implement algorithms. This is where artificial intelligence comes in.

Multidisciplinary Design Optimization (MDO) is the most active field in the design of current complex system engineering, which is possessed with such two difficulties as subsystem information exchange and analytical and computational complexity of systems. Therefore, an improved collaborative optimization algorithm based on ANN (artificial neural network) response surface was proposed dependent on the consistency constraint algorithm and concurrent subspace algorithm. As an optimization method with secondary structure, it satisfied only local constraints in discipline layer, but provided a coordinated mechanism for interdisciplinary conflict in system layer. Finally, it was applied in the multidisciplinary design optimization of autonomous underwater vehicle (AUV). As shown from the result, the MDO convergence stability and reliability of low resistance, low noise and high maneuvering performance of the AUV shape can be ensured by the improved collaborative optimization algorithm, thus verifying the effectiveness of the algorithm.

Heme proteins serve diverse and pivotal biological functions. Therefore, clarifying the mechanisms of these diverse functions of heme is a crucial scientific topic. Distortion of heme porphyrin is one of the key factors regulating the chemical properties of heme. Here, we constructed convolutional neural network models for predicting heme distortion from the tertiary structure of the heme-binding pocket to examine their correlation. For saddling, ruffling, doming, and waving distortions, the experimental structure and predicted values were closely correlated. Furthermore, we assessed the correlation between the cavity shape and molecular structure of heme and demonstrated that hemes in protein pockets with similar structures exhibit near-identical structures, indicating the regulation of heme distortion through the protein environment. These findings indicate that the tertiary structure of the heme-binding pocket regulates the distortion of heme porphyrin, thereby controlling the chemical properties of heme relevant to the protein function; this implies a structure–function correlation in heme proteins.

This work presents an approach for detecting mathematical formulas in scanned document images. The proposed approach is end-to-end trainable. Since many OCR engines cannot reliably work with the formulas, it is essential to isolate them to obtain the clean text for information extraction from the document. Our proposed pipeline comprises a hybrid task cascade network with deformable convolutions and a Resnext101 backbone. Both of these modifications help in better detection. We evaluate the proposed approaches on the ICDAR-2017 POD and Marmot datasets and achieve an overall accuracy of 96% for the ICDAR-2017 POD dataset. We achieve an overall reduction of error of 13%. Furthermore, the results on Marmot datasets are improved for the isolated and embedded formulas. We achieved an accuracy of 98.78% for the isolated formula and 90.21% overall accuracy for embedded formulas. Consequently, it results in an error reduction rate of 43% for isolated and 17.9% for embedded formulas.

Any effort to combat corruption can benefit from an examination of past and projected worldwide trends. In this paper, we forecast the level of corruption in countries by integrating an artificial neural network modeling and time series analysis. The data were obtained from 113 countries from 2007 to 2017. The study is carried out at two levels: (a) global level where all countries are considered as a monolithic group; and (b) cluster level, where countries are placed into groups based on their development-related attributes. For each cluster, we use the findings from our previous study on the cluster analysis of global corruption using machine learning methods that identified the four most influential corruption factors, and we use those as independent variables. Then, using the identified influential factors, we forecast the level of corruption in each cluster using a nonlinear autoregressive recurrent neural network with exogenous inputs (NARX), an artificial neural network technique. The NARX models were developed for each cluster, with the objective function in terms of the Corruption Perceptions Index (CPI). For each model, the optimal neural network is determined by finetuning the hyperparameters. The analysis was repeated for all countries as a single group. The accuracy of the models is assessed by comparing the mean square errors (MSE) of the time series models. The results suggest that the NARX artificial neural network technique yields reliable future values of CPI globally or for each cluster of countries. This can assist policymakers and organizations in assessing the expected efficacies of their current or future corruption control policies from a global perspective as well as for groups of countries.

Multivariate predictive analysis of the Stream-Water (SW) parameters (discharge, water level, temperature, dissolved oxygen, pH, turbidity, and specific conductance) is a pivotal task in the field of water resource management during the era of rapid climate change. The highly dynamic and evolving nature of the meteorological and climatic features have a significant impact on the temporal distribution of the SW variables in recent days making the SW variables forecasting even more complicated for diversified water-related issues. To predict the SW variables, various physics-based numerical models are used using numerous hydrologic parameters. Extensive lab-based investigation and calibration are required to reduce the uncertainty involved in those parameters. However, in the age of data-informed analysis and prediction, several deep learning algorithms showed satisfactory performance in dealing with sequential data. In this research, a comprehensive Explorative Data Analysis (EDA) and feature engineering were performed to prepare the dataset to obtain the best performance of the predictive model. Long Short-Term Memory (LSTM) neural network regression model is trained using over several years of daily data to predict the SW variables up to one week ahead of time (lead time) with satisfactory performance. The performance of the proposed model is found highly adequate through the comparison of the predicted data with the observed data, visualization of the distribution of the errors, and a set of error matrices. Higher performance is achieved through the increase in the number of epochs and hyperparameter tuning. This model can be transferred to other locations with proper feature engineering and optimization to perform univariate predictive analysis and potentially be used to perform real-time SW variables prediction.

Sparse training (ST) aims to improve deep learning by replacing fully connected artificial neural networks (ANNs) with sparse ones. ST is promising but at an early stage, therefore it might benefit to borrow brain-inspired learning paradigms such as epitopological learning (EL) from complex network intelligence theory. EL is a field of network science that studies how to implement learning on networks by changing the shape of their connectivity structure (epitopological plasticity). EL was conceived together with the Cannistraci-Hebb (CH) learning theory according to which: the sparse local-community organization of many complex networks (such as the brain ones) is coupled to a dynamic local Hebbian learning process and contains already in its mere structure enough information to partially predict how the connectivity will evolve during learning. One way to implement EL is via link prediction: predicting the existence likelihood of each nonobserved link in a network. CH theory inspired a network automata rule for link prediction called CH3-L3 that was recently proven to be very effective for general purpose link prediction. Here, starting from CH3-L3 we propose a CH training (CHT) approach to implement epitopological sparse deep learning in ANNs. CHT consists of three parts: kick start pruning, to hint the link predictors; epitopological prediction, to shape the ANN topology; and weight refinement, to tune the synaptic weights values. Experiments on MNIST and CIFAR10 datasets compare the efficiency of CHT and other ST-based algorithms in speeding up the ANN training across epochs. While SET leverages random evolution and RigL adopts gradient information, CHT is the first algorithm in ST that learns to shape sparsity by using the sparse topological organization of the ANN.

Currently, skin cancer is the most commonly diagnosed form of cancer in humans and is one of the leading causes of death in patients with cancer. Biopsy methods are an invasive research method and are not always available for primary diagnosis. Imaging methods have low accuracy and depend on the experience of the dermatologist. Artificial intelligence technologies can match and surpass visual analysis methods in accuracy, but they have the risk of a false negative response when a malignant pigmented lesion can be recognized as benign. One possible way to improve accuracy and reduce the risk of false negatives is to analyze heterogeneous data, combine different preprocessing methods, and use modified loss functions to eliminate the negative impact of unbalanced dermatological data. The paper proposes a multimodal neural network system with a modified cross-entropy loss function that is sensitive to unbalanced heterogeneous dermatological data. The accuracy of recognition in 10 diagnostically significant categories for the proposed system was 85.19%. The novelty of the proposed system lies in the use of cross-entropy loss when training the modified function with the help of weight coefficients. The introduction of weighting factors has reduced the number of false negative forecasts, as well as improved accuracy by 1.02-4.03 percentage points compared to the original multimodal systems. The introduction of the proposed multimodal system as an auxiliary diagnostic tool can reduce the consumption of financial and labor resources involved in the medical industry, as well as increase the chance of early detection of skin cancer.

In recent years, the demand for coffee has increased tremendously. During the production process, green coffee beans are traditionally screened manually for defective beans before they are packed into coffee bean packages; however, this method is not only time-consuming but also increases the rate of human error due to fatigue. Therefore, this paper proposed a lightweight deep convolutional neural network (LDCNN) for the quality detection system of green coffee beans, which combined depthwise separable convolution (DSC), squeeze-and-excite block (SE block), skip block, and other frameworks. To avoid the influence of low parameters of the lightweight model caused by the model training process, rectified Adam (RA), lookahead (LA), and gradient centralization (GC) were included to improve efficiency; the model was also put into the embedded system. Finally, the local interpretable model-agnostic explanations (LIME) model was employed to explain the predictions of the model. The experimental results indicated that the accuracy rate of the model could reach up to 98.38% and the F1 score could be as high as 98.24% when detecting the quality of green coffee beans. Hence, it can obtain higher accuracy, lower computing time, and lower parameters. Moreover, the interpretable model verified that the lightweight model in this work is reliable, providing the basis for screening personnel to understand the judgment through its interpretability, thereby improving the classification and prediction of the model.

Election Algorithm (EA) is a powerful metaheuristics model motivated by phenomena of the socio-political mechanism of the presidential election conducted in many countries. EA is selected as a topic of discussion due to its capability and robustness to carry out complex problems in the random-2SAT logic program. This paper utilizes a hybridized EA assimilated with the Hopfield neural network (HNN) in carrying out random logic program (HNN-R2SATEA). The efficiency of the proposed method was compared with the existing traditional exhaustive search (HNN-R2SATES) model and the recently introduced HNN-R2SATICA model. From the result obtained, clearly proven that based on our proposed hybrid model outperformed other existing model based on the Global Minima Ratio (ZM), Mean Absolute Error (MAE), Bayesian Information Criterion (BIC) and Execution Time (ET). The expected outcome portrays that the EA algorithm outperformed the other two algorithms in doing random-kSAT logic program. The results proved the robustness, effectiveness, and compatibility of the HNN-R2SATEA model.

The neural network technology for real-time cryptographic data protection with symmetric keys (masking codes, neural network architecture and weights matrix) for unmanned aerial vehicles (UAV) onboard communication systems has been developed. It provides hardware and software implementation with high technical and operational characteristics. The development of neural network technology for real-time cryptographic data protection was performed using an integrated approach based on theoretical foundations of neural network cryptographic data protection, new algorithms and structures of neural network data encryption and decryption, modern element base with the possibility of programming for the structure, and computer-aided design of hardware and software tools. The development and implementation of the on-board system of neural network cryptographic data protection in real-time are based on the following principles: variable composition of equipment; modularity; conveyorization and spatial parallelism; specialization and adaptation of hardware and software to data encryption and decryption. The tabular-algorithmic method of calculating the scalar product has been improved, it provides fast calculation of the scalar product of input data for both fixed and floating-point by bringing to the largest common order of weights and forming tables of macro-partial products for them. Components of neural network cryptographic data encryption and decryption have been developed on the processor core supplemented by specialized scalar product calculation modules. The specialized hardware for neural network cryptographic data encryption was developed using VHDL for equipment programming in the Quartus II development environment ver. 13.1 and the appropriate libraries, and implemented on the basis of the FPGA EP3C16F484C6 Cyclone III family.

Structure–function relationships in proteins have been one of the crucial scientific topics. Heme proteins have diverse and pivotal biological functions. Therefore, clarifying their structure–function correlation is significant to understand their functional mechanism and is informative for various fields of science. In this study, we constructed convolutional neural network models for predicting protein functions from the tertiary structures of heme-binding sites (active sites) of heme proteins to examine the structure–function correlation. As a result, we succeeded in the classification of oxygen-binding protein (OB), oxidoreductase (OR), proteins with both functions (OB–OR), and electron transport protein (ET) with high accuracy. Although the misclassification rate for OR and ET was high, the rates between OB and ET and between OB and OR were almost zero, indicating that the prediction model works well between protein groups with very different functions. However, predicting the function of proteins modified with amino acid mutation(s) remains a challenge. Our findings indicate a structure–function correlation in the active site of heme proteins. This study is expected to be applied to the prediction of more detailed protein functions such as catalytic reactions.