The implementation of neural networks (NN) on edge devices enables local processing of wireless data but faces challenges such as high computational complexity and memory requirements when deep neural networks (DNN) are used. Shallow neural networks customized for specific problems are more efficient, requiring fewer resources, and resulting in a lower latency solution. An additional benefit of the smaller network size is that it is suitable for real-time processing on edge devices. The main concern with shallow neural networks is their accuracy performance compared to DNNs. In this paper, we demonstrate that a customized adaptive activation function (AAF) can meet the accuracy of a DNN. We designed an efficient FPGA implementation for a customized segmented spline curve neural network (SSCNN) structure to replace the traditional fixed activation function with an AAF. We compared our SSCNN with different neural network structures such as real-valued time delay neural network (RVTDNN), augmented real-valued time delay neural network (ARVTDNN), and deep neural networks with different parameters. Our proposed SSCNN implementation uses 40% fewer hardware resources and no Block RAMS compared to the DNN with similar accuracy. We experimentally validate this computationally efficient and memory-saving FPGA implementation of SSCNN for digital predistortion of RF power amplifiers using the AMD/Xilinx RFSoC ZCU111. The implemented solution uses less than 3% of the available resources. The solution also enables an increase of the clock frequency to 221.12 MHz, allowing the transmission of wide bandwidth signals.

In this research, we present a comprehensive methodology to categorize perfumes based on their fragrance profiles and subsequently aid in creating innovative odoriferous molecules using advanced neural networks. Drawing from data on Parfumo and the Good Scents Company webpage (Parfumo, 2008; The Good Scents Company, 2021), the study employs sophisticated web scraping techniques to gather diverse perfume attributes. Following this, a k-means algorithm is applied for perfume clustering, paving the way for recommending similar scents to consumers. The process then bridges customer preferences to molecular design by incorporating their feedback into generating new molecules via graph neural networks (GNNs). Through converting the Simple Molecular Input Line Entry System (SMILES) representation into graph structures, the GNN facilitates the creation of new molecular designs attuned to consumer desires. The proposed approach offers promising avenues for consumers to pinpoint similar perfume choices, incorporating feedback, and for manufacturers to conceptualize new fragrant molecules with a high likelihood of market resonance.

In this paper, a class of coupled competitive neural networks, which subject to disturbance and discontinuous activation functions, is concerned. To realize the fixed-time quasi-bipartite synchronization, an aperiodic intermittent controller is initially designed. Subsequently, by combining the fixed-time stability theory and nonsmooth analysis, several criteria are established to ensure the bipartite synchronization in fixed time. Moreover, synchronization error bounds and settling time estimates are provided. Finally, numerical simulations are presented to verify the main results.

In this paper, we present a groundbreaking methodology in the realm of aspect-level sentiment analysis, which capitalizes on the advanced capabilities of graph-based neural networks. Our approach, distinguished as the Aspect Correlation Graph Network (ACGN), represents a significant departure from conventional models. These traditional models often analyze aspects in isolation, failing to capture the intricate web of sentiment relationships that may exist within a single sentence. ACGN, however, is designed to address this gap by employing a sophisticated bidirectional attention mechanism, integrated with positional encoding. This unique combination not only enhances the model's ability to focus on relevant parts of the sentence but also aids in constructing detailed, aspect-focused representations. These representations are particularly crucial for understanding the nuanced interplay of sentiments associated with different aspects. Central to our model's architecture is the incorporation of a graph convolutional network. This network serves as a pivotal component in mapping and analyzing the complex network of sentiment correlations that can exist among various aspects within sentences. Through this integration, ACGN is able to unearth and interpret the subtle and often overlooked sentiment dynamics that traditional models might miss. Our comprehensive evaluations of the Aspect Correlation Graph Network, conducted using the SemEval 2014 datasets, have yielded promising results. These findings demonstrate a clear and significant advancement over the capabilities of existing models. Particularly, the results underscore the critical importance and utility of recognizing and understanding the connections between sentiments of different aspects in text analysis. This insight opens new avenues in the field of sentiment analysis, suggesting a broader application potential of ACGN in various contexts where understanding nuanced sentiment relationships is key. Overall, our study not only introduces a novel approach in aspect-level sentiment analysis but also sets a new standard for future research in this area. By highlighting the integral role of inter-aspect sentiment connections, ACGN paves the way for more sophisticated and accurate sentiment analysis tools, capable of handling the complexities of natural language with greater finesse and precision.

Hopfield Neural Networks (HNNs) are recurrent neural networks used to implement associative memory. Their main feature is their ability to pattern recognition, optimization, or image segmentation. However, sometimes it is not easy to provide the users with good explanations about the results obtained with them due to mainly the large number of changes in the state of neurons (and their weights) produced during a problem of machine learning. There are currently limited techniques to visualize, verbalize, or abstract HNNs. This paper outlines how we can construct automatic video generation systems to explain their execution. This work constitutes a novel approach to get explainable artificial intelligence systems in general and HNNs in particular building on the theory of data-to-text systems and software visualization approaches. We present a complete methodology to build these kinds of systems. Software architecture is also designed, implemented, and tested. Technical details about the implementation are also detailed and explained. Finally, we apply our approach for creating a complete explainer video about the execution of HNNs on a small recognition problem.

The small satellite market continues to grow year after year. A compound annual growth rate of 17% is estimated during the period between 2020 and 2025. Low-cost satellites can send a vast amount of images to be post-processed at the ground to improve the quality and extract detailed information. In this domain lies the resolution enhancement task, where a low-resolution image is converted to a higher resolution automatically. Deep learning approaches to Super-Resolution (SR) reached the state-of-the-art in multiple benchmarks; however, most of them were studied in a single-frame fashion. With satellite imagery, multi-frame images can be obtained at different conditions giving the possibility to add more information per image and improve the final analysis. In this context, we developed and applied to the PROBA-V dataset of multi-frame satellite images a model that recently topped the European Space Agency’s Multi-frame Super Resolution (MFSR) competition. The model is based on proven methods that worked on 2D images tweaked to work on 3D: the Wide Activation Super Resolution (WDSR) family. We show that with a simple 3D CNN residual architecture with WDSR blocks and a frame permutation technique as data augmentation better scores can be achieved than with more complex models. Moreover, the model requires few hardware resources, both for training and evaluation, so it can be applied directly from a personal laptop.

The COVID-19 pandemic situation has created even more difficulties in the quick identification and screening of the COVID-19 patients for the medical specialists. Therefore, a significant study is necessary for detecting COVID-19 cases using an automated diagnosis method, which can aid in controlling the spreading of the virus. In this paper, the study suggests a Deep Convolutional Neural Network-based multi-classification approach (COV-MCNet) using eight different pre-trained architectures such as VGG16, VGG19, ResNet50V2, DenseNet201, InceptionV3, MobileNet, InceptionResNetV2, Xception which are trained and tested on the X-ray images of COVID-19, Normal, Viral Pneumonia, and Bacterial Pneumonia. The results from 3-class (Normal vs. COVID-19 vs. Viral Pneumonia) showed that only the ResNet50V2 model provides the highest classification performance (accuracy: 95.83%, precision: 96.12%, recall: 96.11%, F1-score: 96.11%, specificity: 97.84%) compared to rest of the models. The results from 4-class (Normal vs. COVID-19 vs. Viral Pneumonia vs. Bacterial Pneumonia) demonstrated that the pre-trained model DenseNet201 provides the highest classification performance (accuracy: 92.54%, precision: 93.05%, recall: 92.81%, F1-score: 92.83%, specificity: 97.47%). Notably, the ResNet50V2 (3-class) and DenseNet201 (4-class) models in the proposed COV-MCNet framework showed higher accuracy compared to the rest six models. This indicates that the designed system can produce promising results to detect the COVID-19 cases on the availability of more data. The proposed multi-classification network (COV-MCNet) significantly speeds up the existing radiology-based method, which will be helpful to the medical community and clinical specialists for early diagnosis of the COVID-19 cases during this pandemic.

This paper proposed an optimal tracking control scheme through adaptive dynamic programming(ADP) for a class of partially unknown discrete-time nonlinear systems based on radial basis function neural network(RBF-NN). In order to acquire the unknown system dynamics, we use two RBF-NNs, the one is used to construct the identifier, and the another is used to directly approximate the steady-state control input, where a novel adaptive law is proposed to update neural network weights. While the optimal feedback control and the cost function are derived via feedforward neural networks approximating, it is proposed to regulate the tracking error, the critic network and the actor network are then trained online to obtain the solution of the associated Hamilton–Jacobi–Bellman (HJB) equation being built under the ADP framework. Simulations verify the effectiveness of the optimal tracking control technique using the neural networks.

Clients are increasingly looking for fast and effective means to quickly and frequently survey and communicate the condition of their buildings so that essential repairs and maintenance work can be done in a proactive and timely manner before it becomes too dangerous and expensive. Traditional methods for this type of work commonly comprise of engaging building surveyors to undertake a condition assessment which involves a lengthy site inspection to produce a systematic recording of the physical condition of the building elements, including cost estimates of immediate and projected long-term costs of renewal, repair and maintenance of the building. Current asset condition assessment procedures are extensively time consuming, laborious, and expensive and pose health and safety threats to surveyors, particularly at height and roof levels which are difficult to access. We propose a method for automated detection and localisation of key building defects from images using deep learning and convolution neural networks. The proposed model is based on a pre-trained VGG-16 classifier with Class Activation Mapping (CAM) for object localisation. The model has proven to be robust and able to accurately detect and localise mould growth, stains, and paint deterioration defects arising from dampness in buildings. The approach is being developed with potentials to scale-up to support automated detection of defects and deterioration of buildings in real-time using mobile devices and drones.

In the age of social media, images come from unreliable sources that may change their content to fit a narrative. Automated detection of forged images is a complex task, especially considering recent technological advances in image manipulation software. However, in the existing literature, it is possible to identify two general approaches adopted by detection methods, active and passive. Active techniques preemptively act on an image by inserting structures before any manipulation is made that can verify its authenticity. In contrast, passive methods analyze the content of an image in search of traces of manipulation. In this way, this research proposes a novel approach to image manipulation detection, combining two passive methods through neural networks, creating a generalist approach capable of detecting with greater accuracy than the methods that compose it. Furthermore, this work used a combination of four datasets available in the literature for training and evaluation. After training, the merged approach obtained an accuracy of 89.59% in the set of validation images, significantly higher than the model trained with only unaltered images, which obtained 78.64%, and the two other models trained using images with a feature selection method applied to enhance inconsistencies that obtained 68.02% for Error-Level Analysis images and 50.70% for the method using Discrete Wavelet Transform. In addition to the performance improvement, the proposed approach’s accuracy variation was lower than the other models.

The total tree height (h) and diameter at breast height (dbh) relationship is an essential tool in forest management and planning. The height—diameter (h-dbh) relationship had been studied with several approaches and for several species worldwide. The nonlinear mixed effect modeling (NLMEM) has been extensively used and lately the resilient backpropagation artificial neural network (RBPANN) approach has been a trend topic for modeling this relationship. The artificial neural network (ANN) is a computing system based in artificial intelligence and inspired in biological neural network for supervised learning. In this study the NLMEN and RBPANN approaches were used for modeling the h—dbh relationship for Durango pine species (Pinus durangensis Martínez) in mixed-species forest from Mexico. The total dataset considered 1,000 (11,472 measured trees) randomly selected from 14,390 temporary forest inventory plots and the dataset was randomly divided into two parts; 50% for training and 50% for testing. An unsupervised clustering analysis was used to grouped the dataset into 10 clusters based on k-means clustering method and plot-variables like density, basal area, mean dbh, mean h, quadratic mean diameter, altitude and aspect. The RBPANN was performed for tangent hyperbolicus (RBPANN-tanh), softplus (RBPANN-softplus), and logistic (RBPANN-logistic) activation functions for functions in cross product of the covariate or neurons and the weights for the ANN analysis. For both training and testing, 10 classical statistics (e.g., RMSE, R2, AIC, BIC, logLik) were computed for the residual values and assess the approaches for h—dbh relationship. For training and testing, the ANNs approach outperformed the NLMEM approach, and the RBPANN-tanh has the best performance in both training and testing phases.

Polar mesocyclones (MCs) are small marine atmospheric vortices. The class of intense MCs, called polar lows, are accompanied by extremely strong surface winds and heat fluxes and thus largely influencing deep ocean water formation in the polar regions. Accurate detection of polar mesocyclones in high-resolution satellite data, while challenging, is a time-consuming task, when performed manually. Existing algorithms for the automatic detection of polar mesocyclones are based on the conventional analysis of patterns of cloudiness and involve different empirically defined thresholds of geophysical variables. As a result, various detection methods typically reveal very different results when applied to a single dataset. We develop a conceptually novel approach for the detection of MCs based on the use of deep convolutional neural networks (DCNNs). As a first step, we demonstrate that DCNN model is capable of performing binary classification of 500x500km patches of satellite images regarding MC patterns presence in it. The training dataset is based on the reference database of MCs manually tracked in the Southern Hemisphere from satellite mosaics. We use a subset of this database with MC diameters falling in the range of 200-400 km. This dataset is further used for testing several different DCNN setups, specifically, DCNN built “from scratch”, DCNN based on VGG16 pre-trained weights also engaging the Transfer Learning technique, and DCNN based on VGG16 with Fine Tuning technique. Each of these networks is further applied to both infrared (IR) and a combination of infrared and water vapor (IR+WV) satellite imagery. The best skills (97% in terms of the binary classification accuracy score) is achieved with the model that averages the estimates of the ensemble of different DCNNs. The algorithm can be further extended to the automatic identification and tracking numerical scheme and applied to other atmospheric phenomena characterized by a distinct signature in satellite imagery.

In this article, the evolution in space and in time of the coronavirus pandemic is studied by utilizing a neural network with a self-organizing nature for the spatial analysis of data, and a fuzzy fractal method for capturing the temporal trends of the time series of the countries. Self-organizing neural networks possess the capability for clustering countries in the space domain based on their similar characteristics with respect to their coronavirus cases. In this form enabling finding the countries that are having similar behavior and thus can benefit from utilizing the same methods in fighting the virus propagation. To validate the approach, publicly available datasets of coronavirus cases worldwide have been used. In addition, a fuzzy fractal approach is utilized for the temporal analysis of time series of the countries. Then, a hybrid combination of both the self-organizing maps and the fuzzy fractal approach is proposed for efficient COVID-19 forecasting of the countries. Relevant conclusions have emerged from this study, that may be of great help in putting forward the best possible strategies in fighting the virus pandemic. A lot of the existing works concerned with the Coronavirus have look at the problem mostly from the temporal viewpoint that is of course relevant, but we strongly believe that the combination of both aspects of the problem is relevant to improve the forecasting ability. The most relevant contribution of this article is the proposal of combining neural networks with a self-organizing nature for clustering countries with high similarity and the fuzzy fractal approach for being able to forecast the times series and help in planning control actions for the Coronavirus pandemic.

Deep neural networks (DNN) have successfully delivered a cutting-edge performance in several fields. With the broader deployment of DNN models on critical applications, the security of DNNs becomes an active and yet nascent area. Attacks against DNNs can have catastrophic results, according to recent studies. Poisoning attacks, including backdoor and Trojan attacks, are one of the growing threats against DNNs. Having a wide-angle view of these evolving threats is essential to better understand the security issues. In this regard, creating a semantic model and a knowledge graph for poisoning attacks can reveal the relationships between attacks across intricate data to enhance the security knowledge landscape. In this paper, we propose a DNN Poisoning Attacks Ontology (DNNPAO) that would enhance knowledge sharing and enable further advancements in the field. To do so, we have performed a systematic review of the relevant literature to identify the current state. We collected 28,469 papers from IEEE, ScienceDirect, Web of Science, and Scopus databases, and from these papers, 712 research papers were screened in a rigorous process, and 55 poisoning attacks in DNNs were identified and classified. We extracted a taxonomy of the poisoning attacks as a scheme to develop DNNPAO. Subsequently, we used DNNPAO as a framework to create a knowledge base. Our findings open new lines of research within the field of AI security.

Convolutional Neural Networks combined with autonomous drones are increasingly seen as enablers of partially automating the aircraft maintenance visual inspection process. Such an innovative concept can have a significant impact on aircraft operations. Through supporting aircraft maintenance engineers detect and classify a wide range of defects, the time spent on inspection can significantly be reduced. Examples of defects that can be automatically detected include aircraft dents, paint defects, cracks and holes, and lightning strike damage. Additionally, this concept could also increase the accuracy of damage detection and reduce the number of aircraft inspection incidents related to human factors like fatigue and time pressure. In our previous work, we have applied a recent Convolutional Neural Network architecture known by MASK R-CNN to detect aircraft dents. MASK-RCNN was chosen because it enables the detection of multiple objects in an image while simultaneously generating a segmentation mask for each instance. The previously obtained F1 and F2 scores were 62.67% and 59.35% respectively. This paper extends the previous work by applying different techniques to improve and evaluate prediction performance experimentally. The approaches uses include (1) Balancing the original dataset by adding images without dents; (2) Increasing data homogeneity by focusing on wing images only; (3) Exploring the potential of three augmentation techniques in improving model performance namely flipping, rotating, and blurring; and (4) using a pre-classifier in combination with MASK R-CNN. The results show that a hybrid approache combining MASK R-CNN and augmentation techniques leads to an improved performance with an F1 score of (67.50%) and F2 score of (66.37%)

Applications for Wireless Sensor Networks (WSNs) range from monitoring the environment to automating factories. However, sustained and effective functioning is made more difficult by Sensor Nodes (SNs) limited energy supplies in which optimization is the main issue. So with the aim of increasing the lifespan by decreasing the energy consumption of WSN, Low-Energy Adaptive Clustering Hierarchy (LEACH) protocol with Deep Learning (DL) algorithm is analyzed in this paper. LEACH is a hierarchical mechanism that elects Cluster Heads (CHs) and regularly rotates their positions in order to allocate energy use effectively by using the same amount of energy. However, Deep Learning (DL) method is used to further improve energy optimization. In many applications, the types of Deep Learning methods like Artificial Neural Networks (ANNs) have shown to be very useful. Using this method, WSNs may make more efficient decisions that reduce energy consumption. Data aggregation, duty cycling, and transmission protocols may all be optimized by Deep Learning model's ability to recognize patterns and forecast network behavior. This results in lower energy consumption, a longer lifespan for the network, and better overall performance.

Many engineered approaches have been proposed over the years for solving the hard problem of performing indoor localisation using smartphone sensors. However, specialising these solutions for difficult edge cases remains challenging. Here we propose an end-to-end hybrid multimodal deep neural network localisation system, MM-Loc, relying on zero hand-engineered features, learning them automatically from data instead. This is achieved by using modality-specific neural networks to extract preliminary features from each sensing modality, which are then combined by cross-modality neural structures. We show that our choice of modality-specific neural architectures is capable of estimating the location with good accuracy independently. But for better accuracy, a multimodal neural network fusing the features of early modality-specific representations is a better proposition. Our proposed MM-Loc solution is tested on cross-modality samples characterised by different sampling rates and data representation (inertial sensors, magnetic and WiFi signals), outperforming traditional approaches for location estimation. MM-Loc elegantly trains directly from data unlike conventional indoor positioning systems, which rely on human intuition.

Timely diagnosis of Alzheimer's diseases(AD) is crucial to obtain more practical treatments. In this paper, a novel approach Based on Multi-Level Fuzzy Neural Networks (MLFNN) for early detection of AD is proposed. The focus of study was on the problem of diagnosing AD and MCI patients from healthy people using MLFNN and selecting the best feature(s) and most compatible classification algorithm. In this way, we achieve an excellent performance using only a single feature i.e. MMSE score, by fitting the optimum algorithm to the best area using optimum possible feature(s) namely one feature for a real life problem. It can be said, the proposed method is a discovery that help patients and healthy people get rid of painful and time consuming experiments. Experiments shows the effectiveness of proposed method in current research for diagnosis of AD with one of the highest performance (accuracy rates of 96.6%), ever reported in the literature.

In considering knowledge graphs in a diverse range of domains of interest, graph neural networks have demonstrated significant improvements in node classification and prediction when applied to graph representation with learning node embedding to effectively represent hierarchical properties of graphs. DiffPool is a deep-learning approach using a differentiable graph pooling technique that generates hierarchical representations of graphs. In operation DiffPool is a differentiable graph pooling technique that generates hierarchical representations of graphs. DiffPool learns a differentiable soft cluster assignment for nodes at each layer of a deep graph neural network with nodes mapped sets of clusters. However, control of the learning process is difficult given the complexity and large number of parameters on an `end-to-end’ model. To address this difficulty we propose an novel approach termed FPool which is predicated on the basic approach adopted in DiffPool (where pooling is applied directly to node representations). Methods designed to enhance data classification have been developed and evaluated using a number popular and publicly available sensor data sets. Experimental results for FPool demonstrate improved classification and prediction performance when compared to alternative methods. Moreover, FPool shows an important reduction in the training time over the basic DiffPool framework.

Audio recording in classrooms is a common practice in educational research, with applications ranging from detecting classroom activities to analysing student behaviour. Previous research has employed neural networks for classroom activity detection and speaker role identification. However, these recordings are often affected by background noise that can hinder further analysis, and the literature has only sought to identify noise with general filters and not specifically designed for classrooms. Although the use of high-end microphones and environmental monitoring can mitigate this problem, these solutions can be costly and potentially disruptive to the natural classroom environment. In this context, we propose the development of a neural network model that can specifically detect and filter out background noise in classroom recordings. This model would allow the use of lower quality recordings without compromising analysis capability, thus facilitating data collection in natural educational environments and reducing the costs associated with high-end recording equipment.

This paper studies the finite-time synchronization problem of fractional-order stochastic memristive bidirectional associative memory neural networks (MBAMNNs) with discontinuous jumps. A novel criterion for finite-time synchronization is obtained by utilizing the properties of quadratic fractional-order Gronwall inequality with time delay and the comparison principle. This criterion provides a new approach to analyze the finite-time synchronization problem of neural networks with stochasticity. Finally, numerical simulations are provided to demonstrate the effectiveness and superiority of the obtained results.

Electrical power quality is one of the main elements in power generation systems. At the same time, it is one of the most significant challenges regarding stability and reliability. Due to different switching devices in this type of architecture, different kinds of power generators, and non-linear loads are used for different industrial processes. As a result of this, the need to classify and analyze Power quality disturbance (PQD) to prevent and analyze the degradation of the system reliability affected by the non-linear and non-stationary oscillatory nature. This paper presents A Novel Mul-titasking Deep Neural Network (MDL) for the Classification and Analysis of Multiple Electrical Disturbances. The characteristics are extracted with a specialized and adaptive methodology for non-stationary signals, Empirical Mode Decomposition (EMD). The methodology’s design, devel-opment, and various performance tests are carried out with 28 different difficulty levels, such as severity, disturbance duration time, and noise in the 20 dB to 60 dB signal range. MDL was devel-oped with a diverse data set in difficulty and noise, with a quantity of 4500 records of different samples of multiple electrical disturbances. The analysis and classification methodology has an average accuracy percentage of 95% with multiple disturbances. In addition, an average accuracy percentage of 90% in analyzing important signal aspects for studying electrical power quality such as crest factor, Per Unit voltage analysis, Short Term Flicker Perceptibility (Pst), and Total Harmonic Distortion (THD), among others.

The objective of this article was to forecast the ultimate failure load laminate stacking sequence combination on bonding joints which are exposed to tensile strength by using artificial neural networks. We have glass fiber composite materials with three different sequence combinations ([0°/90°], [±45°], [0°/90°/±45°]). Various adherend thicknesses and also ductile type adhesive was used in the experiment. The bonding geometry is a single lap and has four types of overlap angles 30°, 45°, 60°, 75° respectively. The experimental results demonstrate that composite laminate stacking sequence profoundly affects the bonding joints of failure load. Taking experimental results into account, Levenberg–Marquardt learning algorithm model was used by preferring a three layer forward on ANN so as to discipline network. In order to procure a precise ANN tool, an integrate methodology of experimental method has been used. The outcomes are used to ensure the experimental data’s to the ANN. The method of ANN permits surveying much adequately the probabilities of composite laminate stacking sequence combination using the prevalent ones which are [0°/90°], [±45°] and [0°/90°/±45°]. Testing data and training results were quite well 0.998, 0.997 and 0.998 in turn. Consequences acquired can be used by engineers who are interested in the composite material design to enhance failure load.

Predicting trip purpose from comprehensive and continuous smart card data is beneficial for transport and city planners in investigating travel behaviours and urban mobility. Here we propose a framework, ActivityNET, using machine learning (ML) algorithms to predict passengers’ trip purpose from smart card data and Points-of-Interest (POIs) data. The feasibility of the framework is demonstrated in two phases. Phase I focuses on extracting activities from individuals’ daily travel patterns from smart card data and combining them with POIs using the proposed ‘activity-POIs consolidation algorithm’. Phase II feeds the extracted features into an artificial neural network (ANN) with multiple scenarios and predicts trip purpose under primary activities (home and work) and secondary activities (entertainment, eating, shopping, child drop-offs/pick-ups and part-time work) with high accuracy. As a case study, the proposed ActivityNET framework is applied in Greater London and illustrates a robust competence to predict trip purpose. The promising outcomes demonstrate that the cost-effective framework offers high predictive accuracy and valuable insights into transport planning.

Automated lip reading is a research problem that has developed considerably in recent years. Lip reading is evaluated both visually and audibly in some cases. The lip reading model is a field of use for detecting specific words using images from security cameras, but it is not possible to use audio-visual databases in this situation. It is not possible to obtain the sound input of the pronounced word in all cases. We collected a new Turkish dataset with only the image in this study. The new dataset is produced using Youtube videos, which is an uncontrolled environment. For this reason, images have difficult parameters in terms of environmental factors such as light, angle, color, and personal characteristics of the face. Despite the different features on the human face such as mustache, beard, and make-up, the visual speech recognition problem was developed on 10 classes including single words and two-word phrases using Convolutional Neural Networks (CNN) without any intervention on the data. The proposed study using only-visual data obtained a model which is automated visual speech recognition with a deep learning approach. In addition, since this study uses only-visual data, the computational cost and resource usage is less than in multi-modal studies. It is also the first known study to address the lip reading problem with a deep learning algorithm using a new dataset belonging to the Ural-Altaic languages.

Sparse training (ST) aims to ameliorate deep learning by replacing fully connected artificial neural networks (ANNs) with sparse or ultra-sparse ones, such as brain networks are, therefore it might benefit to borrow brain-inspired learning paradigms from complex network intelligence theory. Here, we launch the ultrasparse advantage challenge, whose goal is to offer evidence on the extent to which ultra-sparse (around 1% connection retained) topologies can achieve any leaning advantage on fully connected. Epitopological learning is a field of network science and complex network intelligence that studies how to implement learning on complex networks by changing the shape of their connectivity structure (epitopological plasticity). One way to implement Epitopological (epi- means new) Learning is via link prediction: predicting the likelihood of nonobserved links to appear in the network. Cannistraci-Hebb learning theory inspired the CH3-L3 network automata rule for link prediction which is effective for general-purpose link prediction. Here, starting from CH3-L3 we propose Epitopological Sparse Meta-deep Learning (ESML) to apply Epitopological Learning to sparse training. In empirical experiments, we find that ESML learns ANNs with ultra-sparse hyperbolic (epi-)topology in which emerges a community layer organization that is meta-deep (meaning that each layer also has an internal depth due to powerlaw node hierarchy). Furthermore, we discover that ESML can in many cases automatically sparse the neurons during training (arriving even to 30% neurons left in hidden layers), this process of node dynamic removal is called percolation. Starting from this network science evidence, we design Cannistraci-Hebb training (CHT), a 4-step training methodology that put ESML at its heart. We conduct experiments on 6 datasets and 3 network structures (MLPs, VGG16, ResNet50) comparing CHT to dynamic sparse training SOTA algorithms and fully connected network. The results indicate that, with a mere 1% of links retained during training, CHT surpasses fully connected networks on VGG16 and ResNet50. This key finding is an evidence for ultra-sparse advantage and signs a milestone in deep learning. CHT acts akin to a gradient-free oracle which adopts CH3-L3 based epitopological learning to guide the placement of new links in the ultra-sparse network topology to facilitate sparse-weight gradient learning, and this in turn reduces the convergence time of ultra-sparse training. Finally, CHT offers first examples of parsimony dynamic sparse training because, in many datasets, it canretain network performance by percolating and significantly reducing the node network size.

Memristive spiking neural networks are a promising emerging technology in the field of deep learning. These models can be implemented using neuromorphic hardware and therefore offer low power consumption and low latency, crucial to edge-computing applications. These systems, however, pose several challenges, including on-chip training and connectivity reduction. The latter is essential to ease the manufacturing complexity of the system. On-chip training can be realized using synaptic plasticity enabled by memristors. In this work, we compare two methods of connectivity reduction applicable to memristive spiking networks: an ensemble-based approach and a probabilistic sparse connectivity approach. We evaluate both of these methods in conjunction with a three-layer spiking neural network on the handwritten and spoken digits classification tasks using two memristive plasticity models and a classical time-dependent plasticity rule. On the handwritten digits recognition task, both methods achieve the F1-score of 0.89–0.93, and yield the 0.80–0.96 F1-score on the spoken digits recognition task.

Gradient based methods are popularly used in training neural networks and can be broadly categorized into first and second order methods. Second order methods have shown to have better convergence compared to first order methods, especially in solving highly nonlinear problems. The BFGS quasi-Newton method is the most commonly studied second order method for neural network training. Recent methods have shown to speed up the convergence of the BFGS method using the Nesterov’s acclerated gradient and momentum terms. The SR1 quasi-Newton method though less commonly used in training neural networks, are known to have interesting properties and provide good Hessian approximations when used with a trust-region approach. Thus, this paper aims to investigate accelerating the Symmetric Rank-1 (SR1) quasi-Newton method with the Nesterov’s gradient for training neural networks and briefly discuss its convergence. The performance of the proposed method is evaluated on a function approximation and image classification problem.

As a relatively high percentage of adenoma polyps are missed, a computer-aided diagnosis (CAD) tool based on deep learning can aid the endoscopist in diagnosing colorectal polyps or colorectal cancer in order to decrease polyps missing rate and prevent colorectal cancer mortality. Convolutional Neural Network (CNN) is a deep learning method and has achieved better results in detecting and segmenting specific objects in images in the last decade than conventional models such as regression, support vector machines or artificial neural networks. In recent years, based on the studies in medical imaging criteria, CNN models have acquired promising results in detecting masses and lesions in various body organs, including colorectal polyps. In this review, the structure and architecture of CNN models and how colonoscopy images are processed as input and converted to the output are explained in detail. In most primary studies conducted in the colorectal polyp detection and classification field, the CNN model has been regarded as a black box since the calculations performed at different layers in the model training process have not been clarified precisely. Furthermore, I discuss the differences between the CNN and conventional models, inspect how to train the CNN model for diagnosing colorectal polyps or cancer, and evaluate model performance after the training process.

Although automated Acute Lymphoblastic Leukemia (ALL) detection is essential, it is challenging due to the morphological correlation between malignant and normal cells. The traditional ALL classification strategy is arduous, time-consuming, often suffers inter-observer variations, and necessitates experienced pathologists. This article has automated the ALL detection task, employing deep Convolutional Neural Networks (CNNs). We explore the weighted ensemble of deep CNNs to recommend a better ALL cell classifier. The weights are estimated from ensemble candidates' corresponding metrics, such as accuracy, F1-score, AUC, and kappa values. Various data augmentations and pre-processing are incorporated for achieving a better generalization of the network. We train and evaluate the proposed model utilizing the publicly available C-NMC-2019 ALL dataset. Our proposed weighted ensemble model has outputted a weighted F1-score of 88.6%, a balanced accuracy of 86.2%, and an AUC of 0.941 in the preliminary test set. The qualitative results displaying the gradient class activation maps confirm that the introduced model has a concentrated learned region. In contrast, the ensemble candidate models, such as Xception, VGG-16, DenseNet-121, MobileNet, and InceptionResNet-V2, separately produce coarse and scatter learned areas for most example cases. Since the proposed ensemble yields a better result for the aimed task, it can experiment in other domains of medical diagnostic applications.

The Sentinel-2 satellite mission offers high resolution multispectral time series image data, enabling the production of detailed land cover maps globally. At this scale, the trade-off between processing time and result quality is a central design decision. Currently, this machine learning task is usually performed using pixelwise classification methods. The radical shift of the computer vision field away from hand engineered image features and towards more automation by representation learning comes with many promises, including higher quality results and less engineering effort. In this paper we assess fully convolutional neural networks architectures as replacements for a Random Forest classifier in an operational context for the production of high resolution land cover maps with Sentinel-2 time series at the country scale. Our contributions include a framework for working with Sentinel-2 L2A time series image data, an adaptation of the U-Net model for dealing with sparse annotation data while maintaining high resolution output, and an analysis of those results in the context of operational production of land cover maps.

The power system has rapidly grown and expanded over the past decades and has been experiencing major changes and challenges. The increase in energy demand and the modern advancements in the smart grid, such as solar and wind energies and electric vehicles, have led to complexity and complications for utilities. A further layer of complexity and difficulty was added by the rapid expansion of behind-the-meter (BTM) photovoltaic (PV) systems with various designs and characteristic features. The rapid increase and the invisible solar power (BTM) have led to fluctuations in power grid stability and reliability and to inefficiency. Accurate forecasting of load generation will help to assure optimal planning, minimize the negative effects of the PV systems, and minimize the operational and maintenance costs. The authors propose a solution that uses combinations of K-means clustering with neural network machine learning models, AMI real-world PV load generation, and weather data to forecast the generation load at customer locations to achieve a 2.49% error between actual and predicted generation load.

Non-invasive, electroencephalography (EEG)-based brain-computer interfaces (BCIs) on motor imagery movements translate the subject’s motor intention into control signals through classifying the EEG patterns caused by different imagination tasks, e.g. hand movements. This type of BCI has been widely studied and used as an alternative mode of communication and environmental control for disabled patients, such as those suffering from a brainstem stroke or a spinal cord injury (SCI). Notwithstanding the success of traditional machine learning methods in classifying EEG signals, these methods still rely on hand-crafted features. The extraction of such features is a difficult task due to the high non-stationarity of EEG signals, which is a major cause for the stagnating progress in classification performance. Remarkable advances in deep learning methods allow end-to-end learning without any feature engineering, which could benefit BCI motor imagery applications. We developed three deep learning models: 1) a long short-term memory (LSTM); 2) a proposed spectrogram-based convolutional neural network model (pCNN); and 3) a recurrent convolutional neural network (RCNN), for decoding motor imagery movements directly from raw EEG signals without (manual) feature engineering. Results were evaluated on our own, publicly available, EEG data collected from 20 subjects and on an existing dataset known as 2b EEG dataset from "BCI Competition IV". Overall, better classification performance was achieved with deep learning models compared to state-of-the art machine learning techniques, which could chart a route ahead for developing new robust techniques for EEG signal decoding. We underpin this point by demonstrating the successful real-time control of a robotic arm using our CNN based BCI.

With the rapid expansion of intelligent resource-constrained devices and high-speed communication technologies, Internet of Things (IoT) has earned a wide recognition as the primary standard for low-power lossy networks (LLNs). Nevertheless, IoT infrastructures are vulnerable to cyber-attacks due to the constraints in computation, storage, and communication capacity of the endpoint devices. From one side, the majority of newly developed cyber-attacks are formed by slightly mutating formerly established cyber-attacks to produce a new attack tending to be treated as a normal traffic through the IoT network. From the other side, the influence of coupling the deep learning techniques with cybersecurity field has become a recent inclination of many security applications due to their impressive performance. In this paper, we provide a comprehensive development of a new intelligent and autonomous deep learning-based detection and classification system for cyber-attacks in IoT communication networks leveraging the power of convolutional neural networks, abbreviated as (IoT-IDCS-CNN). The proposed IoT-IDCS-CNN makes use of the high-performance computing employing the robust CUDA based Nvidia GPUs and the parallel processing employing the high-speed I9-Cores based Intel CPUs. In particular, the proposed system is composed of three subsystems: Feature Engineering subsystem, Feature Learning subsystem and Traffic classification subsystem. All subsystems are developed, verified, integrated, and validated in this research. To evaluate the developed system, we employed the NSL-KDD dataset which includes all the key attacks in the IoT computing. The simulation results demonstrated more than 99.3% and 98.2% of cyber-attacks’ classification accuracy for the binary-class classifier (normal vs anomaly) and the multi-class classifier (five categories) respectively. The proposed system was validated using k-fold cross validation method and was evaluated using the confusion matrix parameters (i.e., TN, TP, FN, FP) along with other classification performance metrics including precision, recall, F1-score, and false alarm rate. The test and evaluation results of the IoT-IDCS-CNN system outperformed many recent machine-learning based IDCS systems in the same area of study.

Interest in drone solutions in forestry applications is growing. Using drones, datasets can be captured flexibly and at high spatial and temporal resolutions when needed. In forestry applications, fundamental tasks include the detection of individual trees, tree species classification, bio-mass estimation, etc. Deep Neural Networks (DNN) have shown superior results when comparing with conventional machine learning methods such as Multi-Layer Perceptron (MLP) in cases of huge input data. The objective of this research was to investigate 3D convolutional neural networks (3D-CNN) to classify three major tree species in a boreal forest: pine, spruce, and birch. The proposed 3D-CNN models were employed to classify tree species in a test site in Finland. The classifiers were trained with a dataset of 3039 manually labelled trees. Then the accuracies were assessed by employing independent datasets of 803 records. To find the most efficient set of feature combination, we compare the performances of 3D-CNN models trained with hyperspectral (HS) channels, RGB channels, and canopy height model (CHM), separately and combined. It is demonstrated that the proposed 3D-CNN model with RGB and HS layers produces the highest classification accuracy. The producer accuracy of the best 3D-CNN classifier on the test dataset were 99.6%, 94.8%, and 97.4% for pines, spruces, and birches, respectively. The best 3D-CNN classifier produced ~5% better classification accuracy than the MLP with all layers. Our results suggest that the proposed method provides excellent classification results with acceptable performance metrics for HS datasets. Our results show that pine class was detectable in most layers. Spruce was most detectable in RGB data, while birch was most detectable in the HS layers. Furthermore, the RGB datasets provide acceptable results for many low-accuracy applications.

In this paper we propose a computational framework that merges mechanistic modeling with deep neural networks obeying the Systems Biology Markup Language (SBML) standard. Over the last 20 years, the systems biology community has developed a large number of mechanistic models in SBML that are currently stored in public databases. With the proposed framework, existing SBML mechanistic models may be upgraded to hybrid systems through the incorporation of deep neural networks into the model core, using a freely available python tool. The so-formed hybrid mechanistic/neural network models are trained with a deep learning algorithm based on the adaptive moment estimation method (ADAM), stochastic regularization and semidirect sensitivity equations. The trained hybrid models are encoded in SBML and stored back in model databases, where they can be further analyzed as regular SBML models. The application of this approach is illustrated with three well-known case studies: the threonine synthesis model in Escherichia coli, the P58IPK signal transduction model, and the Yeast glycolytic oscillations model. The proposed framework is expected to greatly facilitate the widespread use of hybrid modeling techniques for systems biology applications.

Satellite Image Time Series (SITS) is a data set which includes satellite images across several years with a high acquisition rate. Radiometric normalization is a fundamental and important preprocessing method for remote sensing applications using SITS due to the radiometric distortion caused by noise between images. Normalizing the subject image based on the reference image is a general strategy when using traditional radiometric normalization methods to normalize multi-temporal imagery (usually two scenes or three scenes in different time phases). However, these methods are not suitable for calibrating SITS because they cannot minimize the radiometric distortion between any pair of images in SITS. The existing relative radiometric normalization methods for SITS are based on linear assumptions which cannot effectively reduce nonlinear radiometric distortion caused by continuously changing noise in SITS. To overcome this problem and obtain a more accurate SITS, this study proposed a Nonlinear Radiometric Normalization Model (named NMAG) for SITS based on Artificial Neural Networks (ANN) and Greedy Algroithm (GA) . In this method, GA was used to determine the correction order of SITS and calculate the error between the image to be corrected and normalized images, which avoided the selection of a single reference image. ANN was used to obtain the optimal solution of error function, which minimized the radiometric distortion between different images in SITS. SITS composed of 21 Landsat-8 images in Tianjin City from October 2017 to January 2019 were selected to test the method. We compared NMAG with other two contrasting method (refered as CM1 and CM2), and found that the average of root mean square error $(\mu_{RMSE})$ of NMAG $(497.22)$ was significantly smaller than those of CM1 $(641.39)$ and CM2 $(543.47)$, and the accuracy of normalized SITS obtained by using NMAG has increased by 22.4\% and 8.5\% toward CM1 and CM2, respectively. These experimental results confirmed the effectiveness of NMAG in the reduction of radiometric distortion caused by continuously changing noise between images in SITS.

In this paper, Lyapunov-Razumikhin technique, design of state-dependent switching laws, a fixed point theorem and variational methods are employed to derive the existence and the unique existence results of (globally) exponentially stable positive stationary solution of delayed reaction-diffusion cell neural networks under Dirichlet zero boundary value, including the global stability criteria \textbf{in the classical meaning}. Next, sufficient conditions are proposed to guarantee the global stability invariance of ordinary differential systems under the influence of diffusions. New theorems show that the diffusion is a double-edged sword in judging the stability of diffusion systems. Besides, an example is constructed to illuminate that any non-zero constant equilibrium point must be not in the phase plane of dynamic system under Dirichlet zero boundary value, or it must lead to a contradiction. Next, under Lipschitz assumptions on active function, another example is designed to prove that the small diffusion effect will cause the essential change of the phase plane structure of the dynamic behavior of the delayed neural networks via a Saddle point theorem. Finally, a numerical example illustrates the feasibility of the proposed methods.

Novel applications of artificial intelligence for tuning the parameters of industrial machines for optimal performance are emerging at a fast pace. Tuning the combine harvesters and improving the machine performance can dramatically minimize the wastes during harvesting, and it is also beneficial to machine maintenance. Literature includes several soft computing, machine learning and optimization methods that had been used to model the function of harvesters of various crops. Due to the complexity of the problem, machine learning methods had been recently proposed to predict the optimal performance with promising results. In this paper, through proposing a novel hybrid machine learning model based on artificial neural networks integrated with particle swarm optimization (ANN-PSO), the performance analysis of a common combine harvester is presented. The hybridization of machine learning methods with soft computing techniques has recently shown promising results to improve the performance of the combine harvesters. This research aims at improving the results further by providing more stable models with higher accuracy.

This paper presents a comprehensive investigation into the applicability and performance of two prominent growth models, namely the Verhulst model and the Montroll model, in the context of modeling tumor cell growth dynamics. Leveraging the power of Physics-Informed Neural Networks (PINNs), we aim to assess and compare the predictive capabilities of these models against experimental data obtained from the growth patterns of tumor cells. We employ a dataset comprising detailed measurements of tumor cell growth to train and evaluate the Verhulst and Montroll models. By integrating PINNs, we not only account for experimental noise but also embed physical insights into the learning process, enabling the models to capture the underlying mechanisms governing tumor cell growth. Our findings reveal the strengths and limitations of each growth model in accurately representing tumor cell proliferation dynamics. Furthermore, the study sheds light on the impact of incorporating physics-informed constraints on the model predictions. The insights gained from this comparative analysis contribute to advancing our understanding of growth models and their applications in predicting complex biological phenomena, particularly in the realm of tumor cell proliferation.

We observed and studied the phenomenon of the redox front propagation in different polyaniline (PAni) deposited samples interfaced with liquid or solid polymeric electrolyte. The front of electrochemical conversion, in which the insulator and the conductor parts of PAni are interfaced, is studied observing the temporal evolution of the conductivity of PAni samples. The propagation of redox front was studied in electrochemically obtained thick (2 µm) and in thin (50 nm) poly-aniline films, obtained using Langmuir-Blodgett (LB) method. In the configuration that we tested, the speed of the red-ox front propagation for the thin LB films was found to be 200 micron/sec opening the way for the manufacturing large neural networks, realized using PAni based memristive devices, in which the memristance can be quickly changed in the programmable manner. The prototypes of the spiking neuron connections were manufactured on the basis of lithographically developed gold contacts, bridged by electrochemically grown polyaniline and placed under polymer electrolyte layer with only one counter electrode (gate) for the whole manifold of pseudo-two-terminal memristor bridges. The spike propagation was studied in such gold-polyaniline systems. The research opens the possibility of miniature spike or rate-based neural network circuits manufacture, based on metal pads and polyaniline.

Even today, the automatic digitisation of scanned documents in general but especially the automatic optical music recognition (OMR) of historical manuscripts still remain an enormous challenge, since both handwritten musical symbols and text have to be identified. This paper focuses on the Medieval so-called square notation developed in the 11th-12th century, which is already composed of staff lines, staves, clefs, accidentals, and neumes, that are roughly spoken connected single notes. The aim is to develop an algorithm that captures both the neume and pitch, that is melody information that can be used to reconstruct the original writing. Our pipeline is similar to the standard OMR approach and comprises a novel staff line and symbol detection algorithm, based on deep Fully Convolutional Networks (FCN), which perform pixel-based predictions for either staff lines or symbols and their respective types. Then, the staff line detection combines the extracted lines to staves and yields an F1-score of over 99% for both detecting lines and complete staves. For the music symbol detection we choose a novel approach that skips the step to identify neumes and instead directly predicts note components (NCs) and their respective affiliation to a neume. Furthermore, the algorithm detects clefs and accidentals. Our algorithm recognises these symbols with an F1-score of over 96% if the type is ignored and predicts the true symbol sequence of a staff with a diplomatic symbol accuracy rate (dSAR) of about 87%. If only the NCs without their respective connection to a neume, all clefs, and accidentals are of interest the algorithm reaches an harmonic symbol accuracy rate (hSAR) of approximately 90%.

In any document, graphical elements like tables, figures, and formulas contain essential information. The processing and interpretation of such information require specialized algorithms. Off-the-shelf OCR components cannot process this information reliably. Therefore, an essential step in document analysis pipelines is to detect these graphical components. It leads to a high-level conceptual understanding of the documents that makes digitization of documents viable. Since the advent of deep learning, the performance of deep learning-based object detection has improved many folds. In this work, we outline and summarize the deep learning approaches for detecting graphical page objects in the document images. Therefore, we discuss the most relevant deep learning-based approaches and state-of-the-art graphical page object detection in document images. This work provides a comprehensive understanding of the current state-of-the-art and related challenges. Furthermore, we discuss leading datasets along with the quantitative evaluation. Moreover, it discusses briefly the promising directions that can be utilized for further improvements.

Artificial intelligence methods, especially artificial neural networks (ANNs), have recently been successfully used more and more frequently in the mathematical description of physical phenomena in the processing of metallic materials. Results based on measured data of phenomena in real production are often, at least locally, different or more complex than the results of established theories. In general, ANN models, which mostly work as "black boxes", allow us to improve production efficiency by themselves. The fact is, however, that a better understanding of the phenomena under consideration, which is otherwise enabled by classical mathematical approaches in the form of explicit mathematical formulas, could ensure even greater efficiency. In this article, therefore, a general concept is proposed to explain the discussed phenomena in metallic materials. The application of the concept is illustrated on the example of hot extrusion of aluminum alloy 6082, where CAE is used as an artificial neural network to model the phenomenon and predict the key parameters of the phenomenon. ChatGPT is used to explain the results according to the proposed concept. The basic idea is to enable practicing engineers to better understand the considered phenomena and results obtained by ANNs.

The difficulty in detecting tumors in earlier stages is the major cause of mortalities of patients, despite the advancements in treatment and research regarding ovarian cancer. Deep Learning algorithms are applied to serve the purpose of a diagnostic tool by applying them on CT scan images of the ovarian region. The images go through a series of pre-processing techniques and further the tumor is segmented using the UNet model. Instances are then classified into two categories – benign and malignant tumors. Classification is performed using Deep Learning models like CNN, ResNet, DenseNet, Inception-ResNet, VGG16 and Xception along with Machine Learning models such as Random Forest, Gradient Boosting, AdaBoosting, XGBoosting. DenseNet 121 emerges as the best model on this dataset even after applying optimization on the Machine Learning models by obtaining an accuracy of 95.7%. The current work demonstrates the comparison of multiple CNN architectures among themselves and with common Machine Learning algorithms, with and without optimization techniques applied.

Geometrical assessments of human skull have been done based on the anatomical landmarks. Automatic detection of the landmarks, if developed, will be a great help not only medically, but also anthropologically. An automated system with multi-phased deep learning networks to predict three-dimensional coordinate values of cranio-facial landmarks, was developed. From a publicly available database, computed tomography images of craniofacial area were obtained. They were digitally reconstructed into three-dimensional models. Sixteen anatomical landmarks were plotted on each of the models and coordinate values of them were recorded. Three-phased regression deep learning networks were trained respectively with 90 training datasets. For evaluation, 30 testing datasets were employed. Three-dimensional error for the first phase, testing 30 data, was 11.60 pixels in average. (1 pixel = 500 / 512 mm) For the second phase, it was significantly improved to 4.66 pixels. For the third phase, it was significantly progressed to 2.88. This was comparable to the gaps between the landmarks, plotted by two experienced practitioners. Our proposing method of multi-phased prediction, coarse detection first and narrowing down the detection area, may be a possible solution, within the physical limitation of memory and computation.

Due to the ongoing global warming on the Earth, permafrost degradation has been extensively taking place, which poses a substantial threat to civil and industrial facilities and infrastructure elements, as well as to the utilization of natural resources in the Arctic and high-latitude regions. In order to prevent the negative consequences of permafrost thawing under the foundations of constructions, various geophysical techniques for monitoring permafrost have been proposed and applied so far: temperature, electrical, seismic and many others. At the same time, non-stationary electromagnetic methods seem to have significant potential and a number of important practical advantages. We propose a cross-borehole exploration system for the highest localization of target objects in the cryolithozone. A novel mathematical apparatus for three-dimensional modeling of transient electromagnetic (TEM) signals by the vector finite element method has been developed. The original combination of the latter, the Sumudu integral transform and artificial neural networks makes it possible to examine spatially heterogeneous objects of the cryolithozone with a high contrast of geoelectric parameters, significantly reducing computational costs. We have studied the sensitivity of the TEM signals to the boundary between thawed and frozen rocks depending on the inter-borehole distance and spatial orientations of the transmitters and receivers. Numerical simulation results of the TEM monitoring of industrial facilities located on permafrost are considered. The formation of a talik has been shown to significantly manifest itself in the measured electromagnetic responses, which enables timely prevention of industrial disasters and environmental catastrophes.

The angle of seismic excitation is a significant factor of the seismic response of RC buildings. The procedure required for the calculation of the angle for which the potential seismic damage is maximized (critical angle) contains multiple nonlinear time history analyses using in each one of them different angles of incidence. Moreover, the seismic codes recommend the application of more than one accelerograms for the evaluation of seismic response. Thus, the whole procedure becomes time consuming. Herein, a method to reduce the time required for the estimation of the critical angle based on Multilayered Feedforward Perceptron Neural Networks is proposed. The basic idea is the detection of cases in which the critical angle increases the class of seismic damage compared to the class which arises from the application of the seismic motion along the buildings’ structural axes. To this end, the problem is expressed and solved as Pattern Recognition problem. As inputs of networks the ratios of seismic parameters’ values along the two horizontal seismic records' components, as well as appropriately chosen structural parameters, were used. The results of analyses show that the neural networks can reliably detect the cases in which the calculation of the critical angle is essential.

The main goal of the paper is to use a generalized proportional Riemann-Liouville fractional derivative (GPRLFD) to model BAM neural networks and to study some stability properties of the equilibrium. Initially, several properties of the GPRLFD are proved such as the fractional derivative of a squared function. Also some comparison results for GPRLFD are provided. Two types of equilibrium of the BAM model with GPRLFD are defined. In connection with the applied fractional derivative and its singularity at the initial time the Mittag-Leffler exponential stability in time of the equilibrium is introduced and studied. An example is given illustrating the meaning of the equilibrium as well as its stability properties.

Electrolysis stands as a pivotal method for environmentally sustainable hydrogen production. However, the formation of gas bubbles during the electrolysis process poses significant challenges by impeding the electrochemical reactions, diminishing cell efficiency, and dramatically increasing energy consumption. Furthermore, the inherent difficulty in detecting these bubbles arises from the non-transparency of the wall of electrolysis cells. Additionally, these gas bubbles induce alterations in the conductivity of the electrolyte, leading to corresponding fluctuations in the magnetic flux density outside of the electrolysis cell, which can be measured by externally placed magnetic sensors. By solving the inverse problem of the Biot-Savart Law, we can estimate the conductivity distribution as well as the void fraction within the cell. In this work, we study different approaches to solve the inverse problem including Invertible Neural Networks (INNs) and Tikhonov regularization. Our experiments demonstrate that INNs are much more robust to solving the inverse problem than Tikhonov regularization when the level of noise in the magnetic flux density measurements is not known or changes over space and time.

Recent advancements of computer and electronic systems have motivated the extensive use of intelligent systems for automation of agricultural industries. In this study, the temperature variation of the mushroom growing room is modeled through using a multi-layered perceptron (MLP) and radial basis function networks. Modeling has been done based on the independent parameters including ambient temperature, water temperature, fresh air and circulation air dampers, and water tap. According to the obtained results from the networks, the best network for MLP is found to be the second repetition with 12 neurons in the hidden layer and 20 neurons in the hidden layer for radial basis function networks. The obtained results from comparative parameters for two networks showed the highest correlation coefficient (0.966), the lowest root mean square error (RMSE) (0.787) and the lowest mean absolute error (MAE) (0.02746) for radial basis function. Therefore, the neural networks with radial basis function was selected as the optimal predictor for the behavior of the system.

The interpretation of land use and land cover (LULC) is an important issue in the fields of high-resolution remote sensing (RS) image processing and land resource management. Fully training a new or existing convolutional neural network (CNN) architecture for LULC classification requires a large amount of remote sensing images. Thus, fine-tuning a pre-trained CNN for LULC detection is required. To improve the classification accuracy for high resolution remote sensing images, it is necessary to use another feature descriptor and to adopt a classifier for post-processing. A fully connected conditional random fields (FC-CRF), to use the fine-tuned CNN layers, spectral features, and fully connected pairwise potentials, is proposed for image classification of high-resolution remote sensing images. First, an existing CNN model is adopted, and the parameters of CNN are fine-tuned by training datasets. Then, the probabilities of image pixels belong to each class type are calculated. Second, we consider the spectral features and digital surface model (DSM) and combined with a support vector machine (SVM) classifier, the probabilities belong to each LULC class type are determined. Combined with the probabilities achieved by the fine-tuned CNN, new feature descriptors are built. Finally, FC-CRF are introduced to produce the classification results, whereas the unary potentials are achieved by the new feature descriptors and SVM classifier, and the pairwise potentials are achieved by the three-band RS imagery and DSM. Experimental results show that the proposed classification scheme achieves good performance when the total accuracy is about 85%.

A phenotype is the composite of an observable expression of a genome for traits in a given environment. The trajectories of phenotypes computed from an image sequence and timing of important events in a plant’s life cycle can be viewed as temporal phenotypes and indicative of the plant’s growth pattern and vigor. In this paper, we introduce a novel method called FlowerPhenoNet which uses deep neural networks for detecting flowers from multiview image sequences for high throughput temporal plant phenotyping analysis. Following flower detection, a set of novel flower-based phenotypes are computed, e.g., the day of emergence of the first flower in a plant’s life cycle, the total number of flowers present in the plant at a given time, the highest number of flowers bloomed in the plant, growth trajectory of a flower and the blooming trajectory of a plant. To develop a new algorithm and facilitate performance evaluation based on experimental analysis, a benchmark dataset is indispensable. Thus, we introduce a benchmark dataset called FlowerPheno which comprises image sequences of three flowering plant species, e.g., sunflower, coleus, and canna, captured by a visible light camera in a high throughput plant phenotyping platform from multiple view angles. The experimental analyses on the FlowerPheno dataset demonstrate the efficacy of the FlowerPhenoNet.

Object segmentation on remotely-sensed images: aerial (or very high resolution, VHS) images and satellite (or high resolution, HR) images, has been applied to many application domains, especially road extraction in which the segmented objects are served as a mandatory layer in geospatial databases. Several attempts in applying deep convolutional neural network (DCNN) to extract roads from remote sensing images have been made; however, the accuracy is still limited. In this paper, we present an enhanced DCNN framework specifically tailored for road extraction on remote sensing images by applying landscape metrics (LMs) and conditional random fields (CRFs). To improve DCNN, a modern activation function, called exponential linear unit (ELU), is employed in our network resulting in a higher number of and yet more accurate extracted roads. To further reduce falsely classified road objects, a solution based on an adoption of LMs is proposed. Finally, to sharpen the extracted roads, a CRF method is added to our framework. The experiments were conducted on Massachusetts road aerial imagery as well as THEOS satellite imagery data sets. The results showed that our proposed framework outperformed Segnet, the state-of-the-art object segmentation technique on any kinds of remote sensing imagery, in most of the cases in terms of precision, recall, and F1.

Electricity demand forecasting plays a significant role in energy markets. Accurate prediction of electricity demand is the key factor to optimize power generation, consumption, saving energy resources, and determining the energy prices. However, integrating energy mix scenarios, including solar and wind power which are highly non-linear and seasonal, into an existing grid increases uncertainty in generation, adds the challenges for precise forecast. To tackle these challenges, state-of-the-art methods and algorithms have been implemented in literature. We have developed Artificial Intelligence (AI) based deep learning models that can effectively handle the information of long time-series data. Based on the pattern of dataset, four different scenarios were developed and two best scenarios were selected for prediction. Dozens of models were developed and tested in deep AI networks. In the first scenario (Scenario1), data for weekdays excluding holidays was taken and in the second scenario (Scenario2) all the data in the basket was taken. Remaining two scenarios, weekends and holidays were tested and neglected because of their high prediction error. To find the optimal configuration, models were trained and tested within a large space of alternatives called hyper-parameters. In this study, an Aritificial Neural Network (ANN) based Feed-forward Neural Network (FNN) showed the minimum prediction error for Scenario1 while a Recurrent Neural Network (RNN) based Gated Recurrent Network (GRU) showed the minimum prediction error for Scenario2. While comparing the accuracy, the lowest MAPE of 2.47% was obtained from FNN for Scenario1. When evaluating the same testing dataset (non-holidays) of Scenario2, the RNN-GRU model achieved the lowest MAPE of 2.71%. Therefore, we can conclude that grouping of weekdays as Senario1 prepared by excluding the holidays provides better forecasting accuracy compared to the single group approach used in Scenario2, where all the dataset is considered together. However, Scenario2 is equally important to predict the demand for weekends and holidays.

In this paper, we address the problem of car detection from aerial images using Convolutional Neural Networks (CNN). This problem presents additional challenges as compared to car (or any object) detection from ground images because features of vehicles from aerial images are more difficult to discern. To investigate this issue, we assess the performance of two state-of-the-art CNN algorithms, namely Faster R-CNN, which is the most popular region-based algorithm, and YOLOv3, which is known to be the fastest detection algorithm. We analyze two datasets with different characteristics to check the impact of various factors, such as UAV's altitude, camera resolution, and object size. The objective of this work is to conduct a robust comparison between these two cutting-edge algorithms. By using a variety of metrics, we show that YOLOv3 yields better performance in most configurations, except that it exhibits a lower recall and less confident detections when object sizes and scales in the testing dataset differ largely from those in the training dataset.

In this paper, we address the problem of car detection from aerial images using Convolutional Neural Networks (CNN). This problem presents additional challenges as compared to car (or any object) detection from ground images because features of vehicles from aerial images are more difficult to discern. To investigate this issue, we assess the performance of two state-of-the-art CNN algorithms, namely Faster R-CNN, which is the most popular region-based algorithm, and YOLOv3, which is known to be the fastest detection algorithm. We analyze two datasets with different characteristics to check the impact of various factors, such as UAV’s altitude, camera resolution, and object size. The objective of this work is to conduct a robust comparison between these two cutting-edge algorithms. By using a variety of metrics, we show that none of the two algorithms outperforms the other in all cases.

Environmental monitoring systems play a crucial role in assessing environmental quality, detecting limits exceedances, and predicting potential ecological episodes. These systems rely on the measurement of various variables at specific locations and time intervals over an extended period. The concept of environmental monitoring encompasses the assessment of health and safety issues for public and environmental health purposes. Pollution of the atmosphere and water, climate change, and natural disasters are among the consequences of continuous industrial and municipal development and human interference in natural ecosystems. To address these challenges and to protect human lives and the environment, with a special concern on mitigating the ecological effects of industrial development, advanced technical solutions, including the technologies associated with artificial intelligence (artificial neural networks ANNs, machine learning ML) have been developed. These technologies offer powerful tools for analysing the vast amount of data collected by monitoring systems and extracting valuable insights. By applying ANNs and machine learning algorithms, environmental monitoring systems can effectively process and interpret the measured variables to assess environmental quality. Despite challenges and limitations, such as data quality and interpretability of AI models, ongoing research and interdisciplinary collaboration are paving the way for the successful implementation of AI in environmental monitoring, ultimately supporting informed decision-making and sustainable resource management.While several review papers have explored the theory of artificial intelligence (AI), here I aim to review the application of ANNs and ML, in environmental aspects, specifically in automotive and industrial emissions toxicity measurements, as well as atmospheric pollution prevention. By examining the potential of AI in these domains, the paper contributes to understanding the role of advanced technologies in environmental monitoring and protection.

In recent times, using image-based deep learning techniques has been very useful for classifying fruits. However, mango is a particular challenge because its shelf life is directly related to the degree of maturity at the time of harvest, which is determined mainly through the visual assessment of the operators when sorting them, doing it incorrectly, coupled with the blows and compressions exposed during storage and transport, trigger the deterioration in their quality, hence the importance of this operation. In this sense, the purpose of this study focuses on designing a computational model based on Deep Learning that uses a convolutional neural network (CNN) parameterized to select mangoes according to their color tone based on their maturity stage. A dataset was established from 201 labeled images, then a CNN architecture was built, obtaining a model on which 3 criteria were evaluated: the number of convolution and grouping layers, the resolution of the input images and the number of training epochs required. As a result, an input image resolution of 32 x 32 pixels was achieved, the process included a convolution layer followed by pooling that fed another layer that processed the results and ended in an output layer with 2 neurons under the One Hot Encoding representation. The model required 20 epochs to perfect its training, achieving 96.04 % accuracy in mango image selection. This study supports the potential of CNNs in quality analysis through mango color selection, even in uncertainty scenarios.

Traffic shaping is a critical task in software-defined -IoT networks (SDN-IoTs) to efficiently manage network resources and ensure Quality of Service (QoS) for end-users. However, traditional traffic shaping approaches based on queuing theory or static policies may not be effective due to the dynamic and unpredictable nature of network traffic. In this paper, we propose a novel approach that leverages Graph Neural Networks (GNNs) and Multi-arm Bandit algorithms to dynamically optimize traffic shaping policies based on real-time network traffic patterns. Specifically, our approach uses a GNN model to learn and predict network traffic patterns and a Multi-arm Bandit algorithm to optimize traffic shaping policies based on these predictions. We evaluate the proposed approach on three different datasets, including a simulated corporate network (KDD Cup 1999), a collection of network traffic traces (CAIDA), and a simulated network environment with both normal and malicious traffic (NSL-KDD). The results demonstrate that our approach outperforms other state-of-the-art traffic shaping methods, achieving higher throughput, lower packet loss, and lower delay, while effectively detecting anomalous traffic patterns. The proposed approach offers a promising solution to traffic shaping in SDNs, enabling efficient resource management and QoS assurance

Given the high number of vehicle-crash victims, it has been established as a priority to reduce this figure in the transportation sector. For this reason, many of the recent researches are focused on including control systems in existing vehicles, to improve their stability, comfort and handling. These systems need to know in every moment the behavior of the vehicle (state variables), among others, when the different maneuvers are performed, to actuate by means of the systems in the vehicle (brakes, steering, suspension) and, in this way, to achieve a good behavior. The main problem arises from the lack of ability to directly capture several required dynamic vehicle variables, such as roll angle, from low-cost sensors. Previous studies demonstrate that low-cost sensors can provide data in real-time with the required precision and reliability. Even more, other research works indicate that neural networks are efficient mechanisms to estimate roll angle. Nevertheless, it is necessary to assess that the fusion of data coming from low-cost devices and estimations provided by neural networks can fulfill the reliability and appropriateness requirements for using these technologies to improve overall safety in production vehicles. Because of the increasing of computing power, the reduction of consumption and electric devices size, along with the high variety of communication technologies and networking protocols using Internet have yield to Internet of Things (IoT) development. In order to address this issue, this study has two main goals: 1) Determine the appropriateness and performance of neural networks embedded in low-cost sensors kits to estimate roll angle required to evaluate rollover risk situations. 2) Compare the low-cost control unit devices (Intel Edison and Raspberry Pi 3 Model B), to provide the roll angle estimation with this artificial neural network-based approach. To fulfil these objectives an experimental environment has been set up composed of a van with two set of low-cost kits, one including a Raspberry Pi 3 Model B, low cost Inertial Measurement Unit (BNO055 - 37€) and GPS (Mtk3339 - 53€) and the other having an Intel Edison System on Chip linked to a SparkFun 9 Degrees of Freedom module. This experimental environment will be tested in different maneuvers for comparison purposes. Neural networks embedded in low-cost sensor kits provide roll angle estimations very approximated to real values. Even more, Intel Edison and Raspberry Pi 3 Model B have enough computing capabilities to successfully run roll angle estimation based on neural networks to determine rollover risks situation fulfilling real-time operation restrictions stated for this problem.

This study investigates the application of regression neural networks, in particular the fitrnet model, in predicting the hardness of steels. The experiments involve extensive tuning of hyperparameters using Bayesian optimization and employ 5-fold and 10-fold cross-validation schemes. The trained models are rigorously evaluated, and their performances are compared using various metrics such as mean square error (MSE), root mean square error (RMSE), mean absolute error (MAE), and coefficient of determination (R2). The results provide valuable insights into the effectiveness of the models and their ability to generalize to unseen data. In particular, Model 4208 (8-85-141-1) emerges as the top performer with an impressive RMSE of 1.0790 and an R² of 0.9900. The research paper contains an illustrative example that demonstrates the practical application of the developed model in determining the hardenability band for a specific steel grade and shows the effectiveness of the model in predicting and optimizing heat treatment results.

Iron ore processing involves critical steps that affect the quality of the final product. Determining the diameter of the pellets is the necessary initial step for volume measurement and, ultimately, for porosity and bulk density measurement, crucial characteristics for optimizing the burning process in the blast furnace. Traditional measurement methods using mercury present issues related to operator safety and environmental preservation, while the practices described in ISO 4698 standard require lengthy preparation and execution time. In light of environmental needs, operator safety, and the time consumed in tests, implementing a new method based on digital image processing and convolutional neural networks for measuring the diameter of burnt iron ore pellets is proposed. The methodology involves capturing images of the pellets using a high-resolution camera and utilizing digital image processing and neural networks capable of performing pixel-by-pixel object segmentation in the images, providing precise information about the pellets to calculate their volume automatically. The results were compared with those obtained using traditional methods, evaluating their conformity with the ISO 4698 standard. In conclusion, this study offers a new approach to measuring the volume of burnt iron ore pellets, providing accurate and reliable results in compliance with safety and environmental preservation standards.

Accurate estimates of daily rainfall are essential for understanding and modeling the physical processes involved in the interaction between the land surface and the atmosphere. In this study, daily satellite soil moisture observations from the Advanced Microwave Scanning Radiometer - Earth Observing System (AMSR-E) generated by implementing the standard NASA- algorithm are employed for estimating rainfall, firstly, through the use of recently developed approach, SM2RAIN (Brocca et al., 2013) and, secondly, the nonlinear autoregressive network with exogenous inputs (NARX) neural modelling at five climate stations in the Karkheh river basin (KRB), located in southwest Iran. In the SM2RAIN method, the period 1 January 2003 to 31 December 2005 is used for the calibration of algorithm and the remaining 9 months from 1 January 2006 to 30 September 2006 is used for the validation of the rainfall estimates. In the NARX model, the full study period is split into a training (1 January 2003 to 31 September 2005) and a testing (1 September 2005 to 30 September 2006) stage. For the prediction of the rainfall as the desired target (output), relative soil moisture changes from AMSR-E and measured air temperature time series are chosen as exogenous (external) inputs in NARX. The quality of the estimated rainfall data is evaluated by comparing it with observed rainfall data at the five rain gauges in terms of the correlation coefficient R, the RMSE and the statistical bias. For the SM2RAIN method, R ranges between 0.44 and 0.9 for all stations, whereas for the NARX- model the values are generally slightly lower. Moreover, the values of the bias for each station indicate that although SM2RAIN is likely to underestimate large rainfall intensities, due to the known effect of soil moisture saturation, its biases are somewhat lower than those of NARX. In conclusion, the results of the present study show that with the use of AMSR-E soil moisture products in the physically based SM2RAIN- algorithm as well as in the NARX neural network, rainfall for poorly gauged regions can be fairly predicted.

Plant leaf diseases can affect the plants’ leaves to an extent that the plants can collapse and die completely. These diseases may drastically drop the supply of vegetables and fruits to the market, and result in a low agricultural economy. In the literature, different laboratory methods of plant leaf disease detection have been used. These methods were time consuming and could not cover large areas for the detection of leaf diseases. This study infiltrates through the facilitated principles of the Convolutional Neural Networks (CNN) in order to model a network for image recognition and classification of these diseases. Neuroph was used to perform the training of a CNN network that recognized and classified images of the maize leaf diseases that were collected by use of a smart phone camera. A novel way of training and the methodology used, expedite a quick and easy implementation of the system in practice. The developed model was able to recognize 3 different types of maize leaf diseases out of healthy leaves. The Northern Corn Leaf Blight (Exserohilum), Common Rust (Puccinia sorghi) and Gray Leaf Spot (Cerospora) diseases were chosen for this study as they affect most parts of Southern Africa’s maize fields.

This paper presents a novel real time control strategy on four-wheel independent steering system for electric vehicle with stepper motors using neural networks combined with fuzzy logic. The control of the four wheels is based on a variation of Hopfield Neural Network (VHNN) method, in which the input is the error of each steering motor and the output is processed by a hyperbolic tangent function (HTF) feeding the fuzzy logic controller (FLC), which ultimately drives the stepper motor. The whole system is made up of the four aforementioned blocks where they work in sync and inseparable from each other with the common goal of driving all the steering stepper motors at the same time. Simulation results demonstrate that the proposed control method has strong robustness and practicability under different angles and speeds.

This paper presents a robust and efficient fault detection and diagnosis framework for handling small faults and oscillations in synchronous generator (SG) systems. The proposed framework utilizes the Brunovsky form representation of nonlinear systems to mathematically formulate the fault detection problem. A differential-flatness model of SG systems is provided to meet the conditions of the Brunovsky form representation. A combination of high-gain observer and group method of data handling neural network is employed to estimate the trajectory of the system and to learn/ approximate the fault and uncertainties associated functions. The fault detection mechanism is developed based on output residual generation and monitoring so that any unfavourable oscillation and/or fault occurrence can be detected rapidly. Accordingly, an average L1-norm criterion is proposed for rapid decision making of faulty situations. The performance of the proposed framework is investigated for two benchmark scenarios which are actuation fault and fault impact on system dynamics. The simulation results demonstrate the capacity and effectiveness of the proposed solution for rapid fault detection and diagnosis in SG systems in practice, and thus enhancing service maintenance, protection, and life cycle of SGs.

Attackers are perpetually modifying their tactics to avoid detection and frequently leverage legitimate credentials with trusted tools already deployed in a network environment, making it difficult for organizations to proactively identify critical security risks. Network traffic analysis products have emerged in response to attackers’ relentless innovation, offering organizations a realistic path forward for combatting creative attackers. Additionally, thanks to the widespread adoption of cloud computing, Device Operators (DevOps) processes, and the Internet of Things (IoT), maintaining effective network visibility has become a highly complex and overwhelming process. What makes network traffic analysis technology particularly meaningful is its ability to combine its core capabilities to deliver malicious intent detection. In this paper, we propose a novel darknet traffic analysis and network management framework to real-time automating the malicious intent detection process, using a weight agnostic neural networks architecture. It is an effective and accurate computational intelligent forensics tool for network traffic analysis, the demystification of malware traffic, and encrypted traffic identification in real-time. Based on Weight Agnostic Neural Networks (WANNs) methodology, we propose an automated searching neural net architectures strategy that can perform various tasks such as identify zero-day attacks. By automating the malicious intent detection process from the darknet, the advanced proposed solution is reducing the skills and effort barrier that prevents many organizations from effectively protecting their most critical assets.

Cities exemplify the evolving world with changing demographics and climates. Urban green spaces play a crucial role in improving the quality of life of people through their potential to mitigate temperatures. Therefore, comprehending their impact is of para-mount interest. Given the challenges in obtaining temperature data from urban locations, this study develops Artificial Neural Networks (ANNs) to predict daily and hourly temperatures in Valladolid, Spain, with a particular focus on urban allotment gardens and a forested urban park. ANNs were built and evaluated from various combinations of inputs (X), hidden neurons (Y) and outputs (Z) under the practical rule of "making net-works simple, to obtain better results". The best performing model was 6-Y-1 ANN archi-tecture with an impressive result of Root Mean Square Error (RMSE) = 0.42°C in the urban garden called Valle de Arán. However, other five ANN architectures were also tested (7-Y-5; 6-Y-5; 7-Y-1; 3-Y-Z and 2-Y-1). ANNs dedicated to urban temperature analysis hold immense potential for urban planning and research, aiding in under-standing the urban climate, forecasting future temperatures, identifying temperature mitigation strategies and even for managing urban crops

This paper proposes several LabVIEW applications to accomplish the data acquisition, processing, features extraction and real-time classification of the electroencephalographic (EEG) signal detected by the embedded sensor of the NeuroSky Mindwave Mobile headset. The LabVIEW applications are aimed at the implementation of a Brain-Computer Interface system, which is necessary to people with neuromotor disabilities. It is analyzed a novel approach regarding the preparation and automatic generation of the EEG dataset by identifying the most relevant multiple mixtures between selected EEG rhythms (both time and frequency domains of raw signal, delta, theta, alpha, beta, gamma) and extracted statistical features (mean, median, standard deviation, route mean square, Kurtosis coefficient and others). The acquired raw EEG signal is processed and segmented into temporal sequences corresponding to the detection of the multiple voluntary eye-blinks EEG patterns. The main LabVIEW application accomplished the optimal real-time artificial neural networks techniques for the classification of the EEG temporal sequences corresponding to the four states: 0 - No Eye-Blink Detected; 1 - One Eye-Blink Detected; 2 – Two Eye-Blinks Detected and 3 – Three Eye-Blinks Detected. Nevertheless, the application can be used to classify other EEG patterns corresponding to different cognitive tasks, since the whole functionality and working principle could estimate the labels associated with various classes.

The objectives of this paper are to investigate the tradeoffs between a physically constrained neural network and a deep, convolutional neural network and to design a combined ML approach ("VarioCNN"). Our solution is provided in the framework of a cyberinfrastructure that includes a newly designed ML software, GEOCLASS-image, modern high-resolution satellite image data sets (Maxar WorldView data) and instructions/descriptions that may facilitate solving similar spatial classification problems. Combining the advantages of the physically-driven connectionist-geostatistical classification method with those of an efficient CNN, VarioCNN provides a means for rapid and efficient extraction of complex geophysical information from submeter resolution satellite imagery. A retraining loop overcomes the difficulties of creating a labeled training data set. Computational analyses and developments are centered on a specific, but generalizable, geophysical problem: The classification of crevasse types that form during the surge of a glacier system. A surge is a glacial catastrophe, an acceleration of a glacier to typically 100-200 times its normal velocity. GEOCLASS-image is applied to study the current (2016-2024) surge in the Negribreen Glacier System, Svalbard. The geophysical result is a description of the structural evolution and expansion of the surge, based on crevasse types that capture ice deformation in six simplified classes.

In this article, we introduce a new approach to human movement by defining the movement as a static object or a super object in one two-dimensional image. This method can allow researchers to label and describe the total movement as an object isolated from a reference video. This ap-proach allows us to perform various tasks, including finding similar movements in a video, measuring, and comparing movements, generating new similar movements, and defining chore-ography by controlling specific parameters in the human body skeleton. As a result of the pre-sented approach, we can eliminate the need to label images manually, disregard the problem of finding the beginning and the end of a movement, overcome synchronization issues between movements, and perform any deep learning network-based operation that processes super objects in images in general. As part of this article, we will demonstrate two application use cases: one il-lustrates how to verify and score a requested movement. In contrast, the other illustrates how to generate similar movements in the human skeleton space by addressing the challenge of supply-ing sufficient training data for deep learning applications (DL). A Variational Auto Encoder (VAE) simulator and an EfficientNet-B7 classifier architecture embedded within a Siamese twin neural network are presented in this paper in order to demonstrate two use cases. These use cases demonstrated the versatility of our innovative concept in measuring, categorizing, inferring hu-man behavior, and generating gestures for other researchers.

The informational synthesis of neural structures, processes, parameters and characteristics that allow a unified description and modeling as neural machines of natural and artificial neural systems is presented. The general informational parameters as the global quantitative measure of the neural systems computing potential as absolute and relative neural power were proposed. Neural information organizing and processing follows the way in which nature manages neural information by developing functions, functionalities and circuits related to different internal or peripheral components and also to the whole system through a non-deterministic memorization, fragmentation and aggregation of afferent and efferent information, deep neural information processing representing multiple alternations of fragmentation and aggregation stages. The relevant neural characteristics were integrated into a neural machine type model that incorporates unitary also peripheral or interface components as the central ones. The proposed approach allows overcoming the technical constraints in artificial computational implementations of neural information processes and also provides a more relevant description of natural ones.

The intracranial pressure (ICP) signal, as monitored on patients in intensive care units, contains pulses of cardiac origin where P1 and P2 subpeaks can often be observed. When calculable, the ratio of their relative amplitudes is an indicator of the patient’s cerebral compliance. This characterization is particularly informative on the overall state of the cerebrospinal system. We developed a recurrent neural network-based framework for P2/P1 ratio computation that only takes a raw ICP signal as an input. Two tasks are performed, namely pulse classification and subpeak designation. Pulse classification was achieved with an area under the curve of 0.90 on a 4,344-pulse testing dataset, while the peak designation algorithm identified pulses with a P2/P1 ratio > 1 with a 97.3% accuracy. Although it still needs to be evaluated on a larger number of labeled recordings, our automated P2/P1 ratio calculation framework appears to be a promising tool for improving bedside monitoring devices.

An earthquake is a natural event by its general definition. This natural event is a disaster that causes significant damage, loss of life, and other economic effects that will damage the state. The possibility of predicting a natural event such as an earthquake will minimize the reasons mentioned. Data collection, data processing, and data evaluation were carried out in this study. Earthquake forecasting was performed using the data and the RNN (Recurrent Neural Network) method. The study was carried out on seismic data with a magnitude of 3.0 and above belonging to Düzce Province between 1990 and 2022. In order to increase the learning potential of the method, the b and d values of the earthquake were calculated and included in the data set, except for the earthquake magnitude. The determination of earthquakes in a specific time interval in regions of Turkey, the classification of earthquake-related seismic data using artificial neural networks, and the production of predictions for the future reveal the importance of this study.

Tumorigenesis is a complex biological phenomenon that includes extensive genetic and phenotypic heterogeneities and complicated regulatory mechanisms. In the recent few years, our studies demonstrate that tumor-initiating cells are similar to neural stem/progenitor cells in regulatory networks, tumorigenicity and pluripotent differentiation potential. In the review, I will make further discussion on these observations and propose a rule of cell biology by integrating these findings with evidence from developmental biology, tumor biology and evolution, which suggests that neural stemness underlies two coupled cell properties, tumorigenicity and pluripotent differentiation potential. Tumorigenicity and phenotypic heterogeneity in tumor is a result of acquirement of neural stemness in cells. The neural stemness property of tumor-initiating cells can hopefully integrate different concepts/hypotheses underlying tumorigenesis. Neural stem cells/neural progenitors and tumor-initiating cells share regulatory networks; both exhibit neural stemness, tumorigenicity and differentiation potential; both are dependent on expression or activation of ancestral genes (the atavistic effect); both rely primarily on aerobic glycolytic metabolism; both can differentiate into various cells or tissues that are derived from three germ layers, resembling severely disorganized or more severely degenerated process of embryonic development; both are enriched in long genes with more splice variants that provide more plastic scaffolds for cell differentiation, etc. The property of neural stemness might be a key point to understand tumorigenesis and pluripotent differentiation potential, and possibly explain certain pathological observations in tumors that have been inexplicable. Therefore, behind the complexity of tumorigenesis might be a general rule of cell biology, i.e., neural stemness represents the ground state of cell tumorigenicity and pluripotent differentiation potential.

This research investigates the application of deep learning in sentiment analysis of Canadian maritime case law. It offers a framework for improving maritime law and legal analytics policy-making procedures. The automation of legal document extraction takes center stage, underscoring the vital role sentiment analysis plays at the document level. Therefore, this study introduces a novel strategy for sentiment analysis in Canadian maritime case law, combining sentiment case law approaches with state-of-the-art deep learning techniques. The overarching goal is to systematically unearth hidden biases within case law and investigate their impact on legal outcomes. Employing Convolutional Neural Network (CNN) and Long Short-Term Memory (LSTM) based models, this research achieves a remarkable accuracy of 98.05% for categorizing instances. In contrast, Conventional Machine Learning techniques such as Support Vector Machines (SVM) yield an accuracy rate of 52.57%, Naive Bayes at 57.44 %, and Logistic Regression at 61.86%. The superior accuracy of the CNN and STM model combination underscores its usefulness in legal sentiment analysis, offering promising future applications in diverse fields like legal analytics and policy design. These findings mark a significant choice for AI-powered legal tools, presenting more sophisticated and sentiment-aware options for the legal profession.

A chronically stable electrode material with low impedance for recording neural activity and high charge-injection capacity for functional electro-stimulation is desirable for the fabrication of implantable microelectrode arrays that aims to restore impaired or lost neurological functions in humans. For this purpose, we have investigated the electrochemical properties of sputtered ruthenium oxide (RuOx) electrode coatings, deposited on planar microelectrode arrays, using an inorganic model of interstitial fluid (model-ISF) at 37o C as the electrolyte. Through a combination of cyclic voltammetry (CV) and electrochemical impedance spectroscopy (EIS) modelling study we have established the contribution of faradaic reaction as the major charge-injection contributor within the safe neural stimulation potential window of 0.6 V vs Ag|AgCl. We have also established the reversibility of the charge-injection process for sputtered RuOx film by applying constant charge-per-phase current stimulations at different pulse widths and by comparing the magnitudes of leading and trailing access voltages during voltage transient measurements. Finally, the impedance of the sputtered RuOx film was found to be reasonably comparable in both its oxidized and reduced states, although electronic contribution from capacitive double-layer was found to be slightly higher for the completely oxidized film around 0.6 V than its reduced counterpart around -0.6V.

Convolutional neural networks (CNNs), inspired by biological visual cortex systems, are a powerful category of artificial neural networks that can extract the hierarchical features of raw data to greatly reduce the network parametric complexity and enhance the predicting accuracy. They are of significant interest for machine learning tasks such as computer vision, speech recognition, playing board games and medical diagnosis [1-7]. Optical neural networks offer the promise of dramatically accelerating computing speed to overcome the inherent bandwidth bottleneck of electronics. Here, we demonstrate a universal optical vector convolutional accelerator operating beyond 10 Tera-OPS (TOPS - operations per second), generating convolutions of images of 250,000 pixels with 8-bit resolution for 10 kernels simultaneously — enough for facial image recognition. We then use the same hardware to sequentially form a deep optical CNN with ten output neurons, achieving successful recognition of full 10 digits with 900 pixel handwritten digit images with 88% accuracy. Our results are based on simultaneously interleaving temporal, wavelength and spatial dimensions enabled by an integrated microcomb source. This approach is scalable and trainable to much more complex networks for demanding applications such as unmanned vehicle and real-time video recognition.

This research investigates the implementation of complex exponential-based neurons in FPGA, which can pave the way for implementing bio-inspired spiking neural networks to compensate for the existing computational constraints in conventional artificial neural networks. The increasing use of extensive neural networks and the complexity of models in handling big data lead to higher power consumption and delays. Hence, finding solutions to reduce computational complexity is crucial for addressing power consumption challenges. The complex exponential form effectively encodes oscillating features like frequency, amplitude, and phase shift, streamlining the demanding calculations typical of conventional artificial neurons through levering simple phase addition of complex exponential functions. The article implements such a two-neuron and a multi-neuron neural model using Xilinx system generator and Vivado design suite, employing 8-bit, 16-bit, and 32-bit fixed-point data format representations. The study evaluates the accuracy of the proposed neuron model across different FPGA implementations while also providing a detailed analysis of operating frequency, power consumption, and resource usage for the hardware implementations. BRAM-based Vivado designs outperformed Simulink regarding speed, power, and resource efficiency. Specifically, the Vivado BRAM-based approach supported up to 128 neurons, showcasing optimal LUT and FF resource utilization. Such outcomes accommodate choosing the optimal design procedure for implementing spiking neural networks on FPGAs.

Neural stem cell (NSC) based therapies are at the forefront of regenerative medicine strategies to combat illness and injury of the central nervous system (CNS). In addition to their ability to produce new cells, NSCs secrete a variety of products, known as the NSC secretome, that have been shown to ameliorate CNS disease pathology and promote recovery. As pre-clinical and clinical research to harness the NSC secretome for therapeutic purposes advances, a more thorough understanding of the endogenous NSC secretome can provide useful insight into the functional capabilities of NSCs. In this review, we focus on research investigating the autocrine and paracrine functions of the endogenous NSC secretome across life. We also compare the NSC secretome across species, finding signs of conserved parallels between rodent, human and zebrafish NSC secretomes. Throughout development and adulthood, we find evidence that the NSC secretome is a critical component of how endogenous NSCs regulate themselves and their niche. We also find gaps in current literature, most notably in the clinically relevant domain of endogenous NSC paracrine function in the injured CNS. Future investigations to further define the endogenous NSC secretome and its role in CNS tissue regulation are necessary to bolster our understanding of NSC-niche interactions and to aid in the generation of safe and effective NSC-based therapies.

Biological neural network (BNN) is the core brain for creature to accomplish its intelligent behaviors through unique network structure and mechanisms. It also inspires controlling human-designed autonomous agents including robots to generate more advanced intelligent behaviors by mimicking the neural control mechanism of creatures at a deeper and more interactive level. Here, we constructed a whole-brain neural network model of Caenorhabditis elegans (C. elegans), which characterizes the electrochemical processes at the level of cellular synapse. The neural network simulation integrates computational programming and visualization of neurons and synapse connections of C. elegans, containing the specific controllable circuits summarization with their dynamic characteristics within the whole network. To illustrate its particular intelligent control capability in terms of robotics, we introduce the first innovative methodology for applying the BNN model to a 12-legged robot’s movement control based on the established numerical simulation platforms. We accomplished and designed two methods and corresponding encoding processes, one involving orientation control and the other involving locomotion generation, to demonstrate the intelligent control performance of BNN. Both simulation and experiment results indicate that the robot exhibits stronger autonomy and more intelligent moving performance under the control of BNN. We then summarized the contributions for digitalizing the C. elegans’ whole-brain neural network in real-time and utilizing it to control robot in a closed loop to validate its advanced intelligence control ability in terms of a scientific robot.

Speech recognition, or automatic speech recognition (ASR), is a technology designed to convert spoken language into text using software. However, conventional ASR methods involve several distinct components, including language, acoustic, and pronunciation models with dictionaries. This modular approach can be time-consuming and may influence performance. In this study, we propose a method that streamlines the speech recognition process by incorporating a unified recurrent neural network (RNN) architecture. Our architecture integrates a convolutional neural network (CNN) with an RNN and employs a connectionist temporal classification (CTC) loss function. Key experiments were carried out using a dataset comprising 576,656 valid sentences, using erosion techniques. Evaluation of the model performance, measured by the word error rate (WER) metric, demonstrated remarkable results, achieving a WER of 2%. This approach has significant implications for the realm of speech recognition, as it alleviates the need for labor-intensive dictionary creation, enhancing the efficiency and accuracy of ASR systems, and making them more applicable to real-world scenarios. For future enhancements, we recommend the inclusion of dialectal and spontaneous data in the dataset to broaden the model's adaptability. Additionally, fine-tuning the model for specific tasks can optimize its performance for targeted objectives or domains, further enhancing its effectiveness in those areas.

Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs) have been utilised to forecast the foreign exchange market (Forex). However, such models usually exhibit unstable behaviour, as data perturbations often downgrade the functionality of the entire network due to their monolithic architecture. Hence, this study proposes a novel neuroscience-informed modular network applied to the closing prices and the sentiments retrieved from Yahoo Finance and Twitter APIs, aiming to anticipate better price fluctuations in Euro to British Pound Sterling (EUR/GBP) rather than monolithic methods. The proposed model is based on a new modular CNN, replacing pooling layers with orthogonal kernel initialisation RNNs coupled with Monte Carlo Dropout (MCD), namely MCoRNNMCD. It combines two modules: i) a convolutional simple RNN and ii) a convolutional Gated Recurrent Unit (GRU), where orthogonality and MCD are added to reduce the overfitting, assessing each module's uncertainty. These parallel feature extraction modules concatenate their outputs to a final three-layer Artificial Neural Network (ANN) decision-making module. A comprehensive comparison viewing objective evaluation metrics such as the Mean Square Error (MSE) proved that the proposed MCoRNNMCD-ANN outperformed single CNN, LSTM, GRU, and the state-of-the-art hybrid BiCuDNNLSTM, CLSTM, CNN-LSTM, and LSTM-GRU in forecasting hourly EUR/GBP closing price fluctuations.

In recent years, Neural networks are increasingly deployed in various fields to learn complex patterns and make accurate predictions. However, designing an effective neural network model is a challenging task that requires careful consideration of various factors, including architecture, optimization method, and regularization technique. This paper aims to comprehensively overview the state-of-the-art artificial neural network (ANN) generation and highlight key challenges and opportunities in machine learning applications. It provides a critical analysis of current neural network model design methodologies, focusing on the strengths and weaknesses of different approaches. Also, it explores the use of different learning approaches, including convolutional neural networks (CNN), deep neural networks (DNN), and recurrent neural networks (RNN) in image recognition, natural language processing, and time series analysis. Besides, it discusses the benefits of choosing the ideal values for the different components of ANN, such as the number of Input/output layers, hidden layers number, activation function type, epochs number, and model type selection, which help improve the model performance and generalization. Furthermore, it identifies some common pitfalls and limitations of existing design methodologies, such as overfitting, lack of interpretability, and computational complexity. Finally, it proposes some directions for future research, such as developing more efficient and interpretable neural network architectures, improving the scalability of training algorithms, and exploring the potential of new paradigms, such as Spiking Neural Networks, quantum neural networks, and neuromorphic computing.

Computer-aided image diagnostics (CAD) have been used in many fields of diagnostic medicine. It relies heavily on classical computer vision and artificial intelligence. Quantum neural network (QNN) has been introduced by many researchers around the world and presented recently by research corporations such as Microsoft, Google, and IBM. In this paper, the investigation of the validity of using the QNN algorithm for machine-based breast cancer detection was performed. To validate the learnability of the QNN, a series of learnability tests were performed alongside with classical convolutional neural network (CCNN). QNN is built using the Cirq library to perform the assimilation of quantum computation on classical computers. Series of investigations were performed to study the learnability characteristics of QNN and CCNN under the same computational conditions. The comparison was performed for real Mammogram data sets. The investigations showed success in terms of recognizing the data and training. Our work shows better performance of QNN in terms of successfully training and producing a valid model for smaller data set compared to CCNN.

The application of artificial intelligence to point-of-care testing (POCT) disease detection has become a hot research field, in which breath detection which detects the patient's exhaled VOCs combined with sensor arrays of Convolutional Neural Network (CNN) algorithms as a new lung cancer detection is attracting more researchers' attention. However, due to low accuracy, high complexity computation and large number of parameters make the CNN algorithms difficult transplant to the embedded system of POCT devices. A lightweight neural network (LTNet) in this work is proposed to deal with this problem, and meanwhile, achieve high-precision classification of acetone and ethanol gases, which are respiratory markers for lung cancer patients. Compared to currently popular lightweight CNN models such as EfficientNet, LTNet has fewer parameters (32K) and its training weight size is only 0.155 MB. LTNet achieved an overall classification accuracy of 99.06% and 99.14% in the own mixed gas dataset and the University of California (UCI) dataset, which are both higher than that of the five existing models’, and it also offers the shortest training (844.38s and 584.67s) and inference time (23s and 14s) in the same validation set. Compared to the existing CNN models, LTNet is more suitable for resource-limited POCT devices.

Forest fire identification is important for forest resource protection. Effective monitoring of forest fires requires the deployment of multiple monitors with different viewpoints, while most traditional recognition models can only recognize images from a single source. By ignoring the information from images with different viewpoints, these models produce high rates of missed and false alarms. In this paper, we propose a graph neural network model based on the similarity of dynamic features of multi-view images to improve the accuracy of forest fire recognition. The input features of the nodes on the graph are converted into relational features of different gallery pairs by establishing pairs (nodes) representing different viewpoint images and gallery images. The new feature library relationship is used to update the image gallery with dynamic features in order to achieve the estimation of similarity between images and improve the image recognition rate of the model. In addition, to reduce the complexity of image pre-processing process and extract key features in images effectively, this paper also proposes a dynamic feature extraction method for fire regions based on image segment ability. By setting the threshold value of HSV color space, the fire region is segmented from the image, and the dynamic features of successive frames of the fire region are extracted. The experimental results show that, compared with the baseline method Resnet, this paper's method is more effective in identifying forest fires, and its recognition accuracy is improved by 2%. And the scheme of this paper can adapt to different forest fire scenes, with better generalization ability and anti-interference ability.

Characterization of cancer cells and neural stem cells indicates that tumorigenicity and pluripotency are coupled cell properties determined by neural stemness, and tumorigenesis represents a process of progressive loss of original cell identity and gain of neural stemness. This reminds of a most fundamental process required for the development of the nervous system and body axis during embryogenesis, i.e., embryonic neural induction. Neural induction is that, in response to extracellular signals that are secreted by the Spemann-Mangold organizer in amphibians or the node in mammals and inhibit epidermal fate in ectoderm, the ectodermal cells lose their epidermal fate and assume the neural default fate and consequently, turn into neuroectodermal cells. They further differentiate into the nervous system and also some non-neural cells via interaction with adjacent tissues. Failure in neural induction leads to failure of embryogenesis, and ectopic neural induction due to ectopic organizer or node activity or activation of embryonic neural genes causes a formation of secondary body axis or a conjoined twin. During tumorigenesis, cells progressively lose their original cell identity and gain of neural stemness, and consequently, gain of tumorigenicity and pluripotency, due to various intra-/extracellular insults in cells of a postnatal animal. Tumorigenic cells can be induced to differentiation into normal cells and integrate into normal embryonic development within an embryo. However, they form tumors and cannot integrate into animal tissues/organs in a postnatal animal because of lack of embryonic inducing signals. Combination of studies of developmental and cancer biology indicates that neural induction drives embryogenesis in gastrulating embryos but a similar process drives tumorigenesis in a postnatal animal. Tumorigenicity is the manifestation of aberrant occurrence of pluripotent state in a postnatal animal. Pluripotency and tumorigenicity are both but different manifestations of neural stemness in pre- and postnatal stage, respectively, of animal life. The unique property of neural stemness is derived from the evolutionary advantage of neural genes and the neural-biased state of the last common unicellular ancestors of metazoan. Based on these findings, I discuss about some confusion in cancer research, propose to distinguish the causality and associations and discriminate causal and supporting factors involved in tumorigenesis, and suggest revisiting the focus of cancer research.

Artificial Intelligence (AI) made its first appearance in 1956, and since then it has progressively introduced itself in healthcare system and patients’ information and care. AI functions can be grouped under the following headings: Machine Learning (ML), Deep Learning (DL), Artificial Neural Network (ANN), Convolutional Neural Network (CNN), Computer Vision (CV). Chronic limb-threatening ischemia (CLTI) represents the last stage of peripheral artery disease (PAD), and has increased over recent years, together with the rise in prevalence of diabetes and population ageing. Nowadays AI grants the possibility of developing new diagnostic and treatment solutions in the vascular field, given the possibility of accessing clinical, biological, and imaging data. By assessing the vascular anatomy in every patient, as well as the burden of atherosclerosis, and classifying the level and degree of disease, sizing and planning the best endovascular treatment, defining the perioperative complications risk, integrating experiences and resources between different specialties, identifying latent PAD, thus offering evidence-based solutions and guiding surgeons in the choice of the best surgical technique, AI challenges the role of the physician’s experience in PAD treatment.

Partial differential equations (PDEs) are used to describe a wide range of phenomena, such as heat transfer, fluid dynamics, and quantum mechanics. By solving PDEs, we can ob- tain insights into the behavior of the system and make predictions about its future evolution. Conventional numerical methods for obtaining the approximate solutions of PDEs may re- quire extensive computational resources and time, especially for complex PDEs and large- scale problems. The recently developed physics-informed neural network (PINN) has shown promise in a variety of scientific and engineering fields by incorporating physical laws into the loss functions of the neural network (NN). In addition, the training of PINN does not require ground truth data, but it has poor generalization ability to unseen domains. On the other hand, a convolutional neural network (CNN) has fast inference and better generalization ability, but it requires a large amount of training data. Taking the advantages of PINN and CNN by using Legendre multiwavelets (LMWs) as basis functions, we introduce a new method to approach the PDEs in this paper, namely Physics-Informed Legendre Multiwavelets CNN (PiLMWs-CNN), in order to continuously approximate a grid-based state representation that can be handled by a CNN. PiLMWs-CNN enable us to train our models using only physics-informed loss functions without any pre- computed training data, simultaneously providing fast and continuous solutions that gener- alize to previously unknown domains. In particular, the LMWs can simultaneously possess compact support, orthogonality, symmetry, high smoothness, and high approximation order. Compared to orthonormal polynomial (OP) bases, the approximation accuracy can be greatly increased and computation costs can be significantly reduced by using LMWs. We applied PiLMWs-CNN to approximate the damped wave equation, incompressible Navier-Stokes (N- S) equation, and two-dimensional heat conduction equation. The experimental results show that this method provides more accurate, efficient, and fast convergence with better stability when approximating the solution of PDEs.

Human activity recognition (HAR) using wearable sensors is gaining popularity in the healthcare domain, enabling continuous monitoring and timely interventions. The conventional HAR systems are centralised and need data sharing with a server for processing and model training. However, this data sharing leads to various challenges, such as privacy concerns, increased communication and storage costs, inefficient computational algorithms, data scarcity due to unwillingness to share, and difficulty achieving high accuracy and personalised models simultaneously. To overcome these challenges, this paper introduces a privacy and energy-aware federated learning (FL) framework for model training without data sharing. The proposed scheme incorporates spiking neural networks (SNNs), which are particularly suited for resource-constrained devices due to their event-driven information processing. The key idea is to synergise the strengths of SNNs and long short-term memory (LSTM) to create a hybrid model for robust HAR. The proposed hybrid spiking LSTM (S-LSTM) model uses the LSTM as an input layer to capture temporal dependencies efficiently, followed by spiking layers that replace the typical activation using leaky integrate-and-fire neurons. We employ the combination of surrogate gradient learning and backpropagation through time (BPTT) to update the weights of SNN layers in a supervised manner. The performance of the proposed model is evaluated using two publicly available datasets for both indoor and outdoor scenarios. The results show that the proposed S-LSTM performs much better than spike convolutional neural networks (S-CNN) and simple CNN, achieving the accuracy of 97% and 87% for indoor and outdoor datasets, respectively. Personalisation, also known as fine-tuning the global model using local data, is crucial in HAR. It allows the model to adapt to individual behaviour and preferences, making predictions more accurate. The results also show that personalisation improves the performance of participants up to 9% on average. Additionally, analysis with 50% random client participation reduces communication costs by 50% with minimal impact on accuracy.

A scalable optical convolutional neural network (SOCNN) based on free-space optics and Koehler illumination was proposed to address the limitations of the previous 4f correlator system. Unlike Abbe illumination, Koehler illumination provides more uniform illumination and reduces crosstalk. SOCNN allows for scaling up of the input array and the use of incoherent light sources. Hence, the problems associated with 4f correlator systems can be avoided. We analyzed the limitations in scaling the kernel size and parallel throughput and found that SOCNN can offer a multilayer convolutional neural network with massive optical parallelism.

This study uses machine learning to predict the convergence results of the Duffing equation with and without damping. The Duffing equation represents a nonlinear second-order differential equation with interesting behavior in undamped free vibration and forced vibration with damping. Convergence alternates randomly between 1 and -1 in undamped free vibration, depending on initial conditions. For forced vibration with damping, multiple factors influence vibration patterns. We utilize the fourth-order Runge-Kutta method to collect convergence results for both conditions. Machine learning techniques, specifically the long short term memory (LSTM) and LSTM-Neural Network (LSTM-NN) method, are employed to predict these convergence values. The LSTM-NN model is a hybrid approach that combines the LSTM method with the addition of hidden layers of neurons. Both the LSTM and LSTM-NN models are thoroughly explored and analyzed in this research. The research process involves three stages: data preprocessing, training, and verification. The results show that the LSTM-NN model becomes more adept at predicting binary datasets, boasting an impressive accuracy of up to 98%. However, when it comes to predicting multiple solutions, the traditional LSTM method outperforms the LSTM-NN approach.

Speech recognition, also known as automatic speech recognition (ASR), is a technology that enables software to transcribe spoken language into text. However, existing Amharic ASR methods require multiple separate blocks, such as language, acoustic, and pronunciation models with dictionaries, which can be time-consuming and influence performance. This study proposes an approach that replaces much of the speech pipeline with a single recurrent neural network (RNN) architecture. Our proposed architecture is based on a hybrid approach that combines a convolutional neural network (CNN) with a recurrent neural network (RNN) and a connectionist temporal classification (CTC) loss function. We conducted several experiments with noisy audio data that contain 20,000 valid sentences. The model was evaluated using the word error rate (WER) metric, achieving impressive results of 7% WER on noisy data. This approach has significant implications for the field of speech recognition, as it reduces the human effort required to create dictionaries and improves the efficiency and accuracy of ASR systems, making them more practical for real-world applications.

We present a simplified computational rule for the back-propagation formulas for artificial neural networks. In this work, we provide a generic two-step rule for the back-propagation algorithm in matrix notation. Moreover, this rule incorporates both the forward and backward phases of the computations involved in the learning process. Specifically, this recursively computing rule permits the propagation of the changes to all synaptic weights in the network, layer by layer, efficiently. In particular, we use this rule to compute both the up and down partial derivatives of the cost function of all the connections feeding into the output layer.

Background: Disruption of alternative splicing (AS) is frequently observed in cancer and it might represent an important signature for tumor progression and therapy. Exon skipping (ES) represents one of the most frequent AS events and in non-small cell lung cancer (NSCLC) MET exon 14 skipping was shown to be targetable. Methods: We constructed a neural network (NN) specifically designed to detect MET exon 14 skipping events using RNAseq data. Furthermore, for discovery purpose we also developed a sparsely connected autoencoder to identify uncharacterized MET isoforms. Results: The NN had 100% Met exon 14 skipping detection rate, when tested on a manually curated set of 690 TCGA bronchus and lung samples. When globally applied to 2605 TCGA samples, we observed that the majority of false positives was characterized by a blurry coverage of exon 14, but interesting they share a common coverage peak in the second intron and we speculate that this event could be the transcription signature of a LINE1-MET fusion. Conclusions: Taken together our results indicate that neural networks can be an effective tool to provide a quick classification of pathological transcription events and sparsely connected autoencoders could represent the basis for the development of an effective discovery tool.

Real-world time series data sets contain a combination of linear and nonlinear patterns, making the time series forecasting problem more challenging. In this paper, a new hybrid methodology is introduced for forecasting univariate time series data sets using a multiplicative error modeling approach. An autoregressive integrated moving average (ARIMA) model is combined with an autoregressive neural network (ARNN) for improving the predictions of individual forecast models. The proposed multiplicative ARIMA-ARNN model glorifies the chances of capturing the different combinations of linear and nonlinear patterns in time series. The model shows outstanding performance on six standard time-series data sets compared to other widely used single and hybrid forecasting models.

Using smartphone especially android platform has already got eighty percent market shares, due to aforementioned report, it becomes attacker’s primary goal. There is a growing number of private data onto smart phones and low safety defense measure, attackers can use multiple way to launch and to attack user’s smartphones.(e.g. Using different coding style to confuse the software of detecting malware). Existing android malware detection methods use multiple features, like safety sensor API, system call, control flow structure and data information flow, then using machine learning to check whether its malware or not. These feature provide app’s unique property and limitation, that is to say, from some perspectives it might suit for some specific attack, but wouldn’t suit for others. Nowadays most malware detection methods use only one aforementioned feature, and these methods mostly analysis to detect code, but facing the influence of malware’s code confusion and zero-day attack, aforementioned feature extraction method may cause wrong judge. So, it’s necessary to design an effective technique analysis to prevent malware. In this paper, we use the importance of word from apk, because of code confusion, some malware attackers only rename variables, if using general static analysis wouldn’t judge correctly, then use these importance value to go through our proposed method to generate picture, finally using convolutional neural network to see whether the apk file is malware or not.

Deep learning is emerging in many industrial sectors in hand with big data analytics to streamline production. In the biomanufacturing sector, big data infrastructure is lagging comparatively to other industries. A promising approach is to combine Deep Neural Networks (DNN) with prior knowledge in Hybrid Neural Network (HNN) workflows that are less dependent on the quality and quantity of data. This paper reviews published articles over the past 30 years on the topic of HNN applications to bioprocesses. It revealed that HNNs were applied to various bioprocesses, including microbial cultures, animal cells cultures, mixed microbial cultures, and enzyme biocatalysis. HNNs were mainly applied for process analysis, process monitoring, development of software sensors, open- and closed-loop control, batch-to-batch control, model predictive control, intensified design of experiments, quality-by-design, and recently for the development of digital twins. Most previous HNN studies combined shallow Feedforward Neural Networks (FFNNs) with physical laws, such as macroscopic material balance equations, following the semiparametric design principle. Only recently, deep HNNs based on deep FFNNs, Convolution Neural Networks (CNN), Long Short-Term Memory (LSTM) networks and Physics Informed Neural Networks (PINNs) have been reported. The biopharma sector is currently a major driver but applications to biologics quality attributes, new modalities, and downstream processing are significant research gaps.