One of the most effective image processing techniques is the use of convolutional neural networks, where we combine intensity values at grid points in the vicinity of each point. To speed up computations, researchers have developed a dilated version of this technique, in which only some points are processed. It turns out that the most efficient case is when we select points from a sub-grid. In this paper, we explain this empirical efficiency proving that the sub-grid is indeed optimal – in some reasonable sense. To be more precise, we prove that all reasonable optimality criteria, the optimal subset of the original grid is either a sub-grid, or a sub-grid-like set.

Optical artificial neural networks (ONNs) have significant potential for ultra-high computing speed and energy efficiency. We report a new approach to ONNs based on integrated Kerr micro-combs that is programmable, highly scalable and capable of reaching ultra-high speeds, demonstrating the building block of the ONN — a single neuron perceptron — by mapping synapses onto 49 wavelengths to achieve a single-unit throughput of 11.9 Giga-OPS at 8 bits per OP, or 95.2 Gbps. We test the perceptron on handwritten-digit recognition and cancer-cell detection — achieving over 90% and 85% accuracy, respectively. By scaling the perceptron to a deep learning network using off-the-shelf telecom technology we can achieve high throughput operation for matrix multiplication for real-time massive data processing.

The intracranial pressure (ICP) signal, as monitored on patients in intensive care units, contains pulses of cardiac origin where P1 and P2 subpeaks can often be observed. When calculable, the ratio of their relative amplitudes is an indicator of the patient’s cerebral compliance. This characterization is particularly informative on the overall state of the cerebrospinal system. We developed a recurrent neural network-based framework for P2/P1 ratio computation that only takes a raw ICP signal as an input. Two tasks are performed, namely pulse classification and subpeak designation. Pulse classification was achieved with an area under the curve of 0.90 on a 4,344-pulse testing dataset, while the peak designation algorithm identified pulses with a P2/P1 ratio > 1 with a 97.3% accuracy. Although it still needs to be evaluated on a larger number of labeled recordings, our automated P2/P1 ratio calculation framework appears to be a promising tool for improving bedside monitoring devices.

Multiply-accumulate calculations using a memristor crossbar array is an important method to realize neuromorphic computing. However, the memristor array fabrication technology is still immature, and it is difficult to fabricate large-scale arrays with high-yield, which restricts the development of memristor-based neuromorphic computing technology. Therefore, cascading small-scale arrays to achieve the neuromorphic computational ability that can be achieved by large-scale arrays, which is of great significance for promoting the application of memristor-based neuromorphic computing. To address this issue, we present a memristor-based cascaded framework with some basic computation units, several neural network processing units can be cascaded by this means to improve the processing capability of the dataset. Besides, we introduce a split method to reduce pressure of input terminal. Compared with VGGNet and GoogLeNet, the proposed cascaded framework can achieve 93.54% Fashion-MNIST accuracy under the 4.15M parameters. Extensive experiments with Ti/AlOx/TaOx/Pt we fabricated are conducted to show that the circuit simulation results can still provide a high recognition accuracy, and the recognition accuracy loss after circuit simulation can be controlled at around 0.26%.

This research investigates the application of deep learning in sentiment analysis of Canadian maritime case law. It offers a framework for improving maritime law and legal analytics policy-making procedures. The automation of legal document extraction takes center stage, underscoring the vital role sentiment analysis plays at the document level. Therefore, this study introduces a novel strategy for sentiment analysis in Canadian maritime case law, combining sentiment case law approaches with state-of-the-art deep learning techniques. The overarching goal is to systematically unearth hidden biases within case law and investigate their impact on legal outcomes. Employing Convolutional Neural Network (CNN) and Long Short-Term Memory (LSTM) based models, this research achieves a remarkable accuracy of 98.05% for categorizing instances. In contrast, Conventional Machine Learning techniques such as Support Vector Machines (SVM) yield an accuracy rate of 52.57%, Naive Bayes at 57.44 %, and Logistic Regression at 61.86%. The superior accuracy of the CNN and STM model combination underscores its usefulness in legal sentiment analysis, offering promising future applications in diverse fields like legal analytics and policy design. These findings mark a significant choice for AI-powered legal tools, presenting more sophisticated and sentiment-aware options for the legal profession.

Numerous approaches address over-fitting in neural networks: by imposing a penalty on the parameters of the network (L1, L2, etc); by changing the network stochastically (drop-out, Gaussian noise, etc.); or by transforming the input data (batch normalization, etc.). In contrast, we aim to ensure that a minimum amount of supporting evidence is present when fitting the model parameters to the training data. This, at the single neuron level, is equivalent to ensuring that both sides of the separating hyperplane (for a standard artificial neuron) have a minimum number of data points — noting that these points need not belong to the same class for the inner layers. We firstly benchmark the results of this approach on the standard Fashion-MINST dataset, comparing it to various regularization techniques. Interestingly, we note that by nudging each neuron to divide, at least in part, its input data, the resulting networks make use of each neuron, avoiding a hyperplane completely on one side of its input data (which is equivalent to a constant into the next layers). To illustrate this point, we study the prevalence of saturated nodes throughout training, showing that neurons are activated more frequently and earlier in training when using this regularization approach. A direct consequence of the improved neuron activation is that deep networks are now easier to train. This is crucially important when the network topology is not known a priori and fitting often remains stuck in a suboptimal local minima. We demonstrate this property by training a network of increasing depth (and constant width): most regularization approaches will result in increasingly frequent training failures (over different random seeds) whilst the proposed evidence-based regularization significantly outperforms in its ability to train deep networks.

Convolutional neural networks (CNNs), inspired by biological visual cortex systems, are a powerful category of artificial neural networks that can extract the hierarchical features of raw data to greatly reduce the network parametric complexity and enhance the predicting accuracy. They are of significant interest for machine learning tasks such as computer vision, speech recognition, playing board games and medical diagnosis [1-7]. Optical neural networks offer the promise of dramatically accelerating computing speed to overcome the inherent bandwidth bottleneck of electronics. Here, we demonstrate a universal optical vector convolutional accelerator operating beyond 10 Tera-OPS (TOPS - operations per second), generating convolutions of images of 250,000 pixels with 8-bit resolution for 10 kernels simultaneously — enough for facial image recognition. We then use the same hardware to sequentially form a deep optical CNN with ten output neurons, achieving successful recognition of full 10 digits with 900 pixel handwritten digit images with 88% accuracy. Our results are based on simultaneously interleaving temporal, wavelength and spatial dimensions enabled by an integrated microcomb source. This approach is scalable and trainable to much more complex networks for demanding applications such as unmanned vehicle and real-time video recognition.

Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs) have been utilised to forecast the foreign exchange market (Forex). However, such models usually exhibit unstable behaviour, as data perturbations often downgrade the functionality of the entire network due to their monolithic architecture. Hence, this study proposes a novel neuroscience-informed modular network applied to the closing prices and the sentiments retrieved from Yahoo Finance and Twitter APIs, aiming to anticipate better price fluctuations in Euro to British Pound Sterling (EUR/GBP) rather than monolithic methods. The proposed model is based on a new modular CNN, replacing pooling layers with orthogonal kernel initialisation RNNs coupled with Monte Carlo Dropout (MCD), namely MCoRNNMCD. It combines two modules: i) a convolutional simple RNN and ii) a convolutional Gated Recurrent Unit (GRU), where orthogonality and MCD are added to reduce the overfitting, assessing each module's uncertainty. These parallel feature extraction modules concatenate their outputs to a final three-layer Artificial Neural Network (ANN) decision-making module. A comprehensive comparison viewing objective evaluation metrics such as the Mean Square Error (MSE) proved that the proposed MCoRNNMCD-ANN outperformed single CNN, LSTM, GRU, and the state-of-the-art hybrid BiCuDNNLSTM, CLSTM, CNN-LSTM, and LSTM-GRU in forecasting hourly EUR/GBP closing price fluctuations.

In recent years, Neural networks are increasingly deployed in various fields to learn complex patterns and make accurate predictions. However, designing an effective neural network model is a challenging task that requires careful consideration of various factors, including architecture, optimization method, and regularization technique. This paper aims to comprehensively overview the state-of-the-art artificial neural network (ANN) generation and highlight key challenges and opportunities in machine learning applications. It provides a critical analysis of current neural network model design methodologies, focusing on the strengths and weaknesses of different approaches. Also, it explores the use of different learning approaches, including convolutional neural networks (CNN), deep neural networks (DNN), and recurrent neural networks (RNN) in image recognition, natural language processing, and time series analysis. Besides, it discusses the benefits of choosing the ideal values for the different components of ANN, such as the number of Input/output layers, hidden layers number, activation function type, epochs number, and model type selection, which help improve the model performance and generalization. Furthermore, it identifies some common pitfalls and limitations of existing design methodologies, such as overfitting, lack of interpretability, and computational complexity. Finally, it proposes some directions for future research, such as developing more efficient and interpretable neural network architectures, improving the scalability of training algorithms, and exploring the potential of new paradigms, such as Spiking Neural Networks, quantum neural networks, and neuromorphic computing.

This article presents a detailed examination of the methodology and modeling tools utilized to analyze gas flow in pipelines, rooted in the fundamental principles of gas dynamics. The methodology integrates numerical simulations with modern neural network techniques, particularly focusing on the Physics-Informed Neural Network (PINN) method. This innovative approach combines artificial neural networks (ANNs) with physical equations, offering a more efficient and accurate way to model various complex processes and phenomena. The proposed mathematical model, based on the Euler equation, has been meticulously implemented using the Python language. Verification with analytical solutions ensures the accuracy and reliability of the computations. In the research, a comprehensive comparative analysis was conducted between results obtained using the PINN method and those from conventional Computational Fluid Dynamics (CFD) approaches. The analysis highlighted the advantages of the PINN method, which produced smoother pressure and velocity fluctuation profiles while reducing computation time, demonstrating its potential as a transformative modeling tool. The data derived from this study are of paramount importance for ensuring ongoing energy supply reliability and can also be used to create predictive models related to gas behavior in pipelines. The application of modeling techniques for gas flow simulation has the potential to revolutionize the integrity of our energy infrastructure and utilization of gas resources. However, it is crucial to emphasize that the effectiveness of such models relies on continuous monitoring and frequent updates to ensure alignment with real-world conditions. This research not only contributes to a deeper understanding of compressible gas flows but also underscores the crucial role of advanced modeling methodologies in the sustainable management of gas resources for both current and future generations.

Dynamic neural networks (DNN) are types of artificial neural networks (ANN) that are designed to work with sequential data where context in time is important. In contrast to traditional static neural networks that process data in a fixed order, dynamic neural networks use information about past inputs, which is important if dynamic of a certain process is emphasized. They are widely used in natural language processing, speech recognition and time series prediction. In industrial processes, their use is interesting for the prediction of difficult-to-measure process variables. In an industrial process of isomerization, it is crucial to measure the quality attributes affecting the octane number of gasoline. Process analyzers that are commonly used for this purpose are expensive and subject to failures, therefore, in order to achieve continuous production in case of malfunction, mathematical models for estimating the product quality attributes are imposed as a solution. In this paper, mathematical models were developed using dynamic recurrent neural networks (RNN), i.e., their subtype of a long short-term memory (LSTM) architecture. The results of the developed models were compared with the results of several types of other data-driven models developed for an isomerization process, such as multilayer perceptron (MLP) artificial neural networks, support vector machines (SVM) and dynamic polynomial models. The obtained results are satisfactory, which suggests a good possibility of application.

This study uses machine learning to predict the convergence results of the Duffing equation with and without damping. The Duffing equation represents a nonlinear second-order differential equation with interesting behavior in undamped free vibration and forced vibration with damping. Convergence alternates randomly between 1 and -1 in undamped free vibration, depending on initial conditions. For forced vibration with damping, multiple factors influence vibration patterns. We utilize the fourth-order Runge-Kutta method to collect convergence results for both conditions. Machine learning techniques, specifically the long short term memory (LSTM) and LSTM-Neural Network (LSTM-NN) method, are employed to predict these convergence values. The LSTM-NN model is a hybrid approach that combines the LSTM method with the addition of hidden layers of neurons. Both the LSTM and LSTM-NN models are thoroughly explored and analyzed in this research. The research process involves three stages: data preprocessing, training, and verification. The results show that the LSTM-NN model becomes more adept at predicting binary datasets, boasting an impressive accuracy of up to 98%. However, when it comes to predicting multiple solutions, the traditional LSTM method outperforms the LSTM-NN approach.

Creating self-learning algorithms, developing deep neural networks and improving other methods that "learn" for various areas of human activity is the main goal of the theory of machine learning. It helps to replace the human with a machine, aiming to increase the quality of production. The theory of artificial neural networks, which already have replaced the humans in problems of detection of moving objects, recognition of images or sounds, time series prediction, big data analysis and numerical methods remains the most dispersed branch of the theory of machine learning. Certainly, for each area of human activity it is necessary to select appropriate neural network architectures, methods of data processing and some novel tools from applied mathematics. But the universal problem for all these neural networks with specific data is the achieving the highest accuracy in short time. Such problem can be resolved by increasing sizes of architectures and improving data preprocessing, where the accuracy rises with the training time. But there is a possibility to increase the accuracy without time growing, applying optimization methods. In this survey we demonstrate existing optimization algorithms of all types, which can be used in neural networks. There are presented modifications of basic optimization algorithms, such as stochastic gradient descent, adaptive moment estimation, Newton and quasi-Newton optimization methods. But the most recent optimization algorithms are related to information geometry, for Fisher-Rao and Bregman metrics. This approach in optimization extended the theory of classic neural networks to quantum and complex-valued neural networks, due to geometric and probabilistic tools. There are provided applications of all introduced optimization algorithms, what delighted many kinds of neural networks, which can be improved by including any advanced approaches in minimization of the loss function. Afterwards, we demonstrated ways of developing optimization algorithms in further researches, engaging neural networks with progressive architectures. Classical gradient based optimizers can be replaced by fractional order, bilevel and, even, gradient free optimization methods. There is a possibility to add such analogues in graph, spiking, complex-valued, quantum and wavelet neural networks. Besides the usual problems of image recognition, time series prediction, object detection, there are many are other tasks for modern theory of machine learning, such as solving problem of quantum computations, partial differential and integro-differential equations, stochastic processes and Brownian motion, making decisions and computer algebra.

In computer vision and image analysis, Convolutional Neural Networks (CNNs) and other deep learning models are at the forefront of research and development. These advanced models have proven to be highly effective in tasks related to computer vision. One technique that has gained prominence in recent years is the construction of ensembles using Deep CNNs. These ensembles typically involve combining multiple pre-trained CNNs to create a more powerful and robust network. The purpose of this study is to evaluate the effectiveness of building CNN ensembles by combining several advanced techniques. Tested here are CNN ensembles constructed by replacing ReLU layers with different activation functions, employing various data augmentation techniques, and utilizing several algorithms, including some novel ones, that perturb network weights. Experimental results performed across many data sets representing different tasks demonstrate that our proposed methods for building deep ensembles produces superior results. All the resources required to replicate our experiments are available at https://github.com/LorisNanni.

We present a simplified computational rule for the back-propagation formulas for artificial neural networks. In this work, we provide a generic two-step rule for the back-propagation algorithm in matrix notation. Moreover, this rule incorporates both the forward and backward phases of the computations involved in the learning process. Specifically, this recursively computing rule permits the propagation of the changes to all synaptic weights in the network, layer by layer, efficiently. In particular, we use this rule to compute both the up and down partial derivatives of the cost function of all the connections feeding into the output layer.

Optical artificial neural networks (ONNs) have significant potential for ultra-high computing speed and energy efficiency. We report a novel approach to ONNs that uses integrated Kerr optical micro-combs. This approach is programmable and scalable and is capable of reaching ultra-high speeds. We demonstrate the basic building block ONNs — a single neuron perceptron — by mapping synapses onto 49 wavelengths to achieve an operating speed of 11.9 x 10⁹ operations per second, or Giga-OPS, at 8 bits per operation, which equates to 95.2 gigabits/s (Gbps). We test the perceptron on handwritten-digit recognition and cancer-cell detection — achieving over 90% and 85% accuracy, respectively. By scaling the perceptron to a deep learning network using off-the-shelf telecom technology we can achieve high throughput operation for matrix multiplication for real-time massive data processing.

This article presents a forecast model that uses a hybrid architecture of recurrent neural networks (RNN) with surface neural networks (ANN), based on historical records of exported active energy (EAE) and weather data. Two types of models were developed: the first type includes six models that use EAE records and weather variables as inputs, while the second type includes eight models that use only weather variables. Different metrics were applied to assess the performance of these models, and the best model of each type was selected. Finally, a comparison of the performance between the selected models of both types is presented, and they are validated with real data provided by a solar plant, achieving acceptable levels of accuracy. The selected model of the first type has an RMSE of 0.19, MSE of 0.03, MAE of 0.09, a correlation coefficient of 0.96, and a determination coefficient of 0.93. The other selected model of the second type showed lower precision in the metrics (RMSE = 0.24, MSE = 0.06, MAE = 0.10, Corr. Coef. = 0.95, and Det. Coef. = 0.90). Both models demonstrated good performance and acceptable accuracy in forecasting the weekly photovoltaic energy production of the solar plant.

Sense element engagement theory explains how neural networks produce cortical prosthetic vision. A major prediction of the theory can be tested by developing a device which is expected to enable perception of continuous forms in altered visual geometries. The research reported here completes several essential steps in developing this device: (1) replication of simulations that are consistent with the theory using the NEST simulator, which can also be used for full-scale network emulation by a neuromorphic computer; (2) testing whether results consistent with the theory survive increasing the scale and duration of simulations; (3) establishing a method that uses numbers of spikes produced by network neurons to report the number of phosphenes produced by cortical stimulation; and (4) simulating essential functions of the prosthetic device. NEST simulations replicated early results and increasing their scale and duration produced results consistent with the theory. A decision function created using multinomial logistic regression correctly classified the expected number of phosphenes for 2080 spike number distributions for each of three sets of data, half of which arise from simulations expected to yield continuous visual forms on an altered visual geometry. A process for modulating electrical stimulation amplitude based on intermittent population recordings that is predicted to produce continuous visual forms was successfully simulated. The classification function developed using logistic regression will be used to tune this process as the scale of simulations is further increased.

Deep learning models are part of the family of artificial neural networks and, as such, it suffers of catastrophic interference when they learn sequentially. In addition, most of these models have a rigid architecture which prevents the incremental learning of new classes. To overcome these drawbacks, in this article we propose the Self-Improving Generative Artificial Neural Network (SIGANN), a type of end-to-end Deep Neural Network system which is able to ease the catastrophic forgetting problem when leaning new classes. In this method, we introduce a novelty detection model to automatically detect samples of new classes, moreover an adversarial auto-encoder is used to produce samples of previous classes. This system consists of three main modules: a classifier module implemented using a Deep Convolutional Neural Network, a generator module based on an adversarial autoencoder; and a novelty detection module, implemented using an OpenMax activation function. Using the EMNIST data set, the model was trained incrementally, starting with a small set of classes. The results of the simulation show that SIGANN is able to retain previous knowledge with a gradual forgetfulness for each learning sequence. Moreover, SIGANN can detect new classes that are hidden in the data and, therefore, proceed with incremental class learning.

In this paper, we continue the research cycle on the properties of convolutional neural network-based image recognition systems and ways to improve noise immunity and robustness [1]. Currently, a popular research area related to artificial neural networks is adversarial attacks. The effect of adversarial attacks on the image is not highly perceptible to the human eye, also it drastically reduces the neural network accuracy. Image perception by a machine is highly dependent on the propagation of high frequency distortions throughout the network. At the same time, a human efficiently ignores high-frequency distortions, perceiving the shape of objects as a whole. The approach proposed in this paper can improve the image recognition accuracy in the presence of high-frequency distortions, in particular, caused by adversarial attacks. The proposed technique makes it possible to measure up the logic of artificial neural network to that of a human, for whom high-frequency distortions are not decisive in object recognition.

The Physics-Informed Neural Networks (PINNs) improve the efficiency of data utilization by combining physical principles with neural network algorithms and ensure that the predictions are consistent and stable with the physical laws. PINNs opens up a new approach to address inverse problems in fluid mechanics. Based on the single-relaxation-time lattice Boltzmann method (SRT-LBM) with the Bhatnagar-Gross-Krook (BGK) collision operator, the PINN-SRT-LBM model is proposed in this paper for solving the inverse problem in fluid mechanics. The PINN-SRT-LBM model consists of three components. The first component involves a deep neural network that predicts the equilibrium control equations in different discrete velocity directions within SRT-LBM. The second component employs another deep neural network to predict non-equilibrium control equations, enabling inference of the fluid's non-equilibrium characteristics. The third component, a physics informed function translates the outputs of the first two networks into physical infor-mation. By minimizing the residuals of the physical partial differential equations (PDEs), the physics informed function infers relevant macroscopic quantities of the flow. The model evolves two sub-models applicable to different dimensions, named PINN-SRT-LBM-I and PINN-SRT-LBM-II models according to the construction of the physical informed function. The innovation of this work is the introduction of SRT-LBM and discrete velocity models as physical drivers into the neural network through the interpretation function. Therefore, PINN-SRT-LBM allows the neural network to handle inverse problems of various dimensions and focus on problem-specific solving. Experimental results confirm the accurate prediction of flow infor-mation at different Reynolds numbers within the computational domain. Relying on the PINN-SRT-LBM models, inverse problems in fluid mechanics can be solved efficiently.

Convolutional neural networks (CNN) have attracted much attention as a commonly used method for hyperspectral image (HSI) classification in recent years, however, CNNs can only be applied to Euclidean data and have limitations in dealing with relationships due to the limitations of local feature extraction. However, each pixel of a hyperspectral image contains a set of spectral bands that are correlated and interact with each other, and the methods used to process Euclidean data cannot effectively obtain these correlations. In contrast, the graph convolutional network (GCN) can be used in non-Euclidean data, but usually leads to oversmoothing and ignoring local detail features due to the need for superpixel segmentation processing to reduce computational effort. To overcome the above problems, we constructed a network a fusion network based on GCN and CNN, which contains two branches: a graph convolutional network based on superpixel segmentation and a convolutional network with added attention mechanism. The graph convolu-tional branch can extract the structural features and capture the relationships between the nodes, and the convolutional branch can extract the detailed features in the local fine region. Owing to the fact that the features extracted from the two branches are different, the classification performance can be improved by fusing the complementary features extracted from the two branches. To vali-date the proposed algorithm, experiments were conducted on three widely used datasets, namely Indian Pines, Pavia University, and Salinas, and the overall accuracy of 98.78% was obtained in the Indian Pines dataset, and the overall accuracy of 98.99% and 98.69% was obtained in the other two datasets. The results showed that the proposed fusion network can obtain richer features and achieve high classification accuracy.

In this paper, a special recurrent neural network (RNN) called \emph{Long Short-Term Memory (LSTM)} is used to design a virtual load sensor that estimates the mass of heavy vehicles. The estimation algorithm consists of a two-layer LSTM network, with the two layers based on sequence-to-sequence and sequence-to-one logic, respectively. The network estimates the vehicle mass based on the vehicle speed, longitudinal acceleration, engine speed, engine torque, and the accelerator pedal position. The network is trained and tested with a data set collected in a high-fidelity simulation environment called Truckmaker. Training data is generated in acceleration maneuvers in a range of speeds whereas test data is obtained by simulating the vehicle in a Worldwide harmonized light vehicles test cycle (WLTC). Preliminary results show that, with the proposed approach, the heavy-vehicle mass can be estimated as accurately as commercial load sensors in a range of load mass as wide as four tons.

Artificial neural networks are widely established models of computational intelligence that have been tested for effectiveness in a variety of real-world applications. These models require fitting a set of parameters through the use of some optimization technique. However, an issue that researchers often face is finding an efficient range of values for the parameters of the artificial neural network. This paper proposes an innovative technique of generating a promising range of values for the parameters of the artificial neural network. Finding the value field is done by a series of rules for partitioning the original set of values or expanding it, which rules are generated using Grammatical Evolution. After finding a promising interval of values, any optimization technique such as a genetic algorithm can be used to train the artificial neural network on that interval of values. The new technique was tested on a wide range of problems from the relevant literature and the results were extremely promising.

The Necessity of Human resources beyond perception of human understanding towards the Evaluation of video quality or Data screening methodology is conducted based on human perception level since it is concerned with how visual content is perceived by a observer based on observations with his/ her perception on a particular video sequence. Therefore, we considered that the subject has to grade the encoded video sequences under certain test environment conditions based on ITU-Recommendations. Since Human perception is considered as the true judgment and precise measurement of visual content, data screening has became quite essential and quite comfortable to general public due to introduction of User Experience(UX) concept by User Experience community. The translations of A recurrent neural network is based on certain principles, for instance we considered natural language processing which is certainly adaptable towards understanding sequential data and use patterns to predict the consistency within observers. In our research, we adapted principles based on Recurrent Neural Networks while assuming consistency within observers for predicting video quality within data screening environment towards subjective experiments. Moreover,this research work explores the trade offs between Human perception on visual content and consistency of observations within individual observer.

This article explores the transformative role of neural networks in the realm of biology, particularly within microscopy image processing and illustrations. Neural networks have revolutionized cell segmentation and analysis, enabling precise delineation of cell boundaries and tracking of cellular behaviours. They excel in detecting subcellular structures, unravelling intricate organelle interactions. In 3D image reconstruction, neural networks navigate volumetric datasets, enhancing our understanding of spatial cellular architecture. These networks enhance disease diagnosis by identifying anomalies and irregularities, potentially revolutionizing early detection and classification. Moreover, neural networks restore and enhance microscopy images, unveiling hidden details. Lastly, they bridge art and science, fostering captivating biological art and enriching science communication. As neural networks evolve, they promise a future of limitless possibilities, weaving together technology, science, and art to illuminate the microscopic realm in unprecedented ways.

This work describes and compares the backpropagation algorithm with the Extended Kalman filter, a second-order training method which can be applied to the problem of learning neural network parameters and is known to converge in only a few iterations. The algorithms are compared with respect to their effectiveness and speed of convergence using simulated data for both, a regression and a classification task.

The vast majority of autonomous driving systems are limited to applications on roads with clear lane markings and are implemented using commercial grade sensing systems coupled with specialized graphic accelerator hardware. This research reviews an alternative approach for autonomously steering vehicles that eliminates the dependency on road markings and specialized hardware. A combination of machine vision, machine learning and artificial intelligence based on popular pre-trained Convolutional Neural Networks (CNN) and Recurrent Neural Networks (RNN) was used to drive a vehicle along roads lacking lane markings (unmarked roads). The team developed and tested this approach on the ACTor (Autonomous Camus TranspORt) vehicle - an autonomous vehicle development and research platform coupled with a low-cost webcam based sensing system and minimal computational resources. The proposed solution was evaluated on real-world roads and varying environmental conditions. It was found that this solution may be used to successfully navigate unmarked roads autonomously with acceptable road following behavior.

This research investigates the implementation of complex exponential-based neurons in FPGA, which can pave the way for implementing bio-inspired spiking neural networks to compensate for the existing computational constraints in conventional artificial neural networks. The increasing use of extensive neural networks and the complexity of models in handling big data lead to higher power consumption and delays. Hence, finding solutions to reduce computational complexity is crucial for addressing power consumption challenges. The complex exponential form effectively encodes oscillating features like frequency, amplitude, and phase shift, streamlining the demanding calculations typical of conventional artificial neurons through levering simple phase addition of complex exponential functions. The article implements such a two-neuron and a multi-neuron neural model using Xilinx system generator and Vivado design suite, employing 8-bit, 16-bit, and 32-bit fixed-point data format representations. The study evaluates the accuracy of the proposed neuron model across different FPGA implementations while also providing a detailed analysis of operating frequency, power consumption, and resource usage for the hardware implementations. BRAM-based Vivado designs outperformed Simulink regarding speed, power, and resource efficiency. Specifically, the Vivado BRAM-based approach supported up to 128 neurons, showcasing optimal LUT and FF resource utilization. Such outcomes accommodate choosing the optimal design procedure for implementing spiking neural networks on FPGAs.

The informational synthesis of neural structures, processes, parameters and characteristics that allow a unified description and modeling as neural machines of natural and artificial neural systems is presented. The general informational parameters as the global quantitative measure of the neural systems computing potential as absolute and relative neural power were proposed. Neural information organizing and processing follows the way in which nature manages neural information by developing functions, functionalities and circuits related to different internal or peripheral components and also to the whole system through a non-deterministic memorization, fragmentation and aggregation of afferent and efferent information, deep neural information processing representing multiple alternations of fragmentation and aggregation stages. The relevant neural characteristics were integrated into a neural machine type model that incorporates unitary also peripheral or interface components as the central ones. The proposed approach allows overcoming the technical constraints in artificial computational implementations of neural information processes and also provides a more relevant description of natural ones.

When extract building from high resolution remote sensing image with meter/sub-meter accuracy, the shade of trees and interference of roads are the main factors of reducing the extraction accuracy. Proposed a Bayesian Convolutional Neural Networks(BCNET) model base on standard fully convolutional networks(FCN) to solve these problems. First take building with no shade or artificial removal of shade as Sample-A, woodland as Sample-B, road as Sample-C. Set up 3 sample libraries. Learn these sample libraries respectively, get their own set of feature vector; Mixture Gauss model these feature vector set, evaluate the conditional probability density function of mixture of noise object and roofs; Improve the standard FCN from the 2 aspect:(1) Introduce atrous convolution. (2) Take conditional probability density function as the activation function of the last convolution. Carry out experiment using unmanned aerial vehicle(UVA) image, the results show that BCNET model can effectively eliminate the influence of trees and roads, the building extraction accuracy can reach 97%.

The accurate prediction of molecular properties such as lipophilicity and aqueous solubility are of great importance and pose challenges in several stages of the drug discovery pipeline. Machine learning methods like graph-based neural networks (GNNs) have shown exceptionally good performance in predicting these properties. In this work, we introduce a novel GNN architecture, called directed edge graph isomorphism network (D-GIN). It is composed of two distinct sub-architectures (D-MPNN, GIN) and achieves an improvement in accuracy over its sub-architectures employing various learning, and featurization strategies. We argue that combining models with different key aspects help make graph neural networks deeper and simultaneously increase their predictive power. Furthermore, we address current limitations in assessment of deep-learning models, namely, comparison of single training run performance metrics, and offer a more robust solution.

In urban Vehicular Ad Hoc Network (VANET) environments, buildings play a crucial role as they can act as obstacles that attenuate the transmission signal between vehicles. Quantifying the impact of buildings on the transmission quality is essential, especially in critical scenarios involving emergency vehicles, where reliable communication is of utmost importance. In this research, we propose a supervised learning approach based on artificial neural networks (ANNs) to develop a predictive model capable of estimating the level of signal degradation, represented by the bit error rate (BER), based on the obstacles perceived by moving emergency vehicles. By establishing a relationship between the level of signal degradation and the encountered obstacles, our proposed mechanism enables efficient routing decisions to be made prior to the transmission process. Consequently, data packets are routed through paths that exhibit the lowest BER. To gather the necessary training data, we employed SUMO and NS-3 simulations. The simulation results demonstrate that our developed model successfully learns and accurately estimates the BER for new data instances. Overall, our research contributes to enhancing the performance and reliability of communication in urban VANET environments, especially in critical scenarios involving emergency vehicles, by leveraging supervised learning and artificial neural networks to predict signal degradation levels and optimize routing decisions accordingly.

Calligraphy works have high artistic value, but there is a rampant problem of forgery. Indeed, authentication of traditional calligraphy heavily relies on calligraphers' subjective judgment. Therefore, spurred by the recent development of neural networks, this paper proposes a method for authenticating calligraphy works based on an improved EfficientNet network. Specifically, the developed method utilizes the character box algorithm and the centroid algorithm to extract individual calligraphy characters, which are then augmented and used as the training set for the model. The training process employs CBAM and Self-Attention modules to enhance the attention mechanism of the EfficientNet network. The model is tested on authentic works, imitated works, and works from other calligraphers and is compared with other networks. The experimental results demonstrate that the proposed method effectively achieves the authentication of calligraphy works, and the improved CBAM-EfficientNet network and SA-EfficientNet network achieve better authentication performance.

Transfer learning has proven to be a valuable technique for deploying machine learning models on edge devices and embedded systems. By leveraging pre-trained models and fine-tuning them on specific tasks, practitioners can effectively adapt existing models to the constraints and requirements of their application. In the process of adapting an existing model, a practitioner may make adjustments to the model architecture, including the input layers, output layers, and intermediate layers. In this study, we examine the effects of these adjustments on the runtime and energy performance of an edge processor performing inferences. Based on our observations, we make recommendations for how to adjust convolutional neural networks during transfer learning to maintain runtime performance. We observe that the Edge TPU is generally more efficient than a CPU at performing inferences on convolutional neural networks, and continues to outperform a CPU as the depth and width of the convolutional network increases. We explore multiple strategies for adjusting the input and output layers of an existing model and demonstrate important performance cliffs for practitioners to consider when modifying a convolutional neural network model.

With an explosion of data in the nearly every domain, from physical and environmental sciences to econometrics and even social science, we have entered a golden era of time-domain data science. In addition to ever increasing data volumes and varieties of time-series, there's a plethora of techniques available to analyses them. Unsurprisingly, machine learning approaches are very popular. In environmental sciences (ranging from climate studies to space weather/solar physics), machine learning (ML )and deep learning (DL) approaches complement the traditional data assimilation approaches. In solar and astrophysics, multiwavelength time-series or lightcurves along with observations of other messengers like gravitational waves provide a treasure trove of time-domain data for ML/DL approaches for modeling and forecasting. In this regard, deep learning algorithms such as recurrent neural networks (RNNs) should prove extremely powerful for forecasting as it has in several other domains. Such methods agnostic of the physics can only predict transient flares that are represented in the original distribution trained on (eg. coloured noise fluctuations) and not special, isolated episodes (eg. a one-off outburst like a major coronal mass ejection) potentially distinguishing them. Therefore it is vital to test robustness of these techniques ; here we attempt to make a small contribution to this by exploring sensitivity of RNNs with some rudimentary experiments. We test the performance of a very successful class of RNNs, the Long Short Term Memory (LSTM) algorithms with simulated lightcurves. As an example, we focus on univariate forecasting of time-series (or lightcurves) characteristic certain variable astrophysical objects (eg. active galactic nuclei or AGN). Specifically, we explore the sensitivity of training and test losses to key parameters of the LSTM network and data characteristics namely gaps and complexity measured in terms of number of Fourier components. We find that typically, the performances of LSTMs are better for pink or flicker noise type sources. The key parameters on which performance is dependent are batch size for LSTM and the gap percentage of the lightcurves. While a batch size of $10-30$ seems optimal, the most optimal test and train losses are under $10 \%$ of missing data for both periodic and random gaps in pink noise. The performance is far worse for red noise. This compromises detectability of transients. Thus, we show that generative models (time-series simulations) are excellent guides for use of RNN-LSTMs in building forecasting systems

Artificial neural networks are a powerful tool for managing data that is difficult to process and interpret. This paper presents the study of artificial neural networks for information processing generated by an optical encoder based on the polarization of light. A machine learning technique is proposed to train the neural networks, such that the system can predict with remarkable accuracy the angular position in which the rotating element of the neuro-encoder is located, based on information provided by light’s phase shifting arrangements. The proposed neural designs show excellent performance in small angular intervals, and a methodology is proposed to avoid losing this remarkable characteristic in measurements from 0 to 180o or even to 360o. The neuro-encoder is implemented in simulation stage to obtain performance results. This study can be useful to improve capabilities of resolvers or other polyphasic sensors.

The Ocean Color - Simultaneous Marine and Aerosol Retrieval Tool (OC-SMART) is a robust data processing platform that supports a large array of multi-spectral and hyper-spectral sensors. It provides accurate aerosol optical depths and remote sensing reflectances (Rrs estimates) that can be used to generate products such as absorption coefficients due to phytoplankton and detritus/Gelbstoff as well as backscattering coefficients due to particulate matter. The OC-SMART platform yields improved performance in complex environments by utilizing scientific machine learning (SciML) in conjunction with comprehensive radiative transfer computations. This paper expands the capability of OC-SMART by quantifying uncertainties in ocean color retrievals. Bayesian inversion is used to relate measured top of atmosphere radiances and a priori data to estimate posterior probability density functions and associated uncertainties. A framework of the methodology and implementation strategy is presented and uncertainty estimates for Rrs retrievals are provided to demonstrate the approach by applying it to MODIS, OLCI Sentinel-3, and VIIRS sensor data.

In this study, we explore the potential of using a straightforward neural network inspired by the retina model to efficiently restore low-light images. The retina model imitates the neurophysiological principles and dynamics of various optical neurons. Our proposed neural network model reduces the computational overhead compared to traditional signal-processing models while achieving results similar to complex deep learning models from a subjective perceptual perspective. By directly simulating retinal neuron functionalities with neural networks, we not only avoid manual parameter optimization but also lay the groundwork for constructing artificial versions of specific neurobiological organizations.

American Sign Language is a popular language for deaf individuals. Communication is made easier for these people through sign language. However, in a digital era like today, there is a need for these people to be able to communicate online, and even get help from technology to communicate in person with non sign language speakers. This research will present a program able to translate American sign language to plain English. This study aims to use the OpenCV library to recognize hand signals, also a trained model to identify images so that the program can then translate them to words and letters. The program uses a data set of over 2000 images which will be in this case the largest data set available. With over 90\% of accuracy it results in a basic computer program with the largest data set available that would make possible for users to communicate with a wide variety of words and expressions.

Deep Learning is an extremely important research topic in Earth Observation. Current use-cases range from semantic image segmentation, object detection to more common problems found in computer vision such as object identification. Earth Observation is an excellent source for different types of problems and data for Machine Learning in general and Deep Learning in particular. It can be argued that both Earth Observation and Deep Learning as fields of research will benefit greatly from this recent trend of research. In this paper we take several state of the art Deep Learning network topologies and provide a detailed analysis of their performance for semantic image segmentation for building footprint detection. The dataset used is comprised of high resolution images depicting urban scenes. We focused on single model performance on simple RGB images. In most situations several methods have been applied to increase the accuracy of prediction when using deep learning such as ensembling, alternating between optimisers during training and using pretrained weights to bootstrap new models. These methods although effective, are not indicative of single model performance. Instead, in this paper, we present different topology variations of these state of the art topologies and study how these variations effect both training convergence and out of sample, single model, performance.

With the development of data science, more and more machine learning technologies have been designed to solve complicated and challenging real-world tasks containing a large volume of data. And many significant real-world datasets contain data in the form of networks or graphs. Graph Neural Networks is one of the powerful machine learning tools, which could provide a perfect solution to processing a large amount of non-Euclidean data. And because most bio information data in bioinformatics is in the non-Euclidean domain, Graph Neural Networks could then directly be applied to solve problems in bioinformatics. Much research has been done in the field of GNN, and there are also some surveys related to GNN and its applications. However, there has been little research focusing on GNN in bioinformatics, and we think in the future we could better utilize GNN in the field of biology, so we would like to write a literature review to help take a comprehensive look at GNN and their applications in the field of bioinformatics. In this paper, we would first introduce SOTA models in Graph Neural Networks, and second, introduce their applications in bio information. And then we would provide future directions of Graph Neural Networks in bioinformatics.

Over the past decade, recognition of human activities (HAR) has become a vibrant field of research, in particular, the spread in our everyday lives of electronics such as mobile phones, smart cell phones, and video cameras. Furthermore, the advancement in the field of deep methodologies and other paradigms have enabled scientists to enable HAR in many areas, consisting of activities in fitness and wellness. For instance, HAR is one of many resorts to support older people through day-to-day activities to support their cognition and physicality. This study is centered on the key aspects deep learning plays in the development of HAR applications. Although numerous HAR examination studies were carried out previously, there have been no overall studies on this subject, in all the earlier studies there were only specific HAR-related subjects. A detailed review covering all the main subjects in this area is therefore essential. This study discusses the latest developments and works in HAR. It separates the methods and the advantages and disadvantages of each method group. This paper finally addresses many problems in the current HAR subject and provides recommendations for potential study.

Neural networks present the characteristics that the results are strongly dependent on the training data, the weight initialisation, and the hyper-parameters chosen. The determination of the distribution of a statistical estimator, as the Mean Squared Error (MSE) or the accuracy, is fundamental to evaluate the performance of a neural network model (NNM). For many machine learning models, as linear regression, it is possible to analytically obtain information as variance or confidence intervals on the results. Neural networks present the difficulty of being not analytically tractable due to their complexity. Therefore, it is impossible to easily estimate distributions of statistical estimators. When estimating the global performance of an NNM by estimating the MSE in a regression problem, for example, it is important to know the variance of the MSE. Bootstrap is one of the most important resampling techniques to estimate averages and variances, between other properties, of statistical estimators. In this tutorial, the application of two resampling (including bootstrap) techniques to the evaluation of neural networks’ performance is explained from both a theoretical and practical point of view. Pseudo-code of the algorithms is provided to facilitate their implementation. Computational aspects, as the training time, are discussed since resampling techniques always require to run simulations many thousands of times and, therefore, are computationally intensive. A specific version of the bootstrap algorithm is presented that allows the estimation of the distribution of a statistical estimator when dealing with an NNM in a computationally effective way. Finally, algorithms are compared on synthetically generated data to demonstrate their performance.

Over the years, various models have been developed in the stages of the mining process that have allowed predicting and enhancing results, but it is the breakage the variable that connects all the activities of the mining process from the point of view of costs (drilling, blasting, loading, hauling, crushing and grinding). To improve this process, we come up with an idea to develop a breakage prediction model; on the basis of the main variables involved in the drilling and blasting process. For this purpose, we design a computer model based on an Artificial Neuronal Networks (ANN), built by using the most representative variables that come into play with drilling and blasting, such as: the properties of the explosives, the geomechanical parameters of the rock mass, and the design parameters of drilling-blasting. For its experimentation and validation, we have taken the data from a copper mine as reference located in the north of Chile, because of we have the dataset of that ore deposit, which is valid and reliable to evaluate prediction problems based on ANN applied to copper mines. The ANN architecture was of the supervised type, feedforward, with 3 layers and 13 neurons in the only hidden layer, trained with the input data using a dataset with the previously mentioned variables, which then were compared with the breakage results. The model was feed backed in its learning process until it becomes perfected, and is a prediction option that can be used in future blasting of ore deposits with similar characteristics using the same representative variables. Therefore, this is a valid alternative for predicting rock breakage, given that it has been experimentally validated, and has achieved moderately reliable results, providing higher correlation coefficients than traditional models, and with the additional advantage that an ANN model provides, due to its ability to learn and recognize compiled dataset patterns. In this way, using this computer model we can obtain satisfactory results that allow us to predict breakage, providing an alternative for evaluating the costs that this entails.

The paper concludes the first research on mouth-based Emotion Recognition (ER), adopting a Transfer Learning (TL) approach. Transfer Learning results paramount for mouth-based emotion ER, because a few data sets are available, and most of them include emotional expressions simulated by actors, instead of adopting a real-world categorization. Using TL we can use fewer training data than training a whole network from scratch, thus more efficiently fine-tuning the network with emotional data and improving the convolutional neural network accuracy in the desired domain. The proposed approach aims at improving the Emotion Recognition dynamically, taking into account not only new scenarios but also modified situations with respect to the initial training phase, because the image of the mouth can be available even when the whole face is visible only in an unfavourable perspective. Typical applications include automated supervision of bedridden critical patients in an healthcare management environment, or portable applications supporting disabled users having difficulties in seeing or recognizing facial emotions. This work takes advantage from previous preliminary works on mouth-based emotion recognition using CNN deep-learning, and has the further benefit of testing and comparing a set of networks on large data sets for face-based emotion recognition well known in literature. The final result is not directly comparable with works on full-face ER, but valorizes the significance of mouth in emotion recognition, obtaining consistent performances on the visual emotion recognition domain.

This paper presents a study and implementation of a convolutional neural network to identify and recognize humpback whale specimens from the unique patterns of their tails. Starting from a dataset composed of images of whale tails, all the phases of the process of creation and training of a neural network are detailed – from the analysis and pre-processing of images to the elaboration of predictions, using TensorFlow and Keras frameworks. Other possible alternatives are also explained when it comes to tackling this problem and the complications that have arisen during the process of developing this paper.

Biofilms are congregations of bacteria on a surface, and they grow into obstacles for the functionalities of any device or machinery involves anything biological. Biofilms are developed through a biochemical system known as ‘Quorum Sensing’ that accounts for the chemical signaling that direct either biofilm formation or inhibition. Computational models that relate chemical and structural features of compounds to their performance properties have been used to aide in the discovery of active small molecules for many decades. These quantitative structure-activity relationship (QSAR) models are also important for predicting the activity of molecules that can have a range of effectiveness in biological systems. This study uses QSAR methodologies combined with and different machine learning algorithms to predict and assess the performance of several different compounds acting in Quorum Sensing. Through computational probing of the quorum sensing molecular interaction, new design rules can be elucidated for countering biofilms.

Multilabel learning goes beyond standard supervised learning models by associating a sample with more than one class label. Among the many techniques developed in the last decade to handle multilabel learning best approaches are those harnessing the power of ensembles and deep learners. This work proposes merging both methods by combining a set of gated recurrent units, temporal convolutional neural networks, and long short-term memory networks trained with variants of the Adam optimization approach. We examine many Adam variants, each fundamentally based on the difference between present and past gradients, with step size adjusted for each parameter. We also combine Incorporating Multiple Clustering Centers and a bootstrap-aggregated decision trees ensemble, which is shown to further boost classification performance. In addition, we provide an ablation study for assessing the performance improvement that each module of our ensemble produces. Multiple experiments on a large set of datasets representing a wide variety of multilabel tasks demonstrate the robustness of our best ensemble, which is shown to outperform the state-of-the-art. The MATLAB code for generating the best ensembles in the experimental section will be made available at https://github.com/LorisNanni.

Precipitation downscaling is widely employed for enhancing the resolution and accuracy of precipitation products from general circulation models (GCMs). In this study, we propose a novel statistical downscaling method to foster GCMs’ precipitation prediction resolution and accuracy for monsoon region. We develop a deep neural network composed of convolution and Long Short Term Memory (LSTM) recurrent module to estimate precipitation based on well-resolved atmospheric dynamical fields. The proposed model is compared against GCM precipitation product and classical downscaling methods in the Xiangjiang River Basin in South China. Results show considerable improvement compared to the ECMWF-Interim reanalysis precipitation. Also, the model outperforms benchmark downscaling approaches, including 1) quantile mapping, 2) support vector machine, and 3) convolutional neural network. To test the robustness of the model and its applicability in practical forecast, we apply the trained network for precipitation prediction forced by retrospective forecasts from ECMWF model. Compared to ECMWF precipitation forecast, our model makes better use of the resolved dynamical field for more accurate precipitation prediction at lead time from 1 day up to 2 weeks. This superiority decreases along forecast lead time, as GCM’s skill in predicting atmospheric dynamics being diminished by the chaotic effect. At last, we build a distributed hydrological model and force it with different sources of precipitation inputs. Hydrological simulation forced with the neural network precipitation estimation shows significant advantage over simulation forced with the original ERA-Interim precipitation (with NSE value increases from 0.06 to 0.64), and the performance is just slightly worse than the observed precipitation forced simulation (NSE=0.82). This further proves the value of the proposed downscaling method, and suggests its potential for hydrological forecasts.

This study aimed at exploring what role artificial intelligence techniques could play in the futural numerical analysis. In this paper, a convolutional neural network techniques based on modified loss function is proposed as a surrogate of finite element method(FEM). Several surrogate-based physics-informed neural networks(PINNs) are developed to solve representative boundary value problems based on elliptic partial differential equations (PDEs). Results from the proposed surrogate-based approach are in good agreement with ones from conventional FEM. It is found that modification of the loss function could improve the prediction accuracy of the neural network. It is indicated that to some extent the artificial intelligence technique could replace conventional numerical analysis as a great surrogate model.

Deep learning techniques, such as convolutional neural networks (CNN), generative adversarial networks (GAN), and graph neural networks (GNN), have over the past decade changed the ac-curacy of prediction in many diverse fields. In recent years, the application of deep learning tech-niques in computer vision tasks in pathology demonstrated extraordinary potential in assisting clinicians, automating diagnosis, and reducing costs for patients. Formerly unknown pathologi-cal evidence, such as morphological features related to specific biomarkers, copy number varia-tions, and other molecular features, were also able to be captured by deep learning models. In this paper, we review popular deep learning methods and some recent publications about their appli-cations in pathology.

The article presents the operation of the automatic pine sawn timber inspection system, which was developed at the Woodinspector company and offered commercially. The vision inspection system is used to detect various wood defects, including knots, blue stain, and mechanical damage caused by worms. Blue stain is a defect that is difficult to detect based on the color of the wood, because it can be easily confused with wood defects or dirt that do not impair its strength properties. In particular, the issues of detecting blue stain in wood, the use of artificial neural networks and improving the operation of the system in production conditions were discussed.

For years, effort has been devoted to establishing an effective bull breeding soundness evaluation procedure; usual research on the subject is based on BBSE (Bull Breeding Soundness Examination) methodologies, which have significant limitations in terms of their evaluation procedure, such as high cost, time-consuming and administratively difficult. This research focuses on the creation of a prediction model to supplement and/or improve the BBSE approach, through the study of two algorithms, Clustering and Artificial Neural Network (ANN), to find the optimum Machine Learning (ML) approach for our application, with an emphasis on data categorization accuracy. The tool is designed to assist veterinary medicine and farmers in identifying key factors and increasing certainty in their decision-making during the selection of bulls for breeding purposes, providing data from a limited number of factors generated from a deep pairing study of bulls. Zebu Bulls, European Bulls, and Crossbred Bulls were the general groupings. The data utilized in the model's creation (N =359) considers five variables that influence improvement decisions. This approach enhances decision-making by 12%. ANN obtained an accuracy of 90%; Precision was 97% for satisfying, 92% for unsatisfying, and 85% for Bad.

This paper deals with the analysis of the cognitive functions of the cerebral cortex or cortex in the evolutionary context. The triune brain model is presented given its notorious popularity in fields that are not strictly neuroscientific, and its limitations and inaccuracies are also shown. Although it is an outdated theory, this one represented the passage to a new era, because it really was the model of the movement towards understanding the brain in evolutionary terms. Next, the paper illustrates the different areas of the cortex and their functions, as well as the evolution of the brain and the cortex in vertebrates. To date, the expansion of the cortex with the amniotes was placed 300 million years ago, although recent studies place it in the lampreys more than 500 million years ago. Evolutionary changes in mammalian brain and cortex size are discussed, showing the evolution from three to six cell layers and trends in different groups of mammals. Finally, the most recent contributions in the field of research are illustrated, such as the role of neurogenesis and the evolution of the connectivity of the cortex.

Image denoising has been one of the important problems in the field of computer vision, and it has a wide range of practical value in many applications, such as medical image processing, image enhancement, and computational photography. Traditional image denoising methods are usually based on hand-designed features and filters, but these methods perform poorly under complex noise and image structures. In recent years, the rapid development of neural network technology has revolutionized the image-denoising task. This paper introduces the knowledge about neural networks and image denoising, explores the impact of neural networks on image denoising, and how is it possible to denoise images by neural networks. It also summarises other image-denoising methods and finally points out the challenges and problems faced by image-denoising at present. Some possible new development directions are proposed to provide new solutions for image-denoising researchers and to promote the development of the field.

RBF networks are used in a variety of real-world applications such as medical data or signal processing problems. The success of these parametric models lies in the successful adaptation of their parameters using efficient computational techniques. In the current work, a method of adjusting the parameters of these networks using Grammatical Evolution is presented. Grammatical Evolution will be used to successfully discover the most promising range of parameter values and then the training of the parameter set will be achieved using a Genetic Algorithm. The new method was applied to a wide range of data fitting and classification problems, and the results were more than promising.

Snow parameters have traditionally been retrieved using discontinuous, multi-band sensors; however, continuous hyperspectral sensor are now being developed as an alternative. In this paper we investigate the performance of various sensor configurations using machine learning neural networks trained on a simulated dataset. Our results show improvements in accuracy of retrievals of snow grain size and impurity concentration for continuous hyperspectral channel configurations. Retrieval accuracy of snow albedo was found to be similar for all channel configurations.

Tracking temperature changes in certain geographic regions is a current task in modern research on Earth's climate changes. One of the global problems in solving this task is related to the large volume of measured data and the search for appropriate methods for effective determination of changes. The purpose of this research is to track climate temperature changes using a machine learning-based automated change detection method. The presented method includes training of a two-level structure of neural networks, with measured temperatures for a ten-year period of time for a certain geographical region. In the testing phase, the neural structure classifies measured temperatures for two three-year periods, before and after the ten-year time period, respectively, for the same geographic region. An algorithm was developed to visualize the studied regions by creating a map with their geographic coordinates. The classification results in the neural structure outputs are presented and analyzed as possible temperature changes. Suggestions for continuing and expanding the research in the future are discussed.

Retailers must have accurate sales forecasts to operate their businesses efficiently and effectively and to remain competitive in the marketplace. Global forecasting models like RNNs can be a powerful tool for forecasting in retail settings, where multiple time series are often interrelated and influenced by a variety of external factors. By including covariates in a forecasting model, we can better capture the various factors that can influence sales in a retail setting. This can help improve the accuracy of our forecasts and enable better decision-making for inventory management, purchasing, and other operational decisions. In this study we investigate how the accuracy of global forecasting models is affected by the inclusion of different potential demand covariates. To ensure the significance of the study's findings, we used the M5 forecasting competition's openly accessible and well-established dataset. The results obtained from the DeepAR models, which were trained on various combinations of features including time, price, events, and IDs, suggest that individually only the features corresponding to IDs improve the baseline model. However, when all the features are used together, the best performance is achieved, indicating that the individual relevance of each feature is emphasized when the information is given jointly. Comparing the model with features to the model without features, there is an improvement of 1.76\% for MRMSSE and 6.47\% for MMASE.

This report gives a comprehensive summary of two problems about graph convolutional networks (GCNs): over-smoothing and heterophily challenges, and outlines future directions to explore.

Object classification is a crucial task in deep learning, which involves the identification and categorization of objects in images or videos. Although humans can easily recognize common objects, such as cars, animals, or plants, performing this task on a large scale can be time-consuming and error-prone. Therefore, automating this process using neural networks can save time and effort while achieving higher accuracy. Our study focuses on the classification step of human chromosome karyotyping, an important medical procedure that helps diagnose genetic disorders. Traditionally, this task is performed manually by expert cytologists, which is a time-consuming process that requires specialized medical skills. Therefore, automating it through deep learning can be immensely useful. To accomplish this, we implemented and adapted existing preprocessing and data augmentation techniques to prepare the chromosome images for classification. We used ResNet-50 convolutional neural networks and an ensemble approach to classify the chromosomes, obtaining state-of-the-art performance in the tested dataset.

As AI models become more and more common in business and even in our daily lives, it is important to understand what the carbon impact of these models is. Recent papers have shown that this impact can be quite great, i.e., the training of a single high-end model can result in emissions of more than 500t of CO2eq. In this paper we discuss the factors that influence the carbon footprint of AI models, explore what impact different decisions have, and show how the footprint can be reduced. We also examine the footprint of different models to give a guideline on how urgent action is for different organizations.

In this work a new hydroelectric basin modelling approach is described and applied to the Pontecosi basin, Italy. Several types of data sources were used to learn the model: a number of weather stations, satellite observations, Reanalysis dataset and basin data. With the goal of predicting the water level of the basin, the model was composed by three cascade modules. Firstly, different spatial interpolation methods, such as Kriging, Radial Basis Function and Natural Neighbours, were compared and applied to interpolate the weather stations data nearby the basin area to infer the main environmental variables (air temperature, air humidity, precipitation and wind speed) in the basin area. Then, using these variables as inputs, a neural network was trained to predict the mean soil moisture concentration over the area, also to improve the low availability due to satellite orbits. Finally, a non-linear auto regressive exogenous input (NARX) model was trained to simulate the basin level with different prediction horizons, using the data from the previous modules and past basin data (water level, discharge flow rate, turbine flow rate). Accurate predictions of the basin water level were achieved within 1 to 6 hours ahead, with mean absolute errors (MAE) between 2cm and 10cm respectively.

Soil working tools, implements, and machines are inevitable in mechanized agriculture. The soil-tool/machine interaction is a multivariate, dynamic, and intricate process. The accurate interpretation, description, and modeling of a soil-machine interaction is key to providing a solution to sustainable crop production by reducing energy input, excessive soil pulverization, and compaction. The traditional method provides insight into soil-machine interaction but often provides inadequate solutions and lacks broad applicability. Computational intelligence (CI) is a comprehensive class of approaches that rely on approximate information to solve complex problems. The CI method has been extensively studied and applied in soil tillage and traction domain in recent decades. The study critically reviews the CI techniques implemented in soil-machine interactions, especially in the context of tillage, traction, and compaction. The traditional methods and their limitation are discussed. The fundamental of CI methods and a detailed overview of the most popular methods are provided. The study reviews and summarizes the 50 selected articles on soil-machine interaction studies where CI methods were employed. It discusses the strength and limitations of employed CI methods. It also suggests the emergent CI methods and future applications are discussed. The outlined study would serve as a concise reference and a quick and systematic way to understand the applicable CI methods that allow crucial farm management decision-making.

Classical religious texts remain an essential part of human culture due to their undiminished influence on the advancement of civilization. Although their entirely divine origin is questioned repeatedly, explicit or implicit quoting and adherence to their basic guidelines are fundamental in modern society. In this respect, these documents’ inner structure and linguistic style appear to be pivotal. This paper considers the topic from the standpoint of small textual patterns classified using deep learning methods, traditionally applied to analyze short textual material like tweets. We divide the considered documents into small sequential chunks imitating tweets and categorizing them, classifying an entire text. The proposed method demonstrates that the religious text collections correspond to stable ”Twitter”-like structures that adequately reflect stylistic properties. So, concise word combinations seem to be an inborn textual attribute that adequately outlines the proposed multi-source authorship. This approach differs from traditional methods of analyzing classical religious documents, which are based on the consideration and interpretation of relatively long templates. The case study consists of three famous collections of Mosaic authorship in the Old Testament (Hebrew), Pauline authorship in the New Testament (Greek), and Al-Ghazali authorship (Arabic). The obtained results go well with most previously expressed evaluations and complement them with new implications, particularly in the authorship of two famous manuscripts attributed to Al-Ghazali.

Convolutional neural network architectures have become increasingly complex, which has improved the performance slowly on well-known benchmark datasets in the recent years. In this research, we have analyzed the true need for such complexity. We have introduced G-Net light, a lightweight modified GoogleNet with improved filter count per layer to reduce feature overlaps and complexity. Additionally, by limiting the amount of pooling layers in the proposed architecture, we have exploited the skip connections to minimize the spatial information loss. The investigations on the proposed architecture are evaluated on three retinal vessel segmentation publicly available datasets. The proposed G-Net light outperforms other vessel segmentation architectures by reducing the number of trainable parameters..

Optical neural networks (ONNs), or optical neuromorphic hardware accelerators, have the potential to dramatically enhance the computing power and energy efficiency of mainstream electronic processors, due to their ultra-large bandwidths of up to 10’s of terahertz together with their analog architecture that avoids the need for reading and writing data back-and-forth. Different multiplexing techniques have been demonstrated to demonstrate ONNs, amongst which wavelength-division multiplexing (WDM) techniques make sufficient use of the unique advantages of optics in terms of broad bandwidths. Here, we review recent advances in WDM-based ONNs, focusing on methods that use integrated microcombs to implement ONNs. We present results for human image processing using an optical convolution accelerator operating at 11 Tera operations per second. The open challenges and limitations of ONNs that need to be addressed for future applications are also discussed.

Thermal simulations are an important part in the design of electronic systems, especially as systems with high power density become common. In simulation-based design approaches, a considerable amount of time is spent by repeated simulations. In this work, we present a proof-of-concept study of the application of convolutional neural networks to accelerate those thermal simulations. The goal is not to replace standard simulation tools but to provide a method to quickly select promising samples for more detailed investigations. Based on a training set of randomly generated circuits with corresponding Finite Element solutions, the full 3D steady-state temperature field is estimated using a fully convolutional neural network. A custom network architecture is proposed which captures the long-range correlations present in heat conduction problems. We test the network on a separate dataset and find that the mean relative error is around 2 % and the typical evaluation time is 35 ms per sample ( 2 ms for evaluation, 33 ms for data transfer). The benefit of this neural-network-based approach is that, once training is completed, the network can be applied to any system within the design space spanned by the randomised training dataset (which includes different components, material properties, different positioning of components on a PCB, etc.).

The Physics Informed Neural Networks framework is applied to the understanding of the dynamics of Coronavirus of 2019. To provide the governing system of equations used by the framework, the Susceptible-Infected-Recovered-Death mathematical model is used. The study focused on finding the patterns of the dynamics of the disease which involves predicting the infection rate, recovery rate and death rate; thus predicting the active infections, total recovered, susceptible and deceased at any required time. The study used data that was collected on the dynamics of COVID-19 from the Kingdom of Eswatini between March 2020 and September 2021. The obtained results showed less errors thus making highly accurate predictions.

Convolutional Neural Networks (CNNs) have gained prominence in the research literature on image classification over the last decade. One shortcoming of CNNs, however, is their lack of generalizability and tendency to overfit when presented with small training sets. Augmentation directly confronts this problem by generating new data points providing additional information. In this paper, we investigate the performance of more than ten different sets of data augmentation methods, with two novel approaches proposed here: one based on the Discrete Wavelet Transform and the other on the Constant-Q Gabor transform. Pretrained ResNet50 networks are finetuned on each augmentation method. Combinations of these networks are evaluated and compared across three benchmark data sets of images representing diverse problems and collected by instruments that capture information at different scales: a virus data set, a bark data set, and a LIGO glitches data set. Experiments demonstrate the superiority of this approach. The best ensemble proposed in this work achieves state-of-the-art performance across all three data sets. This result shows that varying data augmentation is a feasible way for building an ensemble of classifiers for image classification (code available at https://github.com/LorisNanni).

Creating intelligent systems capable of recognizing emotions is a difficult task, especially when looking at emotions in animals. This paper describes the process of designing a “proof of concept” system to recognize emotions in horses. This system is formed by two elements, a detector and a model. The detector is a fast region-based convolutional neural network that detects horses in an image. The model is a convolutional neural network that predicts the emotions of those horses. These two elements were trained with multiple images of horses until they achieved high accuracy in their tasks. 400 images of horses were collected and labeled to train both the detector and the model while 80 were used to validate the system. Once the two components were validated, they were combined into a testable system that would detect equine emotions based on established behavioral ethograms indicating emotional affect through head, neck, ear, muzzle and eye position. The system showed an accuracy of between 69% and 74% on the validation set, demonstrating that it is possible to predict emotions in animals using autonomous intelligent systems. Such a system has multiple applications including further studies in the growing field of animal emotions as well as in the veterinary field to determine the physical welfare of horses or other livestock.

We confirm that energy dissipation weighting provides the most accurate approach to determining the effective hydraulic conductivity (Keff) of a binary K grid. A deep learning algorithm (UNET) can infer Keff with extremely high accuracy (R2 > 0.99). The UNET architecture could be trained to infer the energy dissipation weighting pattern from an image of the K distribution with high fidelity, although it was less accurate for cases with highly localized structures that controlled flow. Furthermore, the UNET architecture learned to infer the energy dissipation weighting even if it was not trained on this information directly. However, the weights were represented within the UNET in a way that was not immediately interpretable by a human user. This reiterates the idea that even if ML/DL algorithms are trained to make some hydrologic predictions accurately, they must be designed and trained to provide each user-required output if their results are to be used to improve our understanding of hydrologic systems most effectively.

This paper focuses on quantifying the uncertainty in the specific absorption rate values of brain induced by the uncertain positions of the electroencephalography electrodes placed on patient's scalp. To avoid running a large number of simulations, an artificial neural network architecture for uncertainty quantification involving high-dimensional data is proposed in this paper. The proposed method is demonstrated to be an attractive alternative to conventional uncertainty quantification methods because of its considerable advantage in the computational expense and speed.

Complete digraphs are referred to in the combinatorics literature as tournaments. We consider a family of semi-simplicial complexes, that we refer to as ``tournaplexes'', whose simplices are tournaments. In particular, given a digraph G, we associate with it a ``flag tournaplex'' which is a tournaplex containing the directed flag complex of G, but also the geometric realisation of cliques that are not directed. We define several types of filtrations on tournaplexes, and exploiting persistent homology, we observe that flag tournaplexes provide finer means of distinguishing graph dynamics than the directed flag complex. We then demonstrate the power of these ideas by applying them to graph data arising from the Blue Brain Project's digital reconstruction of a rat's neocortex.

In this paper, we present a framework to study the spatial structure of noctilucent clouds formed by ice particles in the upper atmosphere at mid and high latitude during summer. We study noctilucent cloud activity in optical images taken from three different locations and under different atmospheric conditions. In order to identify and distinguish noctilucent cloud activity from other objects in the scene, we employ linear discriminant analysis (LDA) with feature vectors ranging from simple metrics to higher-order local autocorrelation (HLAC), and histogram of oriented gradients (HOG). Finally, we propose a Convolutional Neural Networks (CNN) based method for the detection of noctilucent clouds. The results clearly indicate that the CNN based approach outperforms LDA based methods used in this article. Furthermore, we outline suggestions for future research directions to establish a framework that can be used for synchronizing the optical observations from ground based camera systems with echoes measured with radar systems like EISCAT in order to obtain independent additional information on the ice clouds.

Accurate and fast traffic flow forecasting is vital in intelligent transportation system because many of the advanced features in intelligent transportation systems are based on it. However, existing methods have poor performance regarding accuracy and computational efficiency in long-term traffic flow forecasting under big data. Hence, we propose an improved Long short-term memory (LSTM) Network and its cluster computing implementation in this paper to address the above challenge. We propose a singular point probability LSTM (SDLSTM) algorithm. The method discards the units of the network according to the singular point probability during the training process and amends the SDLSTM by Autoregressive Integrated Moving Average Model (ARIMA) to achieve the accurate prediction of 24-hour traffic flow data. Furthermore, the paper designs a scheme for implementing this method through cluster computing to shorten the calculation time and improve the system's operating speed. Theoretical analysis and experimental results show that SDLSTM gains a higher accuracy rate and better stability in the long-term traffic flow forecasting compared with previous methods.

Convolutional neural networks (CNNs), inspired by biological visual cortex systems, are a powerful category of artificial neural networks that can extract the hierarchical features of raw data to greatly reduce the network parametric complexity and enhance the predicting accuracy. They are of significant interest for machine learning tasks such as computer vision, speech recognition, playing board games and medical diagnosis [1-7]. Optical neural networks offer the promise of dramatically accelerating computing speed to overcome the inherent bandwidth bottleneck of electronics. Here, we demonstrate a universal optical vector convolutional accelerator operating beyond 10 Tera-FLOPS (floating point operations per second), generating convolutions of images of 250,000 pixels with 8-bit resolution for 10 kernels simultaneously — enough for facial image recognition. We then use the same hardware to sequentially form a deep optical CNN with ten output neurons, achieving successful recognition of full 10 digits with 900 pixel handwritten digit images with 88% accuracy. Our results are based on simultaneously interleaving temporal, wavelength and spatial dimensions enabled by an integrated microcomb source. This approach is scalable and trainable to much more complex networks for demanding applications such as unmanned vehicle and real-time video recognition.

The article presents an innovative method for synthesizing radiation patterns efficiently by 1 combining the Taguchi method and neural networks, while validating the results on a 10-element 2 antenna array. The Taguchi method aims to minimize product and process variability, while neural 3 networks are used to model the relationship between antenna design parameters and radiation 4 pattern characteristics. This approach utilizes Taguchi parameters as inputs for the neural network, 5 which is then trained on a dataset generated by the Taguchi method. After training, the network is 6 validated using a real 10-element antenna array. Analytical results demonstrate that this method 7 enables efficient synthesis of radiation patterns with a significant reduction in computation time 8 compared to traditional approaches. Furthermore, validation on the antenna array confirms the 9 accuracy and robustness of the approach, showing a high correlation between predicted performances 10 by the neural network model and actual measurements on the antenna array.In summary, our article 11 highlights that the combined use of the Taguchi method and neural networks, with validation on a 12 real antenna array, offers a promising approach for efficient synthesis of antenna radiation patterns. 13 This approach combines speed, accuracy, and reliability in antenna system design.

Spiking neural networks (SNNs) promise to perform tasks currently done by classical artificial neural networks (ANNs) faster, in smaller footprints and using less energy. Neuromorphic processors are set out to revolutionize computing at a large scale, but the move to edge-computing applications calls for finely-tuned custom implementations to keep pushing towards more efficient systems. To that end, we have examined the architectural design space for executing spiking neuron models on FPGA platforms, focusing on achieving ultra-low area and power consumption. This work presents an efficient clock-driven spiking neuron architecture used for the implementation of both fully-connected cores and 2D convolutional cores, which rely on deep pipelines for synaptic processing and distributed memory for weight and neuron states. With them, we have developed an accelerator for an SNN version of the LeNet-5 network trained on the MNIST dataset. At around 5.5 slices/neuron and only 348 mW, it is able to use 33% less area and 4 times as less power per neuron as current state-of-the-art implementations while keeping low simulation step times.

The emergence of the Internet of Things (IoT) has led to the deployment of various types of sensors in many application fields, including environment monitoring, smart cities, health, industries, and others. The increasing number of connected devices has led to the creation of massive quantities of data that need to be analyzed. Typically, this data is ordered by time, as a time series. In this context, this paper presents a time series prediction model based on Recurrent Neural Networks in order to predict one step ahead. Result obtained through five Internet of Things monitoring datasets, showed that the Recurrent Neural Network obtained better performance that the prediction methods, ARIMA and SVM.

Water resource pollution, particularly in river channels, presents a grave environmental challenge that necessitates a comprehensive and systematic approach encompassing assessment, forecasting, and effective management. This article provides a comprehensive exploration of the methodology and modeling tools employed to scrutinize the process of river channel pollution due to silting, rooted in the fundamental principles of hydrodynamics and pollutant transport dynamics. The study's methodology seamlessly integrates numerical simulations with state-of-the-art neural network techniques, with a specific focus on the Physics-informed neural networks (PINN) method. This innovative approach represents a groundbreaking fusion of artificial neural networks (ANN) and physical equations, offering a more efficient and precise means of modeling a wide array of complex processes and phenomena. The proposed mathematical model, grounded in the Euler equation, has been meticulously implemented using the Ansys Fluent software package, ensuring accuracy and reliability in the computations. In a pivotal phase of the research, a thorough comparative analysis was conducted between the results derived from the PINN method and those obtained through conventional numerical approaches using the Ansys Fluent software package. The outcomes of this analysis revealed the superior performance of the PINN method, characterized by the generation of smoother pressure fluctuation profiles and significantly reduced computation time, underscoring its potential as a transformative modeling tool. The calculated data originating from this study assumes paramount significance in the ongoing battle against river sedimentation. Beyond this immediate application, these findings also serve as a valuable resource for creating predictive materials pertaining to river channel silting, thereby empowering decision-makers and environmental stakeholders with essential information. The utilization of modeling techniques to address pollution concerns in river channels holds the potential to revolutionize risk management and safeguard the integrity of our vital water resources. However, it is imperative to underscore that the effectiveness of such models hinges on ongoing monitoring and frequent data updates, ensuring that they remain aligned with real-world conditions. This research not only contributes to the enhanced understanding and proactive management of river channel pollution due to silting but also underscores the pivotal role of advanced modeling methodologies in the preservation of our invaluable water resources for present and future generations.

Object detection is a popular image processing technique, widely used in numerous applications for detecting and locating objects in images or videos. While being one of the fastest algorithms for object detection, Single-Shot Multibox Detection (SSD) networks are also computationally very demanding, which limits their usage in real-time edge applications. Even though the SSD post-processing algorithm is not the most complex segment of the overall SSD object detection network, it is still computationally demanding and can become a bottleneck with respect to processing latency and power consumption, especially in edge applications with limited resources. When using hardware accelerators to accelerate backbone CNN processing, the SSD post-processing step implemented in software can become critical for high-end applications where high frame rates are required, as this paper shows. To overcome this problem, we propose Puppis, an architecture for hardware acceleration of SSD post-processing algorithm. As experiments will show, our solution will lead to an average SSD post-processing speedup of 34.42 when compared with a software implementation. Furthermore, execution of a complete SSD network will be on average 45.39 times faster than software implementation when the proposed Puppis SSD hardware accelerator is used together with some existing CNN accelerators.

Wet granulation is a frequent process in the pharmaceutical industry. As a starting point for numerous dosage forms, the quality of the granulation not only affects subsequent production steps but also impacts the quality of the final product. It is thus crucial and economical to monitor this operation thoroughly. Here, we report on identifying different phases of a granulation process using a machine learning approach. The phases reflect the water content which in turn influences the processability and quality of the granule mass. We used two kinds of microphones and an acceleration sensor to capture acoustic emissions and vibrations. We trained convolutional neural networks (CNNs) to classify the different phases using transformed sound recordings as the input. We achieved a classification accuracy of up to 90 % using vibrational data and an accuracy of up to 97 % using the audible microphone data. Our results indicate the suitability of using audible sound and machine learning to monitor pharmaceutical processes. Moreover, since recording acoustic emissions is contactless, it readily complies with legal regulations and present Good Manufacturing Practices.

Activated sludge process is a well known method to treat municipal and industrial waste water. In this complex process, the oxygen concentration in the reactors plays a critical role in the plant efficiency. This paper proposes the use of a Long Short-Term Memory (LSTM) network to identify an input-output model suitable for the design of an oxygen concentration controller. The model is identified from easy-accessible measures collected from a real plant. This dataset covers almost a month. The performances achieved with the proposed LSTM model are compared with the ones obtained with a standard AutoRegressive model with eXogenous input (ARX). Both models catch the oscillation and the overall behaviour (ARX ρ=0.833 , LSTM ρ=0.921), but, while the ARX model fails in reaching the correct amplitude (FIT=41.20%), the LSTM presents satisfactory performance (FIT=60.56%).

To drive safely, the driver must be aware of the surroundings, pay attention to the road traffic, and be ready to adapt to new circumstances. Most studies on driving safety focus on detecting anomalies in driver behavior and monitoring the cognitive capabilities of drivers. In our study, we proposed a classifier for basic activities in driving a car, based on a similar approach that could be applied to the recognition of basic activities in daily life, that is, using electrooculographic (EOG) signals and a one-dimensional convolutional neural network (1D CNN). Our classifier achieved an accuracy of 80% for the 16 primary and secondary activities. The accuracy related to primary activities in driving, including crossroad, parking, roundabout was 97.9%, 96.8%, 97.4%, and 99.5%, respectively. The F1 score for secondary driving actions (0.99) was higher than for primary driving activities (0.93–0.94). Furthermore, using the same algorithm, it was possible to distinguish four secondary activities related to activities of daily life and secondary when driving a car.

In the last decade, the genetic engineering world has been shaken up by a relatively new genetic editing tool based on RNA-guided Nucleases (RGNs): the CRISPR/Cas9 system. Since the first report in 1987 and its characterization in 2007 as a bacterial defense mechanism, the interest and research on this system have grown exponentially. CRISPR systems provide immunity to bacteria against invading genetic material; however, with specific modifications in sequence and structure, it becomes a precise editing system that makes it possible to genetically modify almost any organism. There are diverse approaches regarding the refinement of these modifications, such as constructing more accurate nucleases, understanding the cellular context and facing the epigenetic conditions, or re-designing guide RNAs (gRNAs). Considering the critical importance for the correct CRISPR/Cas9 systems performance, our scope will emphasize in the latter approach. Hence, we present an overview of the past and the most recent guide RNA web-based design tools, highlighting their computational architecture and gRNA characteristics evolution through the years. Our study concisely explains the computational approaches that use machine learning techniques, deep neural networks, and large datasets of gRNA/target interactions to make possible both predictions and classifications directed to design, optimize, and create promising gRNAs suitable for future gene therapies.

In the realm of computer vision, semantic segmentation is the task of recognizing objects in images at the pixel level. This is done by performing a classification of each pixel. The task is complex and requires sophisticated skills and knowledge about the context to identify objects’ boundaries. The importance of semantic segmentation in many domains is undisputed. In medical diagnostics, it simplifies the early detection of pathologies, thus mitigating the possible consequences. In this work, we provide a review of the literature on deep ensemble learning models for polyp segmentation and we develop new ensembles based on convolutional neural networks and transformers. The development of an effective ensemble entails ensuring diversity between its components. To this end, we combine different models (HarDNet-MSEG, Polyp-PVT, and HSNet) trained with different data augmentation techniques, optimization methods, and learning rates, which we experimentally demonstrate to be useful to form a better ensemble. Most importantly, we introduce a new method to obtain the segmentation mask which is more suitable for combining transformers in an ensemble. In our extensive experimental evaluation, the proposed ensembles exhibit state-of-the-art performance.

In videos, the human's actions are of three-dimensional (3D) signals. These videos investigate the spatiotemporal knowledge of human behavior. The promising ability is investigated using 3D convolution neural networks (CNNs). The 3D CNNs have not yet achieved high output for their well-established two-dimensional (2D) equivalents in still photographs. Board 3D Convolutional Memory and Spatiotemporal fusion face training difficulty preventing 3D CNN from accomplishing remarkable evaluation. In this paper, we implement Hybrid Deep Learning Architecture that combines STIP and 3D CNN features to enhance the performance of 3D videos effectively. After implementation, the more detailed and deeper charting for training in each circle of space-time fusion. The training model further enhances the results after handling complicated evaluations of models. The video classification model is used in this implemented model. Intelligent 3D Network Protocol for Multimedia Data Classification using Deep Learning is introduced to further understand space-time association in human endeavors. In the implementation of the result, the well-known dataset, i.e., UCF101 to, evaluates the performance of the proposed hybrid technique. The results beat the proposed hybrid technique that substantially beats the initial 3D CNNs. The results are compared with state-of-the-art frameworks from literature for action recognition on UCF101 with an accuracy of 95%.

Imitation Learning is a discipline of Machine Learning primarily concerned with replicating observed behavior of agents known to perform well on a given task, collected in demonstration data sets. In this paper, we set out to introduce a pipeline for collecting demonstrations and training models that can produce motion plans for industrial robots. Object throwing is defined as the motivating use case. Multiple input data modalities are surveyed, and motion capture is selected as the most practicable. Two model architectures operating autoregressively are examined -- feedforward and recurrent neural networks. Trained models execute throws on a real robot successfully, and a battery of quantitative evaluation metrics is proposed, including extrapolated throw accuracy estimates. Recurrent neural networks outperform feedforward ones in most respects, with the best models having an assessed mean throw error on the order of 0.1...0.2 m at distances of 1.5...2.0 m. The data collection, pre-processing, and model training aspects of our proposed approach show promise, but further work is required in developing Cartesian motion planning tools before it is suitable for application in production.

The choice of holiday destinations is highly depended on climate considerations. Nowadays, since the effects of climate crisis are being increasingly felt, the need of accurate weather and climate services for hotels is crucial. Such a service could be beneficial for both the future planning of tourists’ activities and destinations and for hotel managers as it could help in decision making about the planning and expansion of the touristic season, due to a prediction of higher temperatures for a longer time span, thus causing increased revenue for companies in the local touristic sector. The aim of this work is to calculate predictions on climatic variables using statistical techniques as well as Artificial Intelligence (AI) for a specific area of interest utilising data from in situ meteorological station, and produce valuable and reliable localised predictions with the most cost-effective method possible. This investigation will answer the question of the most suitable prediction method for time series data from a single meteorological station that is deployed in a specific location. As a result, an accurate representation of the microclimate in a specific are is achieved. To achieve this high accuracy in situ measurements and prediction techniques are used. As prediction techniques, Seasonal Auto Regressive Integrated Moving Average (SARIMA), AI techniques like the Long-Short-Term-Memory (LSTM) Neural Network and hybrid combinations of the two are used. Variables of interest are divided in the easier to predict temperature and humidity that are more periodic and less chaotic, and the wind speed as an example of a more stochastic variable with no known seasonality and patterns. Our results show that the examined Hybrid methodology performs the best at temperature and wind speed forecasts, closely followed by the SARIMA whereas LSTM perform better overall at the humidity forecast, even after the correction of the Hybrid to the SARIMA model.

The use of convolutional neural networks (CNNs) in image classification has become the standard method of approaching computer vision problems. Here we apply pre-trained networks to classify images of non-breaking, plunging and spilling breaking waves. The CNNs are used as basic feature extractors and a classifier is then trained on top of these networks. The dynamic nature of breaking waves is exploited by using image sequences to gain extra information and improve the classification results. We also see improved classification performance in using pre-computed image features such as the optical flow between image pairs. The inclusion of the dynamic information improves the classification between breaking wave classes. We also provide corrections to the methodology from the article from which the data originates to achieve a more accurate assessment of performance.

In this article, the evolution in space and in time of the coronavirus pandemic is studied by utilizing a neural network with a self-organizing nature for the spatial analysis of data, and a fuzzy fractal method for capturing the temporal trends of the time series of the countries. Self-organizing neural networks possess the capability for clustering countries in the space domain based on their similar characteristics with respect to their coronavirus cases. In this form enabling finding the countries that are having similar behavior and thus can benefit from utilizing the same methods in fighting the virus propagation. To validate the approach, publicly available datasets of coronavirus cases worldwide have been used. In addition, a fuzzy fractal approach is utilized for the temporal analysis of time series of the countries. Then, a hybrid combination of both the self-organizing maps and the fuzzy fractal approach is proposed for efficient COVID-19 forecasting of the countries. Relevant conclusions have emerged from this study, that may be of great help in putting forward the best possible strategies in fighting the virus pandemic. A lot of the existing works concerned with the Coronavirus have look at the problem mostly from the temporal viewpoint that is of course relevant, but we strongly believe that the combination of both aspects of the problem is relevant to improve the forecasting ability. The most relevant contribution of this article is the proposal of combining neural networks with a self-organizing nature for clustering countries with high similarity and the fuzzy fractal approach for being able to forecast the times series and help in planning control actions for the Coronavirus pandemic.

In this paper, the Group Method of Data Handling (GMDH) type of neural networks is used for the inductance calculation of variable inductors. The relation between the inductance of the inductor in the linear and nonlinear regions is investigated, and parameters such as the voltage across the inductor, bias current, and ac current are taken into account. The experimental setup is used for generating the data needed for training the neural network. Over 800 experiments were conducted and were used for training and validation of the neural network results. The results are compared with the reluctance equivalent circuit method, and they show a much better accuracy. The proposed method can be used for the calculation of various magnetic components, and it is not limited to variable inductors.

In this paper, we propose methods for brain tumor detection in MRI images based on ensemble learning. We build upon prior research on ensemble methods by testing the concatenation of pre-trained models: features extracted via transfer learning are merged and segmented by classification algorithms or a stacked ensemble of those algorithms. The proposed approach achieved accuracy scores of 0.98 , outperforming a benchmark VGG-16 model. Considerations to granular computing are given in the paper as well.

As a system becomes more complex, at first, its description and analysis becomes more complicated. However, a further increase in the system’s complexity often makes this analysis simpler. A classical example is Central Limit Theorem: when we have a few independent sources of uncertainty, the resulting uncertainty is very difficult to describe, but as the number of such sources increases, the resulting distribution get close to an easy-to-analyze normal one – and indeed, normal distributions are ubiquitous. We show that such limit theorems make analysis of complex systems easier – i.e., lead to blessing of dimensionality phenomenon – for all the aspects of these systems: the corresponding transformation, the system’s uncertainty, and the desired result of the system’s analysis.

Conventionally, partial differential equations (PDE) problems are solved numerically through discretization process by using finite difference approximations. The algebraic systems generated by this process are then finalized by using an iterative method. Recently, scientists invented a short cut approach, without discretization process, to solve the PDE problems, namely by using machine learning (ML). This is potential to make scientific machine learning as a new sub-field of research. Thus, given the interest in developing ML for solving PDEs, it makes an abundance of an easy-to-use methods that allows researchers to quickly set up and solve problems. In this review paper, we discussed at least three methods for solving high dimensional of PDEs, namely PyDEns, NeuroDiffEq, and Nangs, which are all based on artificial neural networks (ANNs). ANN is one of the methods under ML which proven to be a universal estimator function. Comparison of numerical results presented in solving the classical PDEs such as heat, wave, and Poisson equations, to look at the accuracy and efficiency of the methods. The results showed that the NeuroDiffEq and Nangs algorithms performed better to solve higher dimensional of PDEs than the PyDEns.

This paper presents a quantitative approach to poetry, based on the use of several statistical measures (entropy, informational energy, N-gram, etc.) applied to a few characteristic English writings. We found that English language changes its entropy as time passes, and that entropy depends on the language used and on the author. In order to compare two similar texts, we were able to introduce a statistical method to asses the information entropy between two texts. We also introduced a method of computing the average information conveyed by a group of letters about the next letter in the text. We found a formula for computing the Shannon language entropy and we introduced the concept of N-gram informational energy of a poetry. We also constructed a neural network, which is able to generate Byron-type poetry and to analyze the information proximity to the genuine Byron poetry.

Motivated by the way animals react in nature, we introduce a fully-connected neural-network model, which incorporates the external pattern presented to the system as a fundamental tool of the recognition process. In this new scenario, in the absence of this external field, memories are not attractors inside basins of attraction, as in usual attractor neural networks, although basins may be created according to the external pattern, thus allowing storing a much larger number of memories. The key point consists in calibrating the influence of the external pattern such as to cancel the noise generated by those memories not correlated with the external pattern.We illustrate how this proposal works by including this new contribution in the standard Hopfield model, showing a significant increase in its recognition capacity (typically by a factor $10^{2}$). As an additional feature of this model, we show its ability to react promptly to changes in the external environment, a crucial attribute of living beings. This procedure can be applied to a wide variety of neural-network models to increase considerably their recognition and reaction capabilities.

State-of-the-art remote sensing scene classification methods employ different Convolutional Neural Network architectures for achieving very high classification performance. A trait shared by the majority of these methods is that the class associated with each example is ascertained by examining the activations of the last fully connected layer, and the networks are trained to minimize the cross-entropy between predictions extracted from this layer and ground-truth annotations. In this work, we extend this paradigm by introducing an additional output branch which maps the inputs to low dimensional representations, effectively extracting additional feature representations of the inputs. The proposed model imposes additional distance constrains on these representations with respect to identified class representatives, in addition to the traditional categorical cross-entropy between predictions and ground-truth. By extending the typical cross-entropy loss function with a distance learning function, our proposed approach achieves significant gains across a wide set of benchmark datasets in terms of classification, while providing additional evidence related to class membership and classification confidence.