Deep Learning is an extremely important research topic in Earth Observation. Current use-cases range from semantic image segmentation, object detection to more common problems found in computer vision such as object identification. Earth Observation is an excellent source for different types of problems and data for Machine Learning in general and Deep Learning in particular. It can be argued that both Earth Observation and Deep Learning as fields of research will benefit greatly from this recent trend of research. In this paper we take several state of the art Deep Learning network topologies and provide a detailed analysis of their performance for semantic image segmentation for building footprint detection. The dataset used is comprised of high resolution images depicting urban scenes. We focused on single model performance on simple RGB images. In most situations several methods have been applied to increase the accuracy of prediction when using deep learning such as ensembling, alternating between optimisers during training and using pretrained weights to bootstrap new models. These methods although effective, are not indicative of single model performance. Instead, in this paper, we present different topology variations of these state of the art topologies and study how these variations effect both training convergence and out of sample, single model, performance.

This concept paper gives a narrative about intelligence from insects to the human brain, showing where evolution may have been influenced by the structures in these simpler organisms. The ideas also come from the author's own cognitive model, where a number of algorithms have been developed over time and the precursor structures should be codable to some level. Through developing and trying to implement the design, ideas like separating the data from the function have become architecturally appropriate and there have been several opportunities to make the system more orthogonal. Similarly for the human brain, neural structures may work in-sync with the neural functions, or may be slightly separate from them. Each section discusses one of the neural assemblies with a potential functional result, that cover ideas such as timing or scheduling, structural intelligence and neural binding. Another aspect of self-representation or expression is interesting and may help the brain to realise higher-level functionality based on these lower-level processes.

Dysregulation of high-frequency neuronal oscillations has been implicated in the pathophysiology of schizophrenia. Chronic methamphetamine (METH) use can induce psychosis similar to paranoid schizophrenia. The current study in mice aimed to determine the effect of chronic METH treatment on ongoing and evoked neuronal oscillations. C57BL/6 mice were treated with METH or vehicle control for three weeks and implanted with extradural recording electrodes. Two weeks after the last METH injection, mice underwent three EEG recording sessions to measure ongoing and auditory-evoked gamma and beta oscillatory power in response to an acute challenge with METH (2mg/kg), the NMDA receptor antagonist MK-801 (0.3mg/kg), or saline control. A separate group of mice pretreated with METH showed significantly greater locomotor hyperactivity to an acute METH challenge, confirming long-term sensitisation. Chronic METH did not affect ongoing or evoked gamma or beta power. Acute MK-801 challenge reduced ongoing beta power whereas acute METH challenge significantly increased ongoing gamma power. Both MK-801 and METH challenge suppressed evoked gamma power. Chronic METH treatment did not modulate these acute drug effects. There were minor effects of chronic METH and acute METH and MK-801 on selected components of event-related potential (ERP) waves. In conclusion, chronic METH treatment did not exert neuroplastic effects on the regulation of cortical gamma oscillations in a manner consistent with schizophrenia, despite causing behavioural sensitisation.

With the development of data science, more and more machine learning technologies have been designed to solve complicated and challenging real-world tasks containing a large volume of data. And many significant real-world datasets contain data in the form of networks or graphs. Graph Neural Networks is one of the powerful machine learning tools, which could provide a perfect solution to processing a large amount of non-Euclidean data. And because most bio information data in bioinformatics is in the non-Euclidean domain, Graph Neural Networks could then directly be applied to solve problems in bioinformatics. Much research has been done in the field of GNN, and there are also some surveys related to GNN and its applications. However, there has been little research focusing on GNN in bioinformatics, and we think in the future we could better utilize GNN in the field of biology, so we would like to write a literature review to help take a comprehensive look at GNN and their applications in the field of bioinformatics. In this paper, we would first introduce SOTA models in Graph Neural Networks, and second, introduce their applications in bio information. And then we would provide future directions of Graph Neural Networks in bioinformatics.

Model understanding is critical in many domains, particularly those involved in high-stakes decisions, i.e., medicine, criminal justice, and autonomous driving. Explainable AI (XAI) methods are essential for working with black-box models such as Convolutional Neural Networks. This paper evaluates the traffic sign classifier of Deep Neural Network (DNN) from the Programmable Systems for Intelligence in Automobiles (PRYSTINE) project for explainability. The results of explanations were further used for the CNN PRYSTINE classifier vague kernels` compression. After all, the precision of the classifier was evaluated in different pruning scenarios. The proposed classifier performance methodology was realised by creating the original traffic sign and traffic light classification and explanation code. First, the status of the kernels of the network was evaluated for explainability. For this task, the post-hoc, local, meaningful perturbation-based forward explainable method was integrated into the model to evaluate each kernel status of the network. This method enabled distinguishing high and low-impact kernels in the CNN. Second, the vague kernels of the classifier of the last layer before the fully connected layer were excluded by withdrawing them from the network. Third, the network's precision was evaluated in different kernel compression levels. It is shown that by using the XAI approach for network kernel compression, the pruning of 5% of kernels leads only to a 1% loss in traffic sign and traffic light classification precision. The proposed methodology is crucial where execution time and processing capacity prevail.

Distortion of financial statements is recognized as one of the most important issues in the field of accounting and auditing, which is also one of the most common issues today. In this regard, the present research was conducted, in which stock exchange information was used to investigate, predict, and model accounting distortions. For this purpose, financial performance, non-financial metrics, market-based metrics and commitment, or selection items were reviewed over a 6-year period. For collecting data of distorting companies, database of the Society of Certified Public Accountants in Iran was used and the information was analyzed using data mining methods (decision tree, neural networks, and Bayesian method). The results showed that analysis of financial statements҆ information has a high accuracy in determining and identifying the distorted financial statements. Using this information, it is possible to get better acquainted with the methods of document distortion and to take necessary measures in order to control and prevent administrative violations at national and international levels. Given frequent occurrence of these violations, artificial intelligence models can be used to identify these papers.

Over the past decade, recognition of human activities (HAR) has become a vibrant field of research, in particular, the spread in our everyday lives of electronics such as mobile phones, smart cell phones, and video cameras. Furthermore, the advancement in the field of deep methodologies and other paradigms have enabled scientists to enable HAR in many areas, consisting of activities in fitness and wellness. For instance, HAR is one of many resorts to support older people through day-to-day activities to support their cognition and physicality. This study is centered on the key aspects deep learning plays in the development of HAR applications. Although numerous HAR examination studies were carried out previously, there have been no overall studies on this subject, in all the earlier studies there were only specific HAR-related subjects. A detailed review covering all the main subjects in this area is therefore essential. This study discusses the latest developments and works in HAR. It separates the methods and the advantages and disadvantages of each method group. This paper finally addresses many problems in the current HAR subject and provides recommendations for potential study.

Neural networks present the characteristics that the results are strongly dependent on the training data, the weight initialisation, and the hyper-parameters chosen. The determination of the distribution of a statistical estimator, as the Mean Squared Error (MSE) or the accuracy, is fundamental to evaluate the performance of a neural network model (NNM). For many machine learning models, as linear regression, it is possible to analytically obtain information as variance or confidence intervals on the results. Neural networks present the difficulty of being not analytically tractable due to their complexity. Therefore, it is impossible to easily estimate distributions of statistical estimators. When estimating the global performance of an NNM by estimating the MSE in a regression problem, for example, it is important to know the variance of the MSE. Bootstrap is one of the most important resampling techniques to estimate averages and variances, between other properties, of statistical estimators. In this tutorial, the application of two resampling (including bootstrap) techniques to the evaluation of neural networks’ performance is explained from both a theoretical and practical point of view. Pseudo-code of the algorithms is provided to facilitate their implementation. Computational aspects, as the training time, are discussed since resampling techniques always require to run simulations many thousands of times and, therefore, are computationally intensive. A specific version of the bootstrap algorithm is presented that allows the estimation of the distribution of a statistical estimator when dealing with an NNM in a computationally effective way. Finally, algorithms are compared on synthetically generated data to demonstrate their performance.

Accurately mapping individual tree species in densely forested environments is crucial to forest inventory. When considering only RGB images, this is a challenging task for many automatic photogrammetry processes. The main reason for that is the spectral similarity between species in RGB scenes, which can be a hindrance for most automatic methods. State-of-the-art deep learning methods could be capable of identifying tree species with an attractive cost, accuracy, and computational load in RGB images. This paper presents a deep learning-based approach to detect an important multi-use species of palm trees (Mauritia flexuosa; i.e., Buriti) on aerial RGB imagery. In South-America, this palm tree is essential for many indigenous and local communities because of its characteristics. The species is also a valuable indicator of water resources, which comes as a benefit for mapping its location. The method is based on a Convolutional Neural Network (CNN) to identify and geolocate singular tree species in a high-complexity forest environment, and considers the likelihood of every pixel in the image to be recognized as a possible tree by implementing a confidence map feature extraction. This study compares the performance of the proposed method against state-of-the-art object detection networks. For this, images from a dataset composed of 1,394 airborne scenes, where 5,334 palm-trees were manually labeled, were used. The results returned a mean absolute error (MAE) of 0.75 trees and an F1-measure of 86.9%. These results are better than both Faster R-CNN and RetinaNet considering equal experiment conditions. The proposed network provided fast solutions to detect the palm trees, with a delivered image detection of 0.073 seconds and a standard deviation of 0.002 using the GPU. In conclusion, the method presented is efficient to deal with a high-density forest scenario and can accurately map the location of single species like the M flexuosa palm tree and may be useful for future frameworks.

Identifying fake news on the media has been an important issue. This is especially true considering the wide spread of rumors on the popular social networks such as Twitter. Various kinds of techniques have been proposed for automatic rumor detection. In this work, we study the application of graph neural networks for rumor classification at a lower level, instead of applying existing neural network architectures to detect rumors. The responses to true rumors and false rumors display distinct characteristics. This suggests that it is essential to capture such interactions in an effective manner for a deep learning network to achieve better rumor detection performance. To this end we present a simplified aggregation graph neural network architecture. Experiments on publicly available Twitter datasets demonstrate that the proposed network has performance on a par with or even better than that of state-of-the-art graph convolutional networks, while significantly reducing the computational complexity.

Due to its aggressive and invasive nature glioblastoma (GBM), the most common and aggressive primary brain tumour in adults, remains almost invariably lethal. Significant advances in the last several years have elucidated much of the molecular and genetic complexities of GBM. However, GBM exhibits a vast genetic variation and a wide diversity of phenotypes that has complicated the development of effective therapeutic strategies. This complex pathogenesis makes it necessary the development of experimental models that could be used to further understand the disease, and also to provide a more realistic testing ground for potential therapies. In this report, we describe the process of transformation of primary mouse embryo astrocytes into immortalized cultures with neural stem cell characteristics, that are able to generate of GBM when injected in the brain of C57BL/6 mice, or heterotopic tumours when injected iv. Overall, our results show that oncogenic transformation is a fate for NSC if cultured for long periods in vitro. In addition, since no additional hit is necessary to induce the oncogenic transformation, our model may be used to investigate the pathogenesis of gliomagenesis and to test the effectiveness of different drugs throughout the natural history of GBM.

Long-Term Potentiation(LTP) and Long-Term Depression(LTD) are two major forms of synaptic plasticity, which are also two well-know functional and unit activities involved in high advanced central neural system(CNS) activities, like memory. But we still know little about how the advanced CNS activities are organized in the brain and in the level of organism. Based on the current understanding and experimental evidence of neurology, we propose the term “Information Circulation” to summarize the current understandings for advanced CNS activities, and we define it as separately input neural signals finally converge in different levels of CNS and interact with each other, then neural information are circulated and processed in different levels of CNS to give out orders for next body actions. This review provides a detailed description for the functional organizations of advanced CNS activities in the term of Information Circulation. This article outlines the receiving of outside stimulation and transmission of neural information, especially transmission and procession of visual bioelectrical signals, then we described neural circuits of Information Circulation in advanced CNS activities, the corresponding specificity and dynamic properties of neural circuits, different sensation linkages, and neural synchronization for information circulation to produce consciousness in CNS. In conclusion, Information Circulation is defined as an important signature involved in advanced CNS activities.

Over the years, various models have been developed in the stages of the mining process that have allowed predicting and enhancing results, but it is the breakage the variable that connects all the activities of the mining process from the point of view of costs (drilling, blasting, loading, hauling, crushing and grinding). To improve this process, we come up with an idea to develop a breakage prediction model; on the basis of the main variables involved in the drilling and blasting process. For this purpose, we design a computer model based on an Artificial Neuronal Networks (ANN), built by using the most representative variables that come into play with drilling and blasting, such as: the properties of the explosives, the geomechanical parameters of the rock mass, and the design parameters of drilling-blasting. For its experimentation and validation, we have taken the data from a copper mine as reference located in the north of Chile, because of we have the dataset of that ore deposit, which is valid and reliable to evaluate prediction problems based on ANN applied to copper mines. The ANN architecture was of the supervised type, feedforward, with 3 layers and 13 neurons in the only hidden layer, trained with the input data using a dataset with the previously mentioned variables, which then were compared with the breakage results. The model was feed backed in its learning process until it becomes perfected, and is a prediction option that can be used in future blasting of ore deposits with similar characteristics using the same representative variables. Therefore, this is a valid alternative for predicting rock breakage, given that it has been experimentally validated, and has achieved moderately reliable results, providing higher correlation coefficients than traditional models, and with the additional advantage that an ANN model provides, due to its ability to learn and recognize compiled dataset patterns. In this way, using this computer model we can obtain satisfactory results that allow us to predict breakage, providing an alternative for evaluating the costs that this entails.

The paper concludes the first research on mouth-based Emotion Recognition (ER), adopting a Transfer Learning (TL) approach. Transfer Learning results paramount for mouth-based emotion ER, because a few data sets are available, and most of them include emotional expressions simulated by actors, instead of adopting a real-world categorization. Using TL we can use fewer training data than training a whole network from scratch, thus more efficiently fine-tuning the network with emotional data and improving the convolutional neural network accuracy in the desired domain. The proposed approach aims at improving the Emotion Recognition dynamically, taking into account not only new scenarios but also modified situations with respect to the initial training phase, because the image of the mouth can be available even when the whole face is visible only in an unfavourable perspective. Typical applications include automated supervision of bedridden critical patients in an healthcare management environment, or portable applications supporting disabled users having difficulties in seeing or recognizing facial emotions. This work takes advantage from previous preliminary works on mouth-based emotion recognition using CNN deep-learning, and has the further benefit of testing and comparing a set of networks on large data sets for face-based emotion recognition well known in literature. The final result is not directly comparable with works on full-face ER, but valorizes the significance of mouth in emotion recognition, obtaining consistent performances on the visual emotion recognition domain.

In recent years, artificial neural networks (ANNs) have won numerous contests in pattern recognition, machine learning, and artificial intelligence. The basic unit of an ANN is to mimic neurons in the brain. Neuron in ANNs is expressed as f (wx+b) or f (wx). This structure does not consider the information processing capabilities of dendrites. However, recently, studies shown that dendrites participate in pre-calculation in the brain. Concretely, biological dendrites play a role in the pre-processing to the interaction information of input data. Therefore, it is time to perfect the neuron of the neural network. This paper, add dendrite processing section, presents a novel artificial neuron, according to our previous studies (CR-PNN or Gang transform). The dendrite processing section can be expressed as . Because I perfected the basic unit of ANNs— neuron, there are so many networks to try. This article gives the basic architecture for reference in future research.

The recent worldwide outbreak of the novel corona-virus (COVID-19) opened up new challenges to the research community. Artificial intelligence (AI) driven methods can be useful to predict the parameters, risks, and effects of such an epidemic. Such predictions can be helpful to control and prevent the spread of such diseases. The main challenges of applying AI is the small volume of data and the uncertain nature. Here, we propose a shallow Long short-term memory (LSTM) based neural network to predict the risk category of a country. We have used a Bayesian optimization framework to optimized and automatically design country-specific networks. We have combined the trend data and weather data together for the prediction. The results show that the proposed pipeline outperforms against state-of-the-art methods for 170 countries data and can be a useful tool for such risk categorization. The tool can be used to predict long-duration outbreak of such an epidemic such that we can take preventive steps earlier.

Topology optimization is a pioneering design method that can provide various candidates with high mechanical properties. However, the high-resolution for the optimum structures is highly desired, normally in turn leading to computationally intractable puzzle, especially for the famous Solid Isotropic Material with Penalization (SIMP) method. In this paper, an efficient and high-resolution topology optimization method is proposed based on the Super-Resolution Convolutional Neural Network (SRCNN) technique in the framework of SIMP. The SRCNN includes four processes, i.e. refining, path extraction & representation, non-linear mapping, and reconstruction. The high computational efficiency is achieved by a pooling strategy, which can balance the number of finite element analysis (FEA) and the output mesh in optimization process. To further reduce the high computational cost of 3D topology optimization problems, a combined treatment method using 2D SRCNN is built as another speeding-up strategy. A number of typical examples justify that the high-resolution topology optimization method adopting SRCNN has excellent applicability and high efficiency for 2D and 3D problems with arbitrary boundary conditions, any design domain shape, and varied load.

Equality and incomparability multi-label ranking have not been introduced to learning before. This paper proposes new native ranker neural network to address the problem of multi-label ranking including incomparable preference orders using a new activation and error functions and new architecture. Preference Neural Network PNN solves the multi-label ranking problem, where labels may have indifference preference orders or subgroups which are equally ranked. PNN is a nondeep, multiple-value neuron, single middle layer and one or more output layers network. PNN uses a novel positive smooth staircase (PSS) or smooth staircase (SS) activation function and represents preference orders and Spearman ranking correlation as objective functions. It is introduced in two types, Type A is traditional NN architecture and Type B uses expanding architecture by introducing new type of hidden neuron has multiple activation function in middle layer and duplicated output layers to reinforce the ranking by increasing the number of weights. PNN accepts single data instance as inputs and output neurons represent the number of labels and output value represents the preference value. PNN is evaluated using a new preference mining data set that contains repeated label values which have not experimented on before. SS and PS speed-up the learning and PNN outperforms five previously proposed methods for strict label ranking in terms of accurate results with high computational efficiency.

This paper presents a study and implementation of a convolutional neural network to identify and recognize humpback whale specimens from the unique patterns of their tails. Starting from a dataset composed of images of whale tails, all the phases of the process of creation and training of a neural network are detailed – from the analysis and pre-processing of images to the elaboration of predictions, using TensorFlow and Keras frameworks. Other possible alternatives are also explained when it comes to tackling this problem and the complications that have arisen during the process of developing this paper.

Geospatial Information Systems (GIS) can provide a great environment for using machine learning algorithm for spatial data such as satellite images. Integrating this functionality with artificial intelligence algorithms for analyzing spatial data enables us to predict challenging disasters such as deforestation. Deforestation as an environmental problems has been recorded the most serious threat to environmental diversity and one of the main components of land-use change. In this paper, we investigate spatial distribution of deforestation using artificial neural networks and satellite imagery. We modeled deforestation process using various factors in determining the relationship between deforestation and environmental and socioeconomic factors. Hence, for this purpose, the proximity to roads and habitats, fragmentation of the forest, height from sea level, slope, and soil type are considered in the model. In this research, we modeled land cover changes (forests) to predict deforestation using an artificial neural network due to its significant potential for the development of nonlinear complex models. The procedure involves image registration and error correction, image classification, preparing deforestation maps, determining layers, and designing a multi-layer neural network to predict deforestation. The satellite images for this study are of a region in Hong Kong which are captured from 2012 to 2016. The results of the study demonstrate that neural networks approach for predicting deforestation can be utilized and its outcomes show the areas that destroyed during the research period.

The many-body problem in quantum physics originates from the difficulty of describing the non-trivial correlations encoded in the exponential complexity of the many-body wave function. Motivated by the Giuseppe Carleo's work titled solving the quantum many-body problem with artificial neural networks [Science, 2017, 355: 602], we focus on finding the NNQS approximation of the unknown ground state of a given Hamiltonian $H$ in terms of the best relative error and explore the influences of sum, tensor product, local unitary of Hamiltonians on the best relative error. Besides, we illustrate our method with some examples.

Cognitive neuroscience is the study of how the human brain functions on tasks like decision making, language, perception and reasoning. Deep learning is a class of machine learning algorithms that use neural networks. They are designed to model the responses of neurons in the human brain. Learning can be supervised or unsupervised. Ngram token models are used extensively in language prediction. Ngrams are probabilistic models that are used in predicting the next word or token. They are a statistical model of word sequences or tokens and are called Language Models or Lms. Ngrams are essential in creating language prediction models. We are exploring a broader sandbox ecosystems enabling for AI. Specifically, around Deep learning applications on unstructured content form on the web.

We examine the use of deep learning (neural networks) to predict the movement of the S&P 500 Index using past returns of all the stocks in the index. Our analysis finds that the future direction of the S&P 500 index can be weakly predicted by the prior movements of the underlying stocks in the index. Decomposition of the prediction error indicates that most of the lack of predictability comes from randomness and only a little from nonstationarity. We believe this is the first test of S&P500 market efficiency that uses a very large information set, and it extends the domain of weak-form market efficiency tests.

Biofilms are congregations of bacteria on a surface, and they grow into obstacles for the functionalities of any device or machinery involves anything biological. Biofilms are developed through a biochemical system known as ‘Quorum Sensing’ that accounts for the chemical signaling that direct either biofilm formation or inhibition. Computational models that relate chemical and structural features of compounds to their performance properties have been used to aide in the discovery of active small molecules for many decades. These quantitative structure-activity relationship (QSAR) models are also important for predicting the activity of molecules that can have a range of effectiveness in biological systems. This study uses QSAR methodologies combined with and different machine learning algorithms to predict and assess the performance of several different compounds acting in Quorum Sensing. Through computational probing of the quorum sensing molecular interaction, new design rules can be elucidated for countering biofilms.

With the rapid development of MEMS (Micro-electro-mechanical Systems) fabrication technologies, manifolds microelectrodes with various structures and functions have been designed and fabricated for applications in biomedical research, diagnosis and treatment through electrical stimulation and electrophysiological signal recording. The flexible MEMS microelectrodes exhibit multi-aspect excellent characteristics beyond stiff microelectrodes based on silicon or SU-8, which comprising: lighter weight, smaller volume, better conforming to neural tissue and lower fabrication cost. In this paper, we mainly reviewed key technologies on flexible MEMS microelectrodes for neural interface in recent years, including: design and fabrication technology, flexible MEMS microelectrodes with fluidic channels and electrode-tissue interface modification technology for performance improvement. Furthermore, the future directions of flexible MEMS microelectrodes for neural interface were described including transparent and stretchable microelectrodes integrated with multi-aspect functions and next-generation electrode-tissue interface modifications facilitated electrode efficacy and safety during implantation. Finally, the combinations among micro fabrication techniques with biomedical engineering and nanotechnology represented by flexible MEMS microelectrodes for neural interface will open a new gate to human lives and understanding of the world.

In recent years, advancements in brain science and neuroscience have significantly influenced the field of computer science, particularly in the domain of reinforcement learning (RL). Drawing insights from neurobiology and neuropsychology, researchers have leveraged these findings to develop novel mechanisms for understanding intelligent decision-making processes in the brain. Concurrently, the emergence of quantum computing has opened new frontiers in artificial intelligence, leading to the development of quantum machine learning (QML). This study introduces a novel model that integrates Quantum Spiking Neural Networks (QSNN) and Quantum Long Short-Term Memory (QLSTM) architectures, inspired by the complex workings of the human brain. Specifically designed for reinforcement learning tasks in energy-efficient environments, our approach progresses through two distinct stages mirroring sensory and memory systems. In the initial stage, analogous to the brain’s hypothalamus, low-level information is extracted to emulate sensory data processing patterns. Subsequently, resembling the hippocampus, this information is processed at a higher level, capturing and memorizing correlated patterns. We conduct a comparative analysis of our model against existing quantum models such as Quantum Neural Networks and their classical counterparts, elucidating its unique contributions. Through empirical results, we aim to underscore the effectiveness of our approach in optimizing energy consumption in practical scenarios.

The timely diagnosis of defects at their incipient stage of formation is crucial to extend the life-cycle of technical appliances. This is the case of mechanical-related stress, either due to long aging degradation processes (e.g., corrosion) or in-operation forces (e.g., impact events), which might provoke detrimental damages, such as cracks, disbonding or delaminations, most commonly followed by the release of acoustic energy. The localization of these sources can be successfully fulfilled via adoption of Acoustic Emission (AE)-based inspection techniques through the computation of the Time of Arrival (ToA), namely the time at which the induced mechanical wave released at the occurrence of the acoustic event arrives to the acquisition unit. However, the accurate estimation of the ToA may be hampered by poor Signal-to-Noise ratios (SNRs). In these conditions, standard statistical methods typically fail. In this work, two alternative Deep Learning methods are proposed for ToA retrieval, namely a Dilated Convolutional Neural Network (DilCNN) and a Capsule Neural Network for ToA (CapsToA). These methods have the additional benefit of being portable on resource-constrained microprocessors. Their performance has been extensively studied on both synthetic and experimental data, focusing on the problem of ToA identification for the case of a metallic plate. Results show that the two novel methods can achieve localization errors which are up to 70% more precise than those yielded by conventional strategies, even when the SNR is severely compromised (i.e., down to 2 dB). Moreover, DilCNN and CapsNet have been implemented in a tiny machine learning environment and then deployed on microcontroller units, showing a negligible loss of performance with respect to offline realizations.

Multilabel learning goes beyond standard supervised learning models by associating a sample with more than one class label. Among the many techniques developed in the last decade to handle multilabel learning best approaches are those harnessing the power of ensembles and deep learners. This work proposes merging both methods by combining a set of gated recurrent units, temporal convolutional neural networks, and long short-term memory networks trained with variants of the Adam optimization approach. We examine many Adam variants, each fundamentally based on the difference between present and past gradients, with step size adjusted for each parameter. We also combine Incorporating Multiple Clustering Centers and a bootstrap-aggregated decision trees ensemble, which is shown to further boost classification performance. In addition, we provide an ablation study for assessing the performance improvement that each module of our ensemble produces. Multiple experiments on a large set of datasets representing a wide variety of multilabel tasks demonstrate the robustness of our best ensemble, which is shown to outperform the state-of-the-art. The MATLAB code for generating the best ensembles in the experimental section will be made available at https://github.com/LorisNanni.

In previous work, a quantum mathematical formalism associated an element of experience with a single sensory neuron, as a local reduction of a global mental state. In contrast to the binary objective states of neuronal polarisation/depolarisation, neuronal experience was modeled as a continuous variable, the instantaneous value of which could only be estimated statistically from an ensemble of evoked responses to stereotyped stimulus presentation. In the present work, the quantum operations formalism of energy dissipation through amplitude damping is adopted to explain how smooth evolution of conscious experience might arise from discrete spikes and discontinuous synaptic transmission between neurons.

This study aimed to learn disturbance information to effectively control multiple small unmanned surface vehicles (USVs) for marine research purposes at sea. The learned disturbance information was used to investigate the control of multiple USV swarms based on the leading–following formation technique. Numerous studies have already been conducted on swarm control using drones in the air; however, there has been a paucity of studies on swarm control in the sea. Contrary to swarm control in aviation, the fluid density of the sea is high, and the environment is constantly changing; thus, a robot system would be significantly affected by disturbances. This ultimately influences the maintenance of swarm formations and the accomplishment of missions [1,2]. Therefore, it is difficult to control numerous moving objects while maintaining a formation at sea by using the existing aerial drone swarm control technology. To solve this problem, we learned a long short-term memory (LSTM) model with time-series marine environmental data to predict maritime disturbances. Thereafter, the model was applied to a mission-based position control algorithm of each USV constituting the swarm to design a marine unmanned floating vessel swarm control system that can be applied to various marine environments. Specifically, to effectively control numerous small USVs for marine research purposes, ocean currents are learned using the LSTM algorithm, and the predicted ocean currents are used to generate a swarm USV control system for USV formations. In this study, a control system model of several small USVs equipped with two rear thrusters and a front lateral thruster was designed. The LSTM algorithm was learned using the previous ocean current data, and the velocity of the following ocean currents was predicted. The predictions were then used as a system disturbance to determine the thrust of the controller. To measure ocean currents in the sea when each USV moves, the velocity, azimuth, and position (latitude, longitude) data from the GPS mounted on the USV were used to measure the velocity and direction of the hull's movement. Further, the flow rate was measured using a flow rate sensor on a small USV. The USV movement and position were controlled using an artificial neural network-PID (ANN-PID) controller. This study entailed a comparative analysis of the designed USV model results and those generated by the simulator, including the behavior control rule of the USV swarm and the path of the actual USV swarm at sea. It was verified that the effectiveness of the USV mathematical model and behavior control rules. By comparing the movement path of the swarm USV with and without the disturbance learning algorithm and ANN-PID control algorithm applied to the designed simulator, the position error and maintenance performance of the swarm formation were analyzed, and the application results were compared.

Abstract Background and Objective: Medical Microwave Radiometry (MWR) is used to capture the thermal properties of internal tissues and has usages in breast cancer detection. Our goal in this paper is to improve classification performance and investigate automated neural architecture search methods. Methods: We investigate optimizing the weights of a weight agnostic neural network using bi-population covariance matrix adaptation evolution strategy (BIPOP-CMA-ES) once the topology is found. We compare it against a weight agnostic and cascade correlation neural network. Results: The experiments are conducted on a breast cancer dataset of 4912 patients. Our proposed weight agnostic BIPOP-CMA-ES model achieved the best performance. It obtained an F1-score of 0.9225, accuracy of 0.9219, precision of 0.9228, recall of 0.9217 and topology of 153 connections. Conclusions: The results are an indication of the potential of MWR utilizing a neural network-based diagnostic tool for cancer detection. By separating the tasks of topology search and weight training, we are able to improve the overall performance.

We propose a neural-network-based watermarking method that introduces the quantized activation function that approximates the quantization of JPEG compression. Many neural-network-based watermarking methods have been proposed. Conventional methods have acquired robustness against various attacks by introducing an attack simulation layer between the embedding network and the extraction network. The quantization process of JPEG compression was replaced by the noise addition process in the attack layer of the conventional methods. In this paper, we propose a quantized activation function that can simulate the JPEG quantization standard as it is in order to improve the robustness against the JPEG compression. Our quantized activation function consists of several hyperbolic tangent functions and is applied as an activation function for neural networks. Our network was introduced in the attack layer of ReDMark proposed by Ahmadi et al. to compare it with their method. That is, the embedding and extraction networks had the same structure. We compared the usual JPEG compressed images and the images applying the quantized activation function. The results showed that a network with quantized activation functions can approximate JPEG compression with high accuracy. We also compared the bit error rate (BER) of estimated watermarks generated by our network with those generated by ReDMark. We found that our network was able to produce estimated watermarks with lower BERs than those of ReDMark. Therefore, our network outperformed the conventional method with respect to image quality and BER.

In cutting processes tool condition affects the quality of the manufactured parts. As such an essential component to prevent unplanned downtime and to assure machining quality is having information about the state of the cutting tool. The primary function of it is to alert the operator that the tool has reached or is reaching a level of ware beyond which behavior is unreliable. In this paper the tool condition is being monitored by analyzing the electric current on the main spindle via an artificial intelligence model utilizing a LSTM neural network. In the current study the tool is monitored while working on a cylindrical raw piece made of AA6013 aluminum alloy with a custom polycrystalline diamond tool, for the purposes of monitoring the ware of these tools. Spindle current characteristics were obtained using external measuring equipment to not influence the operation of the machine included in a larger production line. As a novel approach, an artificial intelligence model based on a LSTM neural network is utilized for the analysis of the spindle current obtained during a manufacturing cycle and assessing the tool ware range in real time. The neural network was designed and trained to notice significant char-acteristics of the captured current signal. The conducted research serves as a proof of concept for use of a LSTM neural network-based model as a method of monitoring the condition of cutting tools.

The article presents the operation of the automatic pine sawn timber inspection system, which was developed at the Woodinspector company and offered commercially. The vision inspection system is used to detect various wood defects, including knots, blue stain, and mechanical damage caused by worms. Blue stain is a defect that is difficult to detect based on the color of the wood, because it can be easily confused with wood defects or dirt that do not impair its strength properties. In particular, the issues of detecting blue stain in wood, the use of artificial neural networks and improving the operation of the system in production conditions were discussed.

Abstract：The brain often switches freely between focused attention and divergent thinking, and the default mode network (DMN) is activated during brain rest. The discovery, function and characteristics of default mode network all indicate that learning does not stop when the brain "rests". Mental disease can lead to dysfunctional default mode networks, which affects learning adversely. Therefore, default mode network plays an important role in learning. Neural activities such as β-wave rhythm regulation, "subconscious" divergence mode initiation, hippocampal function and neural replay occur during default mode, which also explain that "rest" promotes learning. This paper reveals the role and neural mechanism of default mode network in learning from the neural level, discusses the essential role default mode network plays in learning, and default mode network enables rest to promote learning.

The digital orthophoto is an image with geometric accuracy and no distortion. It is acquired through a top view of the scene and finds widespread applications in map creation, planning, and related fields.This paper classifies the algorithms for digital orthophoto generation into two groups: explicit methods and implicit methods. Explicit methods rely on traditional geometric methods, obtaining geometric structure presented by explicit parameters with Multi-View Stereo (MVS) theories, as seen in our proposed Top view constrained Dense Matching (TDM). Implicit methods rely on neural rendering, obtaining implicit neural representation of scenes through the training of neural networks, as exemplified by Neural Radiance Fields (NeRF). Both of them obtain digital orthophoto by rendering from a top view perspective. In addition, this paper conducts an in-depth comparative study between explicit and implicit methods. The experiments demonstrate that both algorithms meet the measurement accuracy requirements and exhibit a similar level of quality in terms of generated results. Importantly, the explicit method shows a significant advantage in terms of efficiency, with a time consumption reduction of two orders of magnitude under our latest CUDA version TDM algorithm. Although explicit and implicit methods differ significantly in their representation forms, they share commonalities in the implementation across algorithmic stages. These findings highlight the potential advantages of explicit methods in orthophoto generation, while also providing beneficial references and practical guidance for the fast digital orthophoto generation using implicit methods.

Squash is a sport where referee decisions are essential to the game. However, referee decisions in squash are very subjective in nature. Disputes, both from the players and the audience, regularly happen because the referee made a controversial call. In this study ,we proposed to automate the referee decision process through machine learning. We trained neural networks to predict squash referee decisions using data from 400 referee decisions acquired through extensive video footage reviewing and labeling. Six positional values were extracted, including the attacking player’s position, the retreating player’s position, the ball’s position in the frame, the ball’s projected first bounce, the ball’s projected second bounce, and the attacking player’s racket head position. We calculated nine additional distance values, such as the distance between players and the distance from the attacking player's racket head to the ball's path. Models were trained on Wolfram Mathematica and Python using these values. The best Wolfram Mathematica model achieved an 86% ± 3.03% accuracy, while the best Python model performed a 0.852 ± 0.051 accuracy (85.2% ± 5.1%). These accuracies surpass 85%, demonstrating near-human performances. Our model has great potential for improvement as it's currently trained with limited data (400 referee decisions) and lacks crucial data points such as time and speed. The performance of our model is almost surely going to improve significantly with a larger training data set. Unlike human referees, machine learning models follow a consistent standard, have unlimited attention spans, and make decisions instantly. If the accuracy is improved in the future, the model can potentially serve as an extra refereeing official for both professional and amateur squash matches. Both the analysis of referee decisions in squash and proposal to automate the process using machine learning is unique to this study.

For years, effort has been devoted to establishing an effective bull breeding soundness evaluation procedure; usual research on the subject is based on BBSE (Bull Breeding Soundness Examination) methodologies, which have significant limitations in terms of their evaluation procedure, such as high cost, time-consuming and administratively difficult. This research focuses on the creation of a prediction model to supplement and/or improve the BBSE approach, through the study of two algorithms, Clustering and Artificial Neural Network (ANN), to find the optimum Machine Learning (ML) approach for our application, with an emphasis on data categorization accuracy. The tool is designed to assist veterinary medicine and farmers in identifying key factors and increasing certainty in their decision-making during the selection of bulls for breeding purposes, providing data from a limited number of factors generated from a deep pairing study of bulls. Zebu Bulls, European Bulls, and Crossbred Bulls were the general groupings. The data utilized in the model's creation (N =359) considers five variables that influence improvement decisions. This approach enhances decision-making by 12%. ANN obtained an accuracy of 90%; Precision was 97% for satisfying, 92% for unsatisfying, and 85% for Bad.

Chromatographic separation is a critical technique in the manufacturing processes of both chemical products and biopharmaceuticals. Its principle relies on exploiting the differences in distribution between the stationary and mobile phases to achieve the separation of mixtures. The precision of substance concentration during separation directly impacts the quality and usability of the final product. Therefore, the development of an effective and precise separation control technique has long been a vital concern in the field of chromatographic separation control. Currently, the employment of Simulated Moving Bed (SMB) technology for chromatographic separation, which allows for continuous feed, has been recognized as a cutting-edge technique. SMB is a continuous process technology that enhances the efficiency of adsorbents within the bed. This sequential multi-column SMB technology not only increases production capacity but also reduces the consumption of solvents and water. It is acknowledged as one of the cleanest manufacturing technologies in the biopharmaceutical industry. However, multi-column SMB involves various variables such as flow rates in multiple sections and valve switching times, rendering its system control highly complex. Unlike traditional control objectives that aim to minimize control output errors, the control objective of SMB is to achieve a specific proportion of substance concentration. Consequently, designing and implementing traditional control theories for SMB is challenging, leading to SMB systems largely being controlled based on simple PLC controls. Moreover, these controls are often applicable only to single or a few-column SMB setups, limiting the effectiveness of high-capacity applications. To achieve effective control of SMB, this study employs an adjustable intelligent fuzzy controller with a structure like an approximate neural network (NN) for SMB control research. Simulation results demonstrate that the intelligent controller effectively achieves desirable control outcomes for the SMB system.

To improve the measurement accuracy of three-dimensional rotation angle of the spherical joint, a novel approach is proposed in this study, which combines the magnetic detection by Hall sensor and surface feature identification by eddy current sensor. Firstly, a permanent magnet is embedded in the ball head of spherical joint, and Hall sensors are set and distributed in the ball socket to measure the variation of magnetic flux density when spherical joint rotating, which are related to 3D rotation angle. In order to further improve measurement accuracy and robustness, we also set grooves on the ball head and use eddy current sensors to synchronously identify the rotation angle of the ball head. After the combination of two signals is adopted, a measurement model is established using the RBF neural network by training, and realized real-time measurement of the 3D rotation angle of spherical joint. The feasibility and superiority of this method are validated through experiments. The experimental results indicated that the measurement accuracy is promoted substantially compared to the preliminary measurement scheme based on spherical coding, the average measurement error of single axis is reduced by 9'9". The root mean square errors for the measurement of 3D rotation angles in this proposed method are as follows: the pitch angle α has an error of 1'8", the yaw angle β has an error of 2'15", and the roll angle γ has an error of 29'6".

This paper deals with the analysis of the cognitive functions of the cerebral cortex or cortex in the evolutionary context. The triune brain model is presented given its notorious popularity in fields that are not strictly neuroscientific, and its limitations and inaccuracies are also shown. Although it is an outdated theory, this one represented the passage to a new era, because it really was the model of the movement towards understanding the brain in evolutionary terms. Next, the paper illustrates the different areas of the cortex and their functions, as well as the evolution of the brain and the cortex in vertebrates. To date, the expansion of the cortex with the amniotes was placed 300 million years ago, although recent studies place it in the lampreys more than 500 million years ago. Evolutionary changes in mammalian brain and cortex size are discussed, showing the evolution from three to six cell layers and trends in different groups of mammals. Finally, the most recent contributions in the field of research are illustrated, such as the role of neurogenesis and the evolution of the connectivity of the cortex.

Image denoising has been one of the important problems in the field of computer vision, and it has a wide range of practical value in many applications, such as medical image processing, image enhancement, and computational photography. Traditional image denoising methods are usually based on hand-designed features and filters, but these methods perform poorly under complex noise and image structures. In recent years, the rapid development of neural network technology has revolutionized the image-denoising task. This paper introduces the knowledge about neural networks and image denoising, explores the impact of neural networks on image denoising, and how is it possible to denoise images by neural networks. It also summarises other image-denoising methods and finally points out the challenges and problems faced by image-denoising at present. Some possible new development directions are proposed to provide new solutions for image-denoising researchers and to promote the development of the field.

All kinds of vessels consist of dozens of complex machineries with rotating parts and electric motors that operate continuously in a harsh environment with excess temperature and humidity, vibration, fatigue and load. A breakdown or malfunction in one of these machineries can significantly impact the vessel’s operation and safety and consequently, the safety of the crew and the environment. To maintain operational efficiency and seaworthiness, the shipping industry invests substantial resources in preventive maintenance and repairs. This research presents the economic and technical benefits of predictive maintenance over to traditional preventive maintenance, and repair by replacement approaches in the maritime domain. By leveraging modern technology and Artificial Intelligence, we can analyze real-time operating conditions of machinery, enabling early detection of potential damages and allowing for effective planning of future maintenance and repair activities. In this paper, we propose and develop a Convolutional Neural Network that is fed with raw vibration measurements acquired in a laboratory environment from the ball bearings of a motor. Then, we investigate whether the proposed network can accurately detect the functional state of ball bearings and categorize any possible failures present, contributing to improved maintenance practices in the shipping industry.

RBF networks are used in a variety of real-world applications such as medical data or signal processing problems. The success of these parametric models lies in the successful adaptation of their parameters using efficient computational techniques. In the current work, a method of adjusting the parameters of these networks using Grammatical Evolution is presented. Grammatical Evolution will be used to successfully discover the most promising range of parameter values and then the training of the parameter set will be achieved using a Genetic Algorithm. The new method was applied to a wide range of data fitting and classification problems, and the results were more than promising.

Snow parameters have traditionally been retrieved using discontinuous, multi-band sensors; however, continuous hyperspectral sensor are now being developed as an alternative. In this paper we investigate the performance of various sensor configurations using machine learning neural networks trained on a simulated dataset. Our results show improvements in accuracy of retrievals of snow grain size and impurity concentration for continuous hyperspectral channel configurations. Retrieval accuracy of snow albedo was found to be similar for all channel configurations.

Tracking temperature changes in certain geographic regions is a current task in modern research on Earth's climate changes. One of the global problems in solving this task is related to the large volume of measured data and the search for appropriate methods for effective determination of changes. The purpose of this research is to track climate temperature changes using a machine learning-based automated change detection method. The presented method includes training of a two-level structure of neural networks, with measured temperatures for a ten-year period of time for a certain geographical region. In the testing phase, the neural structure classifies measured temperatures for two three-year periods, before and after the ten-year time period, respectively, for the same geographic region. An algorithm was developed to visualize the studied regions by creating a map with their geographic coordinates. The classification results in the neural structure outputs are presented and analyzed as possible temperature changes. Suggestions for continuing and expanding the research in the future are discussed.

Bundle recommendation is provided to users by combining related items into a bundle, enhancing the user's shopping experience and increase the merchant's sales revenue. Despite the success of existing solutions based on Graph Neural Networks(GNN), there are still some significant challenges: (1)It is demanding to explicitly model multiple complex associations using standard graph neural networks. (2)Numerous additional nodes and edges are introduced to approximate higher-order associations. (3)The user-bundle historical interaction data is highly sparse. In this work, we propose a Global Structural Hypergraph Convolution model for Bundle Recommendation (SHCBR) to address the above problems. Specifically, we jointly incorporate multiple complex interactions between users, items, and bundles(three entities) into a relational hypergraph without introducing additional nodes and edges. Higher-order associations are already involved in the hypergraph structure which alleviate the training burden of neural networks and the dilemma of scarce data. In addition, we design a special matrix propagation rule that captures non-pairwise complex relationships between entities. Using item nodes as links, hypergraph convolution propagation between user and bundle nodes generates the learned representations. Experimental results on two real-world datasets demonstrate that SHCBR outperforms the state-of-the-art baselines by 11.07%-25.66\% on Recall and 16.81%-33.53% on NDCG. We publicly release the codes and datasets at https://github.com/Lxtdzh/SHCBR-master/.

Retailers must have accurate sales forecasts to operate their businesses efficiently and effectively and to remain competitive in the marketplace. Global forecasting models like RNNs can be a powerful tool for forecasting in retail settings, where multiple time series are often interrelated and influenced by a variety of external factors. By including covariates in a forecasting model, we can better capture the various factors that can influence sales in a retail setting. This can help improve the accuracy of our forecasts and enable better decision-making for inventory management, purchasing, and other operational decisions. In this study we investigate how the accuracy of global forecasting models is affected by the inclusion of different potential demand covariates. To ensure the significance of the study's findings, we used the M5 forecasting competition's openly accessible and well-established dataset. The results obtained from the DeepAR models, which were trained on various combinations of features including time, price, events, and IDs, suggest that individually only the features corresponding to IDs improve the baseline model. However, when all the features are used together, the best performance is achieved, indicating that the individual relevance of each feature is emphasized when the information is given jointly. Comparing the model with features to the model without features, there is an improvement of 1.76\% for MRMSSE and 6.47\% for MMASE.

Speaker recognition methods based on convolutional neural networks (CNN) have been widely used in the security field and smart wearable devices. However, the traditional CNN has a large number of hyperparameters that are difficult to be determined, which makes the model easy to fall into local optimum or even fail to converge during the training process. Intelligent algorithms such as particle swarm optimization and genetic algorithm are used to solve the above problems. However, these algorithms have poor performance compared with the current emerging meta-heuristic algorithms. In this study, the dung beetle optimized convolution neural network (DBO-CNN) is proposed to identify the speakers, which is helpful in finding suitable hyperparameters for training. By testing the dataset of 50 people, it was demonstrated that the accuracy of the model was significantly improved by using this approach. Compared with the traditional CNN and CNN optimized by other intelligent algorithms, the accuracy of DBO-CNN has increased by 0.6%~4.8%, and reached 98.3%.

: Innovative solutions are now being researched to manage the ever-increasing amount of data required to optimize the performance of internal combustion engines. Machine Learning ap-proaches have shown to be a valuable tool for signal prediction due to their real-time and cost-effective deployment. Among them, the architecture consisting of Long Short-Term Memory (LSTM) and one-dimensional Convolutional Neural Networks (1DCNN) has emerged as a highly promising and effective option for replacing the role of physical sensors. The architecture com-bines the capacity of LSTM to detect patterns and relationships in smaller segments of the signal with the ability of 1DCNN to detect patterns and relationships in larger segments of the signal. The purpose of this work is to assess the feasibility of substituting a physical device dedicated to calculating the torque supplied by a spark-ignition engine. The suggested architecture was trained and tested using signals from the field during a test campaign conducted under transient operating conditions. The results reveal that LSTM+1DCNN is particularly well suited for signal prediction with considerable variability. It constantly outperforms other architectures used for comparison, with average error percentages of less than 2%, proving the architecture's ability to replace physical sensors.

Teachability has been extensively studied under the context of making industrial robots to be programmable and reprogrammable. However, it is only recently that the artificial intelligence (AI) research community is accelerating the research works with the objective of making humanoid robots and many other robots to be teachable under the context of using natural languages. We human beings spend many years to learn knowledge and skills despite our extraordinary mental capabilities of being teachable with the use of natural languages. Therefore, if we would like to develop human-like robots such as humanoid robots, it is inevitable for us to face the issue of making future humanoid robots to be teachable with the use of natural languages as well. In this paper, we present the key details of a top-down design for achieving a teachable mind which consists of two major processes: the first one is the process which enables humanoid robots to gain innate mental capabilities of transforming incoming signals into meaningful crisp data, and the second one is the process which enables humanoid robots to gain innate mental capabilities of undertaking incremental and deep learning with the main focus of associating conceptual labels in a natural language to meaningful crisp data. These two processes consist of the two necessary and sufficient conditions for future humanoid robots to be teachable with the use of natural languages. In addition, this paper outlines a very likely new finding underlying human brain’s neural systems as well as the obvious mathematics underlying artificial deep neural networks. These outlines provide us the strong reason to separate the study of mind from the study of brain. Hopefully, the content discussed in this paper will help the AI research community to venture into the right direction which is to make future humanoid robots, non-humanoid robots, and many other systems to achieve human-like self-intelligence at cognitive level with the use of natural languages.

The popularity and development of social media has made it more and more convenient to spread rumors, and it has become especially important to detect rumor information from massive amounts of information. Most of the traditional rumor detection methods use content characteristics or propagation structure to mine rumor characteristics, ignoring the fusion characteristics of content and structure and their interaction characteristics. Therefore, a novel rumor detection method based on heterogeneous convolutional networks is proposed. Firstly, this paper constructs a heterogeneous map of joint rumor content and propagation structure to explore the interaction between content and propagation structure during rumor propagation and obtain rumor representation. On that basis, this paper uses the deep residual graph convolutional neural network to construct the content and structure interaction information of the current network propagation model. Finally, this paper uses the two datasets of Twitter15 and Twitter16 to verify the proposed method. Experimental results show that the proposed method has higher detection accuracy than the traditional rumor detection method.

This report gives a comprehensive summary of two problems about graph convolutional networks (GCNs): over-smoothing and heterophily challenges, and outlines future directions to explore.

Background Some patients with mild cognitive impairment (MCI) experience gait disturbances. However, there are few reports on the relationship between gait disturbance and cognitive function in patients with MCI. Therefore, we investigated the neural correlates of gait characteristics related to cognitive dysfunction. Methods Eighty patients diagnosed with MCI from three dementia centres in Gangwon-do, Korea were recruited for this study. We defined MCI as a Clinical Dementia Rating global score of 0.5 or higher, with a memory domain score of 0.5 or greater. The patients were classified as having either early or late MCI based on their Mini Mental Status Examination z-scores. Multiple logistic regression analysis was performed to examine the association between the gait characteristics and cognitive impairment. Analyses included variables such as age, sex, years of education, number of comorbidities, body mass index and height. Results Gait velocity, step count, step length, heel-to-heel base support, swing and stance phase duration, and support time were associated with cognitive function. A decrease in gray matter volume in the right pericalcarine area was associated with gait characteristics related to cognitive dysfunction. An increase in the curvature of gray matter in the right entorhinal, right lateral orbitofrontal, right cuneus, and right and left pars opercularis areas was also associated with gait characteristics related to cognitive dysfunction. Conclusion Since gait impairment is an important factor in determining activities of daily living in patients with mild cognitive impairment, the evaluation of gait and cognitive functions in patients with mild cognitive impairment is important.

To effectively preserve marine environments and manage endangered species, it is necessary to employ efficient, precise, and scalable solutions for environmental monitoring. Ecoacoustics provides several benefits as it enables non-intrusive, prolonged sampling of environmental sounds, making it a promising tool for conducting biodiversity surveys. However, analyzing and interpreting acoustic data can be time-consuming and often demands substantial human supervision. This challenge can be addressed by harnessing contemporary methods for automated audio signal analysis, which have exhibited remarkable performance due to advancements in deep learning research. This paper introduces a research investigation into developing an automatic computerized system to detect dolphin whistles. The proposed method utilizes a fusion of various resnet50 networks integrated with data augmentation techniques. Through extensive experiments conducted on a publically available benchmark, our findings demonstrate that our ensemble yields significant performance enhancements across all evaluated metrics. The MATLAB/PyTorch source code is freely available at: https://github.com/LorisNanni/

Object classification is a crucial task in deep learning, which involves the identification and categorization of objects in images or videos. Although humans can easily recognize common objects, such as cars, animals, or plants, performing this task on a large scale can be time-consuming and error-prone. Therefore, automating this process using neural networks can save time and effort while achieving higher accuracy. Our study focuses on the classification step of human chromosome karyotyping, an important medical procedure that helps diagnose genetic disorders. Traditionally, this task is performed manually by expert cytologists, which is a time-consuming process that requires specialized medical skills. Therefore, automating it through deep learning can be immensely useful. To accomplish this, we implemented and adapted existing preprocessing and data augmentation techniques to prepare the chromosome images for classification. We used ResNet-50 convolutional neural networks and an ensemble approach to classify the chromosomes, obtaining state-of-the-art performance in the tested dataset.

Selecting the optimal activation function for training deep neural networks has always been challenging because it significantly impacts the neural network’s performance and training speed. At this point, researchers are more likely to employ RELU than other well-known activation functions. After RELU, researchers have proposed many activation functions to improve RELU. None of them was capable of surpassing RELU as their most significant rival. SWISH outperformed RELU in several challenging experiments like classification, object detection, and tracking. Replacing RELU units with SWISH, which improves performance in many tasks. This paper proposed a new activation function surpassing Google’s brain’s SWISH function, Which we named AIF. Experiments indicate that our proposed activation function outperforms SWISH in various tasks.

Communication is a key method of expressing one's thoughts and opinions. Amongst many modalities, speech and writing are the most powerful and common forms of human communication. Analysing what and how people think has inherently been an interesting and progressive research domain. This includes bimodal sentiment analysis which is an emerging area in natural language processing (NLP) and has received a great deal of attention in recent years in a variety of areas including social opinion mining, health care, banking, and so on. At present, there are limited studies on bimodal conversational sentiment analysis as it proves to be a challenging area given the complex nature of the way humans express sentiment cues across various modalities. To address this gap, a comparison of the performance of multiple data modality models has been conducted on the MELD dataset, a widely-used dataset for benchmarking sentiment analysis within the research community. Our work then demonstrates the results of combining acoustic and linguistic representations. Lastly, our proposed neural network-based ensemble learning technique is employed over six transformer and deep learning-based models, achieving a State-Of-The-Art (SOTA) accuracy.

Our purpose is to address the biological problem of finding foundations of the organization in the collective activity among cell networks in the nervous system, at the meso/macroscale, giving rise to cognition and consciousness. But in doing so, we encounter another problem related to the interpretation of methods to assess the neural interactions and organization of the neurodynamics, because thermodynamic notions, which have precise meaning only under specific conditions, have been widely employed in these studies. The consequence is that apparently contradictory results appear in the literature, but these contradictions diminish upon the considerations of the specific circumstances of each experiment. After clarifying some of these controversial points and surveying some experimental results, we propose that a necessary condition for cognition/consciousness to emerge is to have available enough energy, or cellular activity; and a sufficient condition is the multiplicity of configurations in which cell networks can communicate, resulting in non-uniform energy distribution, the generation and dissipation of energy gradients due to the constant activity. These ideas may reveal possible fundamental principles of brain organization and how healthy activity may derive to pathological states.

We present a Deep Learning generative model specialized to work with graphs with a regular geometry. It is build on a Variational Autoencoder framework and employs Graph convolutional layers in both encoding and decoding phases. We also introduce a pooling technique (ReNN-Pool), used in the encoder, that allows to downsample graph nodes in a spatially uniform and highly interpretable way. In the decoder, a symmetrical un-pooling technique is used to retrieve the original dimensionality of graphs. Performance of the model are tested on the standard Sprite benchmark dataset, a set of 2D images of video game characters, adequately transforming images data into graphs, and on the more realistic use-case of a dataset of cylindrical-shaped graph data that describe the distributions of the energy deposited by a particle beam in a medium.

As AI models become more and more common in business and even in our daily lives, it is important to understand what the carbon impact of these models is. Recent papers have shown that this impact can be quite great, i.e., the training of a single high-end model can result in emissions of more than 500t of CO2eq. In this paper we discuss the factors that influence the carbon footprint of AI models, explore what impact different decisions have, and show how the footprint can be reduced. We also examine the footprint of different models to give a guideline on how urgent action is for different organizations.

In this work a new hydroelectric basin modelling approach is described and applied to the Pontecosi basin, Italy. Several types of data sources were used to learn the model: a number of weather stations, satellite observations, Reanalysis dataset and basin data. With the goal of predicting the water level of the basin, the model was composed by three cascade modules. Firstly, different spatial interpolation methods, such as Kriging, Radial Basis Function and Natural Neighbours, were compared and applied to interpolate the weather stations data nearby the basin area to infer the main environmental variables (air temperature, air humidity, precipitation and wind speed) in the basin area. Then, using these variables as inputs, a neural network was trained to predict the mean soil moisture concentration over the area, also to improve the low availability due to satellite orbits. Finally, a non-linear auto regressive exogenous input (NARX) model was trained to simulate the basin level with different prediction horizons, using the data from the previous modules and past basin data (water level, discharge flow rate, turbine flow rate). Accurate predictions of the basin water level were achieved within 1 to 6 hours ahead, with mean absolute errors (MAE) between 2cm and 10cm respectively.

Soil working tools, implements, and machines are inevitable in mechanized agriculture. The soil-tool/machine interaction is a multivariate, dynamic, and intricate process. The accurate interpretation, description, and modeling of a soil-machine interaction is key to providing a solution to sustainable crop production by reducing energy input, excessive soil pulverization, and compaction. The traditional method provides insight into soil-machine interaction but often provides inadequate solutions and lacks broad applicability. Computational intelligence (CI) is a comprehensive class of approaches that rely on approximate information to solve complex problems. The CI method has been extensively studied and applied in soil tillage and traction domain in recent decades. The study critically reviews the CI techniques implemented in soil-machine interactions, especially in the context of tillage, traction, and compaction. The traditional methods and their limitation are discussed. The fundamental of CI methods and a detailed overview of the most popular methods are provided. The study reviews and summarizes the 50 selected articles on soil-machine interaction studies where CI methods were employed. It discusses the strength and limitations of employed CI methods. It also suggests the emergent CI methods and future applications are discussed. The outlined study would serve as a concise reference and a quick and systematic way to understand the applicable CI methods that allow crucial farm management decision-making.

Classical religious texts remain an essential part of human culture due to their undiminished influence on the advancement of civilization. Although their entirely divine origin is questioned repeatedly, explicit or implicit quoting and adherence to their basic guidelines are fundamental in modern society. In this respect, these documents’ inner structure and linguistic style appear to be pivotal. This paper considers the topic from the standpoint of small textual patterns classified using deep learning methods, traditionally applied to analyze short textual material like tweets. We divide the considered documents into small sequential chunks imitating tweets and categorizing them, classifying an entire text. The proposed method demonstrates that the religious text collections correspond to stable ”Twitter”-like structures that adequately reflect stylistic properties. So, concise word combinations seem to be an inborn textual attribute that adequately outlines the proposed multi-source authorship. This approach differs from traditional methods of analyzing classical religious documents, which are based on the consideration and interpretation of relatively long templates. The case study consists of three famous collections of Mosaic authorship in the Old Testament (Hebrew), Pauline authorship in the New Testament (Greek), and Al-Ghazali authorship (Arabic). The obtained results go well with most previously expressed evaluations and complement them with new implications, particularly in the authorship of two famous manuscripts attributed to Al-Ghazali.

Convolutional neural network architectures have become increasingly complex, which has improved the performance slowly on well-known benchmark datasets in the recent years. In this research, we have analyzed the true need for such complexity. We have introduced G-Net light, a lightweight modified GoogleNet with improved filter count per layer to reduce feature overlaps and complexity. Additionally, by limiting the amount of pooling layers in the proposed architecture, we have exploited the skip connections to minimize the spatial information loss. The investigations on the proposed architecture are evaluated on three retinal vessel segmentation publicly available datasets. The proposed G-Net light outperforms other vessel segmentation architectures by reducing the number of trainable parameters..

A correctly obtained Land-use/land-cover (LULC) prediction map is essential to under-standing and assessing future patterns. In the study, the LULC map of Urmia/Iran in 2030 was produced using two different prediction methods CA-Markov and Artificial Neural Network (ANN). In general, the study followed a methodology consisting of three steps. In the first steps, Landsat satellite images acquired in 2000, 2010 and 2020 were classified with maximum likelihood algorithm and LULC maps were prepared for each year. In the second stage, to investigate the LULC prediction methods' validation (CA-Markov and ANN) the LULC prediction map of 2020 was produced using the LULC map of 2000 and 2010; In this step, the predicted LULC map of 2020 and the actual LULC map of 2020 were evaluated by correctness, completeness and quality indexes. Finally, The LULC map for 2030 was prepared using all two algorithms and the change map was extracted. The results show that the area of soil and vegetation decreased, and built-up regions increased during the research period. The methods validation results show that the two algorithms are much closer to each other. Nevertheless, in general, ANN has the highest completeness (96.21%) and quality (93.8%) and CA-Markov the most correctness (96.47). This study shows that the CA-Markov algorithm is most successful in predicting the future that had larger areas and a higher percentage in the region (urban and vegetation cover) and the ANN algorithm in predicting phenomena that had smaller levels with fewer percentages (soil and rock).

Optical neural networks (ONNs), or optical neuromorphic hardware accelerators, have the potential to dramatically enhance the computing power and energy efficiency of mainstream electronic processors, due to their ultra-large bandwidths of up to 10’s of terahertz together with their analog architecture that avoids the need for reading and writing data back-and-forth. Different multiplexing techniques have been demonstrated to demonstrate ONNs, amongst which wavelength-division multiplexing (WDM) techniques make sufficient use of the unique advantages of optics in terms of broad bandwidths. Here, we review recent advances in WDM-based ONNs, focusing on methods that use integrated microcombs to implement ONNs. We present results for human image processing using an optical convolution accelerator operating at 11 Tera operations per second. The open challenges and limitations of ONNs that need to be addressed for future applications are also discussed.

Bearings are widely used in various types of electrical machinery and equipment. As their core components, failures will often cause serious consequences . At present, most methods of parameter adjustment are still manual adjustment of parameters. This adjustment method is susceptible to prior knowledge and easy to fall into the local optimal solution, failing to obtain the global optimal solution and requires a lot of resources.Therefore, this paper proposes a new method of bearing fault diagnosis based on wavelet packet transform and convolutional neural network optimized by simulated annealing algorithm.The experimental results show that the method proposed in this paper has a more accurate effect in feature extraction and fault classification compared with traditional bearing fault diagnosis methods. At the same time, compared with the traditional artificial neural network parameter adjustment, this paper introduces the simulated annealing algorithm to automatically adjust the parameters of the neural network, thereby obtaining an adaptive bearing fault diagnosis method. To verify the effectiveness of the method, the Case Western Reserve University bearing database was used for testing, and the traditional intelligent bearing fault diagnosis method was compared. The results show that the method proposed in this paper has good results in bearing fault diagnosis. Provides a new way of thinking in the field of bearing fault diagnosis in parameter adjustment and fault classification algorithms

Thermal simulations are an important part in the design of electronic systems, especially as systems with high power density become common. In simulation-based design approaches, a considerable amount of time is spent by repeated simulations. In this work, we present a proof-of-concept study of the application of convolutional neural networks to accelerate those thermal simulations. The goal is not to replace standard simulation tools but to provide a method to quickly select promising samples for more detailed investigations. Based on a training set of randomly generated circuits with corresponding Finite Element solutions, the full 3D steady-state temperature field is estimated using a fully convolutional neural network. A custom network architecture is proposed which captures the long-range correlations present in heat conduction problems. We test the network on a separate dataset and find that the mean relative error is around 2 % and the typical evaluation time is 35 ms per sample ( 2 ms for evaluation, 33 ms for data transfer). The benefit of this neural-network-based approach is that, once training is completed, the network can be applied to any system within the design space spanned by the randomised training dataset (which includes different components, material properties, different positioning of components on a PCB, etc.).

The Physics Informed Neural Networks framework is applied to the understanding of the dynamics of Coronavirus of 2019. To provide the governing system of equations used by the framework, the Susceptible-Infected-Recovered-Death mathematical model is used. The study focused on finding the patterns of the dynamics of the disease which involves predicting the infection rate, recovery rate and death rate; thus predicting the active infections, total recovered, susceptible and deceased at any required time. The study used data that was collected on the dynamics of COVID-19 from the Kingdom of Eswatini between March 2020 and September 2021. The obtained results showed less errors thus making highly accurate predictions.

One of the most dangerous diseases that threaten people is Cancer. Cancer if diagnosed in earlier stages can be eradicated with its life threatening consequences. In addition, accuracy in prediction plays a major role. Hence, developing a reliable model that contributes much towards the medical community in early diagnosis of Biopsy images with perfect accuracy come to the scenario. The article aims towards development of better predictive models using multi-variate data and high-resolution diagnostic tools in clinical cancer research. This paper proposes the social spider optimization (SSO) algorithm tuned neural network to classify microscopic biopsy images of cancer. The significance of the proposed model relies on the effective tuning of the weights of the NN classifier by the SSO algorithm. The performance of the proposed strategy is analysed with the performance metrics, such as accuracy, sensitivity, specificity, and MCC measures, and are attained to be 95.9181%, 94.2515%, 97.125%, and 97.68% respectively, which shows the effectiveness of the proposed method in effective cancer disease diagnosis.

Convolutional Neural Networks (CNNs) have gained prominence in the research literature on image classification over the last decade. One shortcoming of CNNs, however, is their lack of generalizability and tendency to overfit when presented with small training sets. Augmentation directly confronts this problem by generating new data points providing additional information. In this paper, we investigate the performance of more than ten different sets of data augmentation methods, with two novel approaches proposed here: one based on the Discrete Wavelet Transform and the other on the Constant-Q Gabor transform. Pretrained ResNet50 networks are finetuned on each augmentation method. Combinations of these networks are evaluated and compared across three benchmark data sets of images representing diverse problems and collected by instruments that capture information at different scales: a virus data set, a bark data set, and a LIGO glitches data set. Experiments demonstrate the superiority of this approach. The best ensemble proposed in this work achieves state-of-the-art performance across all three data sets. This result shows that varying data augmentation is a feasible way for building an ensemble of classifiers for image classification (code available at https://github.com/LorisNanni).

Explaining the emergence of behavior and understanding on the basis of neuronal mechanisms is still elusive. One renowned proposal is the Free Energy Principle (FEP), which uses an information-theoretic framework derived from thermodynamic considerations to describe how behavior and understanding would emerge. FEP starts from a whole organism approach, based on mental states and phenomena, mapping them into the neuronal substrate. An alternative approach, the Energy Homeostasis Principle (EHP), initiates a similar explanatory effort, but starting from single neuron phenomena and building up to the whole organism’s behavior and understanding. In this work, we develop the EHP as an alternative but complementary vision to FEP and try to explain how behavior and understanding would emerge from the local requirements of the neurons. Based on EHP and a strict naturalist approach that sees living beings as physical and deterministic systems, we explain scenarios where learning would emerge without the need for volition or goals. Given these starting points, we state several considerations of how we see the nervous system, particularly the role of function, purpose, and the conception of goal-oriented behaviors. We problematize these conceptions, giving an alternative teleology-free framework in which behavior and, ultimately, understanding would still emerge. We reinterpret neural processing explaining basic learning situations up to simple anticipatory behavior. Finally, we end the work with an evolutionary perspective of how this non-goal-oriented behavior appears. We acknowledge that in the current form of our proposal, we are still far from explaining the emergence of understanding. Still, we set the ground for an alternative neuron-based framework to ultimately explain understanding.

Surrogate model based optimization method is widely-used to accelerate the design and optimization process~\cite{marler2004survey}. The input of regression model used in the surrogate model are numbers, which requires users to parametrize the geometries. In this paper, a new parameterization-free surrogate model is introduced and its corresponding uncertainty quantification and sensitivity analysis method are discussed. The input of new surrogate model methods is surface mesh of simulation domain. \Gls{gnns} is used to extract geometric information, and \Gls{cnns} is used to predict contours. This framework bypasses parameterization,as a consequence, uncertainties introduced by manual parameterization is reduced. However, such changes compared with conventional surrogate model methods impose great challenge on uncertainties quantification and sensitivity analysis. Uncertainties quantification in this paper means the error bar of prediction results, which is calculated by Gaussian Process Regression method in current surrogate method. In this paper, a new quantification method achieved by Kullback-Leibler divergence (KLD) is introduced. And the sensitivity analysis is conducted by Automatic Differentiation, which gives a Jacobian matrix of inputs. The method and analysis mentioned above are demonstrated by a low-pressure steam turbine rotator and its exhaust system.

Creating intelligent systems capable of recognizing emotions is a difficult task, especially when looking at emotions in animals. This paper describes the process of designing a “proof of concept” system to recognize emotions in horses. This system is formed by two elements, a detector and a model. The detector is a fast region-based convolutional neural network that detects horses in an image. The model is a convolutional neural network that predicts the emotions of those horses. These two elements were trained with multiple images of horses until they achieved high accuracy in their tasks. 400 images of horses were collected and labeled to train both the detector and the model while 80 were used to validate the system. Once the two components were validated, they were combined into a testable system that would detect equine emotions based on established behavioral ethograms indicating emotional affect through head, neck, ear, muzzle and eye position. The system showed an accuracy of between 69% and 74% on the validation set, demonstrating that it is possible to predict emotions in animals using autonomous intelligent systems. Such a system has multiple applications including further studies in the growing field of animal emotions as well as in the veterinary field to determine the physical welfare of horses or other livestock.

Abstract: The Hanqu Joint Station of the Dingbian Oil Production Plant of Yanchang Oilfield Co., Ltd is located at the edge of the desert in northern China. the bad field conditions and strong sandstorm, the hardware of computer monitoring system of Joint Station is often damaged. At the same time, the core equipment of the joint station three-phase separator oil chamber liquid level is hard to achieve high precision constant value control, the general control algorithm is difficult to meet the control requirements.This paper proposed a design scheme of a oilfield joint station computer monitoring system based on the Siemens S7-300 PLC, the hardware of the monitoring system adopts the redundancy scheme of dual monitoring computers, dual programmable logic controllers (PLCs), and dual industrial Ethernet. The BP neural network PID control algorithm was used to realize constant value control of the oil chamber liquid level of the three-phase separator of the core equipment of the joint station，and realized high control precision. The monitoring system could well adapt to the harsh environment of the scene, and showed high reliability and efficiency.

We confirm that energy dissipation weighting provides the most accurate approach to determining the effective hydraulic conductivity (Keff) of a binary K grid. A deep learning algorithm (UNET) can infer Keff with extremely high accuracy (R2 > 0.99). The UNET architecture could be trained to infer the energy dissipation weighting pattern from an image of the K distribution with high fidelity, although it was less accurate for cases with highly localized structures that controlled flow. Furthermore, the UNET architecture learned to infer the energy dissipation weighting even if it was not trained on this information directly. However, the weights were represented within the UNET in a way that was not immediately interpretable by a human user. This reiterates the idea that even if ML/DL algorithms are trained to make some hydrologic predictions accurately, they must be designed and trained to provide each user-required output if their results are to be used to improve our understanding of hydrologic systems most effectively.

A recent study from our lab revealed that inhibition of cyclooxygenase-2 exclusively reduces the level of PGE2 amongst the prostanoids and hamper the normal development of several structures, strikingly the cranial vault, in chick embryos. In order to unearth the mechanism behind the deviant development of cranial features, the expression pattern of various factors that are known to influence the cranial neural crest cell (CNCC) migration were checked in chick embryo after inhibiting the COX-2 activity using etoricoxib. The compromised level of cell adhesion molecules and their upstream regulators, namely CDH1, CDH2, MSX1, and TGF-β, observed in the etoricoxib treated embryos indicate that COX-2, through its downstream effector PGE2, regulates the expression of these factors perhaps to aid the migration of CNCC. The histological features and levels of FoxD3 as well as PCNA further consolidates the role of COX-2 in migration and survival of CNCC in developing embryo. The results of the current study indicate that the COX-2 plays a pivotal role in orchestrating the proliferation and migration of CNCC during embryonic development of chick.

The main bottleneck when performing computational fluid dynamics (CFD) simulations of combustion systems is the computation and integration of the highly non-linear and stiff chemical source terms. In recent times, machine learning has emerged as a promising tool to accelerate combustion chemistry, involving the use of regression models to predict the chemical source terms as functions of the thermochemical state of the system. However, combustion is a highly nonlinear phenomenon, and this often leads to divergence from the true solution when the neural network representation of chemical kinetics is integrated in time. This is because these approaches minimize the error during training without guaranteeing successful integration with ordinary differential equation (ODE) solvers. In this work, a novel neural ODE approach to combustion modeling, ChemNODE, is developed to address this issue. The source terms predicted by the neural network are integrated during training, and by backpropagating errors through the ODE solver, the neural network weights are adjusted accordingly to minimize the difference between the predicted and actual ODE solutions. It is shown that even when the dimensionality of the thermochemical manifold is trimmed to remove redundant species, the proposed approach accurately captures the correct physical behavior and reproduces the results obtained using the full chemical kinetic mechanism.

The great importance of estimating dissolved oxygen (DO) dictates utilizing proper evaluative models. In this work, a multi-layer perceptron (MLP) network is trained by three capable metaheuristic algorithms, namely multi-verse optimizer (MVO), black hole algorithm (BHA), and shuffled complex evolution (SCE) for predicting the DO using the data of the Klamath River Station, Oregon, US. The records (DO, water temperature, pH, and specific conductance) belonging to the water years 2015 - 2018 (USGS) are used for pattern analysis. The results of this process showed that all three hybrid models could properly infer the DO behavior. However, the BHA and SCE accomplished this task by simpler configurations. Next, the generalization ability of the developed patterns is tested using the data of the 2019 water year. Referring to the calculated mean absolute errors of 1.0161, 1.1997, and 1.0122, as well as Pearson correlation coefficients of 0.8741, 0.8453, and 0.8775, the MLPs trained by the MVO and SCE perform better than the BHA. Therefore, these two hybrids (i.e., the MLP-MVO and MLP-SCE) can be satisfactorily used for future applications.

The evolution in imaging technologies and artificial intelligence algorithms, coupled with improvements in UAV technology, has enabled the use of unmanned aircraft in a wide range of applications. The feasibility of this kind of approach for cattle monitoring has been demonstrated by several studies, but practical use is still challenging due to the particular characteristics of this application, such as the need to track mobile targets and the extensive areas that need to be covered in most cases. The objective of this study was to investigate the feasibility of using a tilted angle to increase the area covered by each image. Deep Convolutional Neural Networks (Xception architecture) were used to generate the models for the experiments, which covered aspects like ideal input dimensions, effect of the distance between animals and sensor, effect of classification error on the overall detection process, and impact of physical obstacles on the accuracy of the model. Experimental results indicate that oblique images can be successfully used under certain conditions, but some practical limitations need to be addressed in order to make this approach appealing.

We recently reported that refocusing attention between temporal and spatial tasks becomes more difficult with increasing age, which could impair daily activities such as driving (Callaghan et al., 2017). Here we investigated the extent to which difficulties in refocusing attention extend to naturalistic settings such as simulated driving. 118 participants in five age groups (18-30; 40-49; 50-59; 60-69; 70-91 years) were compared during simulated driving, where they switched from a spatially focal yet temporally complex task (braking due to traffic ahead) to a spatially more distributed task (reading a motorway road sign). Sequential-Task (switching) performance was compared to Single-Task performance (road sign only) to calculate age-related switch-costs. Electroencephalography was recorded in 34 participants (17 in the 18-30 and 17 in the 60+ years groups) to explore age-related changes in the neural oscillatory signatures of refocusing attention while driving. We indeed observed age-related impairments in attentional refocusing, evidenced by increased switch-costs in response times and by deficient modulation of theta and alpha frequencies. Our findings highlight virtual reality (VR) and Neuro-VR as important methodologies for future psychological and gerontological research.

Myocarditis is the form of an inflammation of the middle layer of the heart wall which is caused by a viral infection and can affect the heart muscle and its electrical system. It has remained as one of the most challenging diagnoses in cardiology. Myocardial is the prime cause of unexpected death in approximately 20% of adults less than 40 years of age. Cardiac MRI (CMR) has been considered as a noninvasive and golden standard diagnostic tool for suspected myocarditis and plays an indispensable role in diagnosing various cardiac diseases. However, the performance of CMR is heavily dependent on the clinical presentation and non-specific features such as chest pain, arrhythmia, and heart failure. Besides, other imaging factors like artifacts, technical errors, pulse sequence, acquisition parameters, contrast agent dose, and more importantly qualitatively visual interpretation can affect the result of the diagnosis. This paper introduces a new deep learning-based model called Convolutional Neural Network-Clustering (CNN-KCL) to diagnose the Myocarditis. The hybrid CNN-KCL method performs the early and accurate diagnosis of Myocarditis. To the best-of-our-knowledge, a Convolutional neural network has never been used before for the diagnosis of Myocarditis. In this study, we used 47 subjects to diagnose myocarditis patients from Tehran's Omid Hospital. The total number of data examined is 10425. Our results demonstrate that CNN-KCL achieves 92.3% in terms of diagnosis myocarditis prediction accuracy which is significantly better than those reported in previous studies.

The article’s goal is to overview challenges and problems on the way from the state of the art CUDA accelerated neural networks code to multi-GPU code. For this purpose, the authors describe the journey of porting the existing in the GitHub, fully-featured CUDA accelerated Darknet engine to OpenCL. The article presents lessons learned and the techniques that were put in place to make this port happen. There are few other implementations on the GitHub that leverage the OpenCL standard, and a few have tried to port Darknet as well. Darknet is a well known convolutional neural network (CNN) framework. The authors of this article investigated all aspects of the porting and achieved the fully-featured Darknet engine on OpenCL. The effort was focused not only on the classification with the use of YOLO1, YOLO2, and YOLO3 CNN models. They also covered other aspects, such as training neural networks, and benchmarks to look for the weak points in the implementation. The GPU computing code substantially improves Darknet computing time compared to the standard CPU version by using underused hardware in existing systems. If the system is OpenCL-based, then it is practically hardware independent. In this article, the authors report comparisons of the computation and training performance compared to the existing CUDA-based Darknet engine in the various computers, including single board computers, and, different CNN use-cases. The authors found that the OpenCL version could perform as fast as the CUDA version in the compute aspect, but it is slower in memory transfer between RAM (CPU memory) and VRAM (GPU memory). It depends on the quality of OpenCL implementation only. Moreover, loosening hardware requirements by the OpenCL Darknet can boost applications of DNN, especially in the energy-sensitive applications of Artificial Intelligence (AI) and Machine Learning (ML).

In this paper, we presented a method of retrieving sea surface wind speed from Sentinel-1 synthetic aperture radar (SAR) horizontal-horizontal (HH) polarization data in extra-wide mode, which have been extensively acquired over the Arctic for sea ice monitoring. In contrast to the conventional algorithm, i.e., using a geophysical model function (GMF) to retrieve sea surface wind by spaceborne SAR, we introduced an alternative method based on physical model guided neural network. Parameters of SAR normalized radar cross section, incidence angle, and wind direction are used as the inputs of the backward propagation (BP) neural network, and the output is the sea surface wind speed. The network is developed based on more than 11,000 HH-polarized EW images acquired in the marginal ice zone (MIZ) of the Arctic and their collocations with scatterometer measurements. Verification of the neural network based on the testing dataset yields a bias of 0.23 m/s and a root mean square error (RMSE) of 1.25 m/s compared to the scatterometer wind speed. Further comparison of the SAR retrieved sea surface wind speed with independent buoy measurements shows a bias and RMSE of 0.12 m/s and 1.42 m/s, respectively. We also analyzed the uncertainty of retrieval when the wind direction data of a reanalysis model are used as inputs to the neural network. By combining the detected sea ice cover information based on the EW data, one can expect to derive simultaneously sea ice and marine-meteorological parameters by spaceborne SAR in a high spatial resolution in the Arctic.

Recently, various relations and criteria have been presented to establish a proper relationship between control systems and control Global Positioning System (GPS)-intelligent buoy system. Given the importance of controlling the position of buoys and the construction of intelligent systems, in this paper, dynamic system modeling is applied to position marine buoys through the improved neural network with a backstepping technique. This study aims at developing a novel controller based on adaptive fuzzy neural network to optimally track the dynamically positioned vehicle on water with unavailable velocities and unidentified control parameters. In order to model the network with the proposed technique, uncertainties and the unwanted disturbances are studied in the neural network. The presented study aims at developing a neural controlling which applies the vectorial back-stepping technique to the surface ships, which have been dynamically positioned with undetermined disturbances and ambivalences. Moreover, the objective function is to minimize the output error for the neural network (NN) based on closed-loop system. The most important feature of the proposed model for the positioning buoys is its independence from comparative knowledge or information on the dynamics and the unwanted disturbances of ships. The numerical and obtained consequences demonstrate that the controller system can adjust the routes and the position of the buoys to the desired objective with relatively few position errors.

Periodic travelling waves are observed in various brain activities including visual, motor, language, sleep, and so on. There are several neural field models describing periodic waves assuming nonlocal interaction and, possibly, inhibition, time delay, or some other properties. In this work we study the influence of asymmetric connectivity functions and of time delay on the emergence of periodic waves and on their properties. Nonlinear wave dynamics is studied, including modulated and aperiodic waves. Multiplicity of waves for the same values of parameters is observed. External stimulation in order to restore wave propagation in a damaged tissue is discussed.

This paper focuses on quantifying the uncertainty in the specific absorption rate values of brain induced by the uncertain positions of the electroencephalography electrodes placed on patient's scalp. To avoid running a large number of simulations, an artificial neural network architecture for uncertainty quantification involving high-dimensional data is proposed in this paper. The proposed method is demonstrated to be an attractive alternative to conventional uncertainty quantification methods because of its considerable advantage in the computational expense and speed.

Complete digraphs are referred to in the combinatorics literature as tournaments. We consider a family of semi-simplicial complexes, that we refer to as ``tournaplexes'', whose simplices are tournaments. In particular, given a digraph G, we associate with it a ``flag tournaplex'' which is a tournaplex containing the directed flag complex of G, but also the geometric realisation of cliques that are not directed. We define several types of filtrations on tournaplexes, and exploiting persistent homology, we observe that flag tournaplexes provide finer means of distinguishing graph dynamics than the directed flag complex. We then demonstrate the power of these ideas by applying them to graph data arising from the Blue Brain Project's digital reconstruction of a rat's neocortex.

Water quality forecasting is increasingly significant for agricultural management and environmental protection. Enormous amounts of water quality data are increasingly being collected by advanced sensors, which leads to an interest in using data-driven models for predicting trends in water quality. However, the unpredictable background noises introduced during water quality monitoring seriously degrade the performance of those models. Meanwhile, artificial neural networks (ANN) with feed-forward architecture lack the capability of maintaining and utilizing the accumulated temporal information, which leads to biased predictions in processing time series data. Hence, we propose a water quality predictive model based on a combination of Kernal Principal Component Analysis (kPCA) and Recurrent Neural Network (RNN) to forecast the trend of dissolved oxygen. Water quality variables are reconstructed based on kPCA method, which aims to reduce the noise from the raw sensory data and preserve actionable information. With the RNN's recurrent connections, our model can make use of the previous information in predicting the trend in the future. Data collected from Burnett River, Australia was applied to evaluate our kPCA-RNN model. The kPCA-RNN model achieved R2 scores up to 0.908, 0.823 and 0.671 for predicting the concentration of dissolved oxygen in the upcoming 1, 2 and 3 hours, respectively. Compared to current data-driven methods like ANN and SVR, the predictive accuracy of the kPCA-RNN model was at least 8 %, 17 % and 21 % better than the comparative models in these 3 cases. The study demonstrates the effectiveness of the kPAC-RNN modeling technique in predicting water quality variables with noisy sensory data.

Speech signals are degraded in real-life environments, product of background noise or other factors. The processing of such signals for voice recognition and voice analysis systems presents important challenges. One of the conditions that make adverse quality difficult to handle in those systems is reverberation, produced by sound wave reflections that travel from the source to the microphone in multiple directions.To enhance signals in such adverse conditions, several deep learning-based methods have been proposed and proven to be effective. Recently, recurrent neural networks, especially those with long and short-term memory (LSTM), have presented surprising results in tasks related to time-dependent processing of signals, such as speech. One of the most challenging aspects of LSTM networks is the high computational cost of the training procedure, which has limited extended experimentation in several cases. In this work, we present a proposal to evaluate the hybrid models of neural networks to learn different reverberation conditions without any previous information. The results show that some combination of LSTM and perceptron layers produce good results in comparison to those from pure LSTM networks, given a fixed number of layers. The evaluation has been made based on quality measurements of the signal's spectrum, training time of the networks and statistical validation of results. Results help to affirm the fact that hybrid networks represent an important solution for speech signal enhancement, with advantages in efficiency, but without a significan drop in quality.

In this paper, we present a framework to study the spatial structure of noctilucent clouds formed by ice particles in the upper atmosphere at mid and high latitude during summer. We study noctilucent cloud activity in optical images taken from three different locations and under different atmospheric conditions. In order to identify and distinguish noctilucent cloud activity from other objects in the scene, we employ linear discriminant analysis (LDA) with feature vectors ranging from simple metrics to higher-order local autocorrelation (HLAC), and histogram of oriented gradients (HOG). Finally, we propose a Convolutional Neural Networks (CNN) based method for the detection of noctilucent clouds. The results clearly indicate that the CNN based approach outperforms LDA based methods used in this article. Furthermore, we outline suggestions for future research directions to establish a framework that can be used for synchronizing the optical observations from ground based camera systems with echoes measured with radar systems like EISCAT in order to obtain independent additional information on the ice clouds.

Automated weeding is an important research area in agrorobotics. Weeds can be removed mechanically or with the precise usage of herbicides. Deep Learning techniques achieved state of the art results in many computer vision tasks, however their deployment on low-cost mobile computers is still challenging. These paper present an advanced version of the system presented in [1]. The described system contains several novelties, compared both with its previous version and related work. It is a part of a project of the automatic weeding machine, developed by Warsaw University of Technology and MCMS Warka Ltd. The obtained model reaches satisfying accuracy at over 10~FPS on the Raspberry Pi 3B+ computer. It was tested for four different plant species at different growth stadiums and lighting conditions. The system performing semantic segmentation is based on Convolutional Neural Networks. Its custom architecture mixes U-Net, MobileNets, DenseNet and ResNet concepts. Amount of needed manual ground truth labels was significantly decreased by the usage of knowledge distillation process, learning final model to mimic an ensemble of complex models on the large database of unlabeled data. Further decrease of the inference time was obtained by two custom modifications: in the usage of separable convolutions in DenseNet block and in the number of channels in each layer. In the authors’ opinion, described novelties can be easily transferred to other agrorobotics tasks.

Boiler efficiency is called to some extent of total thermal energy which can be recovered from the fuel. Boiler efficiency losses are due to four major factors: the dry gas flux, the latent heat of steam in the flue gas, the combustion loss or the loss of unburned fuel, radiation and convection losses. In this research, the thermal behavior of boilers in gas refinery facilities is studied and their efficiency and their losses are calculated. The main part of this research is comprised of analyzing the effect of various parameters on efficiency such as excess air, fuel moisture, air humidity, fuel and air temperature, the temperature of combustion gases, and thermal value of the fuel. Based on the obtained results, it is possible to analyze and make recommendations for optimizing boilers in the gas refinery complex using response-surface method (RSM).

Drosophila circadian circuit is one of the best described neural circuits but is complex enough to obscure our understanding of how it actually works. Animals’ rhythmic behavior, the seemingly simple outcome of their internal clocks, relies on the interaction of heterogeneous clock neurons that are spread across the brain. Direct observations of their coordinated network interactions can bring us forward in understanding the circuit. The current challenge is to observe activity of each of these neurons over a long span of time –hours to days– in live animals. Here we review the progress in circadian circuit interrogation powered by in vivo calcium imaging.

Layout estimation is a challenge of segmenting a cluttered room image into floor, walls and ceiling. We applied Double refinement network proved to be efficient in the depth estimation to generate heat maps for room key points and edges. Our method is the first not using encoder-decoder architecture for the room layout estimation. ResNet50 was utilized as a backbone for the network instead of VGG16 commonly used for the task, allowing the network to be more compact and faster. We designed a special layout score function and layout ranking algorithm for key points and edges output. Our method achieved the lowest pixel and corner errors on the LSUN data set. The input image resolution is 224*224.

With the increase of the Artificial Neural Network (ANN), machine learning has taken a forceful twist in recent times. One of the most spectacular kinds of ANN design is the Convolutional Neural Network (CNN). The Convolutional Neural Network (CNN) is a technology that mixes artificial neural networks and up to date deep learning strategies. In deep learning, Convolutional Neural Network is at the center of spectacular advances. This artificial neural network has been applied to several image recognition tasks for decades and attracted the eye of the researchers of the many countries in recent years as the CNN has shown promising performances in several computer vision and machine learning tasks. This paper describes the underlying architecture and various applications of Convolutional Neural Network.

Accurate and fast traffic flow forecasting is vital in intelligent transportation system because many of the advanced features in intelligent transportation systems are based on it. However, existing methods have poor performance regarding accuracy and computational efficiency in long-term traffic flow forecasting under big data. Hence, we propose an improved Long short-term memory (LSTM) Network and its cluster computing implementation in this paper to address the above challenge. We propose a singular point probability LSTM (SDLSTM) algorithm. The method discards the units of the network according to the singular point probability during the training process and amends the SDLSTM by Autoregressive Integrated Moving Average Model (ARIMA) to achieve the accurate prediction of 24-hour traffic flow data. Furthermore, the paper designs a scheme for implementing this method through cluster computing to shorten the calculation time and improve the system's operating speed. Theoretical analysis and experimental results show that SDLSTM gains a higher accuracy rate and better stability in the long-term traffic flow forecasting compared with previous methods.

In recent years, data providers are generating and streaming a large number of images. More particularly, processing images that contain faces have received great attention due to its numerous applications, such as entertainment and social media apps. The enormous amount of images shared on these applications presents serious challenges and requires massive computing resources to ensure efficient data processing. However, images are subject to a wide range of distortions in real application scenarios during the processing, transmission, sharing, or combination of many factors. So, there is a need to guarantee acceptable delivery content, even though some distorted images do not have access to their original version. In this paper, we present a framework developed to estimate the images' quality while processing a large number of images in real-time. Our quality evaluation is measured using an integration of a deep network with random forests. In addition, a face alignment metric is used to assess the facial features. Experimental results have been conducted on two artificially distorted benchmark datasets, LIVE and TID2013. We show that our proposed approach outperforms the state-of-art methods, having a Pearson Correlation Coefficient (PCC) and Spearman Rank Order Correlation Correlation Coefficient (SROCC) with subjective human scores of almost 0.942 and 0.931 while minimizing the processing time from 4.8ms to 1.8ms.

We propose a deep learning algorithm that directly compensates for luminance degradation owing to the deterioration of organic light emitting diode (OLED) devices to address the burn-in phenomenon of OLED displays. Conventional compensation circuits are encumbered by a high cost of development and manufacturing processes owing to their complexity. However, given that deep learning algorithms are typically mounted on a system on chip (SoC), the complexity of the circuit design is reduced, and the circuit can be reused by re-learning only the changed characteristics of the new pixel device. The proposed approach comprises deep feature generation and multi-stream self-attention, which decipher the importance of the variables, and the correlation between burn-in-related variables. It also utilizes a deep neural network that identifies the nonlinear relationship between the extracted features and luminance degradation. Thereafter, the luminance degradation is estimated from the burn-in-related variables, and the burn-in phenomenon can be addressed by compensating for the luminance degradation. The experimental results revealed that compensation was successfully achieved within an error range of 4.56%, and demonstrate the potential of a new approach that can mitigate the burn-in phenomenon by directly compensating for pixel-level luminance deviation.