Intrinsically Disordered Proteins (IDPs) play crucial roles in signal transduction, cell differentiation, and transcriptional regulation. Many disrupted genes associated with mental disorders are identified as IDPs, and emerging evidence suggests the functional role of IDPs in neuropsychiatric disorders. However, few studies comprehensively explore the functional association between protein disorder properties and different mental diseases. To address this gap, we collected disrupted gene sets for seven mental diseases (MDD, SCZ, BP, ID, AD, ADHD, ASD) and a control gene set from normal brains using DisGeNET and GeneCards databases. The state-of-the-art predictor IUPred2A was then utilized to calculate the disorder properties , followed by a thorough comparison between the disordered proteins in mental diseases and healthy controls. Functional enrichment analysis and protein-protein interaction networks were conducted to investigate the IDPs’ functional roles in psychiatric diseases. Additionally, we demonstrated the role of IDPs in protein binding formations through a case study of alpha-synuclein using protein docking. Our findings reveal that differentially expressed proteins in mental disorders, especially in ASD and ADHD, exhibit more IDPs than those in healthy controls. IDPs in psychiatric disorders are significantly enriched in neurodevelopmental pathways and play an important role in in gene expression regulation. Our results indicate distinct functional patterns of disorder proteins among mental diseases and healthy controls. When comparing the IDPs between ASD and ADHD, we observed that IDPs in ADHD are more likely linked to the synaptic signaling and regulation, while IDPs in ASD are more tendentiously related to diverse and complex biological processes. Our work offers valuable insights into the important functions of IDPs in psychiatric disorders, enhancing our understanding of the molecular mechanisms involved in psychiatric disorders.

Accurate power curve modelling is essential to continuously evaluate the performance of a wind turbine (WT). In this work, we characterize the wind power curves using SCADA data acquired at a frequency of 5 minutes in a wind farm (WF) consisting of 5 WTs. Regarding the non-parametric methods, we select artificial neural networks (ANNs) to make curve estimations. Given that, we have the curves provided by the manufacturer of the WTs given by some very precisely measured pair of wind speed and power points. We can evaluate the difference between the manufacturer characterization and the ones estimated with the data provided by the SCADA system. Before the estimation, we propose a method of filtering the anomalies based on the characteristics provided by the manufacturer. We use three-quarters of the available data for curve estimation and one-quarter for the test. One WT suffered a break in the test part, so we can check how the test estimates reflect this problem in its wind-power curve compared to the estimations obtained in the WTs that worked adequately.

With the development of data science, more and more machine learning technologies have been designed to solve complicated and challenging real-world tasks containing a large volume of data. And many significant real-world datasets contain data in the form of networks or graphs. Graph Neural Networks is one of the powerful machine learning tools, which could provide a perfect solution to processing a large amount of non-Euclidean data. And because most bio information data in bioinformatics is in the non-Euclidean domain, Graph Neural Networks could then directly be applied to solve problems in bioinformatics. Much research has been done in the field of GNN, and there are also some surveys related to GNN and its applications. However, there has been little research focusing on GNN in bioinformatics, and we think in the future we could better utilize GNN in the field of biology, so we would like to write a literature review to help take a comprehensive look at GNN and their applications in the field of bioinformatics. In this paper, we would first introduce SOTA models in Graph Neural Networks, and second, introduce their applications in bio information. And then we would provide future directions of Graph Neural Networks in bioinformatics.

Convolutional Neural Networks (CNNs) have gained prominence in the research literature on image classification over the last decade. One shortcoming of CNNs, however, is their lack of generalizability and tendency to overfit when presented with small training sets. Augmentation directly confronts this problem by generating new data points providing additional information. In this paper, we investigate the performance of more than ten different sets of data augmentation methods, with two novel approaches proposed here: one based on the Discrete Wavelet Transform and the other on the Constant-Q Gabor transform. Pretrained ResNet50 networks are finetuned on each augmentation method. Combinations of these networks are evaluated and compared across three benchmark data sets of images representing diverse problems and collected by instruments that capture information at different scales: a virus data set, a bark data set, and a LIGO glitches data set. Experiments demonstrate the superiority of this approach. The best ensemble proposed in this work achieves state-of-the-art performance across all three data sets. This result shows that varying data augmentation is a feasible way for building an ensemble of classifiers for image classification (code available at https://github.com/LorisNanni).

In the current industrial landscape, increasingly pervaded by technological innovations, the adoption of optimized strategies for asset management is becoming a critical key success factor. Among the various strategies available, the “Prognostics and Health Management” strategy is able to support maintenance management decisions more accurately, through continuous monitoring of equipment health and “Remaining Useful Life” forecasting. In the present study, Convolutional Neural Network-based Deep Neural Network techniques are investigated for the Remaining Useful Life prediction of a punch tool, whose degradation is caused by working surface deformations during the machining process. Surface deformation is determined using a 3D scanning sensor capable of returning point clouds with micrometric accuracy during the operation of the punching machine, avoiding both downtime and human intervention. The 3D point clouds thus obtained are transformed into bidimensional image-type maps, i.e., maps of depths and normal vectors, to fully exploit the potential of convolutional neural networks for extracting features. Such maps are then processed by comparing 15 genetically optimized architectures with the transfer learning of 19 pre-trained models, using a classic machine learning approach, i.e., Support Vector Regression, as a benchmark. The achieved results clearly show that, in this specific case, optimized architectures provide performance far superior (MAPE=0.058) to that of transfer learning which, instead, remains at a lower or slightly higher level (MAPE=0.416) than Support Vector Regression (MAPE=0.857).

Object classification is a crucial task in deep learning, which involves the identification and categorization of objects in images or videos. Although humans can easily recognize common objects, such as cars, animals, or plants, performing this task on a large scale can be time-consuming and error-prone. Therefore, automating this process using neural networks can save time and effort while achieving higher accuracy. Our study focuses on the classification step of human chromosome karyotyping, an important medical procedure that helps diagnose genetic disorders. Traditionally, this task is performed manually by expert cytologists, which is a time-consuming process that requires specialized medical skills. Therefore, automating it through deep learning can be immensely useful. To accomplish this, we implemented and adapted existing preprocessing and data augmentation techniques to prepare the chromosome images for classification. We used ResNet-50 convolutional neural networks and an ensemble approach to classify the chromosomes, obtaining state-of-the-art performance in the tested dataset.

Neural-objective and image optimization approaches for drilling fluid rheology automation are crucial for drilling engineering optimization. A myriad of intelligent computational models are employed to predict and monitor the parameters of mud rheology and filter cake permeability posture using an artificial neural network feedforward (ANN-FF) function, a non-ANN-FF function, an image processing tool, and a model optimization tool. 498 datasets of synthetic-based mud (SBM) flat rheology from a drilling mud laboratory at Nazarbayev University were imported into a user-friendly MATLAB application using image processing and nftool. Apart from the Google TensorFlow Sequential API DNN architecture used, a Levenberg-Marquardt training algorithm with input sigmoid hidden neurons 10, 12, and 18 coupled with a linear output layer was also used to predict the SBM flow index. OBM and SBM filter cakes were processed for void spaces; however, a model study was extrapolated to maximize filtration volume Vf. The study’s findings show that the ANN-FF model employed for rheological property monitoring and prediction had higher steady and exponential levels of accuracy and a correlation coefficient of 0.96-0.99. More so, SBM and OBM image processing presented an area of void spaces of 1790M2 and 1739M2, respectively. The porosity and permeability postures of both SBM and OBM resulted in a significant void space capable of maximizing the flow index. In addition, the single-objective modelling based on a genetic algorithm did validate the experimental rheological data for flow index or filtration volume Vf optimization; the study resulted in the finding that the constrained physics-informed objective function hindered maximizing oil recovery and instead predicted possible formation damage. It is empirical to note that automating flow predictions with neural, objective, and image functions has presented an alternative novel method for non-programmers using MATLAB and Google Colabs that is capable of enhancing mud rheological parameters, drilling efficiency, and hydrocarbon recovery.

Quantum computing (QC) has opened the door to advancements in machine learning (ML) tasks that are currently implemented in the classical domain. Convolutional neural networks (CNNs) are classical ML architectures that exploit data locality and possess a simpler structure than fully-connected multi-layer perceptrons (MLPs) without compromising the accuracy of classification. However, the concept of preserving data locality is usually overlooked in the existing quantum counterparts of CNNs, particularly for extracting multi-features in multidimensional data. In this paper, we present a multidimensional quantum convolutional classifier (MQCC) that performs multidimensional and multi-feature quantum convolution with average and Euclidean pooling and thus adapting the CNN structure to a variational quantum algorithm (VQA). Average pooling is based on the quantum Haar transform (QHT) and Euclidean pooling is based on partial quantum measurement. The experimental work was conducted using multidimensional data to validate the correctness and demonstrate the scalability of the proposed method utilizing both noisy and noise-free quantum simulations. We evaluated the MQCC model with reference to reported work on state-of-the-art quantum simulators from IBM Quantum and Xanadu using a variety of standard ML datasets. The experimental results showed favorable characteristics of our proposed techniques compared to existing work with respect to a number of quantitative metrics such as the number of training parameters, cross-entropy loss, classification accuracy, circuit depth, and quantum gate count.

This article presents a forecast model that uses a hybrid architecture of recurrent neural networks (RNN) with surface neural networks (ANN), based on historical records of exported active energy (EAE) and weather data. Two types of models were developed: the first type includes six models that use EAE records and weather variables as inputs, while the second type includes eight models that use only weather variables. Different metrics were applied to assess the performance of these models, and the best model of each type was selected. Finally, a comparison of the performance between the selected models of both types is presented, and they are validated with real data provided by a solar plant, achieving acceptable levels of accuracy. The selected model of the first type has an RMSE of 0.19, MSE of 0.03, MAE of 0.09, a correlation coefficient of 0.96, and a determination coefficient of 0.93. The other selected model of the second type showed lower precision in the metrics (RMSE = 0.24, MSE = 0.06, MAE = 0.10, Corr. Coef. = 0.95, and Det. Coef. = 0.90). Both models demonstrated good performance and acceptable accuracy in forecasting the weekly photovoltaic energy production of the solar plant.

The conventional method used for calculating pavement condition index (PCI) has two major drawbacks: safety problems during pavement inspection, and human error. This paper proposes a method for removing these problems. The proposed method uses surface deflection data in falling weight Deflectometer test to estimate PCI. The data used in this study were derived from 236 pavement segments taken from Tehran-Qom freeway in Iran. The data set was analyzed using multi layers perceptron (MLP) and radial basis function (RBF) neural networks. These neural networks were optimized by levenberg-marquardt (MLP-LM), scaled conjugate gradient (MLP-SCG), imperialist competitive (RBF-ICA), and genetic (RBF-GA) algorithms. After initial modeling with four neural networks mentioned, the committee machine intelligent systems (CMIS) method was adopted to combine the results and improve the accuracy of the modeling. The results of analysis have been verified by the four criteria of average percent relative error (APRE), average absolute percent relative error (AAPRE), root mean square error (RMSE) and standard error (SD). The best reported results belonged to CMIS, including APRE=2.3303, AAPRE=11.6768, RMSE=12.0056, and SD=0.0210.

The timely diagnosis of defects at their incipient stage of formation is crucial to extend the life-cycle of technical appliances. This is the case of mechanical-related stress, either due to long aging degradation processes (e.g., corrosion) or in-operation forces (e.g., impact events), which might provoke detrimental damages, such as cracks, disbonding or delaminations, most commonly followed by the release of acoustic energy. The localization of these sources can be successfully fulfilled via adoption of Acoustic Emission (AE)-based inspection techniques through the computation of the Time of Arrival (ToA), namely the time at which the induced mechanical wave released at the occurrence of the acoustic event arrives to the acquisition unit. However, the accurate estimation of the ToA may be hampered by poor Signal-to-Noise ratios (SNRs). In these conditions, standard statistical methods typically fail. In this work, two alternative Deep Learning methods are proposed for ToA retrieval, namely a Dilated Convolutional Neural Network (DilCNN) and a Capsule Neural Network for ToA (CapsToA). These methods have the additional benefit of being portable on resource-constrained microprocessors. Their performance has been extensively studied on both synthetic and experimental data, focusing on the problem of ToA identification for the case of a metallic plate. Results show that the two novel methods can achieve localization errors which are up to 70% more precise than those yielded by conventional strategies, even when the SNR is severely compromised (i.e., down to 2 dB). Moreover, DilCNN and CapsNet have been implemented in a tiny machine learning environment and then deployed on microcontroller units, showing a negligible loss of performance with respect to offline realizations.

In videos, the human's actions are of three-dimensional (3D) signals. These videos investigate the spatiotemporal knowledge of human behavior. The promising ability is investigated using 3D convolution neural networks (CNNs). The 3D CNNs have not yet achieved high output for their well-established two-dimensional (2D) equivalents in still photographs. Board 3D Convolutional Memory and Spatiotemporal fusion face training difficulty preventing 3D CNN from accomplishing remarkable evaluation. In this paper, we implement Hybrid Deep Learning Architecture that combines STIP and 3D CNN features to enhance the performance of 3D videos effectively. After implementation, the more detailed and deeper charting for training in each circle of space-time fusion. The training model further enhances the results after handling complicated evaluations of models. The video classification model is used in this implemented model. Intelligent 3D Network Protocol for Multimedia Data Classification using Deep Learning is introduced to further understand space-time association in human endeavors. In the implementation of the result, the well-known dataset, i.e., UCF101 to, evaluates the performance of the proposed hybrid technique. The results beat the proposed hybrid technique that substantially beats the initial 3D CNNs. The results are compared with state-of-the-art frameworks from literature for action recognition on UCF101 with an accuracy of 95%.

In recent years, with the popularity of smart mobile devices, the interaction between devices and users, especially in the form of voice interaction, has become increasingly important. If smart devices can understand more users' emotional states through voice data, more customized services can be provided for users. This paper proposes a novel machine learning model for speech emotion recognition, which combines convolutional neural networks (CNN), long short-term memory neural networks (LSTM), and deep neural networks (DNN), called CLDNN. To make the designed system can recognize the audio signal closer to the human auditory system does, this article uses the Mel frequency cepstral coefficients (MFCCs) of audio data as the input of the machine learning model. First, the MFCCs of the voice signal is extracted as the input of the model, and the feature values of the data are calculated using several local feature learning blocks (LFLB) composed of one-dimensional CNN. Because the audio signals are time-series data, the feature values obtained from LFLBs then input into LSTM layer to enhance the learning on time-series level. Finally, fully connected layers are used for classification and prediction. Three databases RAVDESS, EMO-DB and IEMOCAP are used for the experiments in this paper. The experimental results show that the proposed method can improve the accuracy compared to other related researches in speech emotion recognition.

As AI models become more and more common in business and even in our daily lives, it is important to understand what the carbon impact of these models is. Recent papers have shown that this impact can be quite great, i.e., the training of a single high-end model can result in emissions of more than 500t of CO2eq. In this paper we discuss the factors that influence the carbon footprint of AI models, explore what impact different decisions have, and show how the footprint can be reduced. We also examine the footprint of different models to give a guideline on how urgent action is for different organizations.

The emergence of the Internet of Things (IoT) has led to the deployment of various types of sensors in many application fields, including environment monitoring, smart cities, health, industries, and others. The increasing number of connected devices has led to the creation of massive quantities of data that need to be analyzed. Typically, this data is ordered by time, as a time series. In this context, this paper presents a time series prediction model based on Recurrent Neural Networks in order to predict one step ahead. Result obtained through five Internet of Things monitoring datasets, showed that the Recurrent Neural Network obtained better performance that the prediction methods, ARIMA and SVM.

This paper focuses on quantifying the uncertainty in the specific absorption rate values of brain induced by the uncertain positions of the electroencephalography electrodes placed on patient's scalp. To avoid running a large number of simulations, an artificial neural network architecture for uncertainty quantification involving high-dimensional data is proposed in this paper. The proposed method is demonstrated to be an attractive alternative to conventional uncertainty quantification methods because of its considerable advantage in the computational expense and speed.

Air quality is one of the most concerning problems in major industrialized cities in the world. Prediction of future air quality is highly relevant to public health. In some big cities, multiple air quality measurement stations are deployed at different locations to monitor air pollutants, such as NO2, CO, PM 2.5 and PM 10, over time. At every monitoring time stamp t, we observe one station×feature matrix xt of the pollutant data, which represents a spatio–temporal process. Traditional methods on prediction of air quality typically use data from one station or can only predict a single pollutant (such as PM 2.5) at a time, which ignores the spatial correlation among different stations. Moreover, the air pollution data are typically highly nonstationary, We propose a de-trending graph convolutional LSTM (long short term memory) to continuously predict the whole station×feature matrix in the next 1 to 48 hours, which not only captures the spatial dependency among multiple stations by replacing an inner product with convolution, but also incorporates the de-trending signals (transform a nonstationary process to a stationary one by differencing the data) into our model. Experiments on the air quality data of the city Chengdu and multiple major cities in China demonstrate the feasibility of our method and show promising results.

Neural networks require a large quantity of training spectra and detector responses in order to learn to solve the inverse problem of neutron spectrum unfolding. In addition, due to the under-determined nature of unfolding, non-physical spectra which would not be encountered in usage should not be included in the training set. While physically realistic training spectra are commonly determined experimentally or generated through Monte Carlo simulation, this can become prohibitively expensive when considering the quantity of spectra needed to effectively train an unfolding network. In this paper, we present three algorithms for the generation of large quantities of realistic and physically motivated neutron energy spectra. Using an IAEA compendium of 251 spectra, we compare the unfolding performance of neural networks trained on spectra from these algorithms, when unfolding real-world spectra, to two baselines. We also investigate general methods for evaluating the performance of and optimizing feature engineering algorithms.

Snow parameters have traditionally been retrieved using discontinuous, multi-band sensors; however, continuous hyperspectral sensor are now being developed as an alternative. In this paper we investigate the performance of various sensor configurations using machine learning neural networks trained on a simulated dataset. Our results show improvements in accuracy of retrievals of snow grain size and impurity concentration for continuous hyperspectral channel configurations. Retrieval accuracy of snow albedo was found to be similar for all channel configurations.

This paper provides a methodology for the production of synthetic images for training neural networks to recognise shapes and objects. There are many scenarios in which it is difficult, expensive and even dangerous to produce a set of images that is satisfactory for the training of a neural network. The development of 3D modelling software has nowadays reached such a level of realism and ease of use that it seemed natural to explore this innovative path and to give an answer regarding the reliability of this method that bases the training of the neural network on synthetic images. The results obtained in the two proposed use cases, that of the recognition of a pictorial style and that of the recognition of migrants at sea, leads us to support the validity of the approach, provided that the work is conducted in a very scrupulous and rigorous manner, exploiting the full potential of the modelling software. The code produced, which automatically generates the transformations necessary for the data augmentation of each image, and the generation of random environmental conditions in the case of Blender and Unity3D software, is available under the GPL licence on GitHub. The results obtained lead us to affirm that through the good practices presented in the article, we have defined a simple, reliable, economic and safe method to feed the training phase of a neural network dedicated to the recognition of objects and features, to be applied to various contexts.

With the recent advances in Machine Learning (ML), several models have been successfully applied to financial and accounting data to predict the likelihood of companies’ bankruptcy. However, time series have received little attention in literature with a lack of studies on the application of Deep Learning sequence models as Recurrent Neural Networks (RNN) and the recent Attention-based models in general. In this research work, we investigated the application of Long Short Term Memory (LSTM) networks to exploit time series of accounting data for bankruptcy prediction. The main contributions of our work are the following: a) we have proposed a Multi-head LSTM that models each financial variable in a time window independently and compared it with a single-input LSTM and other traditional ML models. The Multi-head LSTM outperforms all the other models; b) We identify the optimal time series length for bankruptcy prediction to be equal to 4 years of accounting data; c) We made public the dataset we used for the experiments that include data from 8262 different public companies in the American stock market generated in the period between 1999-2018. Furthermore, we prove the efficacy of the Multi-head LSTM models in terms of lower false positives and better division of the two classes.

For autonomous driving, perception is a primary and essential element that fundamentally deals with the insight into the ego vehicle’s environment through sensors. Perception is a challenging task suffering from dynamic objects and continuous environmental changes. The issue gets worse due to interrupting the quality of perception by adverse weather like snow, rain, fog, night light, sand storm, strong daylight, etc. In this work, we have tried to improve camera-based perception accuracy, such as autonomous driving-related object detection in adverse weather. We proposed the improvement of YOLOv8-based object detection in adverse weather through transfer learning using merged data from various harsh weather datasets. Two prosperous open-source datasets (ACDC and DAWN) and their merged dataset were used to detect primary objects on the road in harsh weather. A set of training weights were collected from training on the individual datasets, their merged version, and several subsets of those datasets according to their characteristics. A comparison between the training weights also occurred by evaluating the detection performance on the above-mentioned datasets and their subsets. The evaluation revealed that using custom datasets for training significantly improves the detection performance compared to the YOLOv8 base weights. And using more images through the feature-related data merging technique steadily increases the object detection performance.

Emotion recognition defined as identifying human emotion and is directly related to different fields such as human-computer interfaces, human emotional processing, irrational analysis, medical diagnostics, data-driven animation, human-robot communi- cation and many more. The purpose of this study is to propose a new facial emotional recognition model using convolutional neural network. Our proposed model, “ConvNet”, detects seven specific emotions from image data including anger, disgust, fear, happiness, neutrality, sadness, and surprise. This research focuses on the model’s training accuracy in a short number of epoch which the authors can develop a real-time schema that can easily fit the model and sense emotions. Furthermore, this work focuses on the mental or emotional stuff of a man or woman using the behavioral aspects. To complete the training of the CNN network model, we use the FER2013 databases, and we test the system’s success by identifying facial expressions in the real-time. ConvNet consists of four layers of convolution together with two fully connected layers. The experimental results show that the ConvNet is able to achieve 96% training accuracy which is much better than current existing models. ConvNet also achieved validation accuracy of 65% to 70% (considering different datasets used for experiments), resulting in a higher classification accuracy compared to other existing models. We also made all the materials publicly accessible for the research community at: https://github.com/Tanoy004/Emotion-recognition-through-CNN.

Various empirical models are available to evaluate the temperature effects on relative permeability of the different rock and fluid systems. However, the implementation of limited experimental data points may hinder the applicability of such models to other systems. This study aims to develop new predictive models for k_ro estimation based on multilayer perceptron artificial neural network (MLP-ANN), adaptive neuro-fuzzy inference system (ANFIS), and least squares support vector machine (LSSVM) approaches. A database comprising of 626 data points applied to the model development. The independent variables are temperature, oil viscosity, water viscosity, water saturation ( ), and the absolute permeability. Each variable covers a wide range of variations which increases models’ potential to be applied in various systems with different characteristics. The doubtful experimental data points excluded using a leverage value approach and a sensitivity analysis carried out to determine the quantitative impact of every individual independent variable on the k_ro. Statistical error analyses demonstrated the coefficient of determination (R²) values of 0.985, 0.975, and 0.999 for MLP-ANN, ANFIS, and LSSVM, respectively. The comparative study indicated that the LSSVM had the best performance regarding both graphical and statistical error analyses among the newly proposed models and previously reported models in the literature.

The used car market has a high global economic importance, with more than 35 million cars sold yearly. Accurately predicting prices is a crucial task for both buyers and sellers to facilitate informed decisions in terms of opportunities or potential problems. Although various Machine Learning techniques have been applied to create robust prediction models, a comprehensive approach has yet to be studied. This research introduces two datasets from different markets, one with over 300,000 entries from Germany to serve as a training base for deep prediction models and a second dataset from Romania containing more than 15,000 car quotes used mainly to observe local traits. As such, we include extensive cross-market analyses by comparing the emerging Romanian market versus one of the world’s largest and most developed car markets, Germany. Our study uses several neural network architectures that capture complex relationships between car model features, individual add-ons, and visual features to predict used car prices accurately. Our models achieved a high R2 score exceeding 0.95 on both datasets, indicating their effectiveness in estimating used car prices. Moreover, we experimented with advanced convolutional architectures to predict car prices based solely on visual features extracted from car images. This approach exhibited transfer-learning capabilities, leading to improved prediction accuracy, especially since the Romanian training dataset is limited. Our experiments highlight the most important factors influencing the price, while our findings have practical implications for buyers and sellers in assessing the value of vehicles. At the same time, the insights gained from this study enable informed decision-making and provide valuable guidance in the used car market.

Accurate estimations of actual evapotranspiration (ETa) are fundamental for various environmental issues. Artificial Intelligence-based models are considered a promising alternative to the most common direct ETa estimation techniques and indirect methods by remote sensing (RS)-based surface energy balance models. Artificial Neural Networks (ANN) are proven to be suitable for predicting reference evapotranspiration and ETa based on RS data. The aim of this study is to develop a methodology based on ANNs for estimating daily ETa values using NDVI and land surface temperature, coupled with limited site-specific climatic variables at a large irrigation catchment. Two scenarios were implemented by the ANN model. Data from only the 38 days of satellite overpass dates was selected in Scenario-I, while the 769-day data included the satellite overpass dates and other days located between two satellite overpass acquisitions in Scenario-II. An irrigation scheme in the Mediterranean region of Turkiye was selected, and a total of 38 Landsat images, and local climatic data were used in 2021 and 2022. Results showed that R2 values in Scenario-I and II were acknowledgeably high for training (0.79 and 0.86), testing (0.75 and 0.81), and the entire dataset (0.76 and 0.84), respectively. Results of the new model in two scenarios showed acceptable agreement with ETa-METRIC values. The proposed ANN model demonstrated the potential of obtaining daily ETa using limited climatic data and RS imagery.

Deepfakes are manipulated or altered images, or video, that are created using deep learning models with high levels of photorealism. The two popular methods of producing a deepfake are based on either convolutional neural networks (CNN), or autoencoders. Deepfakes created using CNN comparatively show higher qualities of realism, yet oftentimes leave artifacts and distortions in the generated media that can be detected using machine learning and deep learning algorithms. In recent years, there has been an influx of periocular image and video data because of the increase usage of face masks. By wearing masks, much of what is used for facial recognition is hidden, leaving only the periocular region visible to an observer. This loss of vital information leads to easier misidentification of media, allowing deepfakes to less likely be identified as fake. In this work, feature extraction methods, such as Scale-Invariant Feature Transform (SIFT), Histogram of Oriented Gradients (HOG), and CNN, are used to train an ensemble deep learning model to detect deepfakes in videos on a frame-by-frame level based on the periocular region. Our proposed model is able to distinguish original and manipulated images with accuracies around 98.9 percent, which is an improvement to previous works by combining SIFT and HOG for deepfake detection in convolutional neural networks.

Predicting trip purpose from comprehensive and continuous smart card data is beneficial for transport and city planners in investigating travel behaviours and urban mobility. Here we propose a framework, ActivityNET, using machine learning (ML) algorithms to predict passengers’ trip purpose from smart card data and Points-of-Interest (POIs) data. The feasibility of the framework is demonstrated in two phases. Phase I focuses on extracting activities from individuals’ daily travel patterns from smart card data and combining them with POIs using the proposed ‘activity-POIs consolidation algorithm’. Phase II feeds the extracted features into an artificial neural network (ANN) with multiple scenarios and predicts trip purpose under primary activities (home and work) and secondary activities (entertainment, eating, shopping, child drop-offs/pick-ups and part-time work) with high accuracy. As a case study, the proposed ActivityNET framework is applied in Greater London and illustrates a robust competence to predict trip purpose. The promising outcomes demonstrate that the cost-effective framework offers high predictive accuracy and valuable insights into transport planning.

Eutrophication is a major environmental issue with many negative consequences, such as hypoxia and harmful cyanotoxins production. Monitoring coastal eutrophication is a crucial, especially for island countries like the Republic of Cyprus, which are economically dependent on the touristic sector. Additionally, the open-sea aquaculture industry in Cyprus has been exhibiting an increase in the last decades and environmental monitoring to identify possible signs of eutrophication is mandatory according to the legislation. Therefore, in this modelling study, two different types of Artificial Neural Networks (ANNs) are developed based on in situ-data collected from stations located in the coastal waters of Cyprus. Theses ANNs aim to model the eutrophication phenomenon based on two different data-driven modelling procedures. Firstly, the self-organizing map (SOM) ANN examines several water quality parameters (specifically water temperature, salinity, nitrogen species, ortho-phosphates, dissolved oxygen and electrical conductivity) interactions with the Chlorophyll-a parameter. The SOM model enables us to visualize the monitored parameters relationships and to comprehend complex biological mechanisms related to Chlorophyll-a production. A second feed-forward ANN model is also developed for predicting the Chlorophyll-a levels. Based on this ANN model, several scenarios associated to the eutrophication-related water quality parameters can be extracted. The combination of these two ANNs models is considered a holistic modelling approximation for the identification of eutrophication scenarios, since it enables not only the prediction of the Chlorophyll-a parameter levels, but also the “capturing” of hidden biological mechanisms associated with algal production.

Big data technologies are disruptive technologies that affect every business, including those in the construction industry. The Thai government has also been affected, and attempted to use machine learning techniques with the analytics of big data technologies to predict which construction projects have a winning price over the project budget. However, this technology was never developed, and the government did not implement it because they had data obtained via a traditional data collection process. In this study, traditional data were processed to predict behavior in Thai government construction projects using a machine learning model. The data were collected from the government procurement system in 2019. There were seven input data, including project owner department, type of construction project, bidding method, project duration, project level, winning price over estimated price, and winning price over budget. A range of classification techniques, including an artificial neural network (ANN), a decision tree (DC), and K-nearest neighbor (KNN), were used in this study (ANN). According to the results, after hyperparameter tuning, ANN had the greatest prediction accuracy with 78.9 percent. This study confirms that data from the Thai government procurement system can be investigated using machine learning techniques from big data technologies.

Humans learn from a lot of information sources to make decisions. Once this information is learned in the brain, spatio-temporal associations are made, connecting all these sources (variables) in space and time represented as brain connectivity. In reality, to make a decision, we usually have only part of the infor-mation, either as a limited number of variables, or limited time to make the decision, or both. The brain functions as spatio-temporal associative memory. Inspired by the ability of the human brain, a brain-inspired spatio-temporal associative memory was proposed earlier, that utilizes the NeuCube brain-inspired spiking neural network framework. Here we apply the STAM framework to develop STAM for neuroimaging data, on the cases of EEG and fMRI, resulting in STAM-EEG and STAM-fMRI. The paper shows that once a NeuCube STAM classification model is trained on a complete spatio-temporal EEG or fMRI data, it can be recalled using only part of the time series, or/and only part of the used variables. We evaluate accordingly the temporal association accuracy and spatial association accuracy. This is a pilot study that opens the field for the development of multimodal classification systems on other multimodal neuroimaging data, such as the also shown longitudinal MRI data, trained on complete data, but recalled on partial data collected across different settings, in different labs and clinics, that may vary in terms of variables, time of data collection, and other parameters. The proposed methods will allow also for brain diagnostic/prognostic marker discovery using spatio-temporal neuroimaging data.

The fault diagnosis of wind farms has been proven to be a challenging task and motivates the research activities carried out through this work. Therefore, this paper deals with the fault diagnosis of a wind park benchmark model, and it considers viable solutions to the problem of earlier fault detection and isolation. The design of the fault indicator involves data-driven approaches, as they can represent effective tools for coping with poor analytical knowledge of the system dynamics, noise, uncertainty and disturbances. In particular, the proposed data-driven solutions rely on fuzzy models and neural networks that are used to describe the strongly nonlinear relationships between measurement and faults. The chosen architectures rely on nonlinear autoregressive with exogenous input models, as they can represent the dynamic evolution of the system along time. The developed fault diagnosis schemes are tested by means of a high-fidelity benchmark model, that simulates the normal and the faulty behaviour of a wind farm installation. The achieved performances are also compared with those of a model-based approach relying on nonlinear differential geometry tools. Finally, a Monte-Carlo analysis validates the robustness and the reliability of the proposed solutions against typical parameter uncertainties and disturbances.

The accurate classification of landfill waste diversion plays a critical role in efficient waste management practices. Traditional approaches, such as visual inspection, weighing and volume measurement, and manual sorting, have been widely used but suffer from subjectivity, scalability, and labour requirements. In contrast, machine learning approaches, particularly Convolutional Neural Networks (CNN), have emerged as powerful deep learning models for waste detection and classification. This paper analyses VGG-16, InceptionResNetV2, DenseNet121, Inception V3, and MobileNetV2 models to classify real-life waste when trained on pristine and unadulterated materials, versus samples collected at a landfill site. When training on DiversionNet, the unadulterated material dataset with labels required for landfill modelling, classification accuracy was limited to 49.69% in the real environment. Using real-world samples in the newly formed RealWaste dataset showed that practical applications for deep learning in waste classification is possible, with Inception V3 reaching 89.19% classification accuracy on the full spectrum of labels required for accurate modelling.

Accurate climate data at fine spatial resolution are essential for scientific research and the development and planning of crucial social systems, such as energy and agriculture. Among them, sea surface temperature plays a critical role as the associated El Niño-Southern Oscillation (ENSO) is considered a significant signal of global interannual climate system. In this paper, we propose an implicit neural representation-based interpolation method with temporal information (T_INRI) to reconstruct climate data of high spatial resolution, with sea surface temperature as the research object. Traditional deep learning models for generating high-resolution climate data are only applicable to fixed resolution enhancement scales. In contrast, the proposed T_INRI method is not limited to the enhancement scale provided during the training process and its results indicate that it can enhance low-resolution input by arbitrary scale. Additionally, we discuss the impact of temporal information on the generation of high-resolution climate data, specifically, which month the low-resolution sea surface temperature data is from. Our experimental results indicate that T_INRI is advantageous over traditional interpolation methods under different enhancement scales, and the temporal information can improve T_INRI performance for a different calendar month. We also examined the potential capability of T_INRI in recovering missing grid value. These results demonstrate that the proposed T_INRI is a promising method for generating high-resolution climate data and has significant implications for climate research and related applications.

The neural network technology for real-time cryptographic data protection with symmetric keys (masking codes, neural network architecture and weights matrix) for unmanned aerial vehicles (UAV) onboard communication systems has been developed. It provides hardware and software implementation with high technical and operational characteristics. The development of neural network technology for real-time cryptographic data protection was performed using an integrated approach based on theoretical foundations of neural network cryptographic data protection, new algorithms and structures of neural network data encryption and decryption, modern element base with the possibility of programming for the structure, and computer-aided design of hardware and software tools. The development and implementation of the on-board system of neural network cryptographic data protection in real-time are based on the following principles: variable composition of equipment; modularity; conveyorization and spatial parallelism; specialization and adaptation of hardware and software to data encryption and decryption. The tabular-algorithmic method of calculating the scalar product has been improved, it provides fast calculation of the scalar product of input data for both fixed and floating-point by bringing to the largest common order of weights and forming tables of macro-partial products for them. Components of neural network cryptographic data encryption and decryption have been developed on the processor core supplemented by specialized scalar product calculation modules. The specialized hardware for neural network cryptographic data encryption was developed using VHDL for equipment programming in the Quartus II development environment ver. 13.1 and the appropriate libraries, and implemented on the basis of the FPGA EP3C16F484C6 Cyclone III family.

Ovarian Cancer (OC) is a type of cancer that affects ovaries in women, and is difficult to detect at initial stage due to which it remains as one of the leading causes of cancer death. The ovarian cancer data generated from the Internet of Medical Things (IoMT) was used and a novel approach was proposed for distinguishing the ovarian cancer by utilizing Self Organizing Maps (SOM) and Optimal Recurrent Neural Networks (ORNN). SOM algorithm was utilized for better feature subset selection and was also utilized for separating profitable, understood and intriguing data from huge measures of medical data. In supervised learning techniques, the SOM-based feature selection seems to be a tougher challenge because of the absence of class labels that would guide the search for relevant information to the classifier model. The classification approach can identify ovarian cancer data as benign/malignant. The ovarian cancer detection process can be improved by optimizing the weights of RNN structure using Adaptive Harmony Search Optimization (AHSO). The proposed model in this study can be used to detect cancer at early stages with high accuracy and low Root Mean Square Error (RMSE).

This paper introduces a novel ensemble adjustment Kalman filter (EAKF) that integrates a machine-learning approach. The approach employs nonlinear variable relationships established by a deep neural network (DNN) during the analysis stage of the EAKF. This process nonlinearly projects observation increments into the state variable space. The newly developed DNN-EAKF algorithm can be applied to coupled data assimilation using coupled ocean-atmosphere models. It enhances cross-component updates in strongly coupled data assimilation (SCDA) by diminishing errors in estimating cross-component error covariance arising from insufficient ensemble members, thereby improving the SCDA analysis. This paper employs a conceptual model to conduct twin experiments, validating the DNN-EAKF’s capability to utilize cross-component observation information in SCDA effectively. The approach is anticipated to offer insights for future methodological integrations of machine learning and data assimilation and provide methods for SCDA applications in coupled general circulation models.

Functional Connectivity analysis using Electroencephalography signals is common. The EEG signals are converted to networks by transforming the signals into a correlation matrix and analyzing the resulting networks. Here, four learning models, namely, Logistic Regression, Random Forest, Support Vector Machine, and Recurrent Neural Networks, are implemented on the correlation matrix data to classify them either on their psychometric assessment or the effect of therapy; The EEG data is trail-based/event-related. The classifications based on RNN provided higher accuracy( 74-88%) than the other three models( 50-78%). Instead of using individual graph features, a correlation matrix provides an initial test of the data. When compared with the time-resolved correlation matrix, it offered a 4-5% higher accuracy. The time-resolved correlation matrix is better suited for dynamic studies here; it provides lower accuracy when compared to the correlation matrix, a static feature.

AliAmbra is a project developed to explore and promote high-quality catches of the Amvrakikos Gulf GP to Artas’ wider regions. In addition, this project aimed to implement an integrated plan of action, to form a business identity with high-added value and achieve integrated business services adapted to the special characteristics of the area. The action plan for this project was to actively search for new markets, create a collective identity for the products, promote their quality and added value, engage in gastronomes and tasting exhibitions, dissemination and publicity actions, as well as enhance the quality of the products and markets based on the customer needs. The primary focus of this publication is to observe and analyze the data retrieved from various tasting exhibitions of the AliAmbra project, with a target goal of improving customer experience and product quality.

Total cloud cover (TCC) retrieval from ground-based optical imagery is a problem being tackled by a few generations of researchers. The number of human-designed algorithms for the estimation of TCC grows every year. However, there is not very much progress in terms of quality, mostly due to the lack of systematic approach to the design of the algorithms, to the assessment of their generalization ability, and to the assessment of the TCC retrieval quality. In this study, we discuss the optimization nature of data-driven schemes for TCC retrieval. In order to compare the algorithms, we propose the framework for the assessment of the algorithms characteristics. We present several new algorithms that are based on deep learning techniques: a model for outliers filtering, and a few models for TCC retrieval from all-sky imagery. For training and assessment of data-driven algorithms of this study, we present the Dataset of All-Sky Imagery over the Ocean (DASIO) containing over one million of all-sky optical images of visible sky dome taken in various regions of the World Ocean. The research campaigns contributed to DASIO collection took place in the Atlantic ocean, the Indian ocean the Red and Mediterranean seas, and also in the Arctic ocean. Optical imagery collected during these missions are accompanied by standard meteorological observations of cloudiness characteristics made by experienced observers. We assess the generalization ability of the presented models in several scenarios that differ in terms of the regions selected for the train and validation subsets. As a result, we demonstrate that our models based on convolutional neural networks deliver superior quality compared to all previously published schemes. As a key result, we demonstrate considerable drop of the ability to generalize the training data in case of strong covariate shift between training and validation subsets of imagery which may take place in case of region-aware subsampling.

Recently natural gas (NG) global market attracted much attention in case it is cleaner than oil, and simultaneously in most regions is cheaper than renewable energy sources. However, price fluctuations, environmental concerns, technological development, emerging unconventional resources, energy security challenges, and shipment are some of the forces that made the NG market more dynamic and complex. From a policy-making perspective, it is vital to uncover demand-side future trends. This paper proposed an intelligent forecasting model to forecast NG global demand, however investigating a multi-dimensional purified input vector. The model starts with a data mining (DM) step to purify input features, identify the best time lags, and to pre-process selected input vector. Then a hybrid artificial neural network (ANN) which equipped with genetic optimizer is applied to set up ANN’s characteristics. Among 13 available input features, six features (e.g. Alternative and Nuclear Energy, CO2 Emissions, GDP per Capita, Urban Population, Natural Gas Production, Oil Consumption) selected as the most critical feature via the DM step. Then, the hybrid prediction model is designed to extrapolate the consumption of future trends. The proposed model overcomes competitive models refer to different error based evaluation statistics. Besides, as the model proposed the best input feature set, results compared to the model which used the raw input set, with no DM purification process.

X-band marine radar captures signal reflected from the sea surface. Theoretical studies indicate that the initial unfiltered signal contains meaningful information including parameters related to wind waves. Traditional methods for estimating significant wave height (SWH) rely on the physical laws describing signal reflection from rough surfaces. However, recent studies suggest the feasibility of employing artificial neural networks (ANNs) for SWH approximation. Both classical and ANN-based approaches necessitate costly in situ data. In this study, we propose generating synthetic radar images with specified wind wave parameters using Fourier-based approach and Pierson–Moskowitz wave spectrum as a viable alternative. We generate synthetic images and use them for unsupervised learning approach to train a convolutional component of an ANN. After that, we train a regression ANN based on the previous convolutional component to obtain SWH back from synthetic images. Then, we apply preliminary trained weights for the same regression model to train SWH approximation on the dataset of real sea clutter images. In this study, we demonstrate the increase in the accuracy of SWH estimation from radar images with the preliminary training on synthetic data.

Sleep deficiency impacts the quality of life and may have serious health consequences in the long run. Questionnaire-based subjective assessment of sleep deficiency has many limitations. On the other hand, objective assessment of sleep deficiency is challenging. In this study, we propose a polysomnography-based mathematical model for computing baseline sleep deficiency severity score and then investigated the estimation of sleep deficiency severity using features available only from wearable sensor data including heart rate variability and single-channel electroen-cephalography for a dataset of 500 subjects. We used Monte-Carlo Feature Selection (MCFS) and inter-dependency discovery for selecting the best features and removing multi-collinearity. For developing the Regression model we investigated both the frequentist and the Bayesian ap-proaches. An Artificial Neural Network achieved the best performance of RMSE = 5.47 and an R-squared value of 0.67 for sleep deficiency severity estimation. The developed method is com-parable to conventional methods of Functional Outcome of Sleep Questionnaire and Epworth Sleepiness Scale for assessing the impact of sleep apnea on sleep deficiency. Moreover, the results pave the way for reliable and interpretable sleep deficiency severity estimation using a wearable device in Smart Health.

Recent advances in nanomaterials have been heavily influenced by low-dimensional nanocarbon allotropes. In particular, carbyne has attracted attention for its potential as a true one-dimensional carbon chain with sp1 hybridization. To maximize the capabilities of this material, we employ a focused data-driven inverse design approach based on the carbon nanomaterials genome concept. This involves using deep learning neural network models to identify key descriptors tied to desired properties, enabling property prediction and reverse engineering of nanocarbons. Our iterative approach entails: (i) gathering growth/property data on nanostructures; (ii) identifying informative numerical/categorical predictors; (iii) developing deep learning models mapping descriptors to properties; (iv) refining models with new insights; (v) determining required descriptors/conditions for target properties via inverse mapping; (vi) validating models by synthesizing predicted nanostructures; and (vii) enhancing models with validation data. This allows uncovering hidden growth-property connections, precisely tuning nanocarbons for desired attributes. We introduce new methodologies including exciting synergistic effects, synchronizing atomic vibrations, active screen plasma, energy-driven transformations, surface acoustic micro/nano-manipulation, doping and directed self-assembly to expose relationships and integrate insights into the inverse design flow. This research promises to accelerate discovery of next-generation low-dimensional nanocarbons with exceptional properties and applications.

An approach to the implementation of neural network for real-time cryptographic data protection with symmetric keys oriented on embedded systems is presented. This approach is valuable especially for onboard communication systems in unmanned aerial vehicles (UAV) because of suitability to hardware implementation. In this study we evaluate the possibility to build such a system in hardware implementation at FPGA. The proposed integrated neural network approach for real-time cryptographic data protection was based on theoretical foundations of neural network cryptographic data protection, new algorithms and structures of neural network data encryption and decryption, modern hardware components with programmable structure. The development and implementation of the on-board neural network system for cryptographic data protection in real-time are based on the following principles: variable composition of equipment; modularity; conveyorization and spatial parallelism; hardware and software modification and making them suitable for data encryption and decryption. The tabular-algorithmic method for calculation of the scalar product has been improved that makes it possible to perform fast calculation of the scalar product of fixed-point and floating-point input data by bringing them to the largest common order of weights and forming tables of macro-partial products. Components of neural network cryptographic data encryption and decryption have been developed on the processor core supplemented by specialized scalar product calculation modules. The specialized hardware for neural network cryptographic data encryption was developed using VHDL for equipment programming in the Quartus II development environment ver. 13.1 and the appropriate libraries, and implemented on the basis of the FPGA EP3C16F484C6 Cyclone III family.

Requirements Engineering (RE) is an important step in the whole software development lifecycle. The problem in RE is to determine the class of the software requirements as functional and non-functional. Proper and early identification of these requirements is vital for the entire development cycle. On the other hand, manual government of requirement classes is a timewasting task, and it needs intensive effort. Automatic requirement classification techniques through advanced Machine Learning (ML) strategies are started to be developed to address this problem. Basically, software requirements are generated as natural language explanations and therefore requirement classification may be handled as a particular Natural Language Processing (NLP) problem. From ML point of view, classification strategies require dataset to be used in the training/learning phase. For this goal, we generated a unique Turkish dataset having collected the requiremenst from real-world software projects with 4600 samples. Of these requirements, 3000 and 1600 were labeled as functional and non-functional, respectively. The data set has been evaluated using (i) traditional machine learning algorithms, (ii) deep learning algorithms, and (iii) transformer models respectively. As a result BERTurk was found to be the most successful algorithm to discriminate functional and non-functional requirements in terms of 95% f-score metric.

The paper aims to develop and test an algorithm for classifying Russian-language text reviews of patients’ experiences with medical facilities and physicians, extracted from social media. This is motivated by the limitations of conventional methods of surveying consumers to assess their satisfaction with the quality of services, which are being replaced by automatic processing of text data from social media. This approach enables to get more objective results due to the increased representativeness and independence of the sample of service consumers. The authors have tested machine learning methods using various neural network architectures. A hybrid method was developed to classify text reviews of medical facilities posted by patients on the two most popular physician review websites in Russia. Overall, more than 60,000 reviews were analysed. The main results are as follows: 1) the classification algorithm developed by the authors has a high efficiency, the best result being achieved by the GRU-based architecture (val_accuracy = 0.9271); 2) applying the named entity search method to text messages following their partitioning improved the classification efficiency for each of the classifiers based on artificial neural networks. To further enhance the classification quality, reviews need to be semantically partitioned by target and sentiment and the resulting fragments need to be analysed separately.

To build a representation of what we see, the human brain recruits regions throughout the visual cortex in cascading sequence. Recently, an approach was proposed to evaluate the dynamics of visual perception in high spatiotemporal resolution at the scale of the whole brain. This method combined functional magnetic resonance imaging (fMRI) data with magnetoencephalography (MEG) data using representational similarity analysis and revealed a hierarchical progression from primary visual cortex through the dorsal and ventral streams. To assess the replicability of this method, here we present results of a visual recognition neuro-imaging fusion experiment, and compare them within and across experimental settings. We evaluated the reliability of this method by assessing the consistency of the results under similar test conditions, showing high agreement within participants. We then generalized these results to a separate group of individuals and visual input by comparing them to the fMRI-MEG fusion data of Cichy et al. (2016), revealing a highly similar temporal progression recruiting both the dorsal and ventral streams. Together these results are a testament to the reproducibility of the fMRI-MEG fusion approach and allows for the interpretation of these spatiotemporal dynamic in a broader context.

Electricity demand forecasting plays a significant role in energy markets. Accurate prediction of electricity demand is the key factor to optimize power generation, consumption, saving energy resources, and determining the energy prices. However, integrating energy mix scenarios, including solar and wind power which are highly non-linear and seasonal, into an existing grid increases uncertainty in generation, adds the challenges for precise forecast. To tackle these challenges, state-of-the-art methods and algorithms have been implemented in literature. We have developed Artificial Intelligence (AI) based deep learning models that can effectively handle the information of long time-series data. Based on the pattern of dataset, four different scenarios were developed and two best scenarios were selected for prediction. Dozens of models were developed and tested in deep AI networks. In the first scenario (Scenario1), data for weekdays excluding holidays was taken and in the second scenario (Scenario2) all the data in the basket was taken. Remaining two scenarios, weekends and holidays were tested and neglected because of their high prediction error. To find the optimal configuration, models were trained and tested within a large space of alternatives called hyper-parameters. In this study, an Aritificial Neural Network (ANN) based Feed-forward Neural Network (FNN) showed the minimum prediction error for Scenario1 while a Recurrent Neural Network (RNN) based Gated Recurrent Network (GRU) showed the minimum prediction error for Scenario2. While comparing the accuracy, the lowest MAPE of 2.47% was obtained from FNN for Scenario1. When evaluating the same testing dataset (non-holidays) of Scenario2, the RNN-GRU model achieved the lowest MAPE of 2.71%. Therefore, we can conclude that grouping of weekdays as Senario1 prepared by excluding the holidays provides better forecasting accuracy compared to the single group approach used in Scenario2, where all the dataset is considered together. However, Scenario2 is equally important to predict the demand for weekends and holidays.

The coronavirus disease 2019 (COVID-19) is the fastest transmittable virus caused by severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2). The detection of COVID-19 using artificial intelligence techniques and especially deep learning will help to detect this virus in early stages which will reflect in increasing the opportunities of fast recovery of patients worldwide. This will lead to release the pressure off the healthcare system around the world. In this research, classical data augmentation techniques along with CGAN based on a deep transfer learning model for COVID-19 detection in chest CT scan images will be presented. The limited benchmark datasets for covid-19 especially in chest CT images is the main motivation of this research. The main idea is to collect all the possible images for covid-19 that exists until the very writing of this research and use the classical data augmentations along with CGAN to generate more images to help in the detection of the COVID-19. In this study, five different deep convolutional neural network-based models (AlexNet, VGGNet16, VGGNet19, GoogleNet, and ResNet50) have been selected for the investigation to detect the coronavirus infected patient using chest CT radiographs digital images. The classical data augmentations along with CGAN improve the performance of classification in all selected deep transfer models. The Outcomes show that ResNet50 is the most appropriate deep learning model to detect the COVID-19 from limited chest CT dataset using the classical data augmentation with testing accuracy of 82.91%.

An advanced driver-assistance system (ADAS) is critical to driver-vehicle-interaction systems. Driving behaviour modelling and control significantly improves the global performance of ADAS. A haptic assistive system assists the driver by providing a specific torque on the steering wheel according to the driving-vehicle-road profile to improve the steering control. However, the main problem is designing a compensator dealing with the high-level uncertainties in different driving scenarios with haptic driver assistance, where different personalities and diverse perceptions of drivers are considered. If not properly accounted for, these differences can lead to poor driving performance. This paper focuses on designing a data-driven model-free compensator considering various driving behaviours with a haptic feedback system. A back propagation neural network (BPNN) models driving behaviour based on real driving data (speed, acceleration, vehicle orientation, and current steering angle). Then using the genetic algorithm (GA) and the integral time absolute error (ITEA) criterion to optimize multiple PID compensation parameters for various driving behaviour (such as speeding/braking, lane-keeping and turning), which are then be combined by the fuzzy logic to provide different driving commands. An experiment was conducted with 5 participants in a driving simulator. During the second experiment, seven participants drove in the simulator to evaluate the robustness of the proposed combined GA-PID offline and the Fuzzy-PID controller applied online. The experiment results evaluated the ITAE of lateral displacement and yaw angle, during various driving behaviour. The results validated the proposed method by significantly enhancing the driving performance.

In this article, a method for the moisture mapping of the soil surface of agrophytocenosis was proposed using neural network based on synchronized radar and multispectral optoelectronic data of Sentinel-1,2. To verify the developed method, data from two experimental plots were used. These plots were located two on irrigated soybean crops. The first of them was located on the right bank (1st plot) and the second one on the left bank (2nd plot) of the down part of Volga River. Two experimental soil moisture geo-datasets were done by measurements and geo-referencing points using gravimetric method (1st plot) and with proximal sensing method (2nd plot) using Soil Moisture Sensor ML3-KIT (THETAKIT, Delta). The soil moisture retrieval algorithm was based on the use of a neural network to predict reflection coefficient of an electromagnetic wave from the soil surface, followed by inversion into soil moisture using a dielectric model that takes into account the soil texture. The input parameter of the neural network was the ratio of the microwave radar vegetation index (calculated on the basis of Sentinel-1 data) to the index (calculated on the basis of data of multispectral optoelectronic channels 8 and 11 of Sentinel-2). Such way calculated index reveals showed a significantly greater dependence on soil moisture than on vegetation height that was been used in previous studies. The retrieved values of soil moisture were compared with the soil moisture measured in-situ. The proposed method with a determination coefficient of 0.44-0.65 and a standard deviation of 2.4%-4.2% for the 1st plot as well as with and of the same metrics for the 2nd allows predicting the soil moisture of both a test plots covered by soybean plants, relative to soil moisture measured in-situ. The conducted research created the scientific basis for a new technology for remote sensing the moisture content of the soil surface of agrophytocenosis as an element of the precision farming system and agroecology.

The aim of this study was to build a tool to analyze, using artificial intelligence, the sentiment perception of users who answered two questions from the CSQ – 8 questionnaires with raw Spanish free-text. Their responses are related to mindfulness, which is a novel technique used to control stress and anxiety caused by different factors in daily life. As such, we proposed an online course where this method was applied in order to improve the quality of life of health care professionals in COVID 19 pandemic times. We also carried out an evaluation of the satis-faction level of the participants involved, with a view to establishing strategies to improve fu-ture experiences. To automatically perform this task, we used Natural Language Processing (NLP) models such as swivel embedding, neural networks and transfer learning, so as to classify the inputs into the following 3 categories: negative, neutral and positive. Due to the lim-ited amount of data available - 86 registers for the first and 68 for the second - transfer learning techniques were required. The length of the text had no limit from the user’s standpoint, and our approach attained a maximum accuracy of 93.02 % and 90.53 % respectively based on ground truth labeled by 3 experts. Finally, we proposed a complementary analysis, using com-puter graphic text representation based on word frequency, to help researchers identify relevant information about the opinions with an objective approach to sentiment. The main conclusion drawn from this work is that the application of NLP techniques in small amounts of data using transfer learning is able to obtain enough accuracy in sentiment analysis and text classification stages

One of the most effective image processing techniques is the use of convolutional neural networks, where we combine intensity values at grid points in the vicinity of each point. To speed up computations, researchers have developed a dilated version of this technique, in which only some points are processed. It turns out that the most efficient case is when we select points from a sub-grid. In this paper, we explain this empirical efficiency proving that the sub-grid is indeed optimal – in some reasonable sense. To be more precise, we prove that all reasonable optimality criteria, the optimal subset of the original grid is either a sub-grid, or a sub-grid-like set.

Optical artificial neural networks (ONNs) have significant potential for ultra-high computing speed and energy efficiency. We report a new approach to ONNs based on integrated Kerr micro-combs that is programmable, highly scalable and capable of reaching ultra-high speeds, demonstrating the building block of the ONN — a single neuron perceptron — by mapping synapses onto 49 wavelengths to achieve a single-unit throughput of 11.9 Giga-OPS at 8 bits per OP, or 95.2 Gbps. We test the perceptron on handwritten-digit recognition and cancer-cell detection — achieving over 90% and 85% accuracy, respectively. By scaling the perceptron to a deep learning network using off-the-shelf telecom technology we can achieve high throughput operation for matrix multiplication for real-time massive data processing.

The informational synthesis of neural structures, processes, parameters and characteristics that allow a unified description and modeling as neural machines of natural and artificial neural systems is presented. The general informational parameters as the global quantitative measure of the neural systems computing potential as absolute and relative neural power were proposed. Neural information organizing and processing follows the way in which nature manages neural information by developing functions, functionalities and circuits related to different internal or peripheral components and also to the whole system through a non-deterministic memorization, fragmentation and aggregation of afferent and efferent information, deep neural information processing representing multiple alternations of fragmentation and aggregation stages. The relevant neural characteristics were integrated into a neural machine type model that incorporates unitary also peripheral or interface components as the central ones. The proposed approach allows overcoming the technical constraints in artificial computational implementations of neural information processes and also provides a more relevant description of natural ones.

The intracranial pressure (ICP) signal, as monitored on patients in intensive care units, contains pulses of cardiac origin where P1 and P2 subpeaks can often be observed. When calculable, the ratio of their relative amplitudes is an indicator of the patient’s cerebral compliance. This characterization is particularly informative on the overall state of the cerebrospinal system. We developed a recurrent neural network-based framework for P2/P1 ratio computation that only takes a raw ICP signal as an input. Two tasks are performed, namely pulse classification and subpeak designation. Pulse classification was achieved with an area under the curve of 0.90 on a 4,344-pulse testing dataset, while the peak designation algorithm identified pulses with a P2/P1 ratio > 1 with a 97.3% accuracy. Although it still needs to be evaluated on a larger number of labeled recordings, our automated P2/P1 ratio calculation framework appears to be a promising tool for improving bedside monitoring devices.

Gyroscopes are fascinating instruments with a history of about 200 years. When J.G.F. Bohnenberger presented his machine to his students in 1810 at the University of Tuebingen, Germany, nobody could have foreseen that this fascinating development would be used for complex orientation and positioning. At the University of Stuttgart, Germany, a collection of 160 exhibits is available and in transition for a sustainable future. Here, the systems are digitized in 2D, 2.5D and 3D and are made available for a world-wide community using OpenAccess platforms. The technologies being used are Computed Tomography, Computer Vision, Endoscopy and Photogrammetry. The workflows for combining voxel representations and colored point clouds are described, to create Digital Twins of the tangible assets. Advantages and disadvantages are discussed und work for near future is outlined in this new and challenging field of Tech Heritage digitization.

An earthquake is a natural event by its general definition. This natural event is a disaster that causes significant damage, loss of life, and other economic effects that will damage the state. The possibility of predicting a natural event such as an earthquake will minimize the reasons mentioned. Data collection, data processing, and data evaluation were carried out in this study. Earthquake forecasting was performed using the data and the RNN (Recurrent Neural Network) method. The study was carried out on seismic data with a magnitude of 3.0 and above belonging to Düzce Province between 1990 and 2022. In order to increase the learning potential of the method, the b and d values of the earthquake were calculated and included in the data set, except for the earthquake magnitude. The determination of earthquakes in a specific time interval in regions of Turkey, the classification of earthquake-related seismic data using artificial neural networks, and the production of predictions for the future reveal the importance of this study.

This paper investigates the dynamic properties of artificial neural networks using differential equations and explores the influence of parameters on stability and neural oscillations. By analyzing the equilibrium point of the differential equations, we identify conditions for asymptotic stability and criteria for oscillation in artificial neural networks. Furthermore, we demonstrate how adjusting synaptic weights between neurons can effectively control stability and oscillation. The proposed model offers potential insights into the malfunctioning mechanisms of biological neural networks implicated in neurological disorders like Parkinson's disease tremors and epilepsy seizures, which are characterized by abnormal oscillations.

Multiply-accumulate calculations using a memristor crossbar array is an important method to realize neuromorphic computing. However, the memristor array fabrication technology is still immature, and it is difficult to fabricate large-scale arrays with high-yield, which restricts the development of memristor-based neuromorphic computing technology. Therefore, cascading small-scale arrays to achieve the neuromorphic computational ability that can be achieved by large-scale arrays, which is of great significance for promoting the application of memristor-based neuromorphic computing. To address this issue, we present a memristor-based cascaded framework with some basic computation units, several neural network processing units can be cascaded by this means to improve the processing capability of the dataset. Besides, we introduce a split method to reduce pressure of input terminal. Compared with VGGNet and GoogLeNet, the proposed cascaded framework can achieve 93.54% Fashion-MNIST accuracy under the 4.15M parameters. Extensive experiments with Ti/AlOx/TaOx/Pt we fabricated are conducted to show that the circuit simulation results can still provide a high recognition accuracy, and the recognition accuracy loss after circuit simulation can be controlled at around 0.26%.

Tumorigenesis is a complex biological phenomenon that includes extensive genetic and phenotypic heterogeneities and complicated regulatory mechanisms. In the recent few years, our studies demonstrate that tumor-initiating cells are similar to neural stem/progenitor cells in regulatory networks, tumorigenicity and pluripotent differentiation potential. In the review, I will make further discussion on these observations and propose a rule of cell biology by integrating these findings with evidence from developmental biology, tumor biology and evolution, which suggests that neural stemness underlies two coupled cell properties, tumorigenicity and pluripotent differentiation potential. Tumorigenicity and phenotypic heterogeneity in tumor is a result of acquirement of neural stemness in cells. The neural stemness property of tumor-initiating cells can hopefully integrate different concepts/hypotheses underlying tumorigenesis. Neural stem cells/neural progenitors and tumor-initiating cells share regulatory networks; both exhibit neural stemness, tumorigenicity and differentiation potential; both are dependent on expression or activation of ancestral genes (the atavistic effect); both rely primarily on aerobic glycolytic metabolism; both can differentiate into various cells or tissues that are derived from three germ layers, resembling severely disorganized or more severely degenerated process of embryonic development; both are enriched in long genes with more splice variants that provide more plastic scaffolds for cell differentiation, etc. The property of neural stemness might be a key point to understand tumorigenesis and pluripotent differentiation potential, and possibly explain certain pathological observations in tumors that have been inexplicable. Therefore, behind the complexity of tumorigenesis might be a general rule of cell biology, i.e., neural stemness represents the ground state of cell tumorigenicity and pluripotent differentiation potential.

Attackers are perpetually modifying their tactics to avoid detection and frequently leverage legitimate credentials with trusted tools already deployed in a network environment, making it difficult for organizations to proactively identify critical security risks. Network traffic analysis products have emerged in response to attackers’ relentless innovation, offering organizations a realistic path forward for combatting creative attackers. Additionally, thanks to the widespread adoption of cloud computing, Device Operators (DevOps) processes, and the Internet of Things (IoT), maintaining effective network visibility has become a highly complex and overwhelming process. What makes network traffic analysis technology particularly meaningful is its ability to combine its core capabilities to deliver malicious intent detection. In this paper, we propose a novel darknet traffic analysis and network management framework to real-time automating the malicious intent detection process, using a weight agnostic neural networks architecture. It is an effective and accurate computational intelligent forensics tool for network traffic analysis, the demystification of malware traffic, and encrypted traffic identification in real-time. Based on Weight Agnostic Neural Networks (WANNs) methodology, we propose an automated searching neural net architectures strategy that can perform various tasks such as identify zero-day attacks. By automating the malicious intent detection process from the darknet, the advanced proposed solution is reducing the skills and effort barrier that prevents many organizations from effectively protecting their most critical assets.

This research investigates the application of deep learning in sentiment analysis of Canadian maritime case law. It offers a framework for improving maritime law and legal analytics policy-making procedures. The automation of legal document extraction takes center stage, underscoring the vital role sentiment analysis plays at the document level. Therefore, this study introduces a novel strategy for sentiment analysis in Canadian maritime case law, combining sentiment case law approaches with state-of-the-art deep learning techniques. The overarching goal is to systematically unearth hidden biases within case law and investigate their impact on legal outcomes. Employing Convolutional Neural Network (CNN) and Long Short-Term Memory (LSTM) based models, this research achieves a remarkable accuracy of 98.05% for categorizing instances. In contrast, Conventional Machine Learning techniques such as Support Vector Machines (SVM) yield an accuracy rate of 52.57%, Naive Bayes at 57.44 %, and Logistic Regression at 61.86%. The superior accuracy of the CNN and STM model combination underscores its usefulness in legal sentiment analysis, offering promising future applications in diverse fields like legal analytics and policy design. These findings mark a significant choice for AI-powered legal tools, presenting more sophisticated and sentiment-aware options for the legal profession.

A chronically stable electrode material with low impedance for recording neural activity and high charge-injection capacity for functional electro-stimulation is desirable for the fabrication of implantable microelectrode arrays that aims to restore impaired or lost neurological functions in humans. For this purpose, we have investigated the electrochemical properties of sputtered ruthenium oxide (RuOx) electrode coatings, deposited on planar microelectrode arrays, using an inorganic model of interstitial fluid (model-ISF) at 37o C as the electrolyte. Through a combination of cyclic voltammetry (CV) and electrochemical impedance spectroscopy (EIS) modelling study we have established the contribution of faradaic reaction as the major charge-injection contributor within the safe neural stimulation potential window of 0.6 V vs Ag|AgCl. We have also established the reversibility of the charge-injection process for sputtered RuOx film by applying constant charge-per-phase current stimulations at different pulse widths and by comparing the magnitudes of leading and trailing access voltages during voltage transient measurements. Finally, the impedance of the sputtered RuOx film was found to be reasonably comparable in both its oxidized and reduced states, although electronic contribution from capacitive double-layer was found to be slightly higher for the completely oxidized film around 0.6 V than its reduced counterpart around -0.6V.

Machine learning (ML) has found increasing use in research on energy conversion and storage technologies, in particular, so-called sustainable technologies. While often ML is used to directly optimize the parameters or phenomena of interest in the space of features, in this perspective, we focus on using ML to construct objects and methods that help in or enable the modeling of the underlying phenomena. We highlight the need for machine learning from very sparse and unevenly distributed numeric data in multidimensional spaces in these applications. After a brief introduction of some common regression-type machine learning techniques, we will focus on more advanced ML techniques which use these known methods as building blocks of more complex schemes and thereby allow working with extremely sparse data and also allow generating insight. Specifically, we will highlight the utility of using representations with sub-dimensional functions by combining the high-dimensional model representation ansatz with machine learning methods like neural networks or Gaussian process regressions in applications ranging from heterogeneous catalysis to nuclear energy.

This study investigates the factors influencing the choice of online and offline shopping trips and their impacts on urban transportation, environment, and economy in Tehran, Iran. A questionnaire survey was conducted to collect data from 1,000 active e-commerce users who made successful orders in both online and offline services in the last 20 days of 2021 in areas 2 and 5 of Tehran. A deep neural network model was developed to estimate the type of shopping trip based on 10 indicators, such as age, gender, car ownership, delivery cost, product price, etc. The performance of the model was compared with three other algorithms: MLP, decision tree, and KNN. The results showed that the deep neural network model had the highest accuracy of 95.63%. The most important factors affecting the choice of shopping trips were delivery cost, delivery time, and product price. This study provides insights for transportation planners, e-commerce managers, and policymakers to design effective strategies for reducing transportation costs, pollutant emissions, urban traffic, and increasing user satisfaction and sustainable development.

Convolutional neural networks (CNNs), inspired by biological visual cortex systems, are a powerful category of artificial neural networks that can extract the hierarchical features of raw data to greatly reduce the network parametric complexity and enhance the predicting accuracy. They are of significant interest for machine learning tasks such as computer vision, speech recognition, playing board games and medical diagnosis [1-7]. Optical neural networks offer the promise of dramatically accelerating computing speed to overcome the inherent bandwidth bottleneck of electronics. Here, we demonstrate a universal optical vector convolutional accelerator operating beyond 10 Tera-OPS (TOPS - operations per second), generating convolutions of images of 250,000 pixels with 8-bit resolution for 10 kernels simultaneously — enough for facial image recognition. We then use the same hardware to sequentially form a deep optical CNN with ten output neurons, achieving successful recognition of full 10 digits with 900 pixel handwritten digit images with 88% accuracy. Our results are based on simultaneously interleaving temporal, wavelength and spatial dimensions enabled by an integrated microcomb source. This approach is scalable and trainable to much more complex networks for demanding applications such as unmanned vehicle and real-time video recognition.

This research investigates the implementation of complex exponential-based neurons in FPGA, which can pave the way for implementing bio-inspired spiking neural networks to compensate for the existing computational constraints in conventional artificial neural networks. The increasing use of extensive neural networks and the complexity of models in handling big data lead to higher power consumption and delays. Hence, finding solutions to reduce computational complexity is crucial for addressing power consumption challenges. The complex exponential form effectively encodes oscillating features like frequency, amplitude, and phase shift, streamlining the demanding calculations typical of conventional artificial neurons through levering simple phase addition of complex exponential functions. The article implements such a two-neuron and a multi-neuron neural model using Xilinx system generator and Vivado design suite, employing 8-bit, 16-bit, and 32-bit fixed-point data format representations. The study evaluates the accuracy of the proposed neuron model across different FPGA implementations while also providing a detailed analysis of operating frequency, power consumption, and resource usage for the hardware implementations. BRAM-based Vivado designs outperformed Simulink regarding speed, power, and resource efficiency. Specifically, the Vivado BRAM-based approach supported up to 128 neurons, showcasing optimal LUT and FF resource utilization. Such outcomes accommodate choosing the optimal design procedure for implementing spiking neural networks on FPGAs.

Neural stem cell (NSC) based therapies are at the forefront of regenerative medicine strategies to combat illness and injury of the central nervous system (CNS). In addition to their ability to produce new cells, NSCs secrete a variety of products, known as the NSC secretome, that have been shown to ameliorate CNS disease pathology and promote recovery. As pre-clinical and clinical research to harness the NSC secretome for therapeutic purposes advances, a more thorough understanding of the endogenous NSC secretome can provide useful insight into the functional capabilities of NSCs. In this review, we focus on research investigating the autocrine and paracrine functions of the endogenous NSC secretome across life. We also compare the NSC secretome across species, finding signs of conserved parallels between rodent, human and zebrafish NSC secretomes. Throughout development and adulthood, we find evidence that the NSC secretome is a critical component of how endogenous NSCs regulate themselves and their niche. We also find gaps in current literature, most notably in the clinically relevant domain of endogenous NSC paracrine function in the injured CNS. Future investigations to further define the endogenous NSC secretome and its role in CNS tissue regulation are necessary to bolster our understanding of NSC-niche interactions and to aid in the generation of safe and effective NSC-based therapies.

Biological neural network (BNN) is the core brain for creature to accomplish its intelligent behaviors through unique network structure and mechanisms. It also inspires controlling human-designed autonomous agents including robots to generate more advanced intelligent behaviors by mimicking the neural control mechanism of creatures at a deeper and more interactive level. Here, we constructed a whole-brain neural network model of Caenorhabditis elegans (C. elegans), which characterizes the electrochemical processes at the level of cellular synapse. The neural network simulation integrates computational programming and visualization of neurons and synapse connections of C. elegans, containing the specific controllable circuits summarization with their dynamic characteristics within the whole network. To illustrate its particular intelligent control capability in terms of robotics, we introduce the first innovative methodology for applying the BNN model to a 12-legged robot’s movement control based on the established numerical simulation platforms. We accomplished and designed two methods and corresponding encoding processes, one involving orientation control and the other involving locomotion generation, to demonstrate the intelligent control performance of BNN. Both simulation and experiment results indicate that the robot exhibits stronger autonomy and more intelligent moving performance under the control of BNN. We then summarized the contributions for digitalizing the C. elegans’ whole-brain neural network in real-time and utilizing it to control robot in a closed loop to validate its advanced intelligence control ability in terms of a scientific robot.

Speech recognition, or automatic speech recognition (ASR), is a technology designed to convert spoken language into text using software. However, conventional ASR methods involve several distinct components, including language, acoustic, and pronunciation models with dictionaries. This modular approach can be time-consuming and may influence performance. In this study, we propose a method that streamlines the speech recognition process by incorporating a unified recurrent neural network (RNN) architecture. Our architecture integrates a convolutional neural network (CNN) with an RNN and employs a connectionist temporal classification (CTC) loss function. Key experiments were carried out using a dataset comprising 576,656 valid sentences, using erosion techniques. Evaluation of the model performance, measured by the word error rate (WER) metric, demonstrated remarkable results, achieving a WER of 2%. This approach has significant implications for the realm of speech recognition, as it alleviates the need for labor-intensive dictionary creation, enhancing the efficiency and accuracy of ASR systems, and making them more applicable to real-world scenarios. For future enhancements, we recommend the inclusion of dialectal and spontaneous data in the dataset to broaden the model's adaptability. Additionally, fine-tuning the model for specific tasks can optimize its performance for targeted objectives or domains, further enhancing its effectiveness in those areas.

Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs) have been utilised to forecast the foreign exchange market (Forex). However, such models usually exhibit unstable behaviour, as data perturbations often downgrade the functionality of the entire network due to their monolithic architecture. Hence, this study proposes a novel neuroscience-informed modular network applied to the closing prices and the sentiments retrieved from Yahoo Finance and Twitter APIs, aiming to anticipate better price fluctuations in Euro to British Pound Sterling (EUR/GBP) rather than monolithic methods. The proposed model is based on a new modular CNN, replacing pooling layers with orthogonal kernel initialisation RNNs coupled with Monte Carlo Dropout (MCD), namely MCoRNNMCD. It combines two modules: i) a convolutional simple RNN and ii) a convolutional Gated Recurrent Unit (GRU), where orthogonality and MCD are added to reduce the overfitting, assessing each module's uncertainty. These parallel feature extraction modules concatenate their outputs to a final three-layer Artificial Neural Network (ANN) decision-making module. A comprehensive comparison viewing objective evaluation metrics such as the Mean Square Error (MSE) proved that the proposed MCoRNNMCD-ANN outperformed single CNN, LSTM, GRU, and the state-of-the-art hybrid BiCuDNNLSTM, CLSTM, CNN-LSTM, and LSTM-GRU in forecasting hourly EUR/GBP closing price fluctuations.

In recent years, Neural networks are increasingly deployed in various fields to learn complex patterns and make accurate predictions. However, designing an effective neural network model is a challenging task that requires careful consideration of various factors, including architecture, optimization method, and regularization technique. This paper aims to comprehensively overview the state-of-the-art artificial neural network (ANN) generation and highlight key challenges and opportunities in machine learning applications. It provides a critical analysis of current neural network model design methodologies, focusing on the strengths and weaknesses of different approaches. Also, it explores the use of different learning approaches, including convolutional neural networks (CNN), deep neural networks (DNN), and recurrent neural networks (RNN) in image recognition, natural language processing, and time series analysis. Besides, it discusses the benefits of choosing the ideal values for the different components of ANN, such as the number of Input/output layers, hidden layers number, activation function type, epochs number, and model type selection, which help improve the model performance and generalization. Furthermore, it identifies some common pitfalls and limitations of existing design methodologies, such as overfitting, lack of interpretability, and computational complexity. Finally, it proposes some directions for future research, such as developing more efficient and interpretable neural network architectures, improving the scalability of training algorithms, and exploring the potential of new paradigms, such as Spiking Neural Networks, quantum neural networks, and neuromorphic computing.

Computer-aided image diagnostics (CAD) have been used in many fields of diagnostic medicine. It relies heavily on classical computer vision and artificial intelligence. Quantum neural network (QNN) has been introduced by many researchers around the world and presented recently by research corporations such as Microsoft, Google, and IBM. In this paper, the investigation of the validity of using the QNN algorithm for machine-based breast cancer detection was performed. To validate the learnability of the QNN, a series of learnability tests were performed alongside with classical convolutional neural network (CCNN). QNN is built using the Cirq library to perform the assimilation of quantum computation on classical computers. Series of investigations were performed to study the learnability characteristics of QNN and CCNN under the same computational conditions. The comparison was performed for real Mammogram data sets. The investigations showed success in terms of recognizing the data and training. Our work shows better performance of QNN in terms of successfully training and producing a valid model for smaller data set compared to CCNN.

Optical artificial neural networks (ONNs) — analog computing hardware tailored for machine learning [1,2] — have significant potential for ultra-high computing speed and energy efficiency [3]. We propose a new approach to architectures for ONNs based on integrated Kerr micro-comb sources [4] that is programmable, highly scalable and capable of reaching ultra-high speeds. We experimentally demonstrate the building block of the ONN — a single neuron perceptron — by mapping synapses onto 49 wavelengths of a micro-comb to achieve a high single-unit throughput of 11.9 Giga-FLOPS at 8 bits per FLOP, corresponding to 95.2 Gbps. We test the perceptron on simple standard benchmark datasets — handwritten-digit recognition and cancer-cell detection — achieving over 90% and 85% accuracy, respectively. This performance is a direct result of the record small wavelength spacing (49GHz) for a coherent integrated microcomb source, which results in an unprecedented number of wavelengths for neuromorphic optics. Finally, we propose an approach to scaling the perceptron to a deep learning network using the same single micro-comb device and standard off-the-shelf telecommunications technology, for high-throughput operation involving full matrix multiplication for applications such as real-time massive data processing for unmanned vehicle and aircraft tracking.

This article presents a detailed examination of the methodology and modeling tools utilized to analyze gas flow in pipelines, rooted in the fundamental principles of gas dynamics. The methodology integrates numerical simulations with modern neural network techniques, particularly focusing on the Physics-Informed Neural Network (PINN) method. This innovative approach combines artificial neural networks (ANNs) with physical equations, offering a more efficient and accurate way to model various complex processes and phenomena. The proposed mathematical model, based on the Euler equation, has been meticulously implemented using the Python language. Verification with analytical solutions ensures the accuracy and reliability of the computations. In the research, a comprehensive comparative analysis was conducted between results obtained using the PINN method and those from conventional Computational Fluid Dynamics (CFD) approaches. The analysis highlighted the advantages of the PINN method, which produced smoother pressure and velocity fluctuation profiles while reducing computation time, demonstrating its potential as a transformative modeling tool. The data derived from this study are of paramount importance for ensuring ongoing energy supply reliability and can also be used to create predictive models related to gas behavior in pipelines. The application of modeling techniques for gas flow simulation has the potential to revolutionize the integrity of our energy infrastructure and utilization of gas resources. However, it is crucial to emphasize that the effectiveness of such models relies on continuous monitoring and frequent updates to ensure alignment with real-world conditions. This research not only contributes to a deeper understanding of compressible gas flows but also underscores the crucial role of advanced modeling methodologies in the sustainable management of gas resources for both current and future generations.

The application of artificial intelligence to point-of-care testing (POCT) disease detection has become a hot research field, in which breath detection which detects the patient's exhaled VOCs combined with sensor arrays of Convolutional Neural Network (CNN) algorithms as a new lung cancer detection is attracting more researchers' attention. However, due to low accuracy, high complexity computation and large number of parameters make the CNN algorithms difficult transplant to the embedded system of POCT devices. A lightweight neural network (LTNet) in this work is proposed to deal with this problem, and meanwhile, achieve high-precision classification of acetone and ethanol gases, which are respiratory markers for lung cancer patients. Compared to currently popular lightweight CNN models such as EfficientNet, LTNet has fewer parameters (32K) and its training weight size is only 0.155 MB. LTNet achieved an overall classification accuracy of 99.06% and 99.14% in the own mixed gas dataset and the University of California (UCI) dataset, which are both higher than that of the five existing models’, and it also offers the shortest training (844.38s and 584.67s) and inference time (23s and 14s) in the same validation set. Compared to the existing CNN models, LTNet is more suitable for resource-limited POCT devices.

Dynamic neural networks (DNN) are types of artificial neural networks (ANN) that are designed to work with sequential data where context in time is important. In contrast to traditional static neural networks that process data in a fixed order, dynamic neural networks use information about past inputs, which is important if dynamic of a certain process is emphasized. They are widely used in natural language processing, speech recognition and time series prediction. In industrial processes, their use is interesting for the prediction of difficult-to-measure process variables. In an industrial process of isomerization, it is crucial to measure the quality attributes affecting the octane number of gasoline. Process analyzers that are commonly used for this purpose are expensive and subject to failures, therefore, in order to achieve continuous production in case of malfunction, mathematical models for estimating the product quality attributes are imposed as a solution. In this paper, mathematical models were developed using dynamic recurrent neural networks (RNN), i.e., their subtype of a long short-term memory (LSTM) architecture. The results of the developed models were compared with the results of several types of other data-driven models developed for an isomerization process, such as multilayer perceptron (MLP) artificial neural networks, support vector machines (SVM) and dynamic polynomial models. The obtained results are satisfactory, which suggests a good possibility of application.

This study aimed at exploring what role artificial intelligence techniques could play in the futural numerical analysis. In this paper, a convolutional neural network techniques based on modified loss function is proposed as a surrogate of finite element method(FEM). Several surrogate-based physics-informed neural networks(PINNs) are developed to solve representative boundary value problems based on elliptic partial differential equations (PDEs). Results from the proposed surrogate-based approach are in good agreement with ones from conventional FEM. It is found that modification of the loss function could improve the prediction accuracy of the neural network. It is indicated that to some extent the artificial intelligence technique could replace conventional numerical analysis as a great surrogate model.

Forest fire identification is important for forest resource protection. Effective monitoring of forest fires requires the deployment of multiple monitors with different viewpoints, while most traditional recognition models can only recognize images from a single source. By ignoring the information from images with different viewpoints, these models produce high rates of missed and false alarms. In this paper, we propose a graph neural network model based on the similarity of dynamic features of multi-view images to improve the accuracy of forest fire recognition. The input features of the nodes on the graph are converted into relational features of different gallery pairs by establishing pairs (nodes) representing different viewpoint images and gallery images. The new feature library relationship is used to update the image gallery with dynamic features in order to achieve the estimation of similarity between images and improve the image recognition rate of the model. In addition, to reduce the complexity of image pre-processing process and extract key features in images effectively, this paper also proposes a dynamic feature extraction method for fire regions based on image segment ability. By setting the threshold value of HSV color space, the fire region is segmented from the image, and the dynamic features of successive frames of the fire region are extracted. The experimental results show that, compared with the baseline method Resnet, this paper's method is more effective in identifying forest fires, and its recognition accuracy is improved by 2%. And the scheme of this paper can adapt to different forest fire scenes, with better generalization ability and anti-interference ability.

Characterization of cancer cells and neural stem cells indicates that tumorigenicity and pluripotency are coupled cell properties determined by neural stemness, and tumorigenesis represents a process of progressive loss of original cell identity and gain of neural stemness. This reminds of a most fundamental process required for the development of the nervous system and body axis during embryogenesis, i.e., embryonic neural induction. Neural induction is that, in response to extracellular signals that are secreted by the Spemann-Mangold organizer in amphibians or the node in mammals and inhibit epidermal fate in ectoderm, the ectodermal cells lose their epidermal fate and assume the neural default fate and consequently, turn into neuroectodermal cells. They further differentiate into the nervous system and also some non-neural cells via interaction with adjacent tissues. Failure in neural induction leads to failure of embryogenesis, and ectopic neural induction due to ectopic organizer or node activity or activation of embryonic neural genes causes a formation of secondary body axis or a conjoined twin. During tumorigenesis, cells progressively lose their original cell identity and gain of neural stemness, and consequently, gain of tumorigenicity and pluripotency, due to various intra-/extracellular insults in cells of a postnatal animal. Tumorigenic cells can be induced to differentiation into normal cells and integrate into normal embryonic development within an embryo. However, they form tumors and cannot integrate into animal tissues/organs in a postnatal animal because of lack of embryonic inducing signals. Combination of studies of developmental and cancer biology indicates that neural induction drives embryogenesis in gastrulating embryos but a similar process drives tumorigenesis in a postnatal animal. Tumorigenicity is the manifestation of aberrant occurrence of pluripotent state in a postnatal animal. Pluripotency and tumorigenicity are both but different manifestations of neural stemness in pre- and postnatal stage, respectively, of animal life. The unique property of neural stemness is derived from the evolutionary advantage of neural genes and the neural-biased state of the last common unicellular ancestors of metazoan. Based on these findings, I discuss about some confusion in cancer research, propose to distinguish the causality and associations and discriminate causal and supporting factors involved in tumorigenesis, and suggest revisiting the focus of cancer research.

This paper introduces a novel architecture—bidirectional optical neural network (BONN)—for providing backward connections alongside forward connections in artificial neural networks (ANNs). BONN incorporates laser diodes and photodiodes and exploits the properties of Köhler illumination to establish optical channels for backward directions. Thus, it has bidirectional functionality that is crucial for algorithms such as the backpropagation algorithm. BONN has a scaling limit of 96 × 96 for input and output arrays, and a throughput of 8.5 × 1015 MAC/s. While BONN's throughput may rise with additional layers for continuous input, limitations emerge in the backpropagation algorithm as its throughput doesn't scale with layer count. Successful BONN-based implementation of the backpropagation algorithm requires the development of fast spatial light modulator to accommodate frequent data flow changes. A two-mirror-like BONN and its cascaded extension are alternatives for multilayer emulation, and they help save hardware space and increase the parallel throughput for inference. An investigation into the application of the clustering technique to BONN revealed its potential to help overcome scaling limits and to provide full interconnections for backward directions between doubled input and output ports. BONN's bidirectional nature holds promise for enhancing supervised learning in ANNs and increasing hardware compactness.

Artificial Intelligence (AI) made its first appearance in 1956, and since then it has progressively introduced itself in healthcare system and patients’ information and care. AI functions can be grouped under the following headings: Machine Learning (ML), Deep Learning (DL), Artificial Neural Network (ANN), Convolutional Neural Network (CNN), Computer Vision (CV). Chronic limb-threatening ischemia (CLTI) represents the last stage of peripheral artery disease (PAD), and has increased over recent years, together with the rise in prevalence of diabetes and population ageing. Nowadays AI grants the possibility of developing new diagnostic and treatment solutions in the vascular field, given the possibility of accessing clinical, biological, and imaging data. By assessing the vascular anatomy in every patient, as well as the burden of atherosclerosis, and classifying the level and degree of disease, sizing and planning the best endovascular treatment, defining the perioperative complications risk, integrating experiences and resources between different specialties, identifying latent PAD, thus offering evidence-based solutions and guiding surgeons in the choice of the best surgical technique, AI challenges the role of the physician’s experience in PAD treatment.

Partial differential equations (PDEs) are used to describe a wide range of phenomena, such as heat transfer, fluid dynamics, and quantum mechanics. By solving PDEs, we can ob- tain insights into the behavior of the system and make predictions about its future evolution. Conventional numerical methods for obtaining the approximate solutions of PDEs may re- quire extensive computational resources and time, especially for complex PDEs and large- scale problems. The recently developed physics-informed neural network (PINN) has shown promise in a variety of scientific and engineering fields by incorporating physical laws into the loss functions of the neural network (NN). In addition, the training of PINN does not require ground truth data, but it has poor generalization ability to unseen domains. On the other hand, a convolutional neural network (CNN) has fast inference and better generalization ability, but it requires a large amount of training data. Taking the advantages of PINN and CNN by using Legendre multiwavelets (LMWs) as basis functions, we introduce a new method to approach the PDEs in this paper, namely Physics-Informed Legendre Multiwavelets CNN (PiLMWs-CNN), in order to continuously approximate a grid-based state representation that can be handled by a CNN. PiLMWs-CNN enable us to train our models using only physics-informed loss functions without any pre- computed training data, simultaneously providing fast and continuous solutions that gener- alize to previously unknown domains. In particular, the LMWs can simultaneously possess compact support, orthogonality, symmetry, high smoothness, and high approximation order. Compared to orthonormal polynomial (OP) bases, the approximation accuracy can be greatly increased and computation costs can be significantly reduced by using LMWs. We applied PiLMWs-CNN to approximate the damped wave equation, incompressible Navier-Stokes (N- S) equation, and two-dimensional heat conduction equation. The experimental results show that this method provides more accurate, efficient, and fast convergence with better stability when approximating the solution of PDEs.

Human activity recognition (HAR) using wearable sensors is gaining popularity in the healthcare domain, enabling continuous monitoring and timely interventions. The conventional HAR systems are centralised and need data sharing with a server for processing and model training. However, this data sharing leads to various challenges, such as privacy concerns, increased communication and storage costs, inefficient computational algorithms, data scarcity due to unwillingness to share, and difficulty achieving high accuracy and personalised models simultaneously. To overcome these challenges, this paper introduces a privacy and energy-aware federated learning (FL) framework for model training without data sharing. The proposed scheme incorporates spiking neural networks (SNNs), which are particularly suited for resource-constrained devices due to their event-driven information processing. The key idea is to synergise the strengths of SNNs and long short-term memory (LSTM) to create a hybrid model for robust HAR. The proposed hybrid spiking LSTM (S-LSTM) model uses the LSTM as an input layer to capture temporal dependencies efficiently, followed by spiking layers that replace the typical activation using leaky integrate-and-fire neurons. We employ the combination of surrogate gradient learning and backpropagation through time (BPTT) to update the weights of SNN layers in a supervised manner. The performance of the proposed model is evaluated using two publicly available datasets for both indoor and outdoor scenarios. The results show that the proposed S-LSTM performs much better than spike convolutional neural networks (S-CNN) and simple CNN, achieving the accuracy of 97% and 87% for indoor and outdoor datasets, respectively. Personalisation, also known as fine-tuning the global model using local data, is crucial in HAR. It allows the model to adapt to individual behaviour and preferences, making predictions more accurate. The results also show that personalisation improves the performance of participants up to 9% on average. Additionally, analysis with 50% random client participation reduces communication costs by 50% with minimal impact on accuracy.

A scalable optical convolutional neural network (SOCNN) based on free-space optics and Koehler illumination was proposed to address the limitations of the previous 4f correlator system. Unlike Abbe illumination, Koehler illumination provides more uniform illumination and reduces crosstalk. SOCNN allows for scaling up of the input array and the use of incoherent light sources. Hence, the problems associated with 4f correlator systems can be avoided. We analyzed the limitations in scaling the kernel size and parallel throughput and found that SOCNN can offer a multilayer convolutional neural network with massive optical parallelism.

This study uses machine learning to predict the convergence results of the Duffing equation with and without damping. The Duffing equation represents a nonlinear second-order differential equation with interesting behavior in undamped free vibration and forced vibration with damping. Convergence alternates randomly between 1 and -1 in undamped free vibration, depending on initial conditions. For forced vibration with damping, multiple factors influence vibration patterns. We utilize the fourth-order Runge-Kutta method to collect convergence results for both conditions. Machine learning techniques, specifically the long short term memory (LSTM) and LSTM-Neural Network (LSTM-NN) method, are employed to predict these convergence values. The LSTM-NN model is a hybrid approach that combines the LSTM method with the addition of hidden layers of neurons. Both the LSTM and LSTM-NN models are thoroughly explored and analyzed in this research. The research process involves three stages: data preprocessing, training, and verification. The results show that the LSTM-NN model becomes more adept at predicting binary datasets, boasting an impressive accuracy of up to 98%. However, when it comes to predicting multiple solutions, the traditional LSTM method outperforms the LSTM-NN approach.

Convolutional neural networks (CNNs), inspired by biological visual cortex systems, are a powerful category of artificial neural networks that can extract the hierarchical features of raw data to greatly reduce the network parametric complexity and enhance the predicting accuracy. They are of significant interest for machine learning tasks such as computer vision, speech recognition, playing board games and medical diagnosis [1-7]. Optical neural networks offer the promise of dramatically accelerating computing speed to overcome the inherent bandwidth bottleneck of electronics. Here, we demonstrate a universal optical vector convolutional accelerator operating beyond 10 Tera-FLOPS (floating point operations per second), generating convolutions of images of 250,000 pixels with 8-bit resolution for 10 kernels simultaneously — enough for facial image recognition. We then use the same hardware to sequentially form a deep optical CNN with ten output neurons, achieving successful recognition of full 10 digits with 900 pixel handwritten digit images with 88% accuracy. Our results are based on simultaneously interleaving temporal, wavelength and spatial dimensions enabled by an integrated microcomb source. This approach is scalable and trainable to much more complex networks for demanding applications such as unmanned vehicle and real-time video recognition.

Speech recognition, also known as automatic speech recognition (ASR), is a technology that enables software to transcribe spoken language into text. However, existing Amharic ASR methods require multiple separate blocks, such as language, acoustic, and pronunciation models with dictionaries, which can be time-consuming and influence performance. This study proposes an approach that replaces much of the speech pipeline with a single recurrent neural network (RNN) architecture. Our proposed architecture is based on a hybrid approach that combines a convolutional neural network (CNN) with a recurrent neural network (RNN) and a connectionist temporal classification (CTC) loss function. We conducted several experiments with noisy audio data that contain 20,000 valid sentences. The model was evaluated using the word error rate (WER) metric, achieving impressive results of 7% WER on noisy data. This approach has significant implications for the field of speech recognition, as it reduces the human effort required to create dictionaries and improves the efficiency and accuracy of ASR systems, making them more practical for real-world applications.

Creating self-learning algorithms, developing deep neural networks and improving other methods that "learn" for various areas of human activity is the main goal of the theory of machine learning. It helps to replace the human with a machine, aiming to increase the quality of production. The theory of artificial neural networks, which already have replaced the humans in problems of detection of moving objects, recognition of images or sounds, time series prediction, big data analysis and numerical methods remains the most dispersed branch of the theory of machine learning. Certainly, for each area of human activity it is necessary to select appropriate neural network architectures, methods of data processing and some novel tools from applied mathematics. But the universal problem for all these neural networks with specific data is the achieving the highest accuracy in short time. Such problem can be resolved by increasing sizes of architectures and improving data preprocessing, where the accuracy rises with the training time. But there is a possibility to increase the accuracy without time growing, applying optimization methods. In this survey we demonstrate existing optimization algorithms of all types, which can be used in neural networks. There are presented modifications of basic optimization algorithms, such as stochastic gradient descent, adaptive moment estimation, Newton and quasi-Newton optimization methods. But the most recent optimization algorithms are related to information geometry, for Fisher-Rao and Bregman metrics. This approach in optimization extended the theory of classic neural networks to quantum and complex-valued neural networks, due to geometric and probabilistic tools. There are provided applications of all introduced optimization algorithms, what delighted many kinds of neural networks, which can be improved by including any advanced approaches in minimization of the loss function. Afterwards, we demonstrated ways of developing optimization algorithms in further researches, engaging neural networks with progressive architectures. Classical gradient based optimizers can be replaced by fractional order, bilevel and, even, gradient free optimization methods. There is a possibility to add such analogues in graph, spiking, complex-valued, quantum and wavelet neural networks. Besides the usual problems of image recognition, time series prediction, object detection, there are many are other tasks for modern theory of machine learning, such as solving problem of quantum computations, partial differential and integro-differential equations, stochastic processes and Brownian motion, making decisions and computer algebra.

In the pursuit of energy efficiency and reduced environmental impact, adequate ventilation in enclosed spaces is essential. This study presents a hybrid neural network model designed for real-time monitoring and prediction of environmental variables. The system comprises two phases: An IoT hardware-software platform for data acquisition and decision-making, and a hybrid model combining short-term memory and convolutional recurrent structures. The results are promising and hold potential for integration into parallel processing AI architectures.

In computer vision and image analysis, Convolutional Neural Networks (CNNs) and other deep learning models are at the forefront of research and development. These advanced models have proven to be highly effective in tasks related to computer vision. One technique that has gained prominence in recent years is the construction of ensembles using Deep CNNs. These ensembles typically involve combining multiple pre-trained CNNs to create a more powerful and robust network. The purpose of this study is to evaluate the effectiveness of building CNN ensembles by combining several advanced techniques. Tested here are CNN ensembles constructed by replacing ReLU layers with different activation functions, employing various data augmentation techniques, and utilizing several algorithms, including some novel ones, that perturb network weights. Experimental results performed across many data sets representing different tasks demonstrate that our proposed methods for building deep ensembles produces superior results. All the resources required to replicate our experiments are available at https://github.com/LorisNanni.

We present a simplified computational rule for the back-propagation formulas for artificial neural networks. In this work, we provide a generic two-step rule for the back-propagation algorithm in matrix notation. Moreover, this rule incorporates both the forward and backward phases of the computations involved in the learning process. Specifically, this recursively computing rule permits the propagation of the changes to all synaptic weights in the network, layer by layer, efficiently. In particular, we use this rule to compute both the up and down partial derivatives of the cost function of all the connections feeding into the output layer.

Numerous approaches address over-fitting in neural networks: by imposing a penalty on the parameters of the network (L1, L2, etc); by changing the network stochastically (drop-out, Gaussian noise, etc.); or by transforming the input data (batch normalization, etc.). In contrast, we aim to ensure that a minimum amount of supporting evidence is present when fitting the model parameters to the training data. This, at the single neuron level, is equivalent to ensuring that both sides of the separating hyperplane (for a standard artificial neuron) have a minimum number of data points — noting that these points need not belong to the same class for the inner layers. We firstly benchmark the results of this approach on the standard Fashion-MINST dataset, comparing it to various regularization techniques. Interestingly, we note that by nudging each neuron to divide, at least in part, its input data, the resulting networks make use of each neuron, avoiding a hyperplane completely on one side of its input data (which is equivalent to a constant into the next layers). To illustrate this point, we study the prevalence of saturated nodes throughout training, showing that neurons are activated more frequently and earlier in training when using this regularization approach. A direct consequence of the improved neuron activation is that deep networks are now easier to train. This is crucially important when the network topology is not known a priori and fitting often remains stuck in a suboptimal local minima. We demonstrate this property by training a network of increasing depth (and constant width): most regularization approaches will result in increasingly frequent training failures (over different random seeds) whilst the proposed evidence-based regularization significantly outperforms in its ability to train deep networks.

Background: Disruption of alternative splicing (AS) is frequently observed in cancer and it might represent an important signature for tumor progression and therapy. Exon skipping (ES) represents one of the most frequent AS events and in non-small cell lung cancer (NSCLC) MET exon 14 skipping was shown to be targetable. Methods: We constructed a neural network (NN) specifically designed to detect MET exon 14 skipping events using RNAseq data. Furthermore, for discovery purpose we also developed a sparsely connected autoencoder to identify uncharacterized MET isoforms. Results: The NN had 100% Met exon 14 skipping detection rate, when tested on a manually curated set of 690 TCGA bronchus and lung samples. When globally applied to 2605 TCGA samples, we observed that the majority of false positives was characterized by a blurry coverage of exon 14, but interesting they share a common coverage peak in the second intron and we speculate that this event could be the transcription signature of a LINE1-MET fusion. Conclusions: Taken together our results indicate that neural networks can be an effective tool to provide a quick classification of pathological transcription events and sparsely connected autoencoders could represent the basis for the development of an effective discovery tool.

Optical artificial neural networks (ONNs) have significant potential for ultra-high computing speed and energy efficiency. We report a novel approach to ONNs that uses integrated Kerr optical micro-combs. This approach is programmable and scalable and is capable of reaching ultra-high speeds. We demonstrate the basic building block ONNs — a single neuron perceptron — by mapping synapses onto 49 wavelengths to achieve an operating speed of 11.9 x 10⁹ operations per second, or Giga-OPS, at 8 bits per operation, which equates to 95.2 gigabits/s (Gbps). We test the perceptron on handwritten-digit recognition and cancer-cell detection — achieving over 90% and 85% accuracy, respectively. By scaling the perceptron to a deep learning network using off-the-shelf telecom technology we can achieve high throughput operation for matrix multiplication for real-time massive data processing.

Real-world time series data sets contain a combination of linear and nonlinear patterns, making the time series forecasting problem more challenging. In this paper, a new hybrid methodology is introduced for forecasting univariate time series data sets using a multiplicative error modeling approach. An autoregressive integrated moving average (ARIMA) model is combined with an autoregressive neural network (ARNN) for improving the predictions of individual forecast models. The proposed multiplicative ARIMA-ARNN model glorifies the chances of capturing the different combinations of linear and nonlinear patterns in time series. The model shows outstanding performance on six standard time-series data sets compared to other widely used single and hybrid forecasting models.

Using smartphone especially android platform has already got eighty percent market shares, due to aforementioned report, it becomes attacker’s primary goal. There is a growing number of private data onto smart phones and low safety defense measure, attackers can use multiple way to launch and to attack user’s smartphones.(e.g. Using different coding style to confuse the software of detecting malware). Existing android malware detection methods use multiple features, like safety sensor API, system call, control flow structure and data information flow, then using machine learning to check whether its malware or not. These feature provide app’s unique property and limitation, that is to say, from some perspectives it might suit for some specific attack, but wouldn’t suit for others. Nowadays most malware detection methods use only one aforementioned feature, and these methods mostly analysis to detect code, but facing the influence of malware’s code confusion and zero-day attack, aforementioned feature extraction method may cause wrong judge. So, it’s necessary to design an effective technique analysis to prevent malware. In this paper, we use the importance of word from apk, because of code confusion, some malware attackers only rename variables, if using general static analysis wouldn’t judge correctly, then use these importance value to go through our proposed method to generate picture, finally using convolutional neural network to see whether the apk file is malware or not.

Deep learning is emerging in many industrial sectors in hand with big data analytics to streamline production. In the biomanufacturing sector, big data infrastructure is lagging comparatively to other industries. A promising approach is to combine Deep Neural Networks (DNN) with prior knowledge in Hybrid Neural Network (HNN) workflows that are less dependent on the quality and quantity of data. This paper reviews published articles over the past 30 years on the topic of HNN applications to bioprocesses. It revealed that HNNs were applied to various bioprocesses, including microbial cultures, animal cells cultures, mixed microbial cultures, and enzyme biocatalysis. HNNs were mainly applied for process analysis, process monitoring, development of software sensors, open- and closed-loop control, batch-to-batch control, model predictive control, intensified design of experiments, quality-by-design, and recently for the development of digital twins. Most previous HNN studies combined shallow Feedforward Neural Networks (FFNNs) with physical laws, such as macroscopic material balance equations, following the semiparametric design principle. Only recently, deep HNNs based on deep FFNNs, Convolution Neural Networks (CNN), Long Short-Term Memory (LSTM) networks and Physics Informed Neural Networks (PINNs) have been reported. The biopharma sector is currently a major driver but applications to biologics quality attributes, new modalities, and downstream processing are significant research gaps.