In the intricate domain of text analysis, syntactic dependency trees play a crucial role in unraveling the web of relations among entities embedded in the text. These trees provide a structural roadmap, guiding us through the complex syntax to pinpoint the interactions and connections between different entities. However, the challenge lies in sifting through this intricate structure to extract relevant information, a task that requires precision and discernment. Traditional approaches often rely on rule-based pruning methods to simplify these dependency structures, focusing on certain parts while discarding others. Yet, this approach has its pitfalls, as it can overlook critical nuances and connections that are vital for a comprehensive understanding of the text. Addressing this gap, our research introduces the Syntactic Dependency-Aware Neural Networks (SDANNs), a groundbreaking model designed to harness the full power of the entire dependency tree. This approach marks a significant departure from the conventional methods. Instead of the rigid rule-based pruning, SDANNs implement a more flexible and dynamic 'soft-pruning' technique. This method allows the model to adaptively focus on the sub-structures within the dependency tree that are most relevant for understanding the relationships between entities. By doing so, it ensures that no vital information is overlooked, and all potential connections are considered. The efficacy of SDANNs is not just theoretical but has been empirically validated through extensive testing and evaluations across a wide range of tasks. These tasks include the extraction of complex relations spanning multiple sentences, as well as detailed analyses at the sentence level. In each of these scenarios, SDANNs have demonstrated a remarkable ability to leverage the full structural complexity of dependency trees. This capability sets them apart from existing models, enabling a more nuanced and comprehensive analysis of textual relations. The results of these evaluations consistently show that SDANNs not only meet but significantly exceed the performance of prior models. This superiority is evident in the way SDANNs handle the multifaceted and often subtle interactions within the text, offering insights that were previously inaccessible with conventional methods. In summary, the Syntactic Dependency-Aware Neural Networks represent a significant advancement in the field of text analysis. By fully embracing the complexity of syntactic dependency trees and employing a sophisticated 'soft-pruning' approach, SDANNs open new avenues for exploring and understanding the intricate relationships that exist within written language. This model stands as a testament to the potential of combining advanced neural network architectures with a deep understanding of linguistic structures, paving the way for more accurate, nuanced, and comprehensive analyses of text.

Mammalian cells have evolved to function under Earth’s gravity, but how they respond to microgravity remains largely unknown. Neural stem cells (NSCs) are essential for the maintenance of central nervous system (CNS) functions during development and the regeneration of all CNS cell populations. Here, we examined the behavior of space (SPC)-flown NSCs as they readapted to Earth’s gravity. We found that most of these cells survived the space flight and self-renewed. Yet, some showed autophagy-like behaviors (ALB). To ascertain if the secretome from SPC-flown NSCs contained molecules inducing this behavior, we incubated naïve, non-starved NSCs in a medium containing SPC-NSCs secretome. We found a four-fold increase in the ALB rate. Proteomic analysis of the secretome revealed that the protein of highest content produced by SPC-NSCs was secreted protein acidic and rich in cysteine (SPARC), which triggers endoplasmic reticulum (ER) stress leading to lethal ALB. These results offer novel knowledge on the response of neural cells, particularly NSCs, subjected to space microgravity. Moreover, some secreted proteins have been identified as microgravity sensing, paving a new venue for future research aiming at targeting SPARC metabolism. Although we did not establish a direct relationship between ALB and SPARC as a potential marker, these results represent the first step in the identification of gravity sensing molecules as targets to be modulated and to design effective countermeasures to mitigate intracranial hypertension in astronauts using structure-based-protein design.

Artificial intelligence techniques are now widely used in various agricultural applications, including the detection of devastating diseases such as late blight (Phytophthora infestans) and early blight (Alternaria solani) affecting potato (Solanum Tuberorsum L.) crops. In this paper, we present a mobile application for potato crop pest detection based on deep neural networks. The images were taken from the PlantVillage dataset with a batch of 1000 images for each of the three identified classes. An exploratory analysis of the architectures used for early and late blight diagnosis in potatoes was performed, achieving an accuracy of 98.76%, with MobileNetv2. Based on the results obtained, an offline mobile application was implemented, supported on devices with Android 4.1 or later, also featuring an information section on the 27 pests affecting potato crops and a gallery of symptoms. For future work, segmentation techniques will be used to highlight the damaged region in the potato leaf by evaluating its extent and possibly identifying different types of pests affecting the same plant.

Condition monitoring of rotating shafts is essential for ensuring the reliability and optimal performance of machinery in diverse industries. In this context, as industrial systems become increasingly complex, the need for efficient data processing techniques is paramount. Deep learning has emerged as a dominant approach due to its capacity to capture intricate data patterns and relationships. However, a prevalent challenge lies in the black-box nature of many deep learning algorithms, which often operate without adhering to the underlying physical characteristics intrinsic to the studied phenomena. To address this limitation and enhance the fusion of data-driven methodologies with the fundamental physics of the system under study, this paper leverages physics-informed neural networks (PINNs). Specifically, a simple but realistic numerical case study of an extended Jeffcott rotor model, encompassing damping effects and anisotropic supports for a more comprehensive modelling, is considered. PINNs are used for the estimation of five parameters that characterize the health state of the system. These parameters encompass the radial and angular position of the static unbalance due to the disk installed on the shaft, the stiffness along the principal axes of elasticity, and the non-rotating damping coefficient. The estimation is conducted solely by exploiting the displacement signals from the centre of the disk and, to showcase the efficacy and precision provided by this novel methodology, various scenarios involving different constant rotational speeds are examined. Additionally, the impact of noisy input data is also taken into account within the analysis.

The BP neural network and PSO algorithm are applied into analyzing the reliability of long-span Concrete Filled Steel Tubular (CFST) arch bridges. Firstly, using BP neural network to fit the structural performance function. And then the PSO method was used to calculate the reliability index. A long-span Concrete Filled Steel Tubular arch bridges reliability indices were calculated whether considering the geometric nonlinearity or not. The calculation and analysis results showed that the BP neural network and PSO algorithm compensated the deficiency of the traditional reliability analysis methods, improved the calculation accuracy, provided a new thought and means for the research on the reliability of long-span bridge structure, and well applied to the reliability analysis of long-span Concrete Filled Steel Tubular arch bridges. Additionally, the reliability analysis of long-span Concrete Filled Steel Tubular arch bridges on service limit state must consider the geometric nonlinearity effect, or the results will tend to be unsafe.

Due to the recent trend of unmanned economy, retail stores have gradually reduced their service and cashier manpower. The retail product recognition becomes one of key issues for unmanned shopping. Although the success of deep neural networks has made object recognition feasible in a variety of applications, it still struggles to perform well on a large number of classes of retail products. In this paper, we propose an improved ensemble learning-based approach for retail product recognition. In the proposed approach, the object classification network is first improved through feature extraction and block attention. An ensemble model is then built by integrating multiple network models and using loss selection as model weights. In the experiments, the feasibility of our ensemble learning-based approach has been evaluated through many production items. The results demonstrate the effectiveness of the proposed approach compared with previous retail product recognition methods.

The aims of this paper can be traced back to goals number seven, twelve and thirteen introduced by Agenda 2030, namely "affordable and clean energy", “responsible consumption and production” and "climate action”, respectively. This is due to the fact that the present work supports decarbonization processes acting in a direct way on the electricity consumption achieved by plants pertaining to the waste industry; in fact, this study aims at the realization of a machine learning model for predicting the energy consumption achieved by a mechanical biological treatment (MBT) waste plant located in central Italy, given the distribution of the entering waste. This model is implemented in MATLAB. The model can be used to tune the distribution of the entering waste in order to adapt the plant energy consumption to the capability of the energy sources and can serve as a play-ground model for other energy transformation plants. The results of the study, which feed the literature on the application of artificial intelligence to real industrial plants, could be used to determine energy efficiency actions that could be incorporated into Property’s strategic planning.

In recent years, applying deep learning to solve physics problems has attracted much attention. Data-driven deep learning methods produce operators that can learn solutions to the whole system of partial differential equations. However, the existing methods are only evaluated on simple flow equations (e.g., Burger's equation), and only consider the generalization ability on different initial conditions. In this paper, we construct CFDBench, a benchmark with four classic problems in computational fluid dynamics (CFD): lid-driven cavity flow, laminar boundary layer flow in circular tubes, dam flows through the steps, and periodic Karman vortex street. Each flow problem includes data with various boundary conditions, fluid physical properties, and domain geometries. Compared to existing datasets, the advantages of CFDBench are (1) comprehensive. It contains common physical parameters such as velocity, pressure, and cavity fraction. (2) realistic. It is very suitable for deep learning solutions of fluid mechanics equations. (3) challenging. It has a certain learning difficulty, prompting to find models with strong learning ability. (4) standardized. CFDBench facilitates a comprehensive and fair comparison of different deep learning methods for CFD. We make appropriate modifications to popular deep neural networks to apply them to CFDBench and enable the accommodation of more changing inputs. The evaluation on CFDBench reveals some new shortcomings of existing works and we propose possible directions for solving such problems.

Facial emotion detection is a challenging task that deals with emotion recognition. It has applications in various domains, such as behavior analysis, surveillance systems and human-computer interaction (HCI). Numerous studies have been implemented to detect emotions, including classical machine learning algorithms and advanced deep learning algorithms. For the machine learning algorithm, the hand-crafted feature needs to be extracted, which is a tiring task and requires human effort. Whereas in deep learning models, automated feature extraction is employed from samples. Therefore, in this study, we have proposed a novel and efficient deep learning model based on Neural Architecture Search Network utilizing superior artificial networks such as RNN and child networks. We performed the training utilizing the FER 2013 dataset comprising seven classes: happy, angry, neutral, sad, surprise, fear, and disgust. Furthermore, we analyzed the robustness of the proposed model on CK+ datasets and comparing with existing techniques. Due to the implication of reinforcement learning in the network, most representative features are extracted from the sample network. It extracts all key features without losing the key information. Our proposed model is based on one stage classifier and performs efficient classification. Our technique outperformed the existing models attaining an accuracy of 98.14%, recall of 97.57%, and precision of 97.84%.

Raindrop size distribution (RSD) is an index for reflecting precipitation characteristics. Analyzing the differences of RSD plays an important role in understanding the precipitation microphysical processes and improving quantitative precipitation prediction of radar. In this paper, the RSD data of Dafang (57708) station, Majang (57828) station, and Luodian (57916) station (with an altitude of 1722.7m, 985.0m, and 441.5m, respectively) in Guizhou are analyzed according to the precipitation microphysical characteristics. First,Particles with particle size less than 1mm contributes the highest value to the density of particle number , and decrease with the descending altitude. Second, the GAMMA distribution fit shows a better fit compare to M-P distribution fit, and GAMMA distribution fit increases with the ascending altitude. Third, the mass-weighted average diameter is not sensitive enough to the change of precipitation intensity with correlation coefficients of 54.84%, but there is a obvious relationship between the average volume diameter and the precipitation intensity with correlation coefficients of 69.15%. Then, the precipitation prediction model is established using LSTM neural network after fusing the representative microphysical characteristics of RSD with radar and rain gauge data. The precipitation prediction model is applied to the Dafang (57708) site to predict precipitation for time range of 0-180 minutes, and it is found that for the prediction of convective cloud and stratiform cloud precipitation, the 60-minute prediction results are the most consistent with the actual precipitation, where the correlation coefficients are 92.87% and 92.57%, respectively. Conclusively, the results demonstrate that combining with the RSD base data could improve the reliability of precipitation forecasting.

The definition of artificial intelligence (AI) is the capacity of a computer or machine to mimic or simulate human intelligence. AI has been applied in different areas, including medicine. In cardiology, AI techniques have revolutionized the field of cardiac diagnosis, enabling the detection and prediction of cardiovascular diseases to be more accurate and efficient. This article provides an overview of machine learning, deep learning, and neural networks as they pertain to cardiac diagnosis. We will investigate how these techniques harness the power of data and algorithms to analyze complex patterns in cardiac data, thereby facilitating early detection, risk assessment, and treatment decision-making.

Identification of plants is a challenging task which aims to identify the family, genus, and species level according to morphological features. Automated deep learning-based computer vision algorithms are widely used for identifying plants and can help users to narrow down the possibilities. However, numerous morphological similarities between and within species make the classification difficult. In this paper, we tested a custom convolution neural network (CNN) and vision transformer (ViT) based models using the PyTorch framework to classify plants. We used a large dataset of 88K and 16K images for classifying plants at genus and species levels respectively. Our results show that for classifying plants at the genus level, ViT models perform better compared to CNN-based models ResNet50 and ResNet-RS-420, and other state-of-the-art CNN-based models suggested in previous studies on a similar dataset. The ViT model achieved top accuracy of 83.3% for classifying plants at the genus level. ViT models also perform better for classifying plants at the species level compared to CNN-based models ResNet50 and ResNet-RS-420, with a top accuracy of 92.5%. We show that the correct set of augmentation techniques plays an important role in classification success.

Parametric form findings of free-form space structures and qualitative assessment of their aesthetics are among the concerns of architects. This study aims to evaluate the aesthetic aspect of these structures using ML algorithms based on the expert's experiences. First, various datasets of forms were produced using a parametric algorithm of free-form space structures written in Grasshopper. Then, three multilayer perceptron ANN models were adjusted in their most optimal modes using the results of the preference test based on the aesthetic criteria including simplicity, complexity, and practicality. The results indicate that the ANN models can quantitatively evaluate the aesthetic value of free-form space structures.

Deep neural networks with powerful auto-optimization tools have been widely applied in various research fields, e.g., NLP and computer vision. However, existing neural network architectures typically are constructed using different inductive biases, e.g., preconceptions, expecting to decrease parameter search space during training, reduce computational cost or introduce expert knowledge in the neural network design. As an alternative, Multilayer Perceptron (MLP) provides much better freedom for exploration, has a lower inductive bias than convolutional neural networks (CNNs), and offers good flexibility in learning complex patterns. Even though, such neural architectures are commonly built in a flat Euclidean space, which is not necessarily the optimal space for any data, and is especially not good for modeling hierarchical correlations. Hyperbolic neural networks (HNNs) have gained attention for their ability to capture hierarchical structures present in complex data types like graphs. Recently, there has been an increasing interest to extend HNNs to computer vision tasks, motivated by the observations that images possess rich hierarchical relations. However, this is generally applied by employing a Euclidean backbone for learning higher-level semantic representations and only incorporating a hyperbolic classifier for classification, which, we argue, does not make full use of the advantage of hyperbolic space. Considering the recovery of the attention-free Multilayer Perceptron(MLP), in this paper, we extend it to non-Euclidean space and propose a novel architecture, named Hyperbolic Res-MLP (HR-MLP), that leverages fully hyperbolic layers to learn feature embeddings and perform image classification in an end-to-end fashion. With the help of the proposed Lorentz cross-patch and cross-channel layers, we can directly perform operations in the hyperbolic domain with fewer parameters, making it faster to train and providing comparatively better performance than its Euclidean counterpart. Experiments on CIFAR10, CIFAR100, and MiniImageNet demonstrate a comparable and superior performance when compared to Euclidean baselines. Our code is available at (https://github.com/Ahmad-Omar-Ahsan/HR-MLP)

To describe the hysteresis nonlinearities in smart actuators, numerous models have been presented in the literature, among which the Preisach operator would be the most effective one due to its capability in capturing multi-loop or sophisticated hysteresis curves. When such an operator is coupled with uncertain nonlinear dynamics, especially in noncanonical form, it is a challenging problem to develop techniques to cancel out the hysteresis effects, and at the same time achieve asymptotic tracking performance. To resolve this problem, in this paper, we investigate the problem of iterative inverse-based adaptive control for an uncertain noncanonical nonlinear systems with unknown input Preiasch hysteresis, and a new adaptive version of the closest match algorithm is proposed to compensate for the Preisach hysteresis. With our scheme, the stability and convergence of the closed-loop system can be established. The effectiveness of the proposed control scheme is illustrated by simulation and experiment results.

Keywords: Convolutional Neural Network, Deep Learning, Skin cancer, Generative Adversarial Network

Artificial intelligence is increasingly used in various branches of engineering. In this article, artificial neural networks are used to predict the crush resistance of corrugated packaging. Among the analysed packages were boxes with ventilation openings, packages with perforations, and typical flap boxes, which makes the proposed estimation method very universal. Typical shallow feedforward networks were used, which are perfect for regression problems, mainly when the set of input and output parameters is small, so no complicated architecture or advanced learning techniques are required. The input parameters of the neural networks are selected so as to take into account not only the material used for the production of the packaging, but also the dimensions of the box and the impact of ventilation holes and perforations on the load capacity of individual walls of the packaging. In order to maximize the effectiveness of neural network training process, the group of input parameters was changed so as to eliminate those to which the sensitivity of the model was the lowest. This allowed the selection of the optimal configuration of training pairs for which the estimation error was on the acceptable level. Finally, models of neural networks were selected, for which the training and testing error did not exceed 10%. The demonstrated effectiveness allows to conclude that the proposed set of universal input parameters is suitable for efficient training of a single neural network model capable of predicting the compressive strength of various types of corrugated packaging.

Symbolic semantic understanding of staff images is an important part in music information retrieval. Due to the complex composition of staff symbols and the strong semantic correlation between symbol spaces, it is difficult to understand the pitch and duration of each note during performances. In this paper, we design a semantic understanding system for optical staff symbols. The system uses the YOLOv5 to implement optical staff’s low-level semantic understanding stage, which understands the pitch and duration in natural scales and other symbols that affect the pitch and duration. The proposed note encoding reconstruction algorithm is used to implement high-level semantic understanding stage. Such algorithm understands the logical, spatial, and temporal relationships between natural scales and other symbols based on music theory, and outputs digital codes for the pitch and duration of main notes during performances. The model is trained with a self-constructed SUSN dataset. Experimental results of YOLOv5 show that the precision is 0.989 and the recall is 0.972. For the system, the error rate is 0.031 and the omission rate is 0.021. The paper concludes by analysing the causes of semantic understanding errors and offers recommendations for further research. The results of this paper provide a method for multimodal music artificial intelligence applications such as notation recognition through listening, intelligent score flipping and automatic performance.

With the continuous progress of remote sensing image object detection tasks in recent years, researchers in this field have gradually shifted the focus of their research from horizontal object detection to the study of object detection in arbitrary directions. It is worth noting that some properties are different from the horizontal object detection during oriented object detection that researchers have yet to notice much. This article presents the design of a straightforward and efficient arbitrary-oriented detection system, leveraging the inherent properties of the orientation task, including the rotation angle and box aspect ratio. In the detection of low aspect ratio objects, the angle is of little importance to the orientation bounding box, and it is even difficult to define the angle information in extreme categories. Conversely, in the detection of objects with high aspect ratios, the angle information plays a crucial role and can have a decisive impact on the quality of the detection results. By exploiting the aspect ratio of different targets, this letter proposes a ratio-balanced angle loss that allows the model to make a better trade-off between low-aspect ratio objects and high-aspect ratio objects. The rotation angle of each oriented object, which we naturally embed into a two-dimensional Euclidean space for regression, thus avoiding an overly redundant design and preserving the topological properties of the circular space. The performance of the UCAS-AOD, HRSC2016, and DLR-3K datasets show that the proposed model in this paper achieves a leading level in terms of both accuracy and speed. The code is released at https://github.com/minghuicode/Periodic-Pseudo-Domain.

Deep convolutional neural networks (CNN) are favored methods widely used in medical image processing due to their assured shown performance. Recently, the emergence of new lung diseases and the possibility of early detection of their symptoms has attracted many researchers to classify diseases by training deep CNNs on lung CT images. The trained networks are expected to distinguish between lung indications in different diseases, especially at the early stages of them. With the hope of achieving this purpose, we proposed an efficient deep CNN called AFEX-Net with adaptive feature extraction layers that successfully extract distinguishing features and classify chest CT images. The efficiency of the proposed network has two aspects: it is a lightweight network with low number of parameters and fast training and it has adaptive pooling layers and adaptive activation functions to increase its level of compatibility to the input data. The proposed network has been evaluated on a dataset with more than 10K chest CT slices, while an efficient pre-processing method is developed to remove any bias from the images. Additionally, we evaluated the performance of the proposed model on the public COVID-CTset dataset to prove the generalisability of our model. The obtained results confirm the competence of the proposed network in confronting medical images, where prompt and accurate learning is required.

We observed and studied the phenomenon of the redox front propagation in different polyaniline (PAni) deposited samples interfaced with liquid or solid polymeric electrolyte. The front of electrochemical conversion, in which the insulator and the conductor parts of PAni are interfaced, is studied observing the temporal evolution of the conductivity of PAni samples. The propagation of redox front was studied in electrochemically obtained thick (2 µm) and in thin (50 nm) poly-aniline films, obtained using Langmuir-Blodgett (LB) method. In the configuration that we tested, the speed of the red-ox front propagation for the thin LB films was found to be 200 micron/sec opening the way for the manufacturing large neural networks, realized using PAni based memristive devices, in which the memristance can be quickly changed in the programmable manner. The prototypes of the spiking neuron connections were manufactured on the basis of lithographically developed gold contacts, bridged by electrochemically grown polyaniline and placed under polymer electrolyte layer with only one counter electrode (gate) for the whole manifold of pseudo-two-terminal memristor bridges. The spike propagation was studied in such gold-polyaniline systems. The research opens the possibility of miniature spike or rate-based neural network circuits manufacture, based on metal pads and polyaniline.

Convolutional neural networks (CNN) have attracted much attention as a commonly used method for hyperspectral image (HSI) classification in recent years, however, CNNs can only be applied to Euclidean data and have limitations in dealing with relationships due to the limitations of local feature extraction. However, each pixel of a hyperspectral image contains a set of spectral bands that are correlated and interact with each other, and the methods used to process Euclidean data cannot effectively obtain these correlations. In contrast, the graph convolutional network (GCN) can be used in non-Euclidean data, but usually leads to oversmoothing and ignoring local detail features due to the need for superpixel segmentation processing to reduce computational effort. To overcome the above problems, we constructed a network a fusion network based on GCN and CNN, which contains two branches: a graph convolutional network based on superpixel segmentation and a convolutional network with added attention mechanism. The graph convolu-tional branch can extract the structural features and capture the relationships between the nodes, and the convolutional branch can extract the detailed features in the local fine region. Owing to the fact that the features extracted from the two branches are different, the classification performance can be improved by fusing the complementary features extracted from the two branches. To vali-date the proposed algorithm, experiments were conducted on three widely used datasets, namely Indian Pines, Pavia University, and Salinas, and the overall accuracy of 98.78% was obtained in the Indian Pines dataset, and the overall accuracy of 98.99% and 98.69% was obtained in the other two datasets. The results showed that the proposed fusion network can obtain richer features and achieve high classification accuracy.

In recent years, with the popularity of smart mobile devices, the interaction between devices and users, especially in the form of voice interaction, has become increasingly important. If smart devices can understand more users' emotional states through voice data, more customized services can be provided for users. This paper proposes a novel machine learning model for speech emotion recognition, which combines convolutional neural networks (CNN), long short-term memory neural networks (LSTM), and deep neural networks (DNN), called CLDNN. To make the designed system can recognize the audio signal closer to the human auditory system does, this article uses the Mel frequency cepstral coefficients (MFCCs) of audio data as the input of the machine learning model. First, the MFCCs of the voice signal is extracted as the input of the model, and the feature values of the data are calculated using several local feature learning blocks (LFLB) composed of one-dimensional CNN. Because the audio signals are time-series data, the feature values obtained from LFLBs then input into LSTM layer to enhance the learning on time-series level. Finally, fully connected layers are used for classification and prediction. Three databases RAVDESS, EMO-DB and IEMOCAP are used for the experiments in this paper. The experimental results show that the proposed method can improve the accuracy compared to other related researches in speech emotion recognition.

Automatic and real-time pose estimation is important in monitoring animal behavior, health and welfare. In this paper, we utilized pose estimation for monitoring farrowing process to prevent piglet mortality and preserve the health and welfare of sow. State-of-the-art Deep Learning (DL) methods have lately been used for animal pose estimation. The aim of this paper was to probe the generalization ability of five common DL networks (ResNet50, ResNet101, MobileNet, EfficientNet and DLCRNet) for sow and piglet pose estimation. These architectures predict body parts of several piglets and the sow directly from input video sequences. Real farrowing data from a commercial farm was used for training and validation of the proposed networks. The experimental results demonstrated that MobileNet was able to detect seven body parts of the sow with median test error of 0.61 pixels.

Respiratory rate is an important biomarker that indicates changes in the clinical condition of critically ill patients, so a surveillance tool that can accurately monitor the changing respiratory rate in real time is needed. Through investigating various pairs of machine learning models, we proposed new machine learning model for real-time respiratory rate estimation using photoplethysmogram. New photoplethysmogram-driven respiratory rate dataset(StMary) was collected from surgical intensive care unit of a tertiary referral hospital, using photoplethysmogram signal collector. For 50patients and 50healthy volunteers, 2-minute photoplethysmogram was collected for each subject twice. To evaluate the respiratory rate of subject, it was inputted into the deep neural network model we built, and dataset was splitted into training, validation, testing dataset, then 4-fold cross validation was exploited. Our deep neural network model trained with StMary and two public datasets(BIDMC and CapnoBase) individually, or selectively merged dataset had shown a low error rate in respiration rate measurements. Our model trained with StMary showed low mean absolute error score(1.0273±0.8965), and trained with 3 datasets(CapnoBase, BIDMC and StMary) showed a lower error rate(1.7359±1.6724) than the model trained with CapnoBase and BIDMC(1.9480±1.6751). We could verify the performance of model evaluating respiratory rate from photoplethysmogram, and our dataset could contribute as the clinical research data that supports artificial intelligence models evaluating respiratory rate and surveillance tools to test whether their monitoring function works properly.

Advances in computing technology provide an opportunity to grow and develop an effective crop protection system. This study is a new way of diagnosing plant diseases using a deep and convolutional neural network. Deep learning is the strengthened method of machine learning. It uses a neural network that imitates like a human brain and gains certain information. Although powerful, its training process necessitates millions of tagged data points. The study's Methodology consists of three key stages: data acquisition, pre-processing, and Model Building.

Accurate demand forecasting plays a critical role in most furniture businesses’ operational, tactical, and strategic decisions, as the demand in the furniture business is considered seasonal and becomes more complex in crises. In this work, a neural network model using the Long Short-Term Memory (LSTM) method was developed to forecast the demand for specific product groups. LSTM is a leading deep learning model for time series prediction, particularly seasonal, multi-item, and non-linear situations. The developed model was used to predict the demand based on old data before the Covid-19 pandemic and recent data of the first months of the pandemic as a fast response to the crisis. In addition, a comparison study was conducted between the developed model and the traditional planning inventory used by furniture businesses that provided us with the data. The results showed that the Covid-19 pandemic significantly impacted demand forecasting. Also, the fast response to Covid-19 pandemic has slightly increased the model performance. Finally, the comparison study demonstrated that our model is robust and better than the traditional demand forecasting method. Therefore, the developed model may help the business improve inventory and production planning to create a more flexible supply chain.

The dynamics of suspended sediment involves inherent non-linearity and complexity as a result of the presence of both spatial variability of the basin characteristics and temporal climatic patterns. As a result of this complexity, the conventional sediment rating curve (SRC) and other empirical methods produce inaccurate predictions. Deep neural networks (DNNs) have emerged as one of the advanced modeling techniques capable of addressing inherent non-linearity in hydrological processes over the last few decades. DNN algorithms are used to perform predictive analysis and investigate the interdependencies among the most pivotal water quantity and quality parameters i.e., discharge, suspended sediment concentration (SSC), and turbidity. In this study, the Long short-term memory (LSTM) algorithm of DNNs is used to model the discharge-suspended sediment relationship for the Stony Clove Creek. The simulations were run using primary data on discharge, SSC and turbidity. For the development of the DNN models and examining the effects of input vectors, combinations of different input vectors (namely discharge, and SSC) for the current and previous days are considered. Furthermore, a suitable modelling approach with an appropriate model input structure is suggested based on model performance indices for the training and testing phases. The performance of developed models is assessed using statistical indices such as root mean square error (RMSE), mean absolute error (MAE), and coefficient of determination (R2). Statistically, the performance of DNN-based models in simulating the daily SSC performed well with observed sediment concentration series data. The study demonstrates the suitability of the DNN approach for simulation and estimation of daily SSC, opening up new research avenues for applying hybrid soft computing models in hydrology.

This paper proposes an encryption scheme for high pixel density images. Based on the application of the quantum random walk algorithm, the Long short-term memory (LSTM) can effectively solve the problem of low efficiency of the quantum random walk algorithm in generating large-scale pseudorandom matrices, and further improve the statistical properties of the pseudorandom matrices required for encryption. The LSTM is then divided into columns and fed into the LSTM in order for training. Due to the randomness of the input matrix, the LSTM cannot be trained effectively, so the output matrix is predicted to be highly random. The LSTM prediction matrix of the same size as the key matrix is generated based on the pixel density of the image to be encrypted, which can effectively complete the encryption of the image. In the statistical performance test, the proposed encryption scheme achieves an average information entropy of 7.9992, an average number of pixels changed rate (NPCR) of 99.6231%, an average uniform average change intensity (UACI) of 33.6029% and an average correlation of 0.0032. Finally, various noise simulation tests are also conducted to verify its robustness in real-world applications where common noise and attack interference are encountered.

For a mobile robot, navigation in a densely crowded space can be a challenging and sometimes impossible task, especially with traditional techniques. In this paper, we present a framework to train neural controllers for differential drive mobile robots which must safely navigate a crowded environment while trying to reach a target location. To learn the robot’s policy, we train a convolutional neural network using two reinforcement learning algorithms, Deep Q-Networks (DQN) and Asynchronous Advantage Actor Critic (A3C), and develop a training pipeline that allows to scale the process to several compute nodes. We show that the asynchronous training procedure in A3C can be leveraged to quickly train neural controllers and test them on a real robot in a crowded environment.

Skin detection, the process of distinguishing between skin and non-skin regions in a digital image, is widely used in a variety of applications ranging from hand gesture analysis to body part tracking to facial recognition. Skin detection is a challenging problem that has received a lot of attention from experts and proposals from the research community in the context of intelligent systems, but the lack of common benchmarks and unified testing protocols has hampered fairness among approaches. Comparisons are very difficult. Recently, the success of deep neural networks has had a major impact on the field of image segmentation detection, resulting in various successful models to date. In this work, we survey the most recent research in this field and propose fair comparisons between approaches using several different datasets. The main contributions of this work are: (i) a comprehensive literature review of approaches to skin color detection and a comparison of approaches that may help researchers and practitioners choose the best method for their application; (ii) a comprehensive list of datasets that report ground truth for skin detection; (iii) a framework for evaluating and combining different skin detection approaches. Moreover, we proposed an ensemble of convolutional neural networks and transformers that obtains state of the art performance. All the code is made publicly available at https://github.com/LorisNanni

The simultaneous advances in artificial neural networks and photonic integration technologies have spurred extensive research in optical computing and optical neural networks (ONNs). The potential to simultaneously exploit multiple physical dimensions of time, wavelength and space give ONNs the ability to achieve computing operations with high parallelism and large-data throughput. Different photonic multiplexing techniques based on these multiple degrees of freedom have enabled ONNs with large-scale interconnectivity and linear computing functions. Here, we review the recent advances of ONNs based on different approaches to photonic multiplexing, and present our outlook on key technologies needed to further advance these photonic multiplexing/hybrid-multiplexing techniques of ONNs.

3D reconstruction from a single 2D input is a classic problem in the field of computer vision. With the advancements in deep learning, the performance of 3D reconstruction has also significantly improved. The reconstruction task is more difficult for objects with no textures or complex deformations. This paper serves as a review of recent literature on 3D reconstruction from a single view, with a focus on deep learning methods from 2018 to 2021. Due to lack of standard datasets or 3D shape representation methods, it is hard make direct comparisons between all reviewed methods. However, this paper reviews different approaches for reconstructing 3d shape as depth maps, surface normals, point clouds and meshes; along with various loss functions and evaluation metrics used to train and evaluate these methods.

Deep learning has shown remarkable results in every field, especially in the biomedical field, due to its ability to exploit large-scale datasets. A convolutional neural network (CNN) is a widely used deep learning approach to solve medical imaging problems. Over the past few years, many studies have focused on CNN-based techniques for brain tumor diagnosis. There are, however, still some critical challenges that CNNs face towards clinic application. This study presents a comprehensive review of current literature that involves CNN architectures for brain tumor classification. We compare the key achievements in the performance evaluation metrics of the applied classification algorithms. In addition, this review assesses the clinical effectiveness of the included studies to elaborate on the limitations and directions of this area for future work. No review focusing on the clinical effectiveness of previous works in this field has been published. We believe that this study has the potential to elevate the application of CNN-based deep learning methods in clinical practice and also can be a quick reference for biomedical researchers who are interested in this field.

When computers start to become a dominant part of technology around the 1950s, fundamental questions about reliable designs and robustness were of great relevance. Their development gave rise to the exploration of new questions such as what made brains reliable (since neurons can die) and how computers could get inspiration from neural systems. In parallel, the first Artificial Neural Networks came to life. Since then, the comparative view between brains and computers has been developed in new, sometimes unsuspected directions. With the rise of deep learning and the development of connectomics, an evolutionary look at how both hardware and neural complexity have evolved or designed is required. In this paper, we argue that important similarities have resulted both from convergent evolution (the inevitable outcome of architectural constraints) and inspiration of hardware and software principles guided by toy pictures of neurobiology. Moreover, dissimilarities and gaps originate from the lack of major innovations that have paved the way to biological computing (including brains) that are completely absent within the artificial domain. As it occurs within synthetic biocomputation, we can also ask whether alternative minds can emerge from A.I.\ designs. Here we take an evolutionary view of the problem and discuss the remarkable convergences between living and artificial designs and what are the pre-conditions to achieve artificial intelligence.

The ridge line of the western Pacific subtropical high (WPSHRL) plays an important role in determining the shift of the summer rain belt in eastern China. In this study, we developed a forecast system for the June WPSHRL index based on the latest autumn and winter sea surface temperature (SST). Considering the adverse condition of the small observed sample size, a very simple neural network (NN) model was selected to extract the non-linear relationship between input predictors (SST) and target predictands (WPSHRL) in the forecast system. In addition, some techniques are used to deal with the adverse condition, enhance the stabilization of forecast skills, and analyze the interpretability of the forecast system. The forecast experiments show that the linear correlation coefficient between the predictions from the forecast system and their corresponding observations is around 0.6, and about three-fifths of the observed abnormal years (the years with an obviously high or low WPSHRL index) are successfully predicted. Furthermore, sensitivity experiments show that the forecast system is relatively stable in terms of forecast skill. The above evaluations suggest that the forecast system is valuable in a real application sense.

Graph neural networks (GNNs) build on the success of deep learning models by extending them for use in graph spaces. Transfer learning has proven extremely successful for traditional deep learning problems: resulting in faster training and improved performance. Despite the increasing interest in GNNs and their use cases, there is little research on their transferability. This research demonstrates that transfer learning is effective with GNNs, and describes how source tasks and the choice of GNN impact the ability to learn generalisable knowledge. We perform experiments using real-world and synthetic data within the contexts of node classification and graph classification. To this end, we also provide a general methodology for transfer learning experimentation and present a novel algorithm for generating synthetic graph classification tasks. We compare the performance of GCN, GraphSAGE and GIN across both the synthetic and real-world datasets. Our results demonstrate empirically that GNNs with inductive operations yield statistically significantly improved transfer. Further we show that similarity in community structure between source and target tasks support statistically significant improvements in transfer over and above the use of only the node attributes.

The present study aims to examine the role the social and digital infrastructures might have during the building process of the Smart Regions in the Italian context. Within this framework, it is possible to identify some essential research questions, such as why the same regions are growing faster than the other and which type of effects could be generated from the different connectivity between the regions. Since the Smart Region concept is still composed of technical reports, pilot projects and experiences from a limited number of cities on the international stage, this work it is tried to use a new approach, applying either a neuronal model, the Self-Organizing Maps, and the multivariate regression approach, to extrapolate the existence of possible future conditions for the rising of Smart Regions in Italy, studying the evolution of the used database during the period 2005 – 2016. From the analysis what emerged is that the only bridging social capital dimension, empirically speaking, feed the regional innovation growth because the structure of social relationship facilitates interactions across social, political and economic agents; but there are institutional deficits, most pronounced in Italy and other European countries.

The commonly-used large-scale knowledge bases have been facing challenges in open domain question answering tasks which are caused by the loose knowledge association and weak structural logic of triplet-based knowledge. To find a way out of this dilemma, this work proposes a novel metaknowledge enhanced approach for open domain question answering. We design an automatic approach to extract metaknowledge and build metaknowledge network from Wiki documents. For the purpose of representing the directional weighted graph with hierarchical and semantic features, we present an original graph encoder GE4MK to model the metaknowledge network. Then a metaknowledge enhanced graph reasoning model MEGr-Net is proposed for question answering, which aggregates both relational and neighboring interactions comparing with R-GCN and GAT. Experiments have proved the improvement of metaknowledge over main-stream triplet-based knowledge. We have found that the graph reasoning models and pre-trained language models also have influences on the metaknowledge enhanced question answering approaches.

Evaporation from surface water plays a key role in water accounting of basins, water resources management, and irrigation systems management, so simulating evaporation with high accuracy is very important. In this study, two methods for simulating pan evaporation under different climatic conditions in Iran were developed. In the first method, six experimental relationships (linear, quadratic, and cubic, with two input combinations) were determined for Iran’s six climate types, inspired by a multilayer perceptron neural network (MLP-NN) neuron and optimized with the genetic algorithm. The best relationship of the six was selected for each climate type, and the results were presented in a three-dimensional graph. In the second method, the best overall relationship obtained in the first method was used as the basic relationship, and climatic correction coefficients were determined for other climate types using the genetic algorithm optimization model. Finally, the accuracy of the two methods was validated using data from 32 synoptic weather stations throughout Iran. For the first method, error tolerance diagrams and statistical coefficients showed that a quadratic experimental relationship performed best under all climatic conditions. To simplify the method, two graphs were created based on the quadratic relationship for the different climate types, with the axes of the graphs showing relative humidity and temperature, and with pan evaporation was drawn as contours. For the second method, the quadratic relationship for semi-dry conditions was selected as the basic relationship. The estimated climatic correction coefficients for other climate types lay between 0.8 and 1 for dry, semi-dry, semi-humid, Mediterranean climates, and between 0.4 and 0.6 for humid and very humid climates, indicating that one single relationship cannot be used to simulate pan evaporation for all climatic conditions in Iran. The validation results confirmed the accuracy of the two methods in simulating pan evaporation under different climatic conditions in Iran.

Formaldehyde (HCHO) is one of the most important carcinogenic air contaminants. However, the lack of global surface concentration of HCHO monitoring is currently hindering research on outdoor HCHO pollution. Traditional methods are either restricted to small areas or data- demanding for a global scale of research. To alleviate this issue, we adopted neural networks to estimate surface HCHO concentration with confidence intervals in 2019, where HCHO vertical column density data from TROPOMI, in-situ data from HAPs (harmful air pollutants) monitoring network and ATom mission are utilized. Our result shows that the global surface HCHO average concentration is 2.30 μg/m3. Furthermore, in terms of regions, the concentration in Amazon Basin, Northern China, South-east Asia, Bay of Bengal, Central and Western Africa are among the highest. The results from our study provides a first dataset of the global surface HCHO concentration. In addition, the derived confidence interval of surface HCHO concentration adds an extra layer for the confidence to our results. As a pioneer work in adopting confidence interval estimation into AI-driven atmospheric pollutant research and the first global HCHO surface distribution dataset, our paper will pave the way for the rigorous study on global ambient HCHO health risk and economic loss, thus providing a basis for pollutant controlling policies worldwide.

Diabetic retinopathy is the most common microvascular complication of diabetes mellitus and one of the leading causes of blindness globally. Due to the progressive nature of the disease, earlier detection and timely treatment can lead to substantial reductions in the incidence of irreversible vision-loss. Artificial intelligence (AI) screening systems have offered clinically acceptable and quicker results in detecting diabetic retinopathy from retinal fundus and optical coherence tomography (OCT) images. Thus, this systematic review and meta-analysis of relevant investigations was performed to document the performance of AI screening systems that were applied to fundus and OCT images of patients from diverse geographic locations including North America, Europe, Africa, Asia, and Australia. A systematic literature search on Medline, Global Health, and PubMed was performed and studies published between October 2015 and January 2020 were included. The search strategy was based on the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) reporting guidelines, and AI-based investigations were mandatory for studies inclusion. The abstracts, titles, and full-texts of potentially eligible studies were screened against inclusion and exclusion criteria. Twenty-one studies were included in this systematic review; 18 met inclusion criteria for the meta-analysis. The pooled sensitivity of the evaluated AI screening systems in detecting diabetic retinopathy was 0.93 (95% CI: 0.92-0.94) and the specificity was 0.88 (95% CI: 0.86-0.89). The included studies detailed training and external validation datasets, criteria for diabetic retinopathy case ascertainment, imaging modalities, DR-grading scales, and compared AI results to those of human graders (e.g., ophthalmologists, retinal specialists, trained nurses, and other healthcare providers) as a reference standard. The findings of this study showed that the majority AI screening systems demonstrated clinically acceptable levels of sensitivity and specificity for detecting referable diabetic retinopathy from retinal fundus and OCT photographs. Further improvement depends on the continual development of novel algorithms with large and gradable sets of images for training and validation. If cost-effectiveness ratios can be optimized, AI can become a financially sustainable and clinically effective intervention that can be incorporated into the healthcare systems of low-to-middle income countries (LMICs) and geographically remote locations. Combining screening technologies with treatment interventions such as anti-VEGF therapy, acellular capillary laser treatment, and vitreoretinal surgery can lead to substantial reductions in the incidence of irreversible vision-loss due to proliferative diabetic retinopathy.

The colossal depths of the deep neural network sometimes suffer from ineffective backpropagation of the gradients through all its depths. Whereas, The strong performance of shallower multilayer neural structures prove their ability to increase the gradient signals in the early stages of training which easily gets backpropagated for global loss corrections. Shallow neural structures are always a good starting point for encouraging the sturdy feature characteristics of the input. In this research, a shallow, deep neural structure called PrimeNet is proposed. PrimeNet is aimed to dynamically identify and encourage the quality visual indicators from the input to be used by the subsequent deep network layers and increase the gradient signals in the lower stages of the training pipeline. In addition to this, the layerwise training is performed with the help of locally generated errors which means the gradient is not backpropagated to previous layers, and the hidden layer weights are updated during the forward pass, making this structure a backpropagation free variant. PrimeNet has obtained state-of-the-art results on various image datasets, attaining the dual objective of (1) compact dynamic deep neural structure, which (2) eliminates the problem of backwards-locking. The PrimeNet unit is proposed as an alternative to traditional convolution and dense blocks for faster and memory-efficient training, outperforming previously reported results aimed at adaptive methods for parallel and multilayer deep neural systems.

Deep neural networks (DNNs) have made a huge impact in the field of machine learning by providing unbeatable humanlike performance to solve real-world problems such as image processing and natural language processing (NLP). Convolutional neural network (CNN) and recurrent neural network (RNN) are two typical architectures that are widely used to solve such problems. Time sequence-dependent problems are generally very challenging, and RNN architectures have made an enormous improvement in a wide range of machine learning problems with sequential input involved. In this paper, different types of RNN architectures are compared. Special focus is put on two well-known gated-RNN’s Long Term Short Memory (LSTM) and Gated Recurrent Unit (GRU). We evaluated these models on the task of force estimation system in pouring. In this study, four different models including multi-layers LSTM, multi-layers GRU, single-layer LSTM and single-layer GRU) were created and trained. The result suggests that multi-layer GRU outperformed other three models.

Laboratory medicine has evolved from a mainly manual profession, providing few selected test results to a highly automated and standardized medical discipline, generating millions of test results per year. As next inevitable evolutional step, artificial intelligence (AI) algorithms will need to assist us in structuring and making sense of the masses of diagnostic data collected today. Such systems will be able to connect clinical and diagnostic data and to provide valuable suggestions in diagnosis, prognosis or therapeutic options. They will merge the often so separated worlds of the laboratory and the clinics. When used correctly, it will be a tool, capable of freeing the physicians time so that he/she can refocus on the patient. In this narrative review I therefore aim to provide an overview of what AI is, what applications currently are available in healthcare and in laboratory medicine in particular. I will discuss the challenges and pitfalls of applying AI algorithms and I will elaborate on the question if healthcare workers will be replaced by such systems in the near future.

This study presents the consecutive modification of orange peel (OP) by NaOH and sodium dodecyl sulfate (SDS) for simultaneous elimination of basic dyes from the binary system and modeling the adsorption process using an intelligent tool. The natural and modified biosorbents were characterized by variety of analyses such as: field emission scanning electron microscopy with energy dispersive X-ray, N2 physisorption and Fourier transform infrared spectroscopy techniques. The influence of various variables on dye removal like pH, the quantity of biosorbents, dyes concentration, contact time, and temperature in the binary system were investigated and optimized by an artificial neural network (ANN) model as an intelligent tool. The optimum quantity of the sorbent was found to be 0.30 g for orange peel (OP) and 0.25 g for NaOH-treated OP (NOP) and SDS-decorated NOP (SNOP) at pH = 7. The kinetics and thermodynamics investigations showed that the removal of dyes obeyed the pseudo-second-order kinetic model and were spontaneous and exothermic in nature. Moreover, in order to describe the mechanism of sorption process, desorption studies of dyes were carried out. The desorption percentages of methylene blue (MB) in water and HCl were found to be in the range of 1.93%–4.76% and 18.87%–28.76%, respectively; in addition, the desorption percentages of crystal violet (CV) in water and HCl were obtained to be in the range of 4.11%–7.41% and 32.84%–43.00%, respectively; which could be a recommendation ion exchange or electrostatic attachment of dyes onto biosorbents. The ANN predictions matched with the experimental data very well (0.95308 < R2 < 0.99191 and 0.98335 < R2 < 0.99773 for MB and CV, respectively) which indicated high accuracy of the ANN model. In addition, the relative importance of each parameter was calculated by Garson’s equation.

The forecast of electricity demand has been a recurrent research topic for decades, due to its economical and strategic relevance. Several Machine Learning (ML) techniques have evolved in parallel with the complexity of the electric grid. This paper reviews a wide selection of approaches that have used Artificial Neural Networks (ANN) to forecast electricity demand, aiming to help newcomers and experienced researchers to appraise the common practices and to detect areas where there is room for improvement in the face of the current widespread deployment of smart meters and sensors, which yields an unprecedented amount of data to work with. The review looks at the specific problems tackled by each one of the selected papers, at the results attained by their algorithms, and at the strategies followed to validate and compare the results. This way, it is possible to highlight some peculiarities and algorithm configurations that seem to consistently outperform others in specific settings.

Emotion recognition defined as identifying human emotion and is directly related to different fields such as human-computer interfaces, human emotional processing, irrational analysis, medical diagnostics, data-driven animation, human-robot communi- cation and many more. The purpose of this study is to propose a new facial emotional recognition model using convolutional neural network. Our proposed model, “ConvNet”, detects seven specific emotions from image data including anger, disgust, fear, happiness, neutrality, sadness, and surprise. This research focuses on the model’s training accuracy in a short number of epoch which the authors can develop a real-time schema that can easily fit the model and sense emotions. Furthermore, this work focuses on the mental or emotional stuff of a man or woman using the behavioral aspects. To complete the training of the CNN network model, we use the FER2013 databases, and we test the system’s success by identifying facial expressions in the real-time. ConvNet consists of four layers of convolution together with two fully connected layers. The experimental results show that the ConvNet is able to achieve 96% training accuracy which is much better than current existing models. ConvNet also achieved validation accuracy of 65% to 70% (considering different datasets used for experiments), resulting in a higher classification accuracy compared to other existing models. We also made all the materials publicly accessible for the research community at: https://github.com/Tanoy004/Emotion-recognition-through-CNN.

Recent advancement of deep learning has been elevated the multifaceted nature in various applications of this field. Artificial neural networks are now turning into a genuinely old procedure in the vast area of computer science; the principal thoughts and models are more than fifty years of age. However, in this modern computing era, 3rd generation intelligent models are introduced by scientists. In the biological neuron, actual film channels control the progression of particles over the layer by opening and shutting in light of voltage changes because of inborn current flows and remotely led to signals. A comprehensive 3rd generation, Spiking Neural Network (SNN) is diminishing the distance between deep learning, machine learning, and neuroscience in a biologically-inspired manner. It also connects neuroscience and machine learning to establish high-level efficient computing. Spiking Neural Networks initiate utilizing spikes, which are discrete functions that happen at focuses as expected, as opposed to constant values. This paper is a review of the biological-inspired spiking neural network and its applications in different areas. The author aims to present a brief introduction to SNN, which incorporates the mathematical structure, applications, and implementation of SNN. This paper also represents an overview of machine learning, deep learning, and reinforcement learning. This review paper can help advanced artificial intelligence researchers to get a compact brief intuition of spiking neural networks.

Machine Learning has completely transformed health care system, which transmits medical data through IOT sensors. So it is very important to encrypt them to protect patient data. encrypting medical images from a performance perspective consumes time; hence the use of an auto encoder is essential. An auto encoder is used in this work to compress the image as a vector prior to the encryption process. The digital image passes across description function and a decoder to get back the image in the proposed work; various experiments are carried out on hyper parameters to achieve the highest outcome of the classification. The findings demonstrate that the combination of Mean Square Logarithmic Error as the loss function, ADA grad as an optimizer, two layers for the encoder, and another reverse for the decoder, RELU as the activation function generates the best auto encoder results. The combination of Mean square error (lose function), RMS prop (optimizer), three layers for the encoder and another reverse for the decoder, and RELU (activation function) has the best classification result. All the experiments with different hyper parameter has run almost very close to each other even when changing the number of layers. The running time is between 9 and 16 second for each epoch.

Recently, much attention has been devoted to finding highly efficient and powerful activation functions for CNN layers. Because activation functions inject different nonlinearities between layers that affect performance, varying them is one method for building robust ensembles of CNNs. The objective of this study is to examine the performance of CNN ensembles made with different activation functions, including six new ones presented here: 2D Mexican ReLU, TanELU, MeLU+GaLU, Symmetric MeLU, Symmetric GaLU, and Flexible MeLU. The highest performing ensemble was built with CNNs having different activation layers that randomly replaced the standard ReLU. A comprehensive evaluation of the proposed approach was conducted across fifteen biomedical data sets representing various classification tasks. The proposed method was tested on two basic CNN architectures: Vgg16 and ResNet50. Results demonstrate the superiority in performance of this approach. The MATLAB source code for this study will be available at https://github.com/LorisNanni.

Choline and choline metabolites are essential for all cellular functions. They have also been reported to be crucial for neural development. In this work, we studied the functional characteristics of the choline uptake system in human neural stem cells (hNSCs). Additionally, we investigated the effect of extracellular choline uptake inhibition on the cellular activities in hNSCs. We found that the mRNAs and proteins of choline transporter-like protein 1 (CTL1) and CTL2 were expressed at high levels. Immunostaining showed that CTL1 and CTL2 were localized in the cell membrane and partly in the mitochondria, respectively. The uptake of extracellular choline was saturable and performed by a single uptake mechanism, which was Na+-independent and pH-dependent. We conclude that CTL1 is responsible for extracellular choline uptake, and CTL2 may uptake choline in the mitochondria and be involved in DNA methylation via choline oxidation. Extracellular choline uptake inhibition caused intracellular choline deficiency in hNSCs, which suppressed cell proliferation, cell viability, and neurite outgrowth. Our findings contribute to the understanding of the role of choline in neural development as well as the pathogenesis of various neurological diseases caused by choline deficiency or choline uptake impairment.

This paper presents a new framework to classify floor plan elements and represent them in a vector format. Unlike existing approaches using image-based learning frameworks as the first step to segment the image pixels, we first convert the input floor plan image into vector data and utilize graph neural network. Our framework consists of three steps. (1) image pre-processing and vectorization of the floor plan image. (2) region adjacency graph conversion. (3) graph neural network on converted floor plan graphs. Our approach is able to capture different types of indoor elements including basic elements such as walls, doors, and symbols as well as spatial elements such as rooms and corridors. In addition, the proposed method can also detect element shapes. Experimental results show that our framework can classify indoor elements with an F1 score of 95%, with scale and rotation invariance. Furthermore, we propose a new graph neural network model that takes the distance between nodes into account, which is a valuable feature of spatial network data.

: The significance of heating load (HL) accurate approximation is the primary motivation of this research to distinguish the most efficient predictive model among several neural-metaheuristic models. The proposed models are through synthesizing multi-layer perceptron network (MLP) with ant lion optimization (ALO), biogeography-based optimization (BBO), dragonfly algorithm (DA), evolutionary strategy (ES), invasive weed optimization (IWO), and league champion optimization (LCA) hybrid algorithms. Each ensemble is optimized in terms of the operating population. Accordingly, the ALO-MLP, BBO-MLP, DA-MLP, ES-MLP, IWO-MLP, and LCA-MLP presented their best performance for population sizes of 350, 400, 200, 500, 50, and 300, respectively. The comparison was carried out by implementing a ranking system. Based on the obtained overall scores (OSs), the BBO (OS = 36) featured as the most capable optimization technique, followed by ALO (OS = 27) and ES (OS = 20). Due to the efficient performance of these algorithms, the corresponding MLPs can be promising substitutes for traditional methods used for HL analysis.

Early prediction of thermal loads plays an essential role in analyzing energy-efficient buildings' energy performance. On the other hand, stochastic algorithms have recently shown high proficiency in dealing with this issue. These are the reasons that this work is dedicated to evaluating an innovative hybrid method for predicting the cooling load (CL) in buildings with residential usage. The proposed model is a combination of artificial neural networks and stochastic fractal search (SFS-ANN). Two benchmark algorithms, namely the grasshopper optimization algorithm (GOA) and firefly algorithm (FA), are also considered to be compared with the SFS. The non-linear effect of eight independent factors on the CL is analyzed using each model's optimal structure. Evaluation of the results outlined that all three metaheuristic algorithms (with more than 90 % correlation) can adequately optimize the ANN. In this regard, this tool's prediction error declined by nearly 23, 18, and 36 % by applying the GOA, FA, and SFS techniques. Also, all used accuracy criteria indicated the superiority of the SFS over the benchmark schemes. Therefore, it is inferred that utilizing the SFS along with ANN provides a reliable hybrid model for the early prediction of CL.

The main parameters for calculation of relative humidity are the wet-bulb depression and dry bulb temperature. In this work, easy-to-used predictive tools based on statistical learning concepts, i.e., the Adaptive Network-Based Fuzzy Inference System (ANFIS) and Least Square Support Vector Machine (LSSVM) are developed for calculating relative humidity in terms of wet bulb depression and dry bulb temperature. To evaluate the aforementioned models, some statistical analyses have been done between the actual and estimated data points. Results obtained from the present models showed their capabilities to calculate relative humidity for divers values of dry bulb temperatures and also wet-bulb depression. The obtained values of MSE and MRE were 0.132 and 0.931, 0.193 and 1.291 for the LSSVM and ANFIS approaches respectively. These developed tools are user-friend and can be of massive value for scientists especially, those dealing with air conditioning and wet cooling towers systems to have a noble check of the relative humidity in terms of wet bulb depression and dry bulb temperatures.

Mineral identification is an important part of geological analysis. Traditional identification methods rely on either the experiences of the appraisers or the various measuring instruments and the methods are either easily influenced by the experiences or need too much work. To solve the above problems, there have been studies using image recognition and intelligent algorithms to identify minerals. But the current studies can not identify many minerals, and the accuracy is low. To increase the number of identified mineral categories and the accuracy, we proposed the method that uses both the mineral images and the Mohs hardness in the deep neural networks to identify the minerals. The experimental results showed that the method can reach 94.0% Top-1 accuracy and 99.9% Top-5 accuracy for 28 common minerals. The model that combines image and Mohs hardness together can identify more minerals and increase the accuracy using less training data.

Landslides impact on human activities and socio-economic development especially in mountainous areas. This study focuses on the comparison of the prediction capability of advanced machine learning techniques for rainfall-induced shallow landslide susceptibility of Deokjeokri catchment and Karisanri catchment in South Korea. The influencing factors for landslides i.e. topographic, hydrologic, soil, forest, and geologic factors are prepared from various sources based on availability and a multicollinearity test is also performed to select relevant causative factors. The landslide inventory maps of both catchments are obtained from historical information, aerial photographs and performing field survey. In this study, Deokjeokri catchment is considered as a training area and Karisanri catchment as a testing area. The landslide inventories content 748 landslide points in training and 219 points in testing areas. Three landslide susceptibility maps using machine learning models i.e. Random Forest (RF), Extreme Gradient Boosting (XGBoost) and Deep Neural Network (DNN) are prepared and compared. The outcomes of the analyses are validated using the landslide inventory data. A receiver operating characteristic curve (ROC) method is used to verify the results of the models. The results of this study show that the training accuracy of RF is 0.757 and the testing accuracy is 0.74. Similarly, training accuracy of XGBoost is 0.756 and testing accuracy is 0.703. The prediction of DNN revealed acceptable agreement between susceptibility map and the existing landslides with training and testing accuracy of 0.855 and 0.802, respectively. The results showed that, the DNN model achieved lower prediction error and higher accuracy results than other models for shallow landslide modeling in the study area

Online transactions are becoming more popular in present situation where the globe is facing an unknown disease COVID-19. Now authorities of Countries requested peoples to use cashless transaction as far as possible. Practically it is not always possible to use it in all transactions. Since number of such cashless transactions have been increasing during lockdown period due to COVID-19, fraudulent transactions are also increasing in a rapid way. Fraud can be analysed by viewing a series of customer transactions data that was done in his/her previous transactions. Normally banks or other transaction authorities warned their customers about the transaction If any deviation is noticed by them from available patterns. These authorities think that it is possibly of fraudulent transaction. For detection of fraud during COVID-19, banks and credit card companies are applying various methods such as data mining , decision tree, rule based mining, neural network, fuzzy clustering approach and machine learning methods. These approaches is try to find out normal usage pattern of customers based on their past activities. The objective of this paper is to find out such fraud transactions during such unmanageable situation.Digital payment schemes are often threatened by fraudulent activities. Detecting fraud transaction in during money transfer may save customers from financial loss. Mobile based money transactions are focused in this paper for fraud detection. A Deep Learning (DL) framework is suggested in this paper that monitors and detects fraudulent activities. Implementing and applying recurrent neural network on PaySim generated synthetic financial dataset, deceptive transactions are identified. The proposed method is capable to detect deceptive transactions with an accuracy of 99.87%, F1-Score of 0.99 and MSE of 0.01.

Retinopathy is a human eye disease that causes changes in retinal blood vessels that leads to bleed, leak fluid and vision impairment. Symptoms of retinopathy are blurred vision, changes in color perception, red spots, and eye pain. In this paper, a new methodology based on Convolutional Neural Networks (CNN) is developed and proposed to diagnose and give a decision about the presence of retinopathy. The CNN model is trained by different images of eyes that have retinopathy and those which do not have retinopathy. The performance of the proposed model is compared with the related methods of DREAM, KNN, GD-CNN and SVM. Experimental results show that the proposed CNN performs better.

With the widespread of embedded sensing capabilities of mobile devices, there has been unprecedented development of context-aware solutions. This allows the proliferation of various intelligent applications such as those for remote health and lifestyle monitoring, intelligent personalized services, etc. However, activity context recognition based on multivariate time series signals obtained from mobile devices in unconstrained conditions is naturally prone to imbalance class problems. This means that recognition models tend to predict classes with the majority number of samples whilst ignoring classes with the least number of samples, resulting in poor generalization. To address this problem, we propose to augment the time series signals from inertia sensors with signals from ambient sensing to train deep convolutional neural networks (DCNN) models. DCNN provides the characteristics that capture local dependency and scale invariance of these combined sensor signals. Consequently, we developed a DCNN model using only inertial sensor signals and then developed another model that combined signals from both inertia and ambient sensors aiming to investigate the class imbalance problem by improving the performance of the recognition model. Evaluation and analysis of the proposed system using data with imbalanced classes show that the system achieved better recognition accuracy when data from inertial sensors are combined with those from ambient sensors such as environment noise level and illumination, with an overall improvement of 5.3% accuracy.

This paper analyzes the robust dissipativity of uncertain stochastic generalized neural networks (USGNNs) with Markovian jumping parameters and time-varying delays. In practical applications most of the systems refer to uncertainties, hence, the norm-bounded parameter uncertainties and stochastic disturbance are considered. Then, by constructing an appropriate Lyapunov-Krasovskii functional (LKF) and by employing integral inequalities LMI-based sufficient conditions of the considered systems are established. Numerical simulations are given to show the merit of the presented results.

This paper investigates the use of deep learning techniques to perform energy demand forecasting. Specifically, the authors have adapted a deep neural network originally thought for image classification and composed of a convolutional neural network (CNN) followed by a multilayered fully connected artificial neural network (ANN). The convolutional part of the network was fed with a grid of temperature forecasting data distributed in the area of interest in order to extract a featured temperature. The subsequent ANN is then fed with this calculated temperature along with other data related to the timing of the forecast. The proposed structure was first trained and then used in a real setting aimed to provide the French energy demand forecast using ARPEGE forecasting weather data. The results show that the performance of this approach is in the line of the performance provided by the reference RTE subscription-based service, which opens the possibility to obtain high accuracy forecasting using widely accessible deep learning techniques through open-source machine learning platforms.

Computational simulation using mathematical models is a useful method for understanding the complex behavior of a living system. The majority of studies using mathematical models to reveal biological mechanisms uses one of the two main approaches: the bottom-up or the top-down approach. When we aim to analyze a large-scale network, such as a comprehensive knowledge-integrated model of a target phenomenon, for example a whole-cell model, the variation of analyses is limited to particular kind of analysis because of the size and complexity of the model. To analyze a large-scale regulatory network of neural differentiation, we developed a hybrid method that combines both approaches. To construct a mathematical model, we extracted network motifs, subgraph structures that recur more often in a metabolic network or gene regulatory network than in a random network, from a large-scale regulatory network, detected regulatory motifs among them, and combined these motifs. We confirmed that the model reproduced the known dynamics of HES1 and ASCL1 before and after differentiation, including oscillation and equilibrium of their concentrations. The model also reproduced the effects of overexpression and knockdown of the Id2 gene. Our model suggests that the characteristic change in HES1 and ASCL1 expression in the large-scale regulatory network is controlled by a combination of four feedback loops, including a large loop which has not been focused on. The model extracted by our hybrid method has the potential to reveal the critical mechanisms of neural differentiation. The hybrid method is applicable to other biological events.

In this paper, the fixed-time stabilization problem for a class of uncertain chained system is addressed by using a novel nonsingular recursive terminal sliding mode control approach. A fixed-time controller and an adaptive law are designed to guarantee the uncertain chained form system both Lyapunov stable and fixed-time convergent within the settling time. The advantage of the controller based on the sliding mode is that the settling time does not depend on the system initial state. Furthermore, we use RBF neural network to estimate the uncertainty of the system. Finally, the simulation results demonstrate the performance of the control laws.

Nowadays industrial dryers are used instead of traditional methods for drying. In designing dryers suitable for controlling the process of drying and reaching a high quality product, it is necessary to predict the instantaneous moisture loss during drying. For this purpose, ten mathematical-experimental models with a neural network model based on the kinetic data of pistachio drying are studied. The data obtained from the cabinet dryer will be evaluated at four temperatures of inlet air and different air velocities. The pistachio seeds will be placed in a thin layer on an aluminum sheet on a drying tray and weighed by a scale attached to the computer at different times. In the neural network, data are divided into three parts: educational (60%), validation (20%) and test (20%). Finally, the best mathematical-experimental model using genetic algorithm and the best neural network structure for predicting instantaneous moisture are selected based on the least squared error and the highest correlation coefficient.

This paper presents a novel approach for semantic segmentation of building roof in dense urban environment with Deep Convolution Neural Network (DCNN) using imagery acquired by a Chinese Very High Resolution (VHR) satellite mission, i.e. GaoFen-2 (GF-2). To provide an operational end-to-end work flow for accurate build roof mapping with feature extraction as well as image segmentation, a fully convolutional DCNN with both convolutional and deconvolutional layers is designed to perform the VHR image analysis for labeling pixels. Since the diverse urban patterns and building styles in large areas, sample image data sets of building roof and non-building roof are collected over different metropolitan regions in China. We selected typical cities with dense urban environment in each metropolitan region as study areas for collecting training and test samples. High performance cluster with GPU-mounted workstations is employed to perform the model training and optimization. With the building roof samples collected over different cities, the predictive model with multiple NN layers is developed for building roof labeling. The validation of the building roof map shows that the overall accuracy(OA) and the mean Intersection Over Union( mIOU) of DCNN based segmentation are 94.67%, 0.85 respectively, while CRF-refined segmentation achieved OA of 94.69% and mIOU of 0.83. The results suggest that the proposed approach is a promising solution for building roof mapping with VHR images over large areas across different urban and building patterns. With the operational acquisition of GF2 VHR imagery, it is expected to develop an automated pipeline for operational built-up area monitoring and timely update of building roof map over large areas.

This paper presents a novel approach for indoor acoustic source localization using microphone arrays and based on a Convolutional Neural Network (CNN). The proposed solution is, to the best of our knowledge, the first published work in which the CNN is designed to directly estimate the three dimensional position of an acoustic source, using the raw audio signal as the input information avoiding the use of hand crafted audio features. Given the limited amount of available localization data, we propose in this paper a training strategy based on two steps. We first train our network using semi-synthetic data, generated from close talk speech recordings, and where we simulate the time delays and distortion suffered in the signal that propagates from the source to the array of microphones. We then fine tune this network using a small amount of real data. Our experimental results show that this strategy is able to produce networks that significantly improve existing localization methods based on SRP-PHAT strategies. In addition, our experiments show that our CNN method exhibits better resistance against varying gender of the speaker and different window sizes compared with the other methods.

An effective process monitoring strategy is a requirement for meeting the challenges posed by increasingly complex products and manufacturing processes. To address these needs, this study investigates a comprehensive scheme based on classical machine learning methods, deep learning algorithms, and feature extraction and selection techniques. In a first step, a novel deep learning architecture based on convolutional neural networks (CNN) and gated recurrent units (GRU) is introduced to predict the local weld quality based on mid-wave infrared (MWIR) and near-infrared (NIR) image data. The developed technology is used to discover critical welding defects including lack of fusion (false friends), sagging and lack of penetration, and geometric deviations of the weld seam. Additional work is conducted to investigate the significance of various geometrical, statistical, and spatio-temporal features extracted from the keyhole and weld pool regions. Furthermore, the performance of the proposed deep learning architecture is compared to that of classical supervised machine learning algorithms, such as multi-layer perceptron (MLP), logistic regression (LogReg), support vector machines (SVM), decision trees (DT), random forest (RF) and k-Nearest Neighbors (kNN). Optimal hyperparameters for each algorithm are determined by an extensive grid search. Ultimately, the three best classification models are combined into an ensemble classifier that yields the highest detection rates and achieves the most robust estimation of welding defects among all classifiers studied, which is validated on previously unknown welding trials.

The volume of fluid (VOF) method is a popular technique for the direct numerical simulations of flows involving immiscible fluids. A discrete volume fraction field evolving in time represents the interface, in particular, to compute its geometric properties. The height function method (HF) is based on the volume fraction field, and its estimate of the interface curvature converges with second-order accuracy with grid refinement. Data-driven methods have been recently proposed as an alternative to computing the curvature, with particular consideration for a well-balanced input data set generation and symmetry preservation. In the present work, a two-layer feed-forward neural network is trained on an input data set generated from the height function data instead of the volume fraction field. The symmetries for rotations, reflections, and the anti-symmetry for the phase swapping have been considered to reduce the number of input parameters. The neural network, establishing a correlation between curvature and height function values, can efficiently predict the local interface curvature. We compare the trained neural network to the standard Height Function method to assess its performance and robustness.

Numerous image processing techniques (IPTs) have been employed to detect crack defects, offering an alternative to human-conducted onsite inspections. These IPTs manipulate images to extract defect features, particularly cracks in surfaces produced through Additive Manufacturing (AM). This article presents a vision-based approach that utilizes deep convolutional neural networks (CNNs) for crack detection in AM surfaces. Traditional image processing techniques face challenges with diverse real-world scenarios and varying crack types. To overcome these challenges, our proposed method leverages CNNs, eliminating the need for extensive feature extraction. Annotation for CNN training is facilitated by LabelImg without the requirement for additional IPTs. The trained CNN, enhanced by OpenCV preprocessing techniques, achieves an outstanding 99.54% accuracy on a dataset of 14,982 annotated images with resolutions of 1536 × 1103 pixels. Evaluation metrics exceeding 96% precision, 98% recall, and a 97% F1-score highlight the precision and effectiveness of the entire process.

Deep underground laboratories offer advantages for conducting high-precision observations of weak geophysical signals, benefitting from a low background noise level. It is both valuable and feasible to enhance strong, noisy ground electromagnetic (EM) field data using synchronously recorded underground EM signals, which typically exhibit a high signal-to-noise ratio. In this study, we propose an EM field reconstruction method employing a Long Short-Term Memory (LSTM) recurrent neural network with referenced deep underground EM observations. Initially, a deep learning model was developed to capture the time-varying features of underground multi-component EM fields using the LSTM recurrent neural network. Subsequently, this model was applied to process synchronously observed strong, noisy data from other conventional observation systems, such as those at surface, to achieve noise suppression through signal reconstructions. Both theoretical analysis and practical observational data suggest that the proposed method effectively suppresses noise and reconstructs clean EM signals. This method is efficient and time-saving, representing an effective approach to fully utilizing the advantages of deep underground observation data. Furthermore, this method could be extended to the processing and analysis of other geophysical data.

With the advancement of technology, signal modulation types are becoming increasingly diverse and complex. The phenomenon of signal time-frequency overlap during transmission poses significant challenges for the classification and recognition of mixed signals, including poor recognition capabilities and low generality. This paper presents a recognition model for fine-grained analysis of mixed signal characteristics, proposing a Geometry Coordinate attention mechanism and introducing a low-rank bilinear pooling module to more effectively extract signal features for classification. The model employs a residual neural network as its backbone architecture and utilizes the Geometry Coordinate attention mechanism for time-frequency weighted analysis based on information geometry theory. This analysis targets multiple-scale features within the architecture, producing time-frequency weighted features of the signal. These weighted features are further analyzed through a low-rank bilinear pooling module, combined with the backbone features, to achieve fine-grained feature fusion. This results in a fused feature vector for mixed signal classification. Experiments were conducted on a simulated dataset comprising 39,600 mixed signal time-frequency plots. The model was benchmarked against a baseline using a residual neural network. The experimental outcomes demonstrated an improvement of 9% in the exact match ratio and 5% in the Hamming score. These results indicate that the proposed model significantly enhances the recognition capability and generalizability of mixed signal classification.

Partial Differential Equations (PDEs) constitute a fundamental framework for modeling various physical phenomena across diverse fields such as physics, engineering, and finance. Solving PDEs accurately and efficiently remains a significant challenge, particularly when dealing with complex systems or high-dimensional data. In this paper, we present a novel approach based on deep learning techniques to investigate the effective factors in solving PDEs. Our methodology leverages the expressive power of deep neural networks to learn and represent the underlying relationships between input parameters and the solutions of PDEs. By incorporating domain knowledge into the network architecture and training process, our approach aims to provide insights into the key factors influencing the solution of PDEs. We demonstrate the effectiveness of our method through numerical experiments on various PDEs, showcasing its capability to identify important features and improve predictive accuracy. Our findings suggest that deep learning techniques offer a promising avenue for understanding and analyzing the complex dynamics inherent in PDEs, paving the way for enhanced computational methods in scientific and engineering applications.

Tunicate orthologs in the human genome comprise just 84 genes of the 19,872 protein-coding genes, yet they stand at the base of the Olfactores clade - which radiated to generate thousands of tunicate and vertebrate species. What were the powerful drivers among the 84 that enabled this process? Many of these orthologs are present in gene families. We discuss the biological role of each family and the orthologs’ quantitative contribution to the family. Most important was the evolution of a second type of cadherin. This, a Type II cadherin, had the property of detaching the cell containing that cadherin from cells that expressed the Type I class. The set of such Type II cadherins could now detach and move away from their Type I neighbours, a process which would eventually evolve into the formation of the neural crest, “ the fourth germ layer”, providing a wide range of possibilities for further evolutionary invention. A second important contribution were key additions to the broad development of the muscle and nerve protein and visual perception tool-kits. These developments in mobility and vision provided the basis for the development of the efficient predatory capabilities of the Vertebrata.

COVID-19, caused by the SARS-CoV-2 virus, has spread around the world and killed around 6.9 million people. Rapid and accurate diagnosis is essential for preventing and controlling the disease, reducing transmission and consequently saving lives. RT-PCR is the gold standard test used to detect the disease. However, the test is expensive and the result is time-consuming, which makes mass testing difficult, especially in countries with limited resources. In addition, the test has high analytical specificity and low diagnostic sensitivity, which leads to false-negative results. Several studies in the literature report the presence of hematological and biochemical alterations in infected patients and use these alterations with machine learning algorithms to help diagnose the disease. Therefore, this article presents the results obtained by different neural network architectures based on Adaptive Resonance Theory (ART) for the diagnosis of COVID-19. The study was conducted in two distinct stages: the first consisted of selecting the best ART network among several, using three open-access datasets and comparing the results with the literature. In the second stage, the chosen model was tested on a dataset containing patients from various hospitals in four countries. In addition, the model was subjected to external validation, including data from a country not present during the training and adjustment of the model, in order to validate the robustness and generalization capacity of the model. The results obtained by the ART networks in this study are promising, outperforming not only classical models, but also the deep learning models often used in the literature. Validation on data from different countries strengthens the model’s reliability and effectiveness.

This paper presents a novel approach to sentiment analysis specifically customized for predicting stock market movements, bypassing the need for external dictionaries which are often unavailable for many languages. Our methodology directly analyzes textual data, with a particular focus on context-specific sentiment words within neural network models. This specificity ensures our sentiment analysis is both relevant and accurate in identifying trends in the stock market. We employ sophisticated mathematical modeling techniques to enhance both the precision and interpretability of our models. Through meticulous data handling and advanced machine learning methods, we leverage large datasets from Twitter and financial markets to examine the impact of social media sentiment on financial trends. We achieved an accuracy exceeding 75%, highlighting the effectiveness of our modeling approach, which we further refined into a convolutional neural network model. This achievement contributes valuable insights into sentiment analysis within the financial domain, thereby improving the overall clarity of forecasting in this field.

In this essay, I revisit Neural Networks (NNs) through the lens of evolutionary dynamics. From the two most important features of NNs, I recover the two most general equations of evolutionary dynamics. These two equations may thus serve as a connection between NNs and a body of knowledge that has been built over multiple centuries and has found application across a vast range of disciplines. This connection may therefore also help explain why NNs with the two features in question are applicable across a much broader range of domains than initially envisioned.

End-to-end learned image compression codecs have notably emerged in recent years. These codecs have demonstrated superiority over conventional methods, showcasing remarkable flexibility and adaptability across diverse data domains while supporting new distortion losses. Despite challenges such as computational complexity, learned image compression methods inherently align with learning-based data processing and analytic pipelines due to their well-suited internal representations. The concept of Video Coding for Machines has garnered significant attention from both academic researchers and industry practitioners. This concept reflects the growing need to integrate data compression with computer vision applications. In light of these developments, we present a comprehensive survey and review of lossy image compression methods. Additionally, we provide a concise overview of two prominent international standards, MPEG Video Coding for Machines and JPEG AI. These standards are designed to bridge the gap between data compression and computer vision, catering to practical industry use cases.

Quantum computing (QC) has opened the door to advancements in machine learning (ML) tasks that are currently implemented in the classical domain. Convolutional neural networks (CNNs) are classical ML architectures that exploit data locality and possess a simpler structure than fully-connected multi-layer perceptrons (MLPs) without compromising the accuracy of classification. However, the concept of preserving data locality is usually overlooked in the existing quantum counterparts of CNNs, particularly for extracting multi-features in multidimensional data. In this paper, we present a multidimensional quantum convolutional classifier (MQCC) that performs multidimensional and multi-feature quantum convolution with average and Euclidean pooling and thus adapting the CNN structure to a variational quantum algorithm (VQA). Average pooling is based on the quantum Haar transform (QHT) and Euclidean pooling is based on partial quantum measurement. The experimental work was conducted using multidimensional data to validate the correctness and demonstrate the scalability of the proposed method utilizing both noisy and noise-free quantum simulations. We evaluated the MQCC model with reference to reported work on state-of-the-art quantum simulators from IBM Quantum and Xanadu using a variety of standard ML datasets. The experimental results showed favorable characteristics of our proposed techniques compared to existing work with respect to a number of quantitative metrics such as the number of training parameters, cross-entropy loss, classification accuracy, circuit depth, and quantum gate count.

Ensuring optimal performance and reliability in rotor-bearing systems is crucial for industrial applications. Imbalance and shaft bow in these systems can lead to decreased efficiency and increased vibrations. Early detection and mitigation of a rotor’s faults are essential, and model-based fault identification has gained much attention in the manufacturing industry for years. Over the past two decades, however, the development of fault diagnosis rules with data-driven and artificial intelligence (AI) has become a trend, and in the foreseeable future, AI combined with big data will become mainstream. Nevertheless, the critical role of rotating machinery in manufacturing introduces a challenge, as the availability of fault data is often insufficient. This limitation renders the establishment of diagnostic rules using data-driven methods and AI technologies impractical. In light of these challenges, this study proposes a novel hybrid approach, that combines a physical model with machine learning (ML) techniques for the diagnosis of multi-faults (imbalance and shaft-bow demonstrated) in a Jeffcott rotor. To overcome the lack of real-world faulted, labeled datasets, a physics-based Jeffcott rotor model is first derived and then used to generate abundant fault datasets for ML. Subsequently, simulated data are employed for the training of an artificial neural network (ANN), enabling the network to learn from and analyze the vast array of generated data. The results prove that a well-trained feed-forward neural network (FNN) can accurately isolate and diagnose the imbalance and shaft-bow faults using the simulated data and the real data from the Jeffcott rotor experiment. These physics-based and ML approaches prove effective particularly for multi-fault, offering new possibilities for advanced rotor system monitoring and maintenance strategies in industrial applications.

This paper proposed an optimal tracking control scheme through adaptive dynamic programming(ADP) for a class of partially unknown discrete-time nonlinear systems based on radial basis function neural network(RBF-NN). In order to acquire the unknown system dynamics, we use two RBF-NNs, the one is used to construct the identifier, and the another is used to directly approximate the steady-state control input, where a novel adaptive law is proposed to update neural network weights. While the optimal feedback control and the cost function are derived via feedforward neural networks approximating, it is proposed to regulate the tracking error, the critic network and the actor network are then trained online to obtain the solution of the associated Hamilton–Jacobi–Bellman (HJB) equation being built under the ADP framework. Simulations verify the effectiveness of the optimal tracking control technique using the neural networks.

When dealing with limited sample sizes or missing values in time series data, achieving high accuracy in forecasting becomes challenging. This challenge is particularly common in wind energy forecasting, where many wind farms lack complete data. In this paper, we employ the widely-used ARIMA (Autoregressive Integrated Moving Average) model for time series forecasting. To address the difficulties posed by limited sample sizes and missing values, we adopt a unique approach by utilizing multiple ARIMA models for different datasets. A key distinction from other multiple models methods for time series forecasting lies in each model within our approach using its own dataset, making the fusion process more intricate. To overcome this, we incorporate neural networks and two techniques for model fusion: direct fusion with a learning mechanism and compensation for missing data through training. For each ARIMA model, we introduce a noise estimation method to circumvent the white noise assumptions inherent in ARIMA models. Finally, we apply these methods to wind energy prediction, and our experimental results showcase a significant enhancement in forecasting accuracy.

The speed-density or flow-density relationship is a fundamental aspect of traffic flow theory. Single-regime models often face challenges in consistently fitting empirical data under diverse traffic conditions. This paper addresses the inaccuracies of single-regime models, attributing them not only to their functional forms but also to data dispersion and sample selection bias. To mitigate these issues, the data is segmented into day and night periods, leading to the creation of separate models for each. Calibration results, conducted on data from two freeways, Tehran-Karaj and Tehran-Qom, reveal distinct challenges, with the former exhibiting high dispersion and the latter showing low dispersion. Surprisingly, single-regime models demonstrate notable effectiveness when applied to distinct day and night data. Further analysis, categorizing data into day and night, results in reduced errors (20-60% improvement in R-Square), emphasizing the potential for enhanced accuracy by considering distinct regimes. Overall, the study underscores the significance of regime-specific considerations for accurate traffic flow modeling. Upon thorough evaluation, the study highlights the superiority of the neural network Multi-layer Perceptron (MLP) over traditional models. The research discusses the implications of incorporating day and night parameters into the models, emphasizing their potential for accuracy enhancement.

Considering that cryptocurrencies are now present in practically every financial transaction because they are widely accepted as an alternate means of making payments and exchanging currencies, academics and economists have more opportunities to study cryptocurrency prices. Over the years, investors, traders and investment banks have found it difficult to predict the closing daily price of Ethereum due to its rapid price fluctuation. The daily closing price of cryptocurrency is essential to consider when trading or investing in Ethereum. This report focuses on carrying out a comparative study of the predictive capabilities of deep machine learning algorithms with a stacking ensemble modelling framework using daily historical observations of the price of Ethereum obtained from Coindesk, tweets extracted from Twitter ranging from the 1st of August 2022 to the 8th of August 2022 and other five covariates (closing price lag1, closing price lag2, noltrend, daytype and month) engineered from the closing price of Ethereum. Seven models are used to compute the forecasts for the daily closing price of Ethereum; these are the recurrent neural network, ensemble stacked recurrent neural network, gradient boosting machine, generalized linear model, distributed random forest, deep neural networks and stacked ensemble for gradient boosting machine, generalized linear model, distributed random forest and deep neural networks. The main evaluation metric used is the mean absolute error. According to MAE, RNN forecasts outperform the other model’s forecasts in this study, producing an MAE of 0.0309.

Red mud (RM), a bauxite residue, contains hazardous radioactive wastes and alkaline material and poses severe surface water and groundwater contamination risks, necessitating recycling. Pretreated RM can be used to make adsorbents for water treatment. However, its performance is affected by many factors, resulting in a nonlinear correlation and coupling relationship. This study aimed to identify the best formula for an RM adsorbent using a mathematical model that examines the relationship between 11 formulation types (e.g., pore-assisting agent, component modifier, and external binder) and nine properties (e.g., specific surface area, wetting angle, and Zeta potential, among others). This model was built using a back-propagation neural networks(BP) based on single-factor experimental data and orthogonal test data. The model trained and predicted the established network structure to obtain the optimal adsorbent formula. The RM particle adsorbents had a pH of 10.16, specific surface area(BET) of 48.92 m2·g−1, pore volume of 2.10 cm3·g−1, compressive strength(ST)) of 1.12 KPa, and 24-h immersion pulverization rate(η_m) of 3.72%. In the removal of total phosphorus in flotation tailings backwater, it exhibited a good adsorption capacity (Q)and total phosphorous removal rate(η) of 48.63 mg·g−1 and 95.13%, respectively.

The implementation of neural networks (NN) on edge devices enables local processing of wireless data but faces challenges such as high computational complexity and memory requirements when deep neural networks (DNN) are used. Shallow neural networks customized for specific problems are more efficient, requiring fewer resources, and resulting in a lower latency solution. An additional benefit of the smaller network size is that it is suitable for real-time processing on edge devices. The main concern with shallow neural networks is their accuracy performance compared to DNNs. In this paper, we demonstrate that a customized adaptive activation function (AAF) can meet the accuracy of a DNN. We designed an efficient FPGA implementation for a customized segmented spline curve neural network (SSCNN) structure to replace the traditional fixed activation function with an AAF. We compared our SSCNN with different neural network structures such as real-valued time delay neural network (RVTDNN), augmented real-valued time delay neural network (ARVTDNN), and deep neural networks with different parameters. Our proposed SSCNN implementation uses 40% fewer hardware resources and no Block RAMS compared to the DNN with similar accuracy. We experimentally validate this computationally efficient and memory-saving FPGA implementation of SSCNN for digital predistortion of RF power amplifiers using the AMD/Xilinx RFSoC ZCU111. The implemented solution uses less than 3% of the available resources. The solution also enables an increase of the clock frequency to 221.12 MHz, allowing the transmission of wide bandwidth signals.

With the recent advances in Machine Learning (ML), several models have been successfully applied to financial and accounting data to predict the likelihood of companies’ bankruptcy. However, time series have received little attention in literature with a lack of studies on the application of Deep Learning sequence models as Recurrent Neural Networks (RNN) and the recent Attention-based models in general. In this research work, we investigated the application of Long Short Term Memory (LSTM) networks to exploit time series of accounting data for bankruptcy prediction. The main contributions of our work are the following: a) we have proposed a Multi-head LSTM that models each financial variable in a time window independently and compared it with a single-input LSTM and other traditional ML models. The Multi-head LSTM outperforms all the other models; b) We identify the optimal time series length for bankruptcy prediction to be equal to 4 years of accounting data; c) We made public the dataset we used for the experiments that include data from 8262 different public companies in the American stock market generated in the period between 1999-2018. Furthermore, we prove the efficacy of the Multi-head LSTM models in terms of lower false positives and better division of the two classes.

Photovoltaic systems and wind energy systems are generally used in distributed system. However, zero-sequence and harmonic currents occur due to photovoltaic systems, and positive-sequence harmonic currents occur due to wind turbines. Thus, the sheath current in high voltage cables increases due to these currents and poor power quality in distributed generation plants. Metal parts of the cable are grounded to prevent the negative effects of sheath currents. When long high voltage cable lines are grounded with the method in the literature, high sheath current and voltage occur on the cable sheath. Therefore, the SSBLR method is designed as a new method for the distributed generation system. The SSBLR method has been optimized with prediction methods and multi-objective optimization methods to prevent negative effects of the sheath current. Pareto optimal method is used as multi-objective optimization method, and artificial neural network, hybrid artificial neural network and regression methods are used as prediction method. The most dominant result is searched using different estimation and optimization methods in pareto optimal method. When the artificial neural network-genetic algorithm hybrid method is used as a prediction method, and genetic algorithm used as optimization method, significant decreases are observed in sheath current and voltage.

In this study, tree selection/plantation DSS (Decision Support Systems) were reviewed and assessed based on essential objectives in the available literature. We reviewed existing DSS using multiple data sources and available online resources such as web interfaces. We compared the existing DSS, focusing primarily on five primary objectives in this study, which DSS may address when it comes to the tree selection, including a) Climate resilience, b) Infrastructure/Space Optimization, c) Agroforestry, d) Ecosystem services, e) Urban sustainability. Climate resilience of tree species and urban sustainability is addressed relatively less in existing systems, which can be holistically internalized in future DSS tools Based on this review, DNN (Deep Neural Networks) is recommended to address achieving trade-offs between complex objectives such as maximizing ecosystem services, climate resilience of tree species, maintaining Agroforestry, and other benefits.

This proposed research explores a novel approach to image classification by deploying a complex-valued neural network (CVNN) on a field-programmable gate array (FPGA), specifically for classifying 2D images transformed into polar form. The aim of this research is to address the limitations of existing neural network models in terms of energy and resource efficiency, by exploring the potential of FPGA-based hardware acceleration in conjunction with advanced neural network architectures like CVNNs. The methodological innovation of this research lies in the Cartesian to polar transformation of 2D images, effectively reducing the input data volume required for neural network processing. Subsequent efforts focused on constructing a CVNN model optimized for FPGA implementation, emphasizing the enhancement of computational efficiency and overall performance. The experimental findings provide empirical evidence supporting the efficacy of the image classification system developed in this study. One of the developed models, CVNN_128, achieves an accuracy of 88.3% with an inference time of just 1.6ms and a power consumption of 4.66mW for the classification of the MNIST test dataset consists of 10,000 frames. While there is a slight concession in accuracy compared to recent FPGA implementations that achieve 94.43%, our model significantly excels in classification speed and power efficiency—surpassing existing models by more than a factor of 100. In conclusion, the paper demonstrates the substantial advantages of FPGA-implementation of CVNNs for image classification tasks, particularly in scenarios where speed, resource, and power consumption are critical. The study’s reproducible results and corresponding code are available on GitHub at the following link: https://github.com/mahmad2005/CVNNonFPGA

Seismic velocity inversion is one of the most critical issues in the field of seismic exploration and has long been the focus of numerous experts and scholars. In recent years, the advancement of machine learning technologies has infused new vitality into the research of seismic velocity inversion and yielded a wealth of research outcomes. Typically, seismic velocity inversion based on machine learning lacks control over physical processes and interpretability. Starting from wave theory and the physical processes of seismic data acquisition, this paper proposes a method for seismic velocity model inversion based on Physical Embedding Recurrent Neural Networks. Firstly, the wave equation is a mathematical representation of the physical process of acoustic waves propagating through a medium, and the finite difference method is an effective approach to solving the wave equation. With this in mind, we introduce the architecture of recurrent neural networks to describe the finite difference solution of the wave equation, realizing the embedding of physical processes into machine learning. Secondly, in seismic data acquisition, the propagation of acoustic waves from multiple sources through the medium represents a high-dimensional causal time series (wavefield snapshots), where the influential variable is the velocity model, and the received signals are the observations of the wavefield. This forms a forward modeling process as the forward simulation of the wavefield equation, and the use of error back-propagation between observations and calculations as the velocity inversion process. Through time-lapse inversion and incorporating the causal information of wavefield propagation, the non-uniqueness issue in velocity inversion is mitigated. Through mathematical derivations and theoretical model analyses, the effectiveness and rationality of the method are demonstrated. In conjunction with simulation results for complex models, the method proposed in this paper can achieve velocity inversion in complex geological structures.

Against the backdrop of the current carbon peaking and carbon neutrality policies, higher requirements have been put forward for the upgrading and construction of smart grids. Non-intrusive load monitoring (NILM) technology is a key technology for advanced measurement systems at the end of the power grid. It obtains detailed power information of the load, without the need for traditional hardware deployment. The key step to solve this problem is load decomposition and identification. This paper first utilized Long Short Term Memory-Denoising Autoencoder(LSTM-DAE) to decompose the mixed current signal on the household busbar and obtain the current signals of multiple independent loads that constituted the mixed current. Then, the obtained independent current signals were combined with voltage signals to generate multi-cycle colored Voltage-Current(VI) trajectories, which were color-coded according to the background. These colored VI trajectories with background colors formed a feature library. When the CNN network was used for load recognition, considering the influence of hyperparameters on recognition results, the BOA algorithm was used for optimization, and the optimized CNN network was employed for VI trajectory recognition. Finally, the proposed method was validated using the PLAID dataset. Experimental results showed that the proposed method exhibited better performance in load decomposition and identification.

The power system has rapidly grown and expanded over the past decades and has been experiencing major changes and challenges. The increase in energy demand and the modern advancements in the smart grid, such as solar and wind energies and electric vehicles, have led to complexity and complications for utilities. A further layer of complexity and difficulty was added by the rapid expansion of behind-the-meter (BTM) photovoltaic (PV) systems with various designs and characteristic features. The rapid increase and the invisible solar power (BTM) have led to fluctuations in power grid stability and reliability and to inefficiency. Accurate forecasting of load generation will help to assure optimal planning, minimize the negative effects of the PV systems, and minimize the operational and maintenance costs. The authors propose a solution that uses combinations of K-means clustering with neural network machine learning models, AMI real-world PV load generation, and weather data to forecast the generation load at customer locations to achieve a 2.49% error between actual and predicted generation load.

The study aims to analyze the characteristic parameters of rooftop photovoltaic (PV) power generation on industrial plant buildings in the Ningxia region of China, in order to evaluate the impact of passive design characteristic parameters on the benefits of PV power generation and determine the degree of impact of different passive design characteristic parameters. The methodology involves analyzing the characteristics of existing industrial plant buildings in Ningxia, China, and conducting a series of regional studies on the parameters of plant PV power generation influencing factors. The simulation results revealed that five features, including roof form, PV panel laying pattern, PV panel laying area, azimuth angle, and PV module material, have a significant impact on PV power generation benefits of industrial plant buildings.The study further uses MATLAB 2022b to build a backpropagation neural network model with 5 neurons in the implicit layer to predict the PV power generation benefits. The model calculates the whole-life cycle power generation benefits and the whole-life cycle reduction of carbon emissions as output data. Finally, the Mean Impact Value (MIV) method is employed to select the feature parameter values one by one within their range, and the passive design feature parameters that have the greatest influence on the prediction results are identified as the PV panel laying method, the PV panel tilt angle, and the PV material parameters, respectively.

Applications of gradient method for nonlinear optimization in development of Gradient Neural Network (GNN) and Zhang Neural Network (ZNN) are investigated. Particularly, the solution of the matrix equation AXB=D which changes over time is studied using the novel GNN model, termed as GGNN(A,B,D). The GGNN model is developed applying GNN dynamics on the gradient of the error matrix used in the development of the GNN model. The convergence analysis shows that the neural state matrix of the GGNN(A,B,D) design converges asymptotically to the solution of the matrix equation AXB=D, for any initial state matrix. It is also shown that the convergence result is the least square solution which is defined depending on the selected initial matrix. A hybridization of GGNN with analogous modification GZNN of the ZNN dynamics is considered. The Simulink implementation of presented GGNN models is carried out on the set of real matrices.

We aim to map the ESG patterns of emerging-market companies from 2018 to 2021 in order to determine whether the COVID-19 pandemic exerted any influence on sustainable corporate behavior. Thus, the ESG performances were assessed by applying the Kohonen neural network for clustering purposes at three main levels: i) ESG overall, including country and sectoral perspectives; ii) ESG thematic; and iii) ESG four-folded (stakeholder, perspective, management level, and focus strategic views). Our results show that emerging-market companies focus their ESG efforts on social and governance issues rather than on environmental. However, environmental, and social behavior differ more acutely than governance behavior across clusters. The ESG pillars country-level and eleven market-based economic sectors analysis corroborate the geographic and sector dependence of ESG performance. The thematic-level analysis indicate that operational activities and community issues received more attention, which suggests that emerging-market companies address distinct ESG topics according to their particularities and competitiveness. Furthermore, our empirical findings provide evidence that the ESG behavior of companies has changed over the course of the COVID-19 pandemic. Thus, our findings are relevant to policymakers involved in regulating ESG disclosure practices, investors focused on enhancing their sustainable investment strategies, and firms engaged in improving their ESG involvement.

This paper presents a novel approach for the recognition of multiple sclerosis (MS) using wavelet entropy and a Particle Swarm Optimization (PSO)-based neural network. MS is a complex neurological disorder with diverse clinical manifestations, often challenging to diagnose accurately. In this study, we leverage wavelet entropy (WE) as a feature extraction method to capture intricate patterns within brain imaging data. These extracted features are then employed to train a neural network model optimized through PSO to enhance classification accuracy. Experimental results on a dataset comprising MS and healthy control subjects demonstrate the effectiveness of our proposed approach, viz., WE-PSONN, which achieves a sensitivity of 91.95±1.15, a specificity of 92.36±0.88, a precision of 92.28±0.88, an accuracy of 92.16±0.90. The combination of advanced signal processing techniques and machine learning optimization holds promise for improving the early diagnosis and management of MS, offering potential benefits to both patients and healthcare providers.

In this research, we present a comprehensive methodology to categorize perfumes based on their fragrance profiles and subsequently aid in creating innovative odoriferous molecules using advanced neural networks. Drawing from data on Parfumo and the Good Scents Company webpage (Parfumo, 2008; The Good Scents Company, 2021), the study employs sophisticated web scraping techniques to gather diverse perfume attributes. Following this, a k-means algorithm is applied for perfume clustering, paving the way for recommending similar scents to consumers. The process then bridges customer preferences to molecular design by incorporating their feedback into generating new molecules via graph neural networks (GNNs). Through converting the Simple Molecular Input Line Entry System (SMILES) representation into graph structures, the GNN facilitates the creation of new molecular designs attuned to consumer desires. The proposed approach offers promising avenues for consumers to pinpoint similar perfume choices, incorporating feedback, and for manufacturers to conceptualize new fragrant molecules with a high likelihood of market resonance.

During the 1950s, the Gros Michel species of bananas were nearly wiped out by the incurable Fusarium Wilt, also known as Panama Disease. Originating in Southeast Asia, Fusarium Wilt is a banana pandemic that has been threatening the multi-billion-dollar banana industry worldwide. The disease is caused by a fungus that spreads rapidly throughout the soil and into the roots of banana plants. Currently, the only way to stop the spread of this disease is for farmers to manually inspect and remove infected plants as quickly as possible, whereas it is a time-consuming process. The main purpose of this study is to build a deep Convolutional Neural Network (CNN) using a transfer learning approach to rapidly identify fusarium wilt infections on banana crop leaves. We chose to use the ResNet50 architecture as the base CNN model for our transfer learning approach owing to its remarkable performance in image classification, which was demonstrated through its victory in the ImageNet competition. After its initial training and fine-tuning on a data set consisting of 300 healthy and diseased images, the CNN model achieved near-perfect accuracy of 0.99 and was fine-tuned to adapt the ResNet base model. ResNet50’s distinctive residual block structure could be the reason behind these results. To evaluate this CNN model, 500 test images, consisting of 250 diseased and healthy banana leaf images, were classified by the model. The deep CNN model was able to achieve an accuracy of 0.98 and an F-1 score of 0.98 by correctly identifying the class of 492 of the 500 images. These results show that this DCNN model outperforms existing models such as Sangeetha et al., 2023’s deep CNN model by at least 0.07 in accuracy and is a viable option for identifying Fusarium Wilt in banana crops.