ARTICLE | doi:10.20944/preprints202006.0297.v1
Online: 24 June 2020 (18:02:03 CEST)
Breast Cancer diagnosis is one of the most studied problems in the medical domain. In the medical domain, cancer diagnosis has been studied extensively which instantiates the need of early prediction of cancer disease. For obtaining advance prediction, health records are exploited and given as input to an automated system. This paper focuses on constructing an automated system by employing deep learning based recurrent neural network models. A stacked GRU-LSTM-BRNN is proposed in this paper that accepts health records of a patient for determining possibility of being affected by breast cancer. Proposed model is compared against other baseline classifiers such as stacked Simple-RNN model, stacked LSTM-RNN model, stacked GRU-RNN model. Comparative results obtained in this study indicate that stacked GRU-LSTM-BRNN yield better classification performance for predictions related to breast cancer disease.
ARTICLE | doi:10.20944/preprints202104.0515.v2
Subject: Mathematics & Computer Science, Algebra & Number Theory Keywords: LSTM; convolution; regularization; stock market
Online: 20 April 2021 (21:12:17 CEST)
Many studies in the current literature annotate patterns in stock prices and use computer vision models to learn and recognize these patterns from stock price-action chart images. Additionally, current literature also use Long Short-Term Memory Networks to predict prices from continuous dollar amount data. In this study, we combine the two techniques. We annotate the consolidation breakouts for a given stock price data, and we use continuous stock price data to predict consolidation breakouts. Unlike computer vision models that look at the image of a stock price action, we explore using the convolution operation on raw dollar values to predict consolidation breakouts under a supervised learning problem setting. Unlike LSTMs that predict stock prices given continuous stock data, we use the continuous stock data to classify a given price window as breakout or not. Finally, we do a regularization study to see the effect of L1, L2, and Elastic Net regularization. We hope that combining regression and classification shed more light on stock market prediction studies.
ARTICLE | doi:10.20944/preprints202112.0196.v1
Subject: Mathematics & Computer Science, Information Technology & Data Management Keywords: Speech Rehabilitation; Speech Quality Assessment; LSTM
Online: 13 December 2021 (10:10:36 CET)
The article considers an approach to the problem of assessing the quality of speech during speech rehabilitation as a classification problem. For this, a classifier is built on the basis of an LSTM neural network for dividing speech signals into two classes: before the operation and immediately after. At the same time, speech before the operation is the standard to which it is necessary to approach in the process of rehabilitation. The metric of belonging of the evaluated signal to the reference class acts as an assessment of speech. An experimental assessment of rehabilitation sessions and a comparison of the resulting assessments with expert assessments of phrasal intelligibility were carried out.
ARTICLE | doi:10.20944/preprints201907.0241.v1
Subject: Physical Sciences, Astronomy & Astrophysics Keywords: recurrent neural networks, LSTM, lightcurves, simulation, variability
Online: 22 July 2019 (10:41:25 CEST)
With an explosion of data in the near future, from observatories spanning from radio to gamma-rays, we have entered the era of time domain astronomy. Historically, this field has been limited to modeling the temporal structure with time-series simulations limited to energy ranges blessed with excellent statistics as in X-rays. In addition to ever increasing volumes and variety of astronomical lightcurves, there's a plethora of different types of transients detected not only across the electromagnetic spectrum, but indeed across multiple messengers like counterparts for neutrino and gravitational wave sources. As a result, precise, fast forecasting and modeling the lightcurves or time-series will play a crucial role in both understanding the physical processes as well as coordinating multiwavelength and multimessenger campaigns. In this regard, deep learning algorithms such as recurrent neural networks (RNNs) should prove extremely powerful for forecasting as it has in several other domains. Here we test the performance of a very successful class of RNNs, the Long Short Term Memory (LSTM) algorithms with simulated lightcurves. We focus on univariate forecasting of types of lightcurves typically found in active galactic nuclei (AGN) observations. Specifically, we explore the sensitivity of training and test losses to key parameters of the LSTM network and data characteristics namely gaps and complexity measured in terms of number of Fourier components. We find that typically, the performances of LSTMs are better for pink or flicker noise type sources. The key parameters on which performance is dependent are batch size for LSTM and the gap percentage of the lightcurves. While a batch size of $10-30$ seems optimal, the most optimal test and train losses are under $10 \%$ of missing data for both periodic and random gaps in pink noise. The performance is far worse for red noise. This compromises detectability of transients. The performance gets monotonically worse for data complexity measured in terms of number of Fourier components which is especially relevant in the context of complicated quasi-periodic signals buried under noise. Thus, we show that time-series simulations are excellent guides for use of RNN-LSTMs in forecasting.
ARTICLE | doi:10.20944/preprints201811.0579.v1
Subject: Mathematics & Computer Science, Artificial Intelligence & Robotics Keywords: Deep learning, Cognitive, LSTM, Neural network, Ngrams
Online: 26 November 2018 (10:06:05 CET)
Cognitive neuroscience is the study of how the human brain functions on tasks like decision making, language, perception and reasoning. Deep learning is a class of machine learning algorithms that use neural networks. They are designed to model the responses of neurons in the human brain. Learning can be supervised or unsupervised. Ngram token models are used extensively in language prediction. Ngrams are probabilistic models that are used in predicting the next word or token. They are a statistical model of word sequences or tokens and are called Language Models or Lms. Ngrams are essential in creating language prediction models. We are exploring a broader sandbox ecosystems enabling for AI. Specifically, around Deep learning applications on unstructured content form on the web.
ARTICLE | doi:10.20944/preprints202109.0033.v1
Subject: Engineering, Electrical & Electronic Engineering Keywords: Artificial Intelligence; Low Power Systems; Power Management; LSTM
Online: 2 September 2021 (08:03:24 CEST)
Computer systems that operate in remote locations such as satellites, remote weather stations and autonomous robots are highly limited in the availability of energy for their operation. This work aims to employ artificial intelligence algorithms in energy management in order to obtain the maximum energy yield and the prediction of energy availability to the system. The work presents the main types of algorithms used in artificial intelligence and presents the creation of a prototype that will operate as a low power system powered by batteries and a small solar plate, the prototype will perform inferences on the LSTM neural network algorithm in order to predict the future availability of energy, consequently the management system will carry out the energy distribution in order to obtain the maximum operation of the prototype without total discharge of the batteries. So that the artificial intelligence system could be embedded in the prototype, the TensorFlow Lite framework was used, which allows the inference to be carried out in devices with low consumption and limited processing power.
ARTICLE | doi:10.20944/preprints201910.0376.v1
Subject: Mathematics & Computer Science, Artificial Intelligence & Robotics Keywords: artificial neural network; deep learning; LSTM; speech processing
Online: 31 October 2019 (16:40:30 CET)
Speech signals are degraded in real-life environments, product of background noise or other factors. The processing of such signals for voice recognition and voice analysis systems presents important challenges. One of the conditions that make adverse quality difficult to handle in those systems is reverberation, produced by sound wave reflections that travel from the source to the microphone in multiple directions.To enhance signals in such adverse conditions, several deep learning-based methods have been proposed and proven to be effective. Recently, recurrent neural networks, especially those with long and short-term memory (LSTM), have presented surprising results in tasks related to time-dependent processing of signals, such as speech. One of the most challenging aspects of LSTM networks is the high computational cost of the training procedure, which has limited extended experimentation in several cases. In this work, we present a proposal to evaluate the hybrid models of neural networks to learn different reverberation conditions without any previous information. The results show that some combination of LSTM and perceptron layers produce good results in comparison to those from pure LSTM networks, given a fixed number of layers. The evaluation has been made based on quality measurements of the signal's spectrum, training time of the networks and statistical validation of results. Results help to affirm the fact that hybrid networks represent an important solution for speech signal enhancement, with advantages in efficiency, but without a significan drop in quality.
ARTICLE | doi:10.20944/preprints201808.0163.v1
Subject: Engineering, Electrical & Electronic Engineering Keywords: traffic flow forecasting; cluster computing; LSTM neural networks
Online: 8 August 2018 (08:50:09 CEST)
Accurate and fast traffic flow forecasting is vital in intelligent transportation system because many of the advanced features in intelligent transportation systems are based on it. However, existing methods have poor performance regarding accuracy and computational efficiency in long-term traffic flow forecasting under big data. Hence, we propose an improved Long short-term memory (LSTM) Network and its cluster computing implementation in this paper to address the above challenge. We propose a singular point probability LSTM (SDLSTM) algorithm. The method discards the units of the network according to the singular point probability during the training process and amends the SDLSTM by Autoregressive Integrated Moving Average Model (ARIMA) to achieve the accurate prediction of 24-hour traffic flow data. Furthermore, the paper designs a scheme for implementing this method through cluster computing to shorten the calculation time and improve the system's operating speed. Theoretical analysis and experimental results show that SDLSTM gains a higher accuracy rate and better stability in the long-term traffic flow forecasting compared with previous methods.
ARTICLE | doi:10.20944/preprints201804.0258.v2
Subject: Mathematics & Computer Science, Information Technology & Data Management Keywords: audio classification; multi-resolution analysis; LSTM; auto-ml
Online: 19 July 2018 (05:53:20 CEST)
We describe a multi-resolution approach for audio classification and illustrate its application to the open data set for environmental sound classification. The proposed approach utilizes a multi-resolution based ensemble consisting of targeted feature extraction of approximation (coarse scale) and detail (fine scale) portions of the signal under the action of multiple transforms. This is paired with an automatic machine learning engine for algorithm and parameter selection and the LSTM algorithm, capable of mapping several sequences of features to a predicted class membership probability distribution. Initial results show an improvement in multi-class classification accuracy.
ARTICLE | doi:10.20944/preprints202208.0345.v1
Subject: Mathematics & Computer Science, Artificial Intelligence & Robotics Keywords: Deep learning; solar radiation forecasting; model prediction; solar energy; multi climates data; generalizability; sustainability; Long Short-Term Memory (LSTM); Gated Recurrent Unit (GRU); Convolutional Neural Network (CNN); Hybrid CNN-Bidirectional LSTM; LSTM Autoencoder
Online: 18 August 2022 (10:59:18 CEST)
The sustainability of the planet and its inhabitants is in dire danger and is among the highest priorities on global agendas such as the Sustainable Development Goals (SDGs) of the United Nations (UN). Solar energy -- among other clean, renewable, and sustainable energies -- is seen as essential for environmental, social, and economic sustainability. Predicting solar energy accurately is critical to increasing reliability and stability, and reducing the risks and costs of the energy systems and markets. Researchers have come a long way in developing cutting-edge solar energy forecasting methods. However, these methods are far from optimal in terms of their accuracies, generalizability, benchmarking, and other requirements. Particularly, no single method performs well across all climates and weathers due to the large variations in meteorological data. This paper proposes SENERGY (an acronym for Sustainable Energy), a novel deep learning-based auto-selective approach and tool that, instead of generalising a specific model for all climates, predicts the best performing deep learning model for GHI forecasting in terms of forecasting error. The approach is based on carefully devised deep learning methods and feature sets through an extensive analysis of deep learning forecasting and classification methods using ten meteorological datasets from three continents. We analyse the tool in great detail through a range of metrics and methods for performance analysis, visualization, and comparison of solar forecasting methods. SENERGY outperforms existing methods in all performance metrics including Root Mean Square Error (RMSE), Mean Absolute Error (MAE), Mean Absolute Percentage Error (MAPE), Forecast Skills (FS), Relative Forecasting Error, and the normalised versions of these metrics. The proposed auto-selective approach can be extended to other research problems such as wind energy forecasting and predict forecasting models based on different criteria (in addition to the minimum forecasting error used in this paper) such as the energy required or speed of model execution, different input features, different optimisations of the same models, or other user preferences.
ARTICLE | doi:10.20944/preprints202205.0147.v1
Subject: Mathematics & Computer Science, Artificial Intelligence & Robotics Keywords: SARIMA; Artificial Neural Networks (ANN); LSTM; hybrid methodologies; prediction
Online: 11 May 2022 (05:50:41 CEST)
The choice of holiday destinations is highly depended on climate considerations. Nowadays, since the effects of climate crisis are being increasingly felt, the need of accurate weather and climate services for hotels is crucial. Such a service could be beneficial for both the future planning of tourists’ activities and destinations and for hotel managers as it could help in decision making about the planning and expansion of the touristic season, due to a prediction of higher temperatures for a longer time span, thus causing increased revenue for companies in the local touristic sector. The aim of this work is to calculate predictions on climatic variables using statistical techniques as well as Artificial Intelligence (AI) for a specific area of interest utilising data from in situ meteorological station, and produce valuable and reliable localised predictions with the most cost-effective method possible. This investigation will answer the question of the most suitable prediction method for time series data from a single meteorological station that is deployed in a specific location. As a result, an accurate representation of the microclimate in a specific are is achieved. To achieve this high accuracy in situ measurements and prediction techniques are used. As prediction techniques, Seasonal Auto Regressive Integrated Moving Average (SARIMA), AI techniques like the Long-Short-Term-Memory (LSTM) Neural Network and hybrid combinations of the two are used. Variables of interest are divided in the easier to predict temperature and humidity that are more periodic and less chaotic, and the wind speed as an example of a more stochastic variable with no known seasonality and patterns. Our results show that the examined Hybrid methodology performs the best at temperature and wind speed forecasts, closely followed by the SARIMA whereas LSTM perform better overall at the humidity forecast, even after the correction of the Hybrid to the SARIMA model.
ARTICLE | doi:10.20944/preprints202009.0518.v1
Subject: Mathematics & Computer Science, Information Technology & Data Management Keywords: aquaculture water quality; dissolved oxygen (DO); forecasting; EEMD; LSTM
Online: 22 September 2020 (10:02:09 CEST)
Dissolved Oxygen (DO) concentration is a vital parameter that indicates water quality. DO short term forecasting using time series analysis on data collected from an aquaculture pond is presented here. This can provide data support for an early warning system for an improved management of the aquaculture farm. The conventional forecasting approaches are commonly characterized by low accuracy and poor generalization problems. In this article, we present a novel hybrid DO concentration forecasting method with ensemble empirical mode decomposition (EEMD) based LSTM (Long short-term memory) neural network (NN). With this method, first, the sensor data integrity is improved through linear interpolation and moving average filtering methods of data preprocessing. Next, the EEMD algorithm is applied to decompose the original sensor data into multiple intrinsic mode functions (IMFs). Finally, the feature selection is used to carefully select IMFs that are strongly correlated with the original sensor data and integrate both into inputs for the NN. The hybrid EEMD-based LSTM forecasting model is then constructed. Performance of this proposed model in training and validation sets was compared with the observed real sensor data. To obtain the exact evaluation accuracy of the forecasted results of the hybrid EEMD-based LSTM forecasting model, three statistical performance indices were adopted: MAE, MSE, and RMSE. Results presented for short term (12-hour period) and long term (1-month period) give a strong indication of suitability of this method for forecasting DO values.
REVIEW | doi:10.20944/preprints202208.0031.v1
Subject: Engineering, Electrical & Electronic Engineering Keywords: LSTM; GMDH; ANFIS; Ensemble Learning Models; Wavelet; Time Series Forecasting
Online: 2 August 2022 (03:50:14 CEST)
To improve the monitoring of the electrical power grid it is necessary to evaluate the influence of contamination in relation to leakage current and its progression to a disruptive discharge. In this paper, insulators were tested in a saline chamber to simulate the increase of salt contamination on their surface, and to evaluate the supportability of these components. From the time series forecasting of the leakage current, it is possible to evaluate the development of the fault before a flashover occurs. Choosing which method to use is always a difficult task since some models may have higher computational effort. In this paper, for a complete evaluation, the long short-term memory (LSTM), group method of data handling (GMDH), adaptive neuro-fuzzy inference system (ANFIS), bootstrap aggregation (bagging), sequential learning (boosting), random subspace, and stacked generalization (stacking) ensemble learning models are analyzed. A review and comparison of these well-established methods for time series forecasting is performed. From the results of the best structure of the model, the hyperparameters are evaluated and the Wavelet transform is used to obtain an enhanced model.
ARTICLE | doi:10.20944/preprints202206.0368.v1
Subject: Engineering, Biomedical & Chemical Engineering Keywords: hand gesture classification; transfer learning; three-dimensional convolutional; LSTM network
Online: 27 June 2022 (13:36:40 CEST)
This paper introduces a multi-class hand gesture recognition model developed to identify a set of defined hand gesture sequences in two-dimensional RGB video recordings. The work presents an action detection classifier that looks at both appearance and spatiotemporal parameters of consecutive frames. The classifier utilizes a convolutional-based network combined with a long-short-term memory unit. To leverage the need for a large-scale dataset, the model uses an available dataset to then adopt a technique known as transfer learning to fine-tune the model on the hand gestures of relevance. Validation curves performed over a batch size of 64 indicate an accuracy of 93.95% (± 0.37) with a mean Jaccard index of 0.812 (± 0.105) for 22 participants. The presented model illustrates the possibility of training a model with a small set of data (113,410 fully labelled frames). The proposed pipeline embraces a small-sized architecture that could facilitate its adoption.
ARTICLE | doi:10.20944/preprints202112.0244.v1
Subject: Mathematics & Computer Science, Information Technology & Data Management Keywords: Attention mechanism; CHMM; LSTM; Multi-modal fusion; Human behavior recognition
Online: 14 December 2021 (15:09:15 CET)
The multi-modal data fusion method based on IA-net and CHMM technical proposed is designed to solve the problem that the incompleteness of target behavior information in complex family environment leads to the low accuracy of human behavior recognition.The two improved neural networks(STA-ResNet50、STA-GoogleNet)are combined with LSTM to form two IA-Nets respectively to extract RGB and skeleton modal behavior features in video. The two modal feature sequences are input CHMM to construct the probability fusion model of multi-modal behavior recognition.The experimental results show that the human behavior recognition model proposed in this paper has higher accuracy than the previous fusion methods on HMDB51 and UCF101 datasets. New contributions: attention mechanism is introduced to improve the efficiency of video target feature extraction and utilization. A skeleton based feature extraction framework is proposed, which can be used for human behavior recognition in complex environment. In the field of human behavior recognition, probability theory and neural network are cleverly combined and applied, which provides a new method for multi-modal information fusion.
ARTICLE | doi:10.20944/preprints201908.0073.v1
Subject: Mathematics & Computer Science, Artificial Intelligence & Robotics Keywords: contextual keyword extraction; BERT; word embedding; LSTM; transformers; Deep Learning
Online: 6 August 2019 (09:17:36 CEST)
In this paper we propose a novel self-supervised approach of keywords and keyphrases retrieval and extraction by an end-to-end deep learning approach, which is trained by contextually self-labelled corpus. Our proposed approach is novel to use contextual and semantic features to extract the keywords and has outperformed the state of the art. Through the experiment the proposed approach has been proved to be better in both semantic meaning and quality than the existing popular algorithms of keyword extraction. In addition, we propose to use contextual features from bidirectional transformers to automatically label short-sentence corpus with keywords and keyphrases to build the ground truth. This process avoids the human time to label the keywords and do not need any prior knowledge. To the best of our knowledge, our published dataset in this paper is a fine domain-independent corpus of short sentences with labelled keywords and keyphrases in the NLP community.
ARTICLE | doi:10.20944/preprints202209.0398.v1
Subject: Engineering, Civil Engineering Keywords: river discharge; hydro informatics; water resource; data-driven; deep learning; LSTM
Online: 26 September 2022 (11:30:24 CEST)
River flow prediction is a pivotal task in the field of water resource management during the era of rapid climate change. The highly dynamic and evolving nature of the climatic variables e.g., precipitation has a significant impact on the temporal distribution of the river discharge in recent days making the discharge forecasting even more complicated for diversified water-related issues e.g., flood prediction and irrigation planning. To predict the discharge, various physics-based numerical models are used using numerous hydrologic parameters. Extensive lab-based investigation and calibration are required to reduce the uncertainty involved in those parameters. However, in the age of data-driven predictions, several deep learning algorithms showed satisfactory performance in dealing with sequential data. In this research, Long Short-term Memory (LSTM) neural network regression model is trained using over 80 years of daily data to forecast the discharge time series up to 3 days ahead of time. The performance of the model is found satisfactory through the comparison of the predicted data with the observed data, visualization of the distribution of the errors and Root Mean Squared Error (RMSE) value of 0.09. Higher performance is achieved through the increase in the number of epochs and hyper parameter tuning. This model can be transferred to other locations with proper feature engineering and optimization to perform univariate predictive analysis and potentially be used to perform real-time river discharge prediction.
Subject: Mathematics & Computer Science, Algebra & Number Theory Keywords: motion capture; neural networks; reconstruction; gap filling; FFNN; LSTM; BILSTM; GRU
Online: 3 August 2021 (11:52:46 CEST)
Optical motion capture is a mature contemporary technique for the acquisition of motion data, alas it is non-error-free. Due to technical limitations and occlusions of markers, gaps might occur in such recordings. The article reviews various neural network architectures applied for gap filling problem in motion capture sequences within FBM framework providing the representation for body kinematic structure. The results are compared with interpolation and matrix completion methods. We found out, that for longer sequences simple linear feedforward neural networks can outperform the other, sophisticated architectures. We were also able to identify, that acceleration and monotonicity of input sequence are the parameters that have a notable impact on the obtained results.
ARTICLE | doi:10.20944/preprints202108.0011.v1
Subject: Mathematics & Computer Science, Algebra & Number Theory Keywords: Transformer; spike; neural decoding; CNN; RNN; LSTM; deep learning; information; neuroscience
Online: 2 August 2021 (09:51:43 CEST)
Neural decoding from spiking activity is an essential tool for understanding the information encoded in population neurons, especially in applications like brain-computer interface (BCI). Various quantitative methods have been proposed and have shown superiorities under different scenarios respectively. From the machine learning perspective, the decoding task is to map the high-dimensional spatial & temporal neuronal activity to the low-dimensional physical quantities (e.g., velocity, position). Because of the complex interactions and the abundant dynamics among neural circuits, good decoding algorithms usually have the capability of capturing flexible spatiotemporal structures embedded in the input feature space. Recently, the Transformer-based models are widely used in processing natural languages and images due to its superior performances in handling long-range and global dependencies. Hence, in this work we examine the potential applications of Transformers in neural decoding and introduce two Transformer-based models. Besides adapting the Transformer to neuronal data, we also propose a data augmentation method for overcoming the data shortage issue. We test our models on three experimental datasets and their performances are comparable to the previous state-of-the-art (SOTA) RNN-based methods. In addition, Transformer-based models show increased decoding performances when the input sequences are longer, while LSTM-based models deteriorate quickly. Our research suggests that Transformer-based models are important additions to the existing neural decoding solutions, especially for large datasets with long temporal dependencies.
ARTICLE | doi:10.20944/preprints201906.0051.v1
Subject: Mathematics & Computer Science, Artificial Intelligence & Robotics Keywords: NLP, news media, bias, neural networking, LSTM, information retrieval, filter bubble
Online: 6 June 2019 (13:15:28 CEST)
An article's tone and framing not only influence an audience's perception of a story but may also reveal attributes of author identity and bias. Building upon prior media, psychological, and machine learning research, this neural network-based system detects those writing characteristics in ten news agencies' reporting, discovering patterns that, intentional or not, may reveal an agency's topical perspectives or common contextualization patterns. Specifically, learning linguistic markers of different organizations through a newly released open database, this probabilistic classifier predicts an article's publishing agency with 74% hidden test set accuracy given only a short snippet of text. The resulting model demonstrates how unintentional 'filter bubbles' can emerge in machine learning systems and, by comparing agencies' patterns and highlighting outlets' prototypical articles through an open source exemplar search engine, this paper offers new insight into news media bias.
ARTICLE | doi:10.20944/preprints201905.0228.v1
Subject: Mathematics & Computer Science, Artificial Intelligence & Robotics Keywords: Deep learning, LSTM, Machine learning, Post-filtering, Signal processing, Speech Synthesis
Online: 17 May 2019 (16:16:53 CEST)
Several researchers have contemplated deep learning-based post-filters to increase the quality of statistical parametric speech synthesis, which perform a mapping of the synthetic speech to the natural speech, considering the different parameters separately and trying to reduce the gap between them. The Long Short-term Memory (LSTM) Neural Networks have been applied successfully in this purpose, but there are still many aspects to improve in the results and in the process itself. In this paper, we introduce a new pre-training approach for the LSTM, with the objective of enhancing the quality of the synthesized speech, particularly in the spectrum, in a more efficient manner. Our approach begins with an auto-associative training of one LSTM network, which is used as an initialization for the post-filters. We show the advantages of this initialization for the enhancing of the Mel-Frequency Cepstral parameters of synthetic speech. Results show that the initialization succeeds in achieving better results in enhancing the statistical parametric speech spectrum in most cases when compared to the common random initialization approach of the networks.
ARTICLE | doi:10.20944/preprints202203.0121.v1
Subject: Engineering, Energy & Fuel Technology Keywords: load profiles; electricity; heat; industry; correlation; machine learning; LSTM; prediction; anomaly detection
Online: 8 March 2022 (09:23:34 CET)
The accurate prediction of heat load profiles with a daily resolution is required for a broad range of applications such as potential studies, design, or predictive operation of heating systems. If the heat demand of a consumer is mainly caused by (production) processes independent of the ambient temperature, existing load profile prediction methods fail. To close this gap, this study develops two ex post machine learning models for the prediction of the heat demand with a daily resolution. The selected input features are commonly available to each consumer connected to public natural gas and electricity grids or operating an energy monitoring system: Ambient temperature, weekday, electricity consumption, and heat consumption of the last seven days directly before the predicted day. The study’s database covers electricity and natural gas consumption load profiles from 82 German consumers over a period of two years. Electricity and heat consumption correlate strongly with individual patterns for many consumers. Both, shallow and deep learning algorithms are evaluated. The highest accuracy is achieved by a Long Short Term Memory (LSTM) model with a median of R² of 0.94. The ex post model architecture makes the model suitable for automated anomaly detection in energy monitoring systems.
Subject: Social Sciences, Accounting Keywords: mispricing; motives for mergers and acquisitions; M/B ratio decompositon; LSTM networks
Online: 3 March 2020 (11:02:16 CET)
This study uses a recently developed theory and technique to examine post-acquisition evidence as to the motives for mergers and acquisitions(M&As), and decomposed the M/B ratio into three components: firm-specific error, time-series sector error, and long-run value-to-book. We make a multidimensional grouping according to the frequency of M&As , payment method, proportion of shares acquired, M/B ratio before the merger and total assets of the acquirers before the merge. The results confirm that M&As involve multiple motives, such as market timing, industry and economic shocks, agency and hubris. Using a sample of 2,035 M&As in China, we find that 59% are related to market timing, 68% are related to agency and hubris, 21% are related to industry and economic shocks, 51% are related to multiple motives. Our empirical research finds that the main motives for M&As of listed companies in China are value-decreasing, which have negative impact on the acquirers‘ sustainable development.
ARTICLE | doi:10.20944/preprints202012.0058.v1
Subject: Mathematics & Computer Science, Algebra & Number Theory Keywords: Deep Learning; LSTM Autoencoder; Supervised Learning, Hydraulic Test Rig; Sensor Faults; Component Faults
Online: 2 December 2020 (11:08:40 CET)
Anomaly occurrences in hydraulic machinery may lead to massive systems shut down, jeopardizing the safety of the machinery and its surrounding human operator(s) and environment, and the severe economic implications succeeding the faults and their associated damage. Hydraulics are mostly placed in ruthless environments, where they are consistently vulnerable to many faults. Hence, not only the machines and their components are prone to anomalies, but also the sensors attached to them, which monitor and report their health and behavioral changes. In this work, a comprehensive applicational analysis of anomalies in hydraulic systems extracted from a hydraulic test rig is thoroughly achieved. Firstly, we provided a combination of a new architecture of LSTM autoencoders and supervised machine and deep learning methodologies to perform two separate stages of fault detection and diagnosis. The two phases are condensed by: the detection phase using the LSTM autoencoder. Followed by the fault diagnosis phase represented by the classification schema. The previously mentioned framework is applied to both component and sensor faults in hydraulic systems, deployed in the form of two in-depth applicational experiments. Moreover, a thorough literature review of the past decade related work for the two stages separately is successfully conducted in this paper.
ARTICLE | doi:10.20944/preprints202001.0149.v1
Subject: Mathematics & Computer Science, Artificial Intelligence & Robotics Keywords: Optical Music Recognition; Historical Document Analysis; Medieval manuscripts; neume notation; CNN; LSTM; CTC
Online: 15 January 2020 (12:11:25 CET)
The automatic recognition of scanned Medieval manuscripts still represents a challenge due to degradation, non standard layouts, or notations. This paper focuses on the Medieval square notation developed around the 11th century which is composed of staff lines, clefs, accidentals, and neumes which are basically connected single notes. We present a novel approach to tackle the automatic transcription by applying CNN/LSTM networks that are trained using the segmentation-free CTC-loss-function which considerably facilitates the GT-production. For evaluation, we use three different manuscripts and achieve a dSAR of 86.0% on the most difficult book and 92.2% on the cleanest one. To further improve the results, we apply a neume dictionary during decoding which yields a relative improvement of about 5%.
ARTICLE | doi:10.20944/preprints201809.0481.v1
Subject: Engineering, Other Keywords: Brain-Computer Interfaces, spectrogram-based convolutional neural network model(pCNN), Deep Learning, EEG, LSTM, RCNN
Online: 25 September 2018 (08:58:34 CEST)
Non-invasive, electroencephalography (EEG)-based brain-computer interfaces (BCIs) on motor imagery movements translate the subject’s motor intention into control signals through classifying the EEG patterns caused by different imagination tasks, e.g. hand movements. This type of BCI has been widely studied and used as an alternative mode of communication and environmental control for disabled patients, such as those suffering from a brainstem stroke or a spinal cord injury (SCI). Notwithstanding the success of traditional machine learning methods in classifying EEG signals, these methods still rely on hand-crafted features. The extraction of such features is a difficult task due to the high non-stationarity of EEG signals, which is a major cause for the stagnating progress in classification performance. Remarkable advances in deep learning methods allow end-to-end learning without any feature engineering, which could benefit BCI motor imagery applications. We developed three deep learning models: 1) a long short-term memory (LSTM); 2) a proposed spectrogram-based convolutional neural network model (pCNN); and 3) a recurrent convolutional neural network (RCNN), for decoding motor imagery movements directly from raw EEG signals without (manual) feature engineering. Results were evaluated on our own, publicly available, EEG data collected from 20 subjects and on an existing dataset known as 2b EEG dataset from "BCI Competition IV". Overall, better classification performance was achieved with deep learning models compared to state-of-the art machine learning techniques, which could chart a route ahead for developing new robust techniques for EEG signal decoding. We underpin this point by demonstrating the successful real-time control of a robotic arm using our CNN based BCI.
ARTICLE | doi:10.20944/preprints202112.0031.v1
Subject: Mathematics & Computer Science, Information Technology & Data Management Keywords: Satellite Communication; Signal Propagation; Rain Attenuation; Urban area ground station; SNR, ITU-R; LSTM, Neural network
Online: 2 December 2021 (11:18:57 CET)
Free-space communication is a leading component in global communications. Its advantages relate to a broader signal spread, no wiring, and ease of engagement. Satellite communication services became recently attractive to mega-companies that foresee an excellent opportunity to connect disconnected remote regions, serve emerging machine-to-machine communication, Internet-of-things connectivity, and more. Satellite communication links suffer from arbitrary weather phenomena such as clouds, rain, snow, fog, and dust. In addition, when signals approach the ground station, it has to overcome buildings blocking the direct access to the ground station. Therefore, satellites commonly use redundant signal strength to ensure constant and continuous signal transmission, resulting in excess energy consumption, challenging the limited power capacity generated by solar energy or the fixed amount of fuel. This research proposes LTSM, an artificial recurrent neural network technology that provides a time-dependent prediction of the expected attenuation level due to rain and fog and the signal strength that remained after crossing physical obstacles surrounding the ground station. The satellite transmitter is calibrated accordingly. The satellite outgoing signal strength is based on the predicted signal strength to ensure it will remain strong enough for the ground station to process it. The instant calibration eliminates the excess use of energy resulting in energy savings.
ARTICLE | doi:10.20944/preprints202105.0605.v1
Subject: Mathematics & Computer Science, Algebra & Number Theory Keywords: deep learning; computed tomography; image classification; COVID-19; medical image analysis; pneumonia; CNN, LSTM, medical diagnosis
Online: 25 May 2021 (10:32:29 CEST)
Advancements in deep learning and availability of medical imaging data have led to use of CNN based architectures in disease diagnostic assisted systems. In spite of the abundant use of reverse transcription-polymerase chain reaction (RT-PCR) based tests in COVID-19 diagnosis, CT images offer an applicable supplement with its high sensitivity rates. Here, we study classification of COVID-19 pneumonia (CP) and non-COVID-19 pneumonia (NCP) in chest CT scans using efficient deep learning methods to be readily implemented by any hospital. We report our deep network framework design that encompasses Convolutional Neural Networks (CNNs) and bidirectional Long Short Term Memory (biLSTM) architectures. Our study achieved high specificity (CP: 98.3%, NCP: 96.2% Healthy: 89.3%) and high sensitivity (CP: 84.0%, NCP: 93.9% Healthy: 94.9%) in classifying COVID-19 pneumonia, non-COVID-19 pneumonia and healthy patients. Next, we provide visual explanations for the CNN predictions with gradient-weighted class activation mapping (Grad-CAM). The results provided a model explainability by showing that Ground Glass Opacities (GGO), indicators of COVID-19 pneumonia disease, were captured by our CNN network. Finally, we have implemented our approach in three hospitals proving its compatibility and efficiency.
ARTICLE | doi:10.20944/preprints201911.0113.v1
Subject: Mathematics & Computer Science, Other Keywords: software defined networking; random forest; gain ratio; gru-lstm; anova f-rfe; open flow controller; machine learning
Online: 10 November 2019 (14:27:32 CET)
Recent advancements in Software Defined Networking (SDN) makes it possible to overcome the management challenges of traditional network by logically centralizing control plane and decoupling it from forwarding plane. Through centralized controllers, SDN can prevent security breach, but it also brings in new threats and vulnerabilities. Central controller can be a single point of failure. Hence, flow-based anomaly detection system in OpenFlow Controller can secure SDN to a great extent. In this paper, we investigated two different approaches of flow-based intrusion detection system in OpenFlow Controller. The first of which is based on machine-learning algorithm where NSL-KDD dataset with feature selection ensures the accuracy of 82% with Random Forest classifier using Gain Ratio feature selection evaluator. In the later phase, the second approach is combined with Gated Recurrent Unit Long Short-Term Memory based intrusion detection model based on Deep Neural Network (DNN) where we applied an appropriate ANOVA F-Test and Recursive Feature Elimination feature selection method to improve the classifier performance and achieved an accuracy of 88%. Substantial experiments with comparative analysis clearly show that, deep learning would be a better choice for intrusion detection in OpenFlow Controller.
ARTICLE | doi:10.20944/preprints202207.0039.v1
Subject: Engineering, Electrical & Electronic Engineering Keywords: Autonomous vehicles (A.V.); Anomaly Detection (A.D.); Deep Learning (DL), Symmetry; Long Short-Term Memory (LSTM); False Data Injection (FDI) Attacks
Online: 4 July 2022 (08:14:45 CEST)
Nowadays, technological advancement has transformed traditional vehicles into Au-tonomous Vehicles (A.V.s). In addition, in our daily lives, A.V.s play an important role since they are considered an essential component of smart cities. A.V. is an intelligent vehicle capable of main-taining safe driving by avoiding crashes caused by drivers. Unlike traditional vehicles, which are fully controlled and operated by humans, A.V.s collect information about the outside environment using sensors to ensure safe navigation. Furthermore, A.V.s reduce environmental impact because they usually use electricity to operate instead of fossil fuel, thus decreasing the greenhouse gasses. However, A.V.s could be threatened by cyberattacks, posing risks to human life. For example, re-searchers reported that Wi-Fi technology could be vulnerable to cyberattacks through Tesla and BMW AVs. Therefore, more research is needed to detect cyberattacks targeting the components of A.V.s to mitigate their negative consequences. This research will contribute to the security of A.V.s by detecting cyberattacks at the early stages. First, we inject False Data Injection (FDI) attacks into an A.V. simulation-based system developed by MathWorks. Inc. Second, we collect the dataset generated from the simulation model after integrating the cyberattack. Third, we implement an intelligent symmetrical anomaly detection method to identify FDI attacks targeting the control system of the A.V. through a compromised sensor. We use long short-term memory (LSTM) deep networks to detect FDI attacks in the early stage to ensure the stability of the operation of A.V.s. Our method classifies the collected dataset into two classifications: normal and anomaly data. The ex-perimental result shows that our proposed model's accuracy is 99.95%. To this end, the proposed model outperforms other state-of-the-art models in the same study area.
ARTICLE | doi:10.20944/preprints201803.0121.v1
Subject: Engineering, Electrical & Electronic Engineering Keywords: deep Kalman filter; simultaneous sensor integration and modelling (SSIM); GNSS/IMU integration; recurrent neural network; deep learning; long-short term memory (LSTM)
Online: 15 March 2018 (07:10:32 CET)
The Bayes filters, such as Kalman and particle filters, have been used in sensor fusion to integrate two sources of information and obtain the best estimate of the unknowns. Efficient integration of multiple sensors requires deep knowledge of their error sources and it is not trivial for complicated sensors, such as Inertial Measurement Unit (IMU). Therefore, IMU error modelling and efficient integration of IMU and Global Navigation Satellite System (GNSS) observations has remained a challenge. In this paper, we develop deep Kalman filter to model and remove IMU errors and consequently, improve the accuracy of IMU positioning. In other words, we add modelling step to the prediction and update steps of Kalman filter and the IMU error model is learned during integration. Therefore, our deep Kalman filter outperforms Kalman filter and reaches higher accuracy.
ARTICLE | doi:10.20944/preprints202009.0491.v1
Subject: Mathematics & Computer Science, Artificial Intelligence & Robotics Keywords: Short term load forecasting; STLF; deep learning; RNN; LSTM; GRU; machine learning; SVR; random forest; KNN; energy consumption; energy-intensive manufacturing; time series prediction; industry
Online: 21 September 2020 (04:19:45 CEST)
To minimise environmental impact, avoid regulatory penalties, and improve competitiveness, energy-intensive manufacturing firms require accurate forecasts of their energy consumption so that precautionary and mitigation measures can be taken. Deep learning is widely touted as a superior analytical technique to traditional artificial neural networks, machine learning, and other classical time series models due to its high dimensionality and problem solving capabilities. Despite this, research on its application in demand-side energy forecasting is limited. We compare two benchmarks (Autoregressive Integrated Moving Average (ARIMA), and an existing manual technique used at the case site) against three deep learning models (simple Recurrent Neural Networks (RNN), Long Short-Term Memory (LSTM), and Gated Recurrent Unit (GRU)) and three machine learning models (Support Vector Regression (SVM), Random Forest, and K-Nearest Neighbors (KNN)) for short term load forecasting (STLF) using data from a Brazilian thermoplastic resin manufacturing plant. We use the grid search method to identify the best configurations for each model, and then use Diebold-Mariano testing to confirm the results. Results suggests that the legacy approach used at the case site is the worst performing, and that the GRU model outperformed all other models tested.
ARTICLE | doi:10.20944/preprints202103.0412.v2
Subject: Mathematics & Computer Science, Algebra & Number Theory Keywords: Imaging; Machine learning; Deepfakes; Human pose estimation; Upper body languages; World leader; Deep learning; Computer vision; Recurrent Neural Networks (RNNs); Long Short-term Memory(LSTM); machine learning; Forecasting
Online: 28 April 2021 (12:02:00 CEST)
Recent improvements in deepfake creation have made deepfake videos more realistic. Moreover, open-source software has made deepfake creation more accessible, which reduces the barrier to entry for deepfake creation. This could pose a threat to the people’s privacy. There is a potential danger if the deepfake creation techniques are used by people with an ulterior motive to produce deepfake videos of world leaders to disrupt the order of countries and the world. Therefore, research into the automatic detection of deepfaked media is essential for public security. In this work, we propose a deepfake detection method using upper body language analysis. Specifically, a many-to-one LSTM network was designed and trained as a classification model for deepfake detection. Different models were trained by varying the hyperparameters to build a final model with benchmark accuracy. We achieved 94.39% accuracy on the deepfake test set. The experimental results showed that upper body language can effectively detect deepfakes.
ARTICLE | doi:10.20944/preprints202201.0107.v1
Subject: Engineering, Energy & Fuel Technology Keywords: Very short term load forecasting; VSTLF; Short term load forecasting; STLF; deep learning; RNN; LSTM; GRU; machine learning; SVR; random forest; extreme gradient boosting, energy consumption; ARIMA; time series prediction.
Online: 10 January 2022 (12:17:35 CET)
Commercial buildings are a significant consumer of energy worldwide. Logistics facilities, and specifically warehouses, are a common building type yet under-researched in the demand-side energy forecasting literature. Warehouses have an idiosyncratic profile when compared to other commercial and industrial buildings with a significant reliance on a small number of energy systems. As such, warehouse owners and operators are increasingly entering in to energy performance contracts with energy service companies (ESCOs) to minimise environmental impact, reduce costs, and improve competitiveness. ESCOs and warehouse owners and operators require accurate forecasts of their energy consumption so that precautionary and mitigation measures can be taken. This paper explores the performance of three machine learning models (Support Vector Regression (SVR), Random Forest, and Extreme Gradient Boosting (XGBoost)), three deep learning models (Recurrent Neural Networks (RNN), Long Short-Term Memory (LSTM), and Gated Recurrent Unit (GRU)), and a classical time series model, Autoregressive Integrated Moving Average (ARIMA) for predicting daily energy consumption. The dataset comprises 8,040 records generated over an 11-month period from January to November 2020 from a non-refrigerated logistics facility located in Ireland. The grid search method was used to identify the best configurations for each model. The proposed XGBoost models outperform other models for both very short load forecasting (VSTLF) and short term load forecasting (STLF); the ARIMA model performed the worst.
REVIEW | doi:10.20944/preprints201908.0152.v1
Subject: Mathematics & Computer Science, Artificial Intelligence & Robotics Keywords: deep learning; machine learning model; convolutional neural networks (CNN); recurrent neural networks (RNN); denoising autoencoder (DAE); deep belief networks (DBNs); long short-term memory (LSTM); review; survey; state of the art
Online: 13 August 2019 (09:32:09 CEST)
Deep learning (DL) algorithms have recently emerged from machine learning and soft computing techniques. Since then, several deep learning (DL) algorithms have been recently introduced to scientific communities and are applied in various application domains. Today the usage of DL has become essential due to their intelligence, efficient learning, accuracy and robustness in model building. However, in the scientific literature, a comprehensive list of DL algorithms has not been introduced yet. This paper provides a list of the most popular DL algorithms, along with their applications domains.
REVIEW | doi:10.20944/preprints202105.0254.v1
Subject: Mathematics & Computer Science, Artificial Intelligence & Robotics Keywords: deep learning; deep neural networks; recurrent LSTM models; attention layers; latent variable extraction; domain adaptation; yield and growth prediction in greenhouses; energy optimization in retail refrigerator systems; verification and recognition of expiry date in retail food packaging.
Online: 11 May 2021 (15:46:14 CEST)
This paper provides a review of an emerging field in the food processing sector, referring to efficient and safe food supply chains, ’from farm to fork’, as enabled by Artificial Intelligence (AI). Recent advances in machine and deep learning are used for effective food production, energy management and food labeling. Appropriate deep neural architectures are adopted and used for this purpose, including Fully Convolutional Networks, Long Short-Term Memories and Recurrent Neural Networks, Auto-Encoders and Attention mechanisms, Latent Variable extraction and clustering, as well as Domain Adaptation. Three experimental studies are presented, illustrating the ability of these AI methodologies to produce state-of-the-art performance in the whole food supply chain. In particular, these concern: (i) predicting plant growth and tomato yield in greenhouses, thus matching food production to market needs and reducing food waste or food unavailability; (ii) optimizing energy consumption across large networks of food retail refrigeration systems, through optimal selection of systems that can get shut-down and through prediction of the respective food de-freezing times, during peaks of power demand load; (iii) optical recognition and verification of food consumption expiry date in automatic inspection of retail packaged food, thus ensuring safety of food and people’s health.