preprints.org > computer science and mathematics > information systems > 201909.0295.v1

Working Paper Article Version 1 This version is not peer-reviewed

Version 1 : Received: 25 September 2019 / Approved: 26 September 2019 / Online: 26 September 2019 (09:56:00 CEST)

Control And Data Acquisition (SCADA) systems currently monitor and collect a huge among of data from all kind of processes. In practice, due to sensor failures or to communication errors, in the long-time running, some data may be lost. When it happens, given the nature of these failures, information is lost in bursts, that is, sets of consecutive samples, which besides can be very long. Data completion is a critical step, which must be done with the utmost rigour in order to not propagate errors in the rest of the processing chain stages. Some Big Data techniques do not work if the data series are incomplete, due to the loss of some data. When this occurs it is necessary to fill out the gaps of the historical data with a reliable data completion method. This paper presents an ad-hoc method to replenish the data lost by a SCADA system in case of long bursts. The data correspond to levels of drinking water tanks of a Water Network company that present patterns on a daily and a weekly scale. A method based on tensors is used to take advantage of the data structure. A specially designed \textit{tensorization} is employed to deal with bursts of missed data, applying a twice tensor decomposition and a signal continuity correction. Statistical tests are realized, which consist of apply the data reconstruction algorithms, by deliberately removing bursts of data in verified historical database servers, to be able to evaluate the real effectiveness of the tested methods. For this application, the presented approach outperforms the other techniques found in the literature.

water networks; scada data; tensor completion; tensor decomposition

Computer Science and Mathematics, Information Systems

* All users must log in before leaving a comment