An Integrative Systems Model for Oil and Gas Pipeline Data Prediction and Monitoring Using a Machine Intelligence and Sequence Learning Neural Technique

: O il and gas pipeline vandalism is a recurrent problem in oil rich zones of Nigeria and its West African neighbors and remains a challenge for multinationals to set ahead control measures to avert possible damages to operations both in infrastructure and business profit margins. In this paper, an integrative systems model comprising of a machine intelligence technique called Hierarchical Temporal Memory (HTM) and a sequence learning neural network called the Online-Sequential Extreme Learning Machine (OS-ELM) is proposed for monitoring and prediction of pipeline pressure data. The system models the continual prediction of pipeline oil/gas pressure signals useful for secure monitoring and control to avert acts of vandalism in oil and gas installations. The HTM uses a spatial pooler operated in temporal aggregated fashion and is defined as HTM-SP. The OS-ELM technique uses an explicit hierarchical training scheme so that the best cost estimates may be found after a stipulated number of trial runs. We study the performance of three OS-ELM neural activations: the sigmoid (sig), sinusoidal (sin) and radial basis function (rbf) activations. The results indicate improvement factors of 1.297, 1.297 and 1.300 of the HTM-SP over the OS-ELM sigmoid, sinusoidal and radial basis activations respectively.


Introduction
One of the most troublesome challenges faced by oil and gas operators in Nigeria particularly the South-South regions have been the recurrent issue of pipeline vandalism.This menace to oil and gas installations often results to the high level of insecurity and disruptions to service found in such regions for which the primary causes have been linked to the militant nature of such areas.The consequences of these disruptions are burst or damaged pipelines, fire and consequent explosions.
In this regard, measures such as the use of surveillance cameras, intruder alarms etc, including the use of stern looking militarized personnel have been put in place by industry operators in association with the Government to forestall any threats to oil and gas facilities.Notwithstanding these measures, disruptions to oil and gas operations still remain a possibility.This particular challenge has been attributed to the rigid nature of these security measures which is largely due to the limitation in design and operation.If the pressure flow of oil and gas can be effectively monitored and predicted, then it may be possible by way of automatic control devices to avert the disruptions to service and minimize if not eliminate in its entirety the use of human intervention.experts and robotics for smart monitoring and sensing in onshore and offshore oil fields is also a very promising approach as in [3,4]; a discussion of this line of applications can also be found in [5][6][7].More recently, the advantages of cortical-like processing agents have been shown to have the potential for effective continual monitoring of pipeline [8,9].These agents use neuroscience and biological ideas closely intertwined with software to implement very useful applications for commerce and industry.This research paper specifically proposes the use of a variant of an emerging state-of-the-art tool for machine intelligence called the Hierarchical Temporal Memory (HTM) for predictive monitoring of oil and gas pipeline pressure data.The HTM is based on a suite of biological constrained soft-computing techniques called the Cortical Learning Algorithms (CLA) or simply HTM-CLA [10].HTM-CLA possess the desirable property of online (continual) learning which is vital to systems that change through time and that requires automatic decision making.A model of how the system will perform in a real time system is also presented as an integrative system.Also presented is a comparative study of the performances of the aforementioned technique with a well established technique called the Online Sequential Extreme Learning Machine (OS-ELM).The OS-ELM have been shown to perform relatively well on a number of tasks and is claimed to be extremely fast and superior to some other sequence learning algorithms [11].However, we show in this study that the proposed HTM technique will on the average outperform the OS-ELM for the task of online (continual) prediction of a real world pipeline pressure data but the differences are not substantial so that the OS-ELM may still have the potential to be used as front-end in the integrated monitoring system.

Hierarchical Temporal Memory
Hierarchical Temporal Memory (HTM) is specifically a constrained machine intelligence neural network technique for continual learning tasks [12]; its principle is based on the formation of Sparse Distributed Representations (SDR), the use of the SDR to form invariant representations spatially and temporally, and then learning to make continual predictions from these representations using the theory of biology and neuroscience [12,13].Most if not all conventional Artificial Neural Networks (ANN) do not posses all these important functional properties or operations; one important point to note here is that most ANN requires separate training and testing dataset instead of continually learning and predicting on the training dataset.
In a typical HTM network, a Spatial Pooler (SP) stage is used to generate Sparse Distributed Representations (SDR); these SDR is compared with real world sensory input or synthetic sensory-like data to generate a matching set of SDR and then a Temporal Pooler (TP) stage is used for making predictions on the matching SDR set formed by the HTM-SP.These SDR are the basic data structures of any HTM neural network and capture the adaptive learning units used in the neocortex -the seat of intelligence in the brain.The idea of SDR was based on an earlier work on the notion of sparse coding earlier proposed in [14,15].An instance of the HTM neuron model is as shown in Figure 1.This neuron model is largely inspired by neuroscience studies describing activity-dependent synaptogenesis -a theory which proposes that the growth and origin of biological synapses is stimulated by an external sensory signal [16].In the diagram of Fig. 4, the proximal synapses is designated as green blobs; these blobs are linearly summed to produce a feed-forward activation while a set of corresponding distal synapses are designated by segments of blue blobs describing feedback and context experiences that are or-ed (logically summed) to generate a spiking neural activation when they exceed a pre-specified recognition threshold (denoted by a Sigma sign).There is the notion that feedback and context experiences are formed using the distal connections.

Spatial Pooling in HTM
In HTM, spatial pooling is performed using the notion of SDR followed by competitive Hebbian learning rules, a Homeostatic excitability control, and an overlapping mechanism for deriving candidate or winner SDR patterns via inhibition [17].SDR are formed by activating or deactivating a set of potential synapses or connecting neuron links.These synapses are grouped into a set of mini-columns and are spread out in a hypercube based on a set of predefined rules.
Let us consider a group of mini-columns with a set of potential connecting logical synapses or neurons.These potential connections may be initialized accordingly as in Equation ( 1): where, j = HTM neuron location index in the mini-column i = mini-column index j x = location of the jth input neuron (synapses) in the input space c i x = location centre of potential neurons (synapses) of ith mini-column in a hypercube of input space  = edge length of j  = fraction of inputs within the hypercube of input space that are potential connections ij  = represents a uniformly distributed random number between 0 and 1

I = an indicator function
The indicator function is usually described by a logical conditioning rule as in Equation ( 2): If a set of connected synapses are described in terms of a binary matrix, W, then its formation may be computed by conditioning its associated synapses to a permanence activation rule as: where, ij D = independent and identically distributed (i.i.d) dendrite synaptic permanence values from the jth input to the ith mini-column c  = synaptic permanence threshold The i.i.d synapse permanence values are described by Equation ( 4) as: Where a natural topology exists, neighborhood mini-columns may be inhibited in accordance to the relation given in Equation ( 5) otherwise a global inhibition parameter is used.
where, i y = is the ith HTM-SP mini-column j y = is the jth HTM-SP mini-column j i, = mini-column indexes  = inhibition radius control parameter For creating associations with input patterns, feed-forward inputs to each of the generative spatial mini-columns are computed using a matching technique called the overlap; this concept is diagrammatically illustrated in Figure 2. The overlap is computed as:  From Equation ( 6), we can calculate the activation of each SP mini-column as: where, s = target activation density (sparsity) The HTM-SP uses a learning rule inspired by competitive Hebbian learning for reinforcing dendrite permanence values [17].The learning rule can be calculated from the formula given in Equation (9) as: where,  p = positive permanence value increment  p = negative permanence value increment 1  t A = activation state at time step, t Finally, boost updating in HTM-SP follows the homeostatic excitability control mechanism comparable to that observed in cortical neurons [19].Boosting is accomplished in HTM-SP using the expressions in Equation (10).  t a i = the current activity of the ith mini-column at time step t.  = a positive parameter that controls the strength of the adaptation effect As mentioned in [17], "such calculations have been used in previous models of homeostatic synaptic plasticity" as in [20,21].

Temporal Classifier
In the proposed HTM system, feed-back associations are built from the HTM Spatial Pooler (SP) SDR using a temporal overlap classifier.The Temporal classifier uses the overlap technique which is similar to Equation ( 6); however predictions are made by performing a match between a set of past observation sequences in the SDR (used as context) and its current observation sequence.The temporal overlaps through time are obtained using Equation (11).
where, c N = Number of past sample sequences in the SDR used as context k = size of the temporal aggregated (bi-variate) sequence of the SDR through time t j = temporal aggregation index number sp j t W = the bi-variate SDR after temporal aggregation 2.1.3.Temporal Classifier A Temporal Aggregation Technique (TAT) is used in the HTM-SP to build a cause-and-effect data sequence from a univariate integer sequence of the SDR formed in the spatial pooling step and then used for an overlapping temporal classification (OTC); the assumption here is that the previous sequence is the cause of the next sequence.The proposed HTM-SP technique is a variant of a software tool developed earlier in [22] and can be found in [23].In HTM-SP, adding more variables increases the degree-of-freedom for making effective overlap matches.The temporal aggregation procedure used in the predictions is as follows:  Form a single-column vector matrix of length 1:N having a with a width of 1, where N represents the number of sampled sequences of the SDR obtained from the HTM-SP stage.
The elements in this matrix contain the indexes for temporal aggregation. For each element in the matrix formed in Step 1 greater than 1, perform a modulus operation such that if a remainder exists for the considered element we skip that element, otherwise we select the element; this operation results in single-column vector matrix of length approximately equal to 1:N/2.The elements in this matrix contain the set of even indexes in the matrix obtained from Step1 at time instance, t.We call this set At(1).
 For all elements in the set At(1), form a concatenation of At(1) with At(1) 1-step behind as {At(1) At-1(1)}; this concatenation represent the temporal aggregator index set.We call this set of indexes At(agg).
 Using At(agg) as index sequence, extract SDR patterns obtained from the HTM-SP stage in a temporal aggregated fashion and then perform overlap temporal classification through time.

The Online Sequential Extreme Learning Machine
In standard neural network paradigm, learning data sequence(s) or block(s) online implies continually learning the input-output associations sequence by sequence or chunk-by-chunk, in such a way and manner that the predictions have to vary at each time step and are formed as anew at next time step.The Online Sequential Extreme Learning Machine (OS-ELM) is one neural machine learning technique that conforms to this concept.The OS-ELM technique was developed in [11] and has been shown to be faster with good generalization over a similar sequence learning network called the Resource Allocation Network (RAN) proposed in [24] and its associated variants [23][24][25][26][27][28][29].
However, from experience with working with the OS-ELM on real world data, we discovered that OS-ELM may not perform favorably well generating models with large standard errors particularly for univariate series i.e. sequential data without any associated labels.These large deviations may be the result of local learning.In the following sub-section (sub-section 2.2.1) we propose an explicit hierarchical training scheme that affords a global solution space.

Explicit Hierarchical Training of the OS-ELM Neural System
As mentioned previously, the OS-ELM may suffer from a local learning bias.To perform a search for a global solution space and generate reliable predictions, an explicit hierarchical training scheme is used based on a revised OS-ELM functional class coined OSELMrev which includes a Mean Absolute Percentage Error (MAPE) update to calculate the test prediction errors.The original OS-ELM functional class can be found in [30].However, this scheme basically uses a first order simulation run to generate predictions which encourages local learning but this may likely hamper knowledge discovery.
In order to overcome the aforementioned limitation, an n-order simulation is proposed here which is based on the aforementioned scheme.This encourages the discovery of a global prediction space and requires that several meta-iteration runs akin to a Monte Carlo simulation be made to obtain a best fitted prediction sequence set.By best fitted prediction is meant that single column-wise prediction sequence from a multitude of columnar prediction sequences that gave the least MAPE value.
We perform the following algorithmic steps to derive our global prediction space as follows: Algorithm 1.The OS-ELM is very sensitive to the way the data is presented to it.If data is not well labelled, it may generate results that are very far away from the expectation even with small values of errors.Thus, care must be taken to ensure that a univariate sequence does not hamper the OS-ELM performance.
For the HTM-SP this is not a problem as the Temporal Aggregation Technique (TAT) described in a previous sub-section (sub-section 2.1.3)takes care of this limitation.A sample scheme of the encoding process for the OS-ELM is as shown in Table 1.In this table, the first column represents the pressure (sequence) signals obtained from a pipeline pressure sensor while the second column is the helper label code -in this case the time stamp in hours.The sequence signals may also be any other measurable physical quantity as long as it can be sequentially monitored.The entire dataset used for the simulations is provided in Appendix A. suggested in [31] for monitoring temperature sequences; these aforementioned operations are performed by the second operator block (OP-2).

Results
The continual learning/predictive performance of the Hierarchical Temporal Memory Spatial Pooler (HTM-SP) is presented and also compared with the Online Sequential Extreme Learning Machine (OS-ELM).

Performance metrics and parameter tuning
For comparing the results we use the Mean Absolute Percentage Error (MAPE) to evaluate the predictions in both techniques.The MAPE is less sensitive to outliers particularly for large values in the predictions.The MAPE is computed as: where, y = the observed input data sequence y ˆ= the model's predictions of y

Comparative Performances of both techniques
The comparative performances of the HTM-SP with the OS-ELM considering all three activations and using the improvement factor metric introduced earlier in subsection 3.1, Equation ( 13) is as shown in Table 3.In this case, if we propose the HTM-SP as our candidate technique, the results clearly indicates that the HTM-SP will outperform the OS-ELM by a factor of 1.297, 1.297, and 1.300 over the OS-ELM using the sig, sin and rbf activations respectively.Note that reversing the candidacy will obviously not give the OS-ELM any advantage.

Discussion
In this research paper, we have proposed a machine intelligence system called the Hierarchical Temporal Memory (HTM) for real-time monitoring and prediction of pressure signals in a production field useful for securing oil and gas pipelines.We have compared our technique with the Online Sequential Extreme Learning Machine (OS-ELM) and have shown through the MAPE error performances that our technique betters the existing OS-ELM technique.The disadvantage of the OS-ELM is the requirement for training and testing data and in the need for extensive tweaking of several parameters.In the proposed HTM technique, there is no need for such requirement and we have to only tune one parameter.This is an obvious advantage when dealing with real-time automatic learning/monitoring and control system.
The improvement factor metric introduced in this research indicates that the HTM-SP machine intelligence technique is slightly superior over the machine learning one based on the used dataset.However, this is not to say the OS-ELM did not farewell either.Also the OS-ELM sig and sin activations are highly stable compared to the rbf activation that seem to generalize better.

Conclusions
In this research paper, we have proposed an integrative systems approach to the monitoring of real world oil and gas pipeline data using techniques of machine intelligence and machine learning.
These techniques include an emerging machine intelligence technique for streaming data analytics called the Hierarchical Temporal Memory (HTM) and a sequence learning neural network technique called the Online Sequential Extreme Learning Machine (OS-ELM).
The HTM technique used a spatial pooling process coined (HTM-SP) with temporal aggregation of learned sparse distributed sequences to make its predictions whereas the OS-ELM technique employed an explicit hierarchical training scheme to compute a global prediction space.
The performances of both techniques as real time security monitoring agents in oil and gas installations are promising.Future research directions should include the integration with Real Time Embedded Operating Systems (RTOS) and a study of the computational cost with the use of full and scaled down versions of both techniques to enhance computational processing speed.

Figure 2 .
Figure 2.An illustrative concept of the Overlap in an HTM-SP; adapted from [18].

ia
= time averaged activation over the last T SDR inputs, T = an integer number denoting the number of Monte Carlo trials to obtain a reasonable activation estimate.

Figure 3 .
Figure 3.An Integrative Systems Model for real time pipeline monitoring and control

Figure 4 .Figure 5 .
Figure 4. Moving Average plot of the HTM-SP at look-ahead of 1-step

Table 1 .
A sample encoding scheme for the pipeline dataset gives least error will pass the predicted signal that activates the CONTROL/ALERT block via OP-2 block.The CONTROL/ALERT block will typically be activated once the pressure line drops below a threshold pressure value which may be defined explicitly or by a reference pressure signal tapped from a consensus of continually predicted pressure signals as block that feeds to the CONTROL/ALERT block through a second operator block (OP-2) i.e. the processor block that

Preprints (www.preprints.org) | NOT PEER-REVIEWED | Posted: 9 August 2018 doi:10.20944/preprints201808.0194.v1Table 1 .
MAPE Performance of OS-ELM activation function variants.The HTM-SP performance result when the look-ahead parameter is set to 1 and 2 is as shown in the moving MAPE plot of Figures4 and 5respectively.In Figure4, the MAPE values averages at around 0.23 with a peak value of about 0.32 and minimum value of 0.14 while in Figure5the corresponding average, peak and minimum values are 0.24, 0.36 and 0.05 respectively.It is possible to achieve much lower values by tweaking the other HTM-SP parameters but we do not consider this form of experimentation.

Table 1 .
Improvement of the HTM-SP over the OS-ELM activation function variants.