Integrated Approach to Optimizing Selection and Placement of Water Pipeline Condition Monitoring Technologies

1. Introduction

Condition Assessment (CA) and Condition Monitoring (CM) of vast and dispersed underground infrastructure assets in water supply systems are essential to prevent catastrophic events in compromised strategic assets. Routine assessments and monitoring are fundamental aspects of identifying early distress warnings and structural anomalies, which when addressed can extend the Remaining Useful Life (RUL) of this vital infrastructure component. Although Barfuss [1] reported that utilities serving 30% of the population of the United States and Canada have seen an overall reduction in water main failure of 20% in the past five years, there is an estimated water main failure every two minutes in the US, which represents an approximate loss of drinking water of 6 billion gallons per day [2]. These types of infrastructure failures not only represent an economic loss from the spill of treated drinking water, but also represent costly and disruptive emergency repairs [3].

Due to the extensive inventories of pipelines, valves, meters, tanks, reservoirs, and other crucial elements of water systems, it can be challenging to monitor and measure each asset, unless the high risk of failure makes it utterly necessary. However, it is widely understood that the water utility sector often lags in adopting advanced and innovative technologies, thereby compromising operating efficiency, cost management, and overall level of service [4]. The results of a recent survey provide an overall judgement on this topic. Barfuss [1] reports that 43.5% of utilities in the US and Canada (17.1% of the estimated total water mains in both countries) conduct regular CA of water mains; suggesting that an even lower percentage may perform consistent CM. Furthermore, Culshaw and Kersey [5] explained that due to blissful ignorance and the complexities of adopting innovative or advanced technologies, certain circumstances have led stakeholders to forego the assessment of system deterioration, opting instead for the liability of corrective actions and system renewal. Consequently, CM of underground pipelines has remained a time-consuming and less rewarding undertaking [3].

Recent technological advances in ease of installation, power requirements, wireless connectivity, data science, and development of new sensing approaches make it possible to virtually assess and monitor almost any type of underground pipeline. Despite these advancements, the adoption of CM technologies does not imply the selection, purchase, and use of a device [6]. The fundamental challenge relies on the design of strategies for optimal selection and placement of monitoring technology. When performed under optimal conditions, monitoring provides increased system reliability by detecting and analyzing the development of failures in the early stages, thus providing time for key decision-making aimed at minimizing any loss [7].

Previous research has primarily focused on determining the optimal placement of pressure sensors [8,9,10,11,12,13,14,15,16,17,18] and identifying the assets that require renewal, along with the appropriate technologies for this purpose [19,20,21,22]. In contrast, comparatively fewer studies have focused on strategies for the selection of CA and CM technologies [23,24,25]. Furthermore, determining the optimal placement and selection of technologies for monitoring infrastructure assets generally involves Decision Support Systems (DSS). In this regard, Hangan et al. [26] explained that, until 2022, the volume of published studies on DSS and water topics has been relatively low compared to other water-related topics such as the Internet of Things (IoT), big data, and anomaly detection.

To address both optimal selection and placement of CM technology for water supply systems, this article introduces a procedure that screens sensor technologies based on their performance attributes, uses a recently introduced hierarchical strategy to rank technologies, and explores the use of various Machine Learning (ML) algorithms to identify which pipelines are located in vulnerable areas prone to failures and identify the maintenance or physical attributes of water pipeline data that contribute to failures. An integrating algorithm proposed as part of this study combines the results of technology selection and sensor location to achieve optimal results. Therefore, the objectives are to provide a summary on the topic of CA, CM, technology selection and placement; a detailed description of the proposed approach for optimal selection and placement via an integrative algorithm; and verify its applicability through a synthetic case scenario.

Table 1 lists the acronyms and abbreviations used throughout this article. The remaining contents are structured as follows: Section 2 provides a comprehensive review of the literature and a summary of CA and CM technologies applicable to water pipelines. Section 3 briefly discusses the published strategies for placement and selection of CA and CM technologies. Section 4 provides a detailed description of the proposed integrative approach to optimal technology placement and selection. Section 6 includes a discussion highlighting the benefits and limitations of the new approach and providing alternatives for future research. Section 7 provides concluding remarks.

2. Condition Assessment and Monitoring Technologies

At its core, CA evaluates the deterioration of the system using technology to determine the physical integrity of a water pipeline and to determine whether a pipeline meets the needs required by a utility in terms of RUL, overall deterioration, the ability to withstand internal pressures and external loading, water tightness, and hydraulic capacity [27,28]. Traditional methodologies such as walking the pipe, sounding or hammer testing, and internal visual inspections are primitive and do not provide detailed information on the condition of the structure to manage risk and make informed decisions [3,29]. Most utility water pipelines are pressure conduits, which have more complexities than gravity pipes; therefore are more difficult to inspect. Some of the complexities include lower visibility, limited accessibility, and possible service interruptions [30]. The technological developments have aimed to overcome these limitations by conducting field or desktop analyses and direct or indirect evaluations [28]. Liu and Kleiner [31] explained that visual inspections and Non-Destructive Testing (NDT) are examples of direct assessment; whereas water audits and soil resistivity measurements are examples of indirect assessment. These technologies allow utilities to implement an asset management strategy in an attempt to identify pipelines in need of renewal and narrow the number of assets that require replacement [27].

CM differs from Structural Health Monitoring (SHM), yet it is an integral component of SHM along with other four disciplines, including NDT, statistical analysis of damage identification and damage prognosis [7,32]. CM encompasses more than the acquisition and installation of a sensor or device [6]. For example, for remote monitoring of water assets, it can incorporate various key elements and technologies, including sensors, communications, data management and interpretation, and power requirements [33]. Large water utilities consider monitoring an approach to strategic priority to improve data-driven decision making [34]. Typically, water utilities install sensors at critical locations in their supply systems to monitor water pressures through Supervisory Control and Data Acquisition (SCADA) [35], which is useful for the initial indirect desktop CM of watermains; however, this is typically not enough as vulnerable underground infrastructure follows various failure mechanisms and can experience unexpected catastrophic failures that result in costly, disruptive, and reactive replacements [3]. Although it is evident that CM is paramount for system resiliency and reliability, it is often considered an unwanted and uneconomical aspect of the asset life cycle [7]. In fact, monitoring the condition of underground assets has generally remained a time-consuming and less rewarding task [3].

CM entails an orderly determination of the current system condition. It enables understanding the remaining useful life, facilitates analysis of developing failures, and defines the severity of distress signals [7]. CA and CM differ primarily in their focus. CA is generally characterized by the reporting of a condition in a discrete single state with specific data collection, whereas CM is characterized by the reporting of multiple condition states in a somewhat continuous manner with broader data collection. Some researchers indicate that CM includes the acquisition of data in an automatic and unattended mode to increase structural performance awareness [6,36]; however, others adopt a broader interpretation that includes common single-state CA technologies that provide time-lapse monitoring and benchmarking using periodic inspections [25,37]. Moreover, the US Environment Protection Agency (EPA) defines CA as the direct inspection, investigation, and indirect monitoring of the structural and operational performance of an asset [38]. In some cases, this gray area between CA with NDT and CM has been a topic of debate within industry organizations [39]. Given these circumstances, published literature reviews on CM of water assets can include CA technologies with periodic inspections for time-lapse monitoring [26,37,40,41].

As shown in Figure 1, CA and CM technologies can be classified according to the sensing modality (acoustic or non-acoustic methods) [42,43], system implementation (hardware or software-based methods) [43,44], measurement approach (direct or indirect methods) [37], data generation (measurement-based or model-based) [45] and interrogation frequency (near real-time, periodic or as-needed methods) [25,46].

Summaries of the main advantages and disadvantages of the classified CA and CM technologies based on their sensing modality are shown in Table 2 (acoustic-based) and Table 3 (non-acoustic-based). A summary of the advanced software-based methods that are part of the system implementation (software-based) and data generation (Model-based) classifications is shown in Table 4. In the following sections, this study provides a method for classifying these technologies by interrogation frequency and combining them with optimal CM technologies.

3. Techniques for Placement of Condition Monitoring Technologies

In addition to developing innovative approaches or advancing existing technology, recent research on CA and CM has also focused on optimal sensor location to better address failure detection and reduction [43]. Building from a definition by Sela and Amin [93], optimal sensor placement can be defined as finding the locations where anomaly detection is maximized with the least number of sensors in a cost-effective approach. The techniques for optimal sensor placement are considered by many researchers as paramount for satisfactory results. With the rationale behind optimizing placement of CM sensors being diverse, Table 5 summarizes some of the frequent reasons.

As clearly explained by Hunt [7], the essence of a CM strategy involves building management abilities to ascertain possible future failures and the most appropriate CM equipment to detect such failures. In industry practice, the management strategies for defining where to install CM technology often include components of asset management principles coupled with risk-based approaches. For example, in a Water Research Foundation (WRF) report by Reed et al. [25], the authors proposed a strategy applicable to large diameter pipelines of 30 inches and larger ( 750 mm) that included the following five stages: (1) Selection and prioritization of pipelines for monitoring, (2) Screening and monitoring, (3) Investigation and assessment, (4) Remedial action, and (5) Future monitoring plan. The authors explain that in stage 2 water pipelines are evaluated to determine if a detailed investigation is required. At stage 3 all pipelines are re-evaluated to determine if remedial action is required. When a pipeline requires remedial action, it continues to stage 4 and its RUL is extended. In contrast, when a pipeline does not require remedial action, a future long-term monitoring plan is designed with periodic or near-real-time CM technologies. Alternatively, published scholarly articles tend to conduct simulated analyses for relatively small networks where sensors are optimized for areas with the majority of simulated leaks without considering risk-based asset management principles. A case in point is the study by Yang and Wang [9], who used a hydraulic model based on EPANET Net3 to generate leak events and determine Detection Coverage Rate (DCR) and Total Detection Sensitivity (TDS) indexes for pressure sensors. Another case is the study by ChenLei et al. [8] who proposed a method to optimally locate pressure sensors using data collected from an experimental laboratory setting. The authors developed a ML model to spatially group pipeline network nodes and pressure anomalies using an improved version of DBSCAN (Density-Based Spatial Clustering of Applications with Noise) called OPTICS (Ordering Points to Identify the Clustering Structure). The authors identified the optimal number of sensors, but failed at considering non-spatial attributes often used in risk-based asset management principles such as pipe material and diameter. A summary of the relevant studies on the optimal placement of CM sensors is presented in Table 6.

Table 6 shows that the subject of effective sensor placement has been investigated, especially for pressure sensors. Despite significant progress, the literature points to further research requirements. For example, Menapace et al. [101] explained that sensor placement optimization models tend to excel in one application without considering various criteria. The study also suggested that models may require considering additional network complexities without creating significant computing limitations. Gupta and Kulat [42] also explained that optimal multiparameter sensor placement is required and Santos-Ruiz et al. [16] indicated that additional work must account for sensor location prioritization based on asset management principles. Stanczyk and Burszta [43] concluded that since the implementation of a precise placement of a CM sensor is necessary, it remains a subject of continuous development. Furthermore, a recent study by Du et al. [18], explained that Pressure Sensor Placement (PSP) strategies often lack robustness, as they are prone to background noise and inaccuracies. The authors analyzed the subject by quantifying various metrics in randomly generated leaks or bursts through simulations, suggesting that the strategy could benefit engineering practice by using historical pipe burst and operational data from a real-world system.

4. Framework for Optimal Selection and Placement of CM Technology

One of the main components of this study is to provide a framework that optimally integrates the selection and placement of CM technology for water pipelines that considers multiple technologies, risk-based asset management principles, and robust use of historical pipe failure data. This section introduces such framework from a conceptual approach, details on the associated components, and data sources. Despite its straightforward structure, the framework requires minimum viable system data on topological conditions, physical properties, and historical operational records.

The framework provides an alternative formulation to achieve optimal selection and placement of CM technologies. It is developed in part from general guidelines by Wang [46] and Liu and Kleiner [36], and based on the recommendations of research by Zarghamee et al. [102] who suggested that the placement and selection of optimal condition assessment and monitoring technology should be based on risk-based asset management approaches. The resulting concept of the framework is synthesized into five components named the R-E-R-A-V approach (Redundant, Established, Reliable, Accurate, and Viable). The fundamental ideas for each component are explained below.

• Redundant: Various technologies and back-up systems should be considered.
• Established: Methods should be reliable and with a proven track record.
• Reliable: Technologies developed through rigorous testing and evaluation in real-world systems.
• Accurate: Results should provide minimal false positives, false negatives, and errors.
• Viable: Cost-effective and financially feasible approach given the unique limitations of utilities.

Based on the R-E-R-A-V principle, the process to determine the optimal type and placement of CM sensors includes a Technology Readiness Level (TRL) evaluation to define established and reliable technologies. The process uses a Spherical Fuzzy Analytic Hierarchy Process (SFAHP), an OPTICS algorithm, and a k-Nearest Neighbors (kNN) model to define accurate and viable technologies. Finally, an integrative algorithm utilizes information obtained from previous components to select which pipe sections are required to be monitored. The last step uses an array of various scored and ranked CM technology, assuring system redundancy. A summary of the various components included in the framework is shown in Figure 2.

The following subsections provide details of the framework’s principles for the selection and placement of CM technology for water pipelines. As each component of the proposed approach is a data source (DS_1 to DS_6) on the integrative Algorithm 1.

4.1. Approach to Optimal Placement of CM Technologies

The proposed framework provides a multimodel procedure for identifying the optimal placement of CM technology using data usually available in water utilities. Particularly, the approach benefits from well-structured spatial and non-spatial data sets consisting of basic topological and physical information of assets, including, but not limited to, pipe material, diameter, length, installation year, longitude and latitude. Additionally, data on historical maintenance activities related to pipe failures (pipe leaks and breaks) is utilized to define monitoring priority areas. These data sources are part of the usual Likelihood of Failure (LoF) and Consequence of Failure (CoF) components of risk analyses; therefore, they allow for mutually integrated evaluations of risk-based asset management. Most utilities collect and display these data through Geographical Information System (GIS) platforms and Computerized Maintenance Management System (CMMS).

Note that in the water industry the terms leak, break, and burst can have different interpretations [103]; therefore, successful results and implementation of this approach in real-world systems require investigations of how data are collected. This approach uses the term main failures as an umbrella term based on the definition by the International Water Association (IWA) [104]. The definition includes detected water leaks in the distribution and transmission system that require renewal or repair.

Initially, spatial CMMS and GIS data (DS_1 and DS_2 in Figure 2) are combined to determine failure clusters at the network or DMA levels. These failure clusters represent problematic areas where one or more deterioration factors seem to cause general degradation of pipe materials. There are countless factors affecting the RUL of pipelines, which for the most part include pipe material, age, diameter, soil type and humidity, and depth of cover. Since in real networks it may not be possible to have reliable data on such a diverse number of features, a spatial classification of failure clusters is an appropriate method for grouping locations actively affected by deteriorating factors. To account for this, an OPTICS classification is proposed (DS_3 in Figure 2). The OPTICS algorithm was first introduced by Ankerst et al. [103] as an improved version of DBSCAN to address the high sensitivity of manually set parameters. Unlike DBSCAN, OPTICS does not automatically assign clusters but generates a reachability plot from which cluster structures can be extracted.

OPTICS is a well-known algorithm in the ML and data science fields; therefore, it is often included natively or integrated via third-party plugins in GIS software used by water utilities [104,105]. The steps involved in performing a data classification using OPTICS have been covered by various authors ChenLei et al. [8], Ankerst et al. [103]; however, in general it involves the following minimum steps. Once data is loaded to an Integrated Development Environment (IDE) and prepared by performing general scrubbing techniques, the OPTICS algorithm runs using two key parameters: minimum domain points to form dense regions (MinPts) and a distance metric (e.g., Euclidean, Cosine, or Manhatan). Although a maximum epsilon radius (max_eps) can be specified, for most cases it is set to infinity to allow the algorithm to explore varying degrees of densities. The algorithm computes reachability distances and produces a density-based ordering of data. Once the algorithm ranks each data point, a reachability plot can be used to visualize density variations and define a suitable reachability distance with an epsilon cut-off point that encompasses desired clusters.

An additional ML model based on kNN is proposed to associate pipe characteristics and historical maintenance data to discern possible new or additional failures (DS_4 in Figure 2). Although other attempts to predict water main failures have been limited to the evaluation of Logistic Regression (LR), Support Vector Machine (SVM), Classification Tree (CT), and Random Forest (RF) algorithms [90], there is reported research on the use of kNN as a fundamental classifier for the CM of water assets, with a demonstrated ability to determine the location of anomalies in water systems [43,88,106,107].

The proposed kNN model provides automated decisions from known data patterns of training data by considering a multidimensional set of spatial and non-spatial features (DS_1 and DS_2 in Figure 2). Some of these features include, pipe material, pipe size, average age at break, pipe location, and proximity to a failure cluster previously determined with OPTICS. The process of developing a kNN model requires common data scrubbing techniques, normalization, and one-hot-encoding for addressing missing observations, produce fair comparisons between features, and replace categorical variables. The data set is split into training and testing portions for model development and validation. The algorithm is run by selecting an appropriate number of neighbors to compare (k) and a distance metric (e.g., Euclidean, Manhattan, Minkowski or Hamming)[108]. Defining the k value can induce different degrees of bias and variance, therefore, the literature recommends heuristic approaches and options such as the Certainty Factor (kNN-CF), Locally Informative-kNN, Globally Informative-kNN, K-local Hyperplane Distance Nearest Neighbor (HkNN), k Nearest Neighboring Trajectories (k-NNT) and Correlation Matrix kNN (CM-kNN) [109]. Despite the previous approaches to minimize the effects of selecting a k value, the recent published literature confirms that appropriate performance can be achieved when the kNN model is cross-validated by experimental tuning [107].

The last step considered for optimal placement of CM technologies integrates the basic topological and physical pipeline data (DS_1 and DS_2 in Figure 2) with the preferred technology alternatives derived from the components for optimal CM technologies. These data are used as part of Algorithm 1.

Algorithm 1 RERAV-bassed Integrative sensor placement and technology selection.

4.2. Approach to Selection of CM Technologies

The framework incorporates sensor-based technologies that have achieved sufficient technological maturity, commercial viability, and can be deployed in wired or wireless configurations. The sensors in this framework are capable of providing CM data at near real-time or periodic frequencies. Consequently, the proposed framework does not incorporate certain CA technologies or data-driven CM methods (listed in Table 4), since those methods and technologies are primary suited for as-needed inspections or to analyze data patterns from preexisting sensor deployments. The approach evaluates technologies based on their TRL (DS_5 Figure 2).

Because unverified technology can result in unreliable condition assessment and monitoring data, inaccurate anomaly detection, unnecessary repairs, missed failures [102], verifying technology reliability before selection is crucial. The degree of reliability and technological maturity of CM technology has been evaluated using innovation indicators such as the TRL [76]. The method was developed by the American National Aeronautics and Space Administration (NASA) on the concepts of flight readiness reviews [110,111]. The TRL concept was later adopted by the US Department of Defense (DoD) as a technology acquisition tool, the European Union (EU) as an innovation policy tool, and other agencies worldwide [110]. The TRL approach provides scales to measure the maturity level of technologies from initial basic principles to final systems proven in operational environments. Since the lack of specific terminology for pipeline-related technologies makes it difficult to assign specific TRL levels to CM technologies with the definitions provided by NASA or the EU, it is prudent to adopt the TRL definitions established by the Pipeline Research Council International (PRCI) [112]. The PRCI TRL definitions are also complemented with other TRL approaches used in civil engineering, such as the TRL guidelines established by the US Department of Transportation (USDOT) [113]. Therefore, a proposed TRL system for evaluating CM technologies used for water supply systems is shown in Figure 3.

Based on the published literature on TRL, their application to pipelines and their use for evaluating CM technology, the proposed companion definitions of TRL are provided as follows [110,111,112,113]. In TRL 1 basic scientific and engineering principles of monitoring pipeline conditions are studied and reported in a conceptual approach. TRL 2 elaborates on the conceptual approach to formulate an application proven through analysis or reference to similar CM technologies. TRL 3 defines a technology that demonstrates feasibility through functionality testing using physical modeling with pipeline system representations in laboratory experiments or simulations. In TRL 4, the performance, reliability, scalability, applicability, benefits, and risks of CM technologies are evaluated through relevant but generic laboratory tests that integrate hardware, sensing components, communications, data acquisition and processing, and sensed element. TRL 5 is about validating the CM technology in simulated environments that resemble real applications. At this stage, the technology qualification process demonstrates performance and reliability in the anticipated operating conditions and environment of the sensed element. TRL 6 relates to a technology that is designed and assembled to function as a production-ready device or a full-scale prototype deployed in the system to be monitored. At this stage, the technology demonstrates functional system reliability without being in the anticipated field environment. TRL 7 defines a technology that is fully-functional in the operational and environmental conditions of a system as part of a filed qualification and demonstration program. TRL 8 relates to the successful application of a production-ready CM technology for less than three years in the actual operational and environmental conditions of a real system. TRL 9 is reached when a production-ready CM technology has been deployed and in operation for more than three years with acceptable predefined levels of accuracy, reliability, and risk.

The implementation of a TRL approach often requires an assessment using scoring sheets with guiding questionnaires and supporting details or evidence about the technology. The scoring sheet is designed to obtain yes or no answers with supporting evidence in accordance with the definition given for each level. For example, in TRL 1 the USDOT TRL system asks: "Do basic scientific principles support the concept?" and "Has the technology development methodology or approach been developed?" [113]. Appropriate knowledge on the level of technological evolution provided by the TRL assessments for each CM provides a defined approach to the selection of optimal and mature technologies. However, other factors, such as costs and budgets, are also associated with decision-making and technology selection by water utilities.

Since the different options of CM technology have differences in their deployment and overall functionality within the physical and operational conditions of water utilities, a decision-making tool is necessary to further define an optimal approach (DS_6 in Figure 2). SFAHP is a Multi-Criteria Analysis for System Characteristics (MCA-SC), and is one of the newest iterations in traditional Analytic Hierarchy Process (AHP). SFAHP has been shown to outperform other variants of AHP in prioritizing the significance of water pipeline deterioration factors [114].

Traditional AHP fuzzy approaches often rely on triangular fuzzy numbers to define membership functions. In contrast, SFAHP employs Spherical Fuzzy Sets (SFS) to incorporate a three-component structure that captures membership, non-membership, and an explicit hesitancy degree. These values determine how well an element belongs to a set and quantify the uncertainty of the evaluator’s judgment of an alternative with respect to a criterion due to lack of complete information or confidence in the decision. SFS was introduced by Kutlu Gundogdu and Kahraman [115] in 2019, and has been applied in AHP to select optimal technologies and prioritize the criticality of pipeline deterioration [114,116].

A hierarchical structure of the problem (CM selection) is developed for SFAHP by defining technology alternatives and various criteria. The process follows the steps proposed by Kutlu Gundogdu and Kahraman [115], and applied in other related studies [114,116]. Subsequently, a spherical fuzzy linguistic evaluation scale is performed by developing pairwise comparison matrices and performing a consistency check by calculating the Consistency Ratio (CR). Additional steps include calculating spherical fuzzy logical weights using the Spherical Weighted Arithmetic Mean (SWAM) operator, aggregating spherical fuzzy weights, defuzzifying criteria weights, and normalizing. Lastly, the SFAHP score is calculated by arithmetic addition of global weights, and the final score is defuzzified.

Let A1 to A4 be verified TRL 8 and TRL 9 CM technologies (alternatives), and C1 to C5 various sample system or platform characteristics (criteria), including installation, operation, communications, and cost aspects (see Table 7) The hierarchical structure of the problem would then be defined as shown in Figure 4.

Using a hierarchical structure similar to that presented in Figure 4 and the SFAHP calculation process outlined in Kutlu Gundogdu and Kahraman [115], the selection approach allows decision-makers to incorporate technology functionalities and deployment options.

After defining a preferred alternative through SFAHP, additional verifications are needed to confirm suitability of the selected technology based on its operational limits and the characteristics of the infrastructure in which it will be deployed. For example, CM technologies typically used for concrete pipelines should not be selected to be deployed in other pipe materials. Therefore, in such cases, it is necessary to develop a dataset that contains the operational limits of CM technologies. These operating limits are generally available in the published scientific literature and through manufacturers’ technical specifications. Some sources have also been compiled and are available in technical industry reports.

With the aforementioned sources of information, an integrative Algorithm 1 automatically determines which assets require condition monitoring and assigns the most appropriate technology. The algorithm integrates the following sources of information DS_1: base DMA GIS information (topological and physical), DS_2: base DMA CMMS information (record of failures), DS_3: results of failure prone areas with OPTICS algorithm, DS_4: results of failure prediction model with kNN, DS_5: TRL evaluation and operational limits of CM technologies, DS_6: results of the SFAHP ranking of CM technologies. Algorithm 1 works by evaluating the vulnerability of each pipe segment using two approaches, (1) its proximity to general degradation factors that occur in areas prone to failures previously indexed with an OPTICS algorithm, and (2) the likelihood of future failures as expressed from known failure factors indexed in a kNN model. If a pipe section is found to be vulnerable and likely to have failures, the pipe is classified as being part of the utility distribution or transmission system. CM technologies are selected for each qualifying pipe using the operational limits of the technology. Only the first SFAHP ranked technology is assigned.

5. Approach Implementation Through a Case Scenario

An additional objective of this study is to verify the proposed framework using synthetic network and system data. In this section, the approach described in Section 4 is applied to assign optimal CM technology using the integrative Algorithm 1, together with the results from the OPTICS and kNN models. Since this section uses a synthetic approach, the selection of technologies with TRL and SFAHP is treated as predefined rather than formulated de novo for this scenario.

Given the widespread use and reproducibility of EPANET Net3 within the body of knowledge, this section develops the scenario using preexisting Net3 system information with additional characteristics. Net3 is a well-studied system example from the EPANET manual, and is derived from the North Marin Water District (NMWD) in Novato, California [117]. It has been used to study water quality, chlorine residual, and disinfection formation modeling; however, it is also frequently used to study sensor placement [9,18,96,97]. Net3 provides information on selected topological and physical attributes, such as pipe diameter, network geometry; however, it lacks specific information on pipe material, installation dates, and history of failures. Furthermore, since hydraulic modeling with Net3 does not require consistent scaling of pipe lengths on a map, they are displayed at varying scales in EPANET. Given that the proposed OPTICS requires a consistent scale of data points and the kNN model requires input from topological and operational data, some attributes of the standard Net3 system were supplemented as shown in Table 8 for the layers related to pipes and coordinates. Other layers of Net3, such as reservoirs, tanks, and pumps, were not considered for this case.

A GIS version of Net3 was developed by exporting a shapefile using the inp2shp command of the InpTools package. The shapefiles were imported into QGIS for the addition of supplemental information indicated in Table 8. The pipe materials were assigned replicating the expansion of the system and the common pipe diameter ranges for each material. Materials such as Ductile Iron Pipe (DIP) were assigned to pipe segments located roughly at the center of the system, and materials such as High Density Polyethylene (HDPE) or Polyvinyl Chloride (PVC) assigned predominantly to the edges of the system. Pipe materials were not assigned to segments of the Net3 system directly related to tanks, reservoirs, rivers, and pumps. Figure 5 illustrates the materials and diameters for each pipe segment in Net3 as assigned for this case.

The dates when pipe segment installation occurred were assigned based on typical material use throughout time from 1920 to 2025, following a hypothetical expansion of the system. Therefore, Cast Iron Pipes (CIP) were mostly assigned installation dates from 1920 to 1950, DIP segments were assigned installation dates from 1950 to 2000, PVC and Molecularly Oriented PVC (PVCO) were assigned installation dates from 1980 to 2025, and Concrete Pressure Pipes (CPP) were assigned installation dates from 1920 to 2025. The installation date assignments also followed a hypothetical system expansion as shown in Figure 6. The extracted length of the linestrings was extracted from the map geometry and used as the pipe length attribute in lieu of the standard Net3 length, which does not coincide with the map scale.

A layer containing polygons that encircled all the pipe segments was developed to locate the system pipe failures. These pipe failures were randomly generated using the QGIS random point generator tool for polygons. The number of points that represent pipe failures was determined based on the average number of failures per 100 km a year in North America, which as reported by Barfuss [1] is equal to 7.7 breaks/100 km-yr. Each resulting failure point was assigned to the nearest pipe in the Net3 system in QGIS.

Information on the analytic geometry of each pipe failure point and pipe segment was extracted from QGIS and imported into R to run the OPTICS algorithm using the dbscan package by Hahsler et al. [118]. Since for this approach, the aim of OPTICS clustering is to develop a new data feature that describes the proximity to failure clusters which represent areas where various deterioration factors cause general degradation of pipe materials, and given that this new feature is employed in the kNN model; the dataset for OPTICS was partitioned. The incident dates were taken to partition the data into approximately 70% for model training and 30% for testing. Because the dataset for pipe failures (DS_2) included incident dates from this approach data of pipe failures from 1984 to 2024, a training set from 1985 to 2012 was created and used for clustering. The OPTICS MinPts chosen were 5 and the resulting reachability and clustering plots are shown in Figure 7.

The minimum Euclidean distance $\delta$ of each pipe segment centroid ($x,y$) from any cluster hull centroid ($c_{kx},c_{ky}$) was calculated using Eq. 1. Then, for each pipe segment centroid ($x,y$) in DS_1, $\delta (x,y)$ is compared to the maximum $\delta (x^{\prime },y^{\prime })\in ,D{S}_{\mathit{1}}$, where ($x^{\prime },y^{\prime }$) is a dummy variable representing all pipe segments in the dataset. When calculating $\varphi (x,y)$, or the Cluster Distance Factor (CDF), with Eq. 2, the measure is inverted so that higher values correspond to pipe segments closer to cluster hull centroids. The resulting $\varphi$-values for each pipe segment were aggregated into DS_1 as an independent variable of the kNN model.

$$\delta (x,y)=\underset{1\le k\le K}{min}\sqrt{{(x-c_{kx})}^2+{(y-c_{ky})}^2}\mathrm{where}x,y\in \mathbb{N}$$

(1)

$$\varphi (x,y)=1-{\displaystyle \frac{\delta (x,y)}{{\displaystyle \underset{(x^{\prime },y^{\prime })\in \mathcal{D}}{max}\delta (x^{\prime },y^{\prime })}}}$$

(2)

The kNN model was developed by dividing the pipe failure rates dataset (DS_2) into approximately 70% for training and 30% for testing based on the failure incident date criteria. Since failures for this case were recorded from 1985 to 2024, the training dataset included pipe segments with and without a history of failures from 1985 to 2012. Likewise, the test data set included pipe segments and their failure history from 2013 to 2024. As the kNN dependent variable, the model used a true/false categorical feature indicating whether each pipe segment has a history of failures. For independent variables, the kNN model calculates a mixed multidimensional Euclidean distance of normalized features including the CDF, total number of failures, asset age, pipe material, and pipe size. A summary of the parameters used to tune the ML techniques for determining the optimal position of CM technology is included in Table 9.

The performance measures of the predicting kNN model include accuracy, sensitivity, specificity, precision, and negative predictive value. These measures are calculated per Eqs. 3, to 7, where $TP$ is True Positive, $TN$ is True Negative, $FP$ is False Positive, and $FN$ is False Negative. The performance outcome of the actual and predicted results from the developed kNN model is summarized with the confusion matrix shown in Figure 8. As observed in the resulting performance measures (Table 10), the model particularly excels in appropriately predicting which pipe segments will not have failures.

$$\mathrm{Accuracy}={\displaystyle \frac{TP+TN}{TP+TN+FP+FN}}$$

(3)

$$\mathrm{Sensitivity}={\displaystyle \frac{TP}{TP+FN}}$$

(4)

$$\mathrm{Specificity}={\displaystyle \frac{TN}{TN+FP}}$$

(5)

$$\mathrm{Precision}={\displaystyle \frac{TP}{TP+FP}}$$

(6)

$$\mathrm{Negative}\mathrm{Predictive}\mathrm{Value}={\displaystyle \frac{TN}{TN+FN}}$$

(7)

The results of the OPTICS-kNN model provide a clear identification of the pipe segments to monitor to prevent approximately 95.7% of pipe failures for a period of 11 years. Since there are 121 pipe segments in the Net3 shapefile and 31 of them were identified for monitoring by the kNN model, approximately 25% of the pipe segments require selection and deployment of CM technology. As shown in Figure 9, the pipes that require monitoring are of various materials, sizes, and lengths. Based on these characteristics, the results of a TRL evaluation, and the SFAHP ranking, an optimal CM technology can be selected. Evident TRL 9 technologies such as permanent surface mounted VAM (noise logging) sensors can be strategically deployed for metallic pipe segments such as the ones depicted on the inset map in Figure 9.

In real-world systems, an SFAHP ranking of technologies provides a preferable set of CM technologies according to factors such as cost, scalability, and wireless connectivity, among others. Suppose that the pipe segments to be monitored are the ones selected in the results of the kNN model shown in Figure 9, and that permanent VAM noise logger technology is ranked with first preference by SFAHP, Algorithm 1 works by evaluating each pipe segment in the system, filtering the segments likely to fail, determining which corresponds to the distribution and transmission systems, and assigning the permanent VAM noise logger technology according to its operational limits and the characteristics of the pipe segment (mainly material and diameter).

6. Discussion

Previous studies offer only partial solutions to the challenge of optimal selection and placement of CM technology that considers operational conditions, maintenance history, technological capabilities, failure patterns, and the needs of water utilities. Often, published research focuses on selecting or placing technologies and does not address multiple factors with minimal data sources. Typically, addressing this diverse challenge requires a vast amount of data, including soil type, groundwater levels, depths of cover, hydraulic flow, system pressures, and additional site-specific parameters. Furthermore, these methods often require extensive post-processing activities to clean, integrate, and analyze patterns in various datasets.

In contrast, the framework and components introduced in this paper address the challenge of a streamlined approach using minimal network data commonly available at most water utilities. Through the RERAV-based approach and an integrative Algorithm 1, an optimal solution can be derived using standard water system information and standard data handling. By integrating data from CM technology performance attributes with topological, physical, and historical network data, the framework provides the following benefits.

• Suitability for analyzing entire water system networks at once, or by DMA, pipe size ranges (distribution or transmission systems), neighborhood, and so forth.
• Ability to reevaluate technology readiness and eventually include emerging methods that currently have the potential to revolutionize CM of underground infrastructure (e.g., QST and SSM).
• Adaptability to meet the needs and goals of different types and sizes of water utilities by systematically ranking CM methods based on their functionality and varying operational environments.
• Possibility of providing more or less importance to technology costs, wireless connectivity, scalability, power requirements, installation complexity, and additional criteria of importance to utilities.
• Applicability to most water utilities due to its lower special data requirements. The approach utilizes data features typically employed for regular water audits and loss control studies.
• Ability to discern which pipelines in vulnerable areas have a higher chance of failure, thus requiring specific CM based on its physical, topological, and material characteristics.

The approach introduced in this paper is not a fully automated process, it requires human intervention to define some criteria, especially those related to the SFAHP ranking of CM technologies. Despite the need to provide expert knowledge for decision-making, the methodology is considered to have reduced levels of complexity, since its implementation can be achieved with well-known ML algorithms and standard data sources. Furthermore, an automated solution for any water utility is not practical since utilities have their unique resource challenges, requiring case-by-case adjustments of AHP criteria to suit those particular needs.

Future research to explore applicability in real-world systems and conduct optimization methods to improve robustness by identifying noise-generating factors that can lead to unreliable results. Additionally, it is encouraged to explore the application of the introduced REVAV-based approach for the location of sensors directly on system appurtenances (e.g., hydrants and valves), which are generally the locations where CM technologies are deployed.

7. Conclusions

Although studies and industry reports suggest a modest improvement in overall water main infrastructure conditions, the deterioration of large, dispersed, and strategic water assets continues to be a cause of significant economic losses due to unforeseen failures of unsupervised assets. Addressing this issue requires methods to determine both suitable technology and precise positioning based on the conditions and performance of the system. However, current strategies traditionally rely on complex algorithms and vast amounts of data from sources often of insufficient quality or not readily available at utilities. Furthermore, current research has focused on separate analyses for the choice of sensor and optimal sensor placement, employing modeled hydraulic data while disregarding historical system performance. A successful condition monitoring program must incorporate accurate technology selection and precise technology placement, effectively detecting most early warnings of compromised strategic assets, preventing leaks and breaks, prioritizing repairs, and ultimately reducing infrastructure maintenance costs.

This study provided a comprehensive review of CA and CM technologies, along with common approaches to selecting and placing technology. To the authors’ knowledge, no previous research has proposed a CDF derived from the centroids of OPTICS cluster convex hulls as a measure to evaluate pipe segment vulnerability due to proximity to failure prone areas. Additionally, a unified approach for positioning and selecting optimal CM technology for water supply systems has not been documented in the existing body of knowledge. The demonstration of the usefulness of OPTICS, CDF, and the kNN model using minimal physical, topological and maintenance data through a synthetic benchmark (Net3) highlights the feasibility of this innovative approach. Hence, building on earlier work, this study documents an innovative approach grounded in a REVAV-based framework, that integrates structured decision-making techniques and ML algorithms. The study expands its innovative reach by introducing an integrative algorithm that uses asset management principles to automatically assign CM technology to pipelines at the highest risk of failure. Finally, due to the minimal data requirements, and standard GIS and CMMS data sources, the framework and methods presented in this article are highly adaptable to real-world systems.