Improvement Measures for Bridge Inspection Efficiency Using Spatial Information Technology

The economic development and infrastructure of a nation are closely interrelated. In addition, public trust in national infrastructure facilities is closely linked to the preservation of the advantages provided by these facilities to the public. Since the 1970s, Korea has achieved exponential economic growth over a short period of time and the number of infrastructure facilities has increased correspondingly. This compressed economic development has been underpinned by the national infrastructure, whose safety and usability have been excluded from the scope of the development. However, after around 30 years, structural deterioration coupled with general insensitivity to safety in today’s society has considerably reduced public trust in using the infrastructure. Realistically, policies that mainly focus on developing new technologies related to infrastructure construction have led to practical limitations that discourage the development of technologies for maintenance or inspection. Furthermore, current maintenance works face certain limitations caused by various reasons: insufficient budget, increasing number of infrastructure facilities requiring maintenance, shortage of manpower, and rapidly increasing number of aging infrastructure facilities. To overcome these limitations, a new approach is required that is different from general inspection methods under the existing rules and regulations. In this context, this study aimed to explore the efficiency of bridge inspection and maintenance by unmanned aerial vehicles (UAVs) that could observe inaccessible areas, could be conveniently and easily controlled, and could offer high economic benefits. To this end, various tests were performed on elevated bridges, and suitable UAV images were obtained. The obtained UAV images were inspected by using machine vision technology, thereby excluding subjective evaluations by humans. Methods for enhancing the objectivity of the inspection were also discussed. The test results showed that both the efficiency and objectivity of the proposed method were better than those of the existing bridge maintenance and inspection methods.


Introduction
Bridges are an important example of SOC(social overhead capital).They are directly related to public safety from a socioeconomic perspective and are critical infrastructure components for transport and logistics, which are integral to economic activities.Recently, in Korea, the level of infrastructure safety perceived by the people has significantly dropped because of natural disasters as well as accidents resulting from human error.Therefore, it is imperative to secure the safety of SOC.In fact, public infrastructure safety is a common concern not only in Korea but also in many other countries that have achieved dramatic economic development over a short period of time.In the case of Korea, infrastructure facilities were built mainly during the 1970s and 1980s, when the compressed economic development occurred.Infrastructure facilities that were built 30 or more years ago are generally classified as aged infrastructure.A study revealed that aged infrastructure facilities accounted for over 11% of all infrastructure facilities as of 2017 [1,2].Therefore, it is clear that the systematic maintenance, repair, and reinforcement of aged infrastructure facilities, which are related to securing the national safety network, are critical problems.So far, good maintenance has kept infrastructure facilities free from accidents.However, going forward, the safety of infrastructure may face considerable risk because of the increasing number of aged infrastructure facilities, greater damage inflicted by natural disasters, and inefficient safety and maintenance works.Therefore, to ensure infrastructure safety, which is essential for building a content society, changes must be promptly predicted in the future and new countermeasures must be developed in response [3][4][5].To fulfill this objective, an investigation was conducted with regard to the types and methods of current bridge maintenance works.Based on the abovementioned investigation, an analysis was performed to identify the root cause of the lack of inspection of many infrastructure facilities despite the existence of relevant laws and regulations.Based on the analysis results, the UAV-image-based inspection of infrastructure facilities was discussed as a new approach that is different from the existing inspection methods which are dependent on subjective and naked-eye inspections.The possibility of replacing the conventional inspection methods with the new image-based maintenance method, which is based on key spatial information technologies such as UAV imaging and machine vision, was explored.

Consideration of Bridge Maintenance Works
As of 2016, there were 30,983 existing bridges in Korea.Among these, 10,358 bridges (considered on Types 1 and 2) are known to be registered in the Facility Management System of the Korea Infrastructure Safety and Technology Corporation.These bridges are subjected to inspections under two categories-safety inspection and precise safety diagnosis-in accordance with the Special Act on the Safety Control of Installations and relevant guidelines (Detailed name: Guidelines on Safety Inspection and Precise Safety Diagnosis, Ministry of Land, Infrastructure and Transport) on infrastructure safety management.The primary purpose of the safety inspection is to check the current status of infrastructure facilities using simple tools and carry out a condition assessment.The safety inspection comprises three sub-inspections: regular inspection, close inspection, and emergency inspection.Close inspection consists of early and regular inspections, whereas emergency inspection consists of damage and special inspections.A regular inspection involves an investigation of the general external condition of the accessible areas of the bridge.Naked eyes and assistive tools such as telescopes and mirrors are used, and the inspection is conducted at least once every 6 months.In principle, an early inspection shall be conducted by a qualified senior engineer who can perform close and emergency inspections, and the initial close inspection of new or modified infrastructure facilities shall be conducted within 6 months after the completion of construction or from the date of granting approval for use (including temporary use).A regular close inspection involves an accurate assessment of the current status of infrastructure facilities, an assessment of changes from the initial or previously recorded status, and a confirmation of whether infrastructure facilities satisfy the requirements for use.In addition, a closer naked-eye inspection, simple measuring instruments, and bridge inspection vehicles are used if needed.A regular close inspection is conducted at least once every 2 years.Figure 1 shows a photograph of an inspection using an inspection vehicle when conducting a close inspection.

Figure 1. Close inspection using bridge inspection vehicle
An emergency inspection is conducted when it is considered needed by a managing body or when a managing body is requested to do so by the heads of relevant administrations.The inspection is composed of damage and special inspections.A damage inspection is a nonscheduled inspection designed to immediately check the damage to the infrastructure caused by disasters or accidents.A special inspection is conducted at a level similar to a close inspection and is conducted in cases where a defect such as foundation subsidence or scouring is suspected or where the continuous usability of infrastructure facilities that are restricted from use needs to be assessed.A precise safety diagnosis is conducted when it is considered needed to prevent disasters, ensure safety of infrastructure facilities, etc., by a managing body based on the results of a safety inspection.The purpose of a precise safety diagnosis is to check, analyze, and assess the structural safety, causes of defects, etc.; to identify physical and functional defects in infrastructure facilities; and to adequately and promptly address the problem.In addition, the diagnosis aims to suggest methods for repair and reinforcement.The features of each bridge inspection type are briefly described in Figure 2  As explained above, under the Special Act on the Safety Control of Installations, all bridge infrastructure facilities that correspond to Types 1 and 2 are subjected to inspection depending on the period and methods.In this section, implications from the analysis of the current status of general maintenance works, including inspection, are discussed.This section also presents the study's background and purpose.Although inspection is mandated by relevant rules and regulations, there are many practical difficulties in complying with these rules and regulations.In this context, three main causes were analyzed through a preliminary investigation, and they are described in detail in the following subsections.

Rapid Increase in Number and Aging of Infrastructure Facilities
The Korea Expressway Corporation (KEC) expects the total number of infrastructure facilities in Korea to increase by 40% over the next 10 years (Figure 3).The number of bridge infrastructure facilities under the supervision of the KEC was 9,180 in 2014, and it is expected to reach 12,986 in 2025, which is an increase of 3,142.Furthermore, with an increasing number of infrastructure facilities becoming aged, it is expected that there will be 9,576 bridges that are 30 or more years old in 10 years, accounting for approximately 30.9% of the total bridge infrastructure facilities.Further, 21,737 bridges will be under this category in 20 years, accounting for approximately 70.2% of the total bridge infrastructure facilities.Compared to other countries, infrastructure facilities are aging at an extremely faster rate in Korea, and the cause of this phenomenon is thought to be the compressed economic development of the past in Korea [1].

Shortage of Manpower
According to an investigation, the average age of the general workforce in the maintenance sector was 44 years as of December 2016.Because the maintenance of infrastructure facilities is viewed as a "3D (dirty, dangerous, and difficult)" job, it is not easy to recruit new workers.Because of such circumstances, the industry may inevitably face a shortage of manpower once the current workforce reaches the age of retirement.The gap in and shortage of the maintenance workforce by generation in the context of the increasing number of total infrastructure facilities imply that the industry, which is heavily dependent on manpower, is faced with a great challenge.

Increase in Maintenance Costs
The third cause is attributed to the close interrelation between the industry and national policies.A decrease in the SOC project budgets obviously leads to a decrease in the infrastructure maintenance budget.In contrast, maintenance costs are expected to increase by 14% on average every year (Table 1).In particular, costs for inspection and diagnosis are expected to increase threefold in the next 5 years from KRW 111.9 billion in 2014 to KRW 241.6 billion in 2020.This is the budget estimated according to the increasing number of infrastructure facilities.Despite the current problems because of which an increase in maintenance costs is inevitable, the budgets for national SOC projects are decreasing-KRW 22.5 trillion in 2015 to KRW 21.9 trillion in 2016, which is a decrease of approximately KRW 580 billion [2].

Implications
There are considerable data and evidence to confirm that the current methods of bridge infrastructure inspection are faced with numerous practical challenges.If practical measures are not devised, it would be impossible to conduct adequate number of inspections in the future because of decreasing budgets and the increasing number of infrastructure facilities.It is therefore considered necessary to transition from the current manpower-based inspection to information and communication technology-based inspections.Recently, geo-spatial information technologies such as UAV imaging and machine vision have been introduced [6][7][8][9][10].This study therefore aims to investigate whether the active utilization of such technologies can be an ultimate solution to the problems described above.

Determination of Maintenance Efficiency through UAV Image Analysis and 3D Viewer
In this section, the possibility of replacing the conventional maintenance methods with new methods using UAV images and maintenance efficiency is discussed.To this end, the selection of study bridges, UAV operation, UAV image processing, and the characterization of a 3D viewer are explained.Figure 4 shows a flowchart of the processes carried out in this study.In addition, details of the test bed used in this study, UAV operation, and image matching are described in detail in this section.Table 2 lists the specifications of the UAV used in the study.This UAV is a model designed specifically for the maintenance work of bridges and other infrastructure facilities with an on-top gimbal and is manufactured by Leica.The UAV was operated for two days between November 20 and December 8, 2016, beginning at 8 a.m. each morning.The operation hours of the UAV were within the hours of minimal traffic around the bridge and minimal temperature fluctuation due to sunrise, which could result in the creation of air currents.Around 500 short images were obtained during the 2-h test operation.

Establishing Flight Plan and Measuring Ground Control Points (GCPs)
Prior to the operation, the flight trajectory of the UAV was set considering many variables such as the optimal altitude for image recording, camera angle of view, route, duplication rate, and direction of sunlight.In the study, ground control station-based Aibotix software called AiProFlight was used to set the flight route.Figure 5 shows the UAV flight plan setup.A total of 11 GCPs were installed for the study purpose.Depending on the altitude of the bridge, three GCPs were installed on the bridge deck and the remaining eight were installed directly on the ground below the bridge.The GCPs were measured by VRS (Virtual Reference System) observation according to aerial photogrammetric regulations.Figure 6 shows the installation points of the GCPs and the scene of the installation.

UAV Image Processing
For processing the obtained UAV images, the PIX4D Mapper software was used.For the coordinate system, the international system of WGS94 coordinates was used, and the variable values of the camera used for recording were considered for the processing.Figure 7 shows a flowchart for the general UAV image processing.

Analysis of Machine-Vision-Based Inspection Items
Next, the study aimed to explore technical measures to improve the efficiency of bridge maintenance and inspection using the obtained UAV images and machine vision technology.To this end, a preliminary investigation was conducted on the outcomes derived from an inspector's subjective assessment or items recorded in the inspection log.According to the investigation results, the primary items in the bridge inspection included crack detection, white coated analysis, and water leak check.It was confirmed that these items were inspected based on subjective assessment.Image analysis was performed in these three major areas of inspection using machine vision technologythe most used technology in the field of image analysis in recent times.Based on the assumptions that each item of the inspection could be converted into data and information through analysis and that UAV images could be obtained in a regular and consistent manner, it was assumed that automated detection and prediction would be possible through machine learning in the future.This study formed the basic phase of the proposed study plan and aimed to perform the image analysis for each item of inspection subjected to UAV imaging.

Machine-Vision-Based UAV Image Analysis
As explained above, a specific software was developed to support the bridge inspection through machine vision technology for short images obtained by the UAV (Figure 9).As of now, this C ++based software imports images of the bridge members, creates an algorithm for inspection as a function, and finally displays the corresponding inspection works on each input image, which can be printed as the output.This process was mainly divided into two parts.The first part was the extraction of information channels required prior to image analysis, and the second part was the application of the algorithm required to extract candidates for inspection.The first stage of the inspection item (crack, white coated, and water leak) analysis was the conversion of the color format, which was carried out for all the three major inspection items.The Lab color space was formed by nonlinearly converting the CIE XYZ color spaces in accordance with the antagonism theory of human color vision [11].The Lab color space represents the color space of CIE 1976 Lab and is used by converting the CIE XYZ color space.This is a transformation based on the cubic function, and the Lab color space has a nonlinear relationship with the actual wavelengths of light.In other words, the gap between two different colors is designed to be proportionate to the actual chrominance perceived by human sight.The formal and invert conversion of the Lab color format is as follows: Here, () is as follows: ,  ,  are the normalized values of CIE XYZ for the standard white color.Inverse transformation is performed as follows: Following the first stage of the analysis performed for the difference in brightness, Gaussian adaptive filtering was applied to extract the priority candidates for each inspection item in relation to these images and the binarization process was carried out subsequently.Binarization is the most solid segmentation method.It is frequently used for dividing an image to extract a desired area or a specific object in the image, making it an adequate method for the application.In the binarization, the original image was converted into a grayscale image, and the background and objects were separated using threshold values.In the application, threshold values were applied to the fixed area of the entire image.The grayscale image was then inputted to scan the entire image.If the pixel value were greater than the threshold value, the pixel value at the same position in the resulting image was designated in white color; else, the value was designated in black color.In the image of the bridge, cracks were generally indicated in a darker color than the other areas of the bridge.To use this feature, the color format conversion of stage 101 was first carried out.The recorded image was in RGB(A) color format, which was then changed to the Lab color format.The Lab color format consisted of a three-channel color format: channel L for brightness and channels a and b for chrominance.To use channel L, data were extracted from channel L (Figure 11) in stage 102.The extracted channel L data were then used in the subsequent stages, i.e., 103, 105, and 107.In stage 103, the Gaussian adaptive threshold method was applied on the channel L data.The Gaussian adaptive threshold method calculated the area weighted average by using the Gaussian load within the local area as the threshold value.Cracks had darker brightness values than the other areas and were designated as candidate areas if the brightness value was below the threshold value.With regard to the set value for the Gaussian adaptive threshold, the size of the local area was set as 45 × 45 for experimental reasons.After completing stage 103, binary images were obtained in stage 104 and set as threshold candidate 1.In stage 105, the contour of the area was extracted using the channel L data.When extracting the contour, the area was detected based on the similarity of the brightness value, and the extraction was carried out by setting the data on the contour of the identical area as boundaries.In stage 106, for the extracted area boundaries, when the width of the area was below the threshold value, the detected area was considered to be false or inaccurate (because of noise in the image, etc.) and was then removed.The resulting image is shown in Figure 12.The data for the area extracted through the two stages were used as crack candidate 2. In stage 107, histogram equalization was carried out using the contrast-limited adaptive histogram equalization (CLAHE) method to minimize the effects of external brightness.The CLAHE method was used because it could minimize the impact of the difference in regional illumination when carrying out histogram equalization.Using the resulting image obtained from the histogram equalization in stage 108, a local average filter was applied to each area to compare the average brightness and the current pixel brightness.A darker image was detected by comparing the entire pixels with the local average, and it was designated as crack candidate 3 (Figure 5).After extracting crack candidates 1, 2, and 3, all the detected intersections in each candidate were measured, and in stage 109, the areas with boundaries equal to or greater than a certain width and the areas with darker brightness values than the surrounding areas were assumed to be cracks.of inspectors and explore such a possibility through comparative analysis.Therefore, only the cracks with a width of 0.5 mm or greater were taken into consideration, and the comparison was carried out only in the quarter span of the lower part of the bridge.

Figure 16. Flowchart for water leak detection
To detect the leak in the image (Figure 16), the data for the brightness channel were used as the primary data, as in the case of the crack detection.The leak area also had a darker brightness value than the other areas.Therefore, to detect the leak area, the channel L data (Figure 17), which indicated the brightness in the Lab color format, were extracted through stages 701 and 702 (Figure 7).The data extracted from channel L were then used in the subsequent stages, i.e., 703 and 705.In stages 703 and 704, the binary images were extracted using the same method used in the crack detection.Darker areas were detected in the binary images to obtain the resulting image, as shown in Figure 18, and were used as leak candidate 1.In stage 705, the data representing the edges in the image were extracted using the Canny edge detection algorithm.For the edges in the bridge image, the threshold values were set as 0 and 150 for experimental purposes, and the resulting image was obtained, as shown in Figure 19.Because the data representing the edges included data for all the edges in the image, a top-hat morphological filter was applied in stage 706 to detect the edges of the comparatively darker area.When using a morphological filter, a filter kernel with a size of 5 × 5 was used.Figure 20 shows the edge image to which the filter was applied and the image was then used as leak candidate 2.
The intersection of the areas detected in leak candidates 1 and 2 was measured, and the leak areas with darker brightness values than the surrounding areas and the edges in the darker areas were detected in stage 707.The resulting image is shown in Figure 21.The white coated area in the bridge image had lighter brightness values than the surrounding areas.To use this feature, as in the cases of crack and leak detections, the channel L data (Figure 23) obtained in stage 1302 were extracted for use in the Lab color format in stage 1301 (Figure 13).As in the other cases, in stage 1303, the channel L data obtained from stage 1302 were used to apply the Gaussian adaptive threshold method.As the efflorescence area had a lighter brightness value than the surrounding areas, the data was extracted only when the value exceeds the threshold value, and a binary image was obtained in stage 1304, as shown in Figure 24.This image was designated as efflorescence candidate 1.When efflorescence occurred, more edges were found in the efflorescence area than in the other areas.Therefore, the edges were detected through the Canny edge detection in stage 1305.The detected edges were designated as efflorescence candidate 2, as shown in Figure 25.Then, the intersection of the designated efflorescence candidates 1 and 2 was measured, and the efflorescence areas with comparatively lighter brightness values than the surrounding areas as well as many edges within the efflorescence areas were detected in stage 1306.The detected efflorescence areas are shown in Figure 26.The computer vision technology described above could be effectively applied to bridge inspection [12].This result was confirmed by field technicians performing on-site inspection, but the number of technicians was not sufficiently large to quantify the result.Accordingly, further study is required to validate the technology.
The study results show that the proposed UAV image acquisition and analysis method for inaccessible bridge members overcomes the limitations of the existing bridge inspection methods and thus may successfully replace them.To confirm this possibility, a cost-benefit analysis was performed, whose details are listed in Table 3.For the cost-benefit analysis, particular members were selected because of the characteristics of the target bridge.The cost-benefit analysis for a single bridge cannot be extended to every bridge.Especially, because Wonjudaegyo Bridge, which is the target bridge in this study, is an elevated bridge, our analysis might produce many unavoidable differences.When an elevated bridge is inspected, inaccessible members like the pylon, pier, and bridge bearing always require an inspection car and a crane.The cost for existing methods that use such equipment is two or three times that for the drone-based inspection method, and the existing methods also require more working hours.Moreover, as the UAV system is easier to transport and install, two or more bridges can be inspected in a single day, thereby boosting the advantages of the drone-based method.

Conclusions
The aim of this study was to enhance the efficiency of bridge maintenance and inspection.Korea is facing the problem of simultaneous deterioration of many infrastructure facilities because of its compressed economic growth.The shortage of manpower for maintenance and inspection is also aggravating the situation.In addition, the corresponding budget is continuously decreasing along with its reducing importance in policy agendas.Considering this background, the current maintenance and inspection methods have clear limitations, which undoubtedly threaten public safety.This study considered the use of a UAV, which has advantages in terms of economy, convenience, and data acquisition in inaccessible areas, as an alternative to the existing methods.We selected Wonjudaegyo Bridge as the test bed and used a rotary-wing UAV named Aibotix, which is manufactured by Leica.The images obtained by the UAV were displayed in a 3D viewer after an image matching process.Thus, the maintenance work could be carried out from any location.Machine vision technology, which is currently one of the most remarkable image analysis technologies, was applied to inspect the matched images.Crack detection, efflorescence analysis, and leak check were carried out, and the results were analyzed based on the bridge inspection list.We also developed software that included functions for each element technology to monitor the UAVbased maintenance and inspection of bridges.Finally, a cost-benefit analysis was performed to compare the UAV-based inspection method with existing methods.The proposed UAV-based method was found to be better in terms of economy, convenience, and objectivity.

Figure 3 .
Figure 3. Increase in infrastructure facilities

Figure 6 .
Figure 6.GCP surveying and installation points

Figure 7 .
Figure 7. Process for image matching

Figure 8 .
Figure 8. Results of image matching

Figure 9 .
Figure 9. Development of bridge inspection software based on UAV images

Figure 15 .
Figure 15.Resulting image obtained from crack detection

Figure 18 .Figure 21 .
Figure 18.Resulting image obtained from Gaussian adaptive threshold method

Table 1 .
Increase in maintenance costs

Table 3 .
Details of cost-benefit analysis