Towards a Long-Term UAV Monitoring Framework/Strategy for Post-mining Effects: Prosper-Haniel Case

1. Introduction

Mining activities have a strong impact on the surrounding environment even many years after the cessation of mining [1,2]. Kratzsch [3] points out the impact of mining operations on soil, water and air. Increasingly, land subsidence can be observed as a result of previous mining activities, creating sinkholes or subsidence basins, among other things. Research work on this issue has been carried out since around the second half of the 19^th century [4]. This has resulted in a number of theories aimed at providing a method of predicting the impact of underground mining on the surface: Keinhorst [5], Bals [6], Lehmann et al. [7], Awiershin [8], Sann [9], Knothe [10], Litwiszyn [11], Berry [12], Sashurin [13], Bush [14,15,16], Suchowerska Iwaniec et al. [17] and Xia et al. [18,19]. At present, radar interferometry, among other methods, is used to monitor the Earth’s movements. Numerous research papers have documented land subsidence processes in mining areas using radar interferometry in countries such as China [20,21,22,23,24], the Czech Republic [25,26], Ethiopia [34], Germany [27,28,29,30,33,35], Iran [31], Poland [36,37,38,39,40,41,42,43,44], Republic of South Africa [45] and Russia [32]. It is noteworthy that the European Ground Motion Service [46] provides all users with access to analyses based on radar images from the Sentinel-1 space mission.

As Wójcik [47] points out, the mining activity only directly affects the lithosphere. However, it results in interconnected impacts and reveals changes in other areas of the Earth’s crust. Observation of the water environment is a key aspect of geo-monitoring deterioration processes, as water resources, whether groundwater, surface water, flowing water or standing water, have an impact on the surrounding environment: animal habitats, plants, soils, as well as the landscape and climate [1]. Through land subsidence, floodplains can be created [35]. An important issue described in Melchers et al. [48] and also in the article by Westermann et al. [49] is the rise in groundwater levels after the cessation of mining operations, which has a significant impact on water management in the area. It is possible to observe the effects of mining operations on the atmosphere, as spontaneous combustion occurs within the deposited mine tailings on the hillsides, causing pollution and releasing gases into the atmosphere. Spreckels et al. [1] points to the necessity of continuous environmental monitoring in mining areas during and after the cessation of mining activities. Kretschmann [2] refers to the monitoring of mining activities as a perpetual task due to the long-term nature of the issue. The literature describes the effects of land subsidence in mining areas from: Belgium [52,53], Columbia [51], China [54,55], Czech Republic [56], United Kingdom [50,51], Netherlands [57], Germany [33,48,50,51,58,59,60], Poland [61,62,63] and Republic of South Africa [50].

A prime example is subsidence, which can cause extensive damage to the environment and human activity: damage to streets, telecommunication, gas, water, sewage, electricity networks and buildings (Figure 1). Subsidence lakes are one of the consequences of mining activities, resulting from long-term subsidence of the ground surface. It is worth noting, however, that the groundwater level does not change over a given period of time, which is why, first, waterlogging occurs and finally water reservoirs are formed (Figure 2) [64]. The complexity of the post-mining processes results in the use of innovative, state-of-the-art research methods and instruments. Pawlik et al. [65] and Rudolph et al. [66] present various methods for geomonitoring of post-mining processes in use: 3D geological structure modelling, multispectral satellite imagery, multispectral drone imagery, earth movement data, mobile GIS, weather data and expert knowledge. In this paper, research will be carried out based on the use of UAV (Unmanned Aerial Vehicle). So far, the data acquired with the UAV have made it possible to study the state of soils [67,68,69,70] and to pore over the moisture content of the state of soils with the results of multispectral analyses using vegetation indices [71]. As shown by Pawlik et al. [72], the multispectral camera of the DJI Phantom 4 Multispectral drone has similar characteristics to the sensor placed on the space mission satellites, Sentinel-2. Therefore, drones with a multispectral camera can be used to calculate vegetation indices. A review and description of these cues, which can be used to observe post-mining processes, was performed by Pawlik et al. [73]. A thermal imaging camera can be used to search for fire [74,75,76,77], to monitor post-mining dumps [78] and for the inspection of oil, gas and mining industry [79,80,81,82,83]. A review of the use of UAVs for monitoring mining areas was performed by Ren et al. [84].

2. Materials and Methods

2.1. Research Area

The research area is located in the Ruhr region of western Germany (Figure 3), on the site of the closed Prosper-Haniel coal mine. Coordinates of the extent: upper left: 6.8543345 E 51.5880290N and down right: 6.8613088 E 51.5847676 N. Mining operations under the Kirchheller Heide area began in 1976 [85]. However, the main mining activity was in the 1990s and lasted until 2018 [86]. Operations at the Prosper-Haniel mine ceased in December 2018 [85]. There are forms of nature conservation in the study area, for example protected natural areas and also bird protection areas. It is a forested area, with the Schwarzbach and Elsbach streams running through the central part of the area, as well as the established subsidence lakes Weihnachtssee and Pfingstssee. Pawlik et al. [87] observed on the basis of a spatio-temporal analysis of satellite images based on vegetation indices that vegetation changes occurred between 2002 and 2012.

2.2. Materials

This subsection will present the research instruments used.

2.2.1. DJI Phantom 4 Multispectral

The drone produced by DJI enables a drone flight lasting up to 27 minutes on one battery [88]. It has six camera sensors: One RGB camera and five lenses dedicated to different spectral bands (Table 1). Due to its weight of 1.5 kg, a remote pilot license of competency is required to fly the drone in the European Union [89,90,91]. The built-in solar sensor allows the detection of solar radiation, “maximizing the accuracy and consistency of data collection at different times of the day” [88]. With the implemented RTK (Real Time Kinematic) module, centimeter-accuracy images can be obtained, because this module acquires location corrections from satellite positioning in real time.

Multispectral analysis of images obtained by drone flights is based on spectral band analysis. Before performing a multispectral analysis, the characteristics of the multispectral camera sensor in question should be known. Unmanned Aerial Vehicles (UAVs), commonly known as drones or copters, have evolved into unmanned aerial vehicles equipped with various sensors (e.g., GNSS receivers and cameras). By using UAVs, it is possible to flexibly and cost-effectively capture image data of small to medium size (up to 10 ha) [92].

2.2.2. DJI Mavic 2 Enterprise Advance Thermal

DJI Mavic 2 Enterprise Advance Thermal allows you to fly the drone about 28 minutes [93]. Its weight is 909g [Ibidem], which means that drone licenses are required to fly the drone [89,90,91]. In this version, the drone has two cameras: RGB and thermal. The RGB camera is characterized by a ½” CMOS sensor with a resolution of 48MP. The accuracy of the thermal camera is ±2℃ [93]. Also, as in the case of the DJI P4 MS drone, an RTK module is implemented.

2.3. Methodology

This subchapter will present and describe the steps that are performed when drone flight (Figure 4).

2.3.1. Pre-Planning of the Flights

Since UAV surveying is nowadays an accepted and much used method, the pre-planning will not be overly discussed in this article. After reviewing the platforms, sensors, resolutions and accuracies used for the desired monitoring results, these must be checked against the conditions on site. This includes, for example, local topography, aviation obstructions, and possible restrictions on flight operations, such as airfields or residential development. The applicable regulations of the respective country apply here; Germany follows the EU Rules for Unmanned Aircraft Systems [94] in this regard. The legal basis at national level (LuftVG1, LuftVO2, LuftVZO3) has been adapted by the "Act on the Adaptation of National Regulations to the Commission Implementing Regulation (EU) 2019/947 of May 24, 2019 on the Rules and Procedures for the Operation of Unmanned Aerial Vehicles" [95].

2.3.2. Preparation of Drone Flights

Directly before the flight, it is still necessary to check whether any short-term NOTAMs (NOtice To AirMan) are available [96]. These are notices to airspace users informing them of special features in the region, such as a music festival or the holding of military exercises, which in turn may result in an airspace closure, also for UAVs. In addition to the legal framework conditions, the flight weather must also be checked before each flight. For this purpose, there are, for example, various apps and online offers (e.g. UAV Forecast) [97] in which you can directly insert the parameters of the UAV manufacturer (e.g. maximum wind strength). The actual conditions on site must be checked again directly before the flight (portable anemometer, thermometer, also barometer). To ensure correct georeferencing of UAV images, there are two methods [98]: Direct georeferencing (via a GNSS module linked to the camera) and indirect georeferencing via pre-measured and marked ground control points (GCP). The use of a Positioning RTK [99] module can improve the accuracy of the GNSS chips typically installed in all UAVs from several meters to a few centimeters. Most professionally used drones now have RTK either on board as standard (e.g. the DJI Phantom 4 Multispectral used here) or available as an optional module (e.g. the DJI Mavic 2 Enterprise Advanced used here). Although both methods lead to good georeferencing, in particular the combination of RTK and GCP use produces the best results [100]. Due to the remote location of the study area and the resulting lack of mobile Internet access (Figure 6), it was not possible to use the RTK network "Satellite Positioning Service SAPOS", which is freely available in North-Rhine Westphalia [101]. The wide distribution of GCP was rejected due to the large area, poor visibility from the air, as well as lengthy surveying traverses, attributed to the impossibility of RTK-GNSS measurements. As a solution, a dedicated RTK base station (Figure 5) was established on a previously highly accurate surveyed point at the edge of the survey area. A Trimble R12 GNSS rover was reconfigured accordingly for this purpose and was able to send its own correction data via a WiFi connection as an NTRIP (Networked Transport of RTCM via Internet Protocol) caster to the controller of the UAVs, which in turn passed it on directly to the drones. A few previously measured GCP then served to verify the georeferencing of the later results. The accuracy in the centimeter range could be confirmed with this solution and costly corrections in post-processing could be avoided.

2.3.3. Drone Flights

The drone flight begins with selecting the planned area of the flight, then the flight parameters are important (Table 2).

Figure 7 shows the drone flight planning in the DJI GSP application. All listed flight parameters enable the assumed accuracy of the final results to be achieved. During the flight, the operator is obliged to maintain visual contact with the drone and monitor the weather (e.g. wind speed). Of course, the drone operator should monitor the situation on an ongoing basis and, in the event of an emergency, land it immediately.

2.3.4. Post-Processing

For the processing of the raw data from the UAVs, the software Agisoft Metashape was used. However, the procedure of the photogrammetric analysis (Figure 8) used here is basically the same for most comparable commercial (e.g. ArcGIS Drone2Map) or free (e.g. OpenDroneMap) software products. Usually, only the names, parameters and implementations of the different work steps change.

The advantage of Metashape over comparable products is that the work steps can be carried out individually one after the other. This makes it easy to optimize the parameters in the intermediate steps and gives you much better control over the overall result. Thus, for the creation of a 3D model, it makes sense to generate a high-resolution 3D mesh, while for an orthophoto, a lower-resolution, 2.5D DEM (Digital Elevation Modell) is also sufficient.

Multispectral

For the processing of the multispectral data, these were assessed according to the workflow with regard to their basic quality, sorted and loaded into the program. As an additional step, the data was then radiometrically calibrated to obtain a result that is as independent as possible of external conditions (position of the sun, cloud cover). For this purpose, the images were reflectance calibrated using the sun sensor permanently installed in the DJI Phantom 4 Multispectral used and an additional calibration panel with which images were taken before and after [106].The reflectance values of the panel were determined in the laboratory for all wavelengths from 250 to 950 nm, so that a radiometric correction value for the spectral ranges used (red, green, blue, RedEdge, NIR) could be calculated and applied under the given lighting conditions on the basis of these images [107]:

$F_i=\frac{{\mathsf{\rho }}_i}{\mathrm{a}\mathrm{v}\mathrm{g}\left(\mathrm{L}\mathrm{i}\right)}$ (Formula 1)

where Fi is the reflection calibration factor for band I, ρi is the average reflectance of the calibration panel for the i-th band (from the provided calibration data of the panel) and avg(Li) is the average value of radiance for the pixels within the panel for band i.

With the calibrated data, the subsequent processes up to the creation of the orthophoto could be carried out. Since no 3D model was needed, the images were only aligned, a rough, georeferenced mesh was created, and from this, a DEM and finally a multispectral orthophoto were derived. Using these products as a foundation, additional derivatives can be generated in ArcGIS Pro (e.g., DTM – Digital Terrain Model, vegetation indices), and a selection of these is presented in Chapter 3.

Thermal Infrared and RGB

The used DJI Mavic 2 Enterprise Advanced has two cameras, capturing a high-resolution RGB image and a slightly lower-resolution thermal infrared image at each trigger point. Both receive the same image coordinate via the modular RTK module. First, the RGB and thermal images must be sorted, as they are processed separately. The RGB images can be used without further pre-processing to create high resolution RGB orthophotos, DEMs and DTMs. The thermal images are in gray values by default, where each gray value within the image represents an absolute temperature value. These can be analyzed with special software (DJI Thermal Analysis Tool) based on the environmental parameters (ambient temperature, humidity, shooting distance). Since the scaling of the gray values changes slightly from dark = cold to light = warm from shot to shot, the absolute temperature values are inevitably lost during orthophoto generation and the associated color balancing. However, the information about the relative temperature differences is retained and allows conclusions to be drawn about various monitoring aspects, for example for the identification of water surfaces and coastline. After the processing steps described above, a gray-scale image with relative temperatures is obtained as an orthophoto, which can be colored differently depending on the purpose (cf. Figure 8).

3. Results

After the flight has been carried out, all data and measurement points are implemented in the software to achieve the originally intended goals. The results of drone flight development are:

-

individual images,

-

orthophoto map,

-

calculated vegetation indices on the basis of multispectral drone flights,

-

thermal orthophoto map,

-

digital terrain model (DTM),

-

digital surface model (DSM).

The first result obtained was an orthophoto map (Figure 9), which makes it possible to visualize the area with a better spatial resolution than satellite images, as well as a first visual assessment and interpretation.

The results presented can be used for analysis and data interpretation by implementing other data into the computational process. In this chapter, the authors aim to present the key possibilities of UAV use for the geo-monitoring of post-mining processes.

3.1. Updating and Revising Land Classifications

For the state of North Rhine-Westphalia (NRW), a land use classification is available on the Open-NRW geoportal website via the Web Map Service (WMS). It is worth noting that the classification is based on the acquired satellite images of the Sentinel-2 mission and the study is valid 01.04.2022. In the study area, there are different classes:

-

Vegetation,

-

Grass,

-

Shrub,

-

Hardwood,

-

Street,

-

Water.

Thanks to the use of drone flights, better spatial resolution and interpretability of the results obtained from the drone flights can be achieved. Figure 10 shows a summary of the received orthophotomap dated 28.03.2023 and the WMS Landbedeckung service from the OpenNRW portal. The combination enables a detailed analysis of the vegetation condition and verification of its conformity with the class presented by the WMS service. For the classification, a support vector machine (SVM) has been applied over the multispectral dataset. A supervised machine learning algorithm is applied using a training dataset created from a small area of pixels used as training data. The SVM’s duty is to detect a separating hyperplane (defined as the class boundary) that segregates the features space into two classes with the largest margin for each class [108]. An optimization problem is solved by structural risk minimization for identifying the aforementioned hyperplane. Only the samples that are closest to the class boundaries are required to train the classifier, the so-called support vectors. Essentially, the localization of the hyperplane in the feature space is largely influenced by those training samples. Therefore, the classification results of the support vector machine can achieve high accuracy even when only a small number of training samples are available [108,109]. Figure 10 (left side) demonstrates the classification result using 500 number of samples to define each class, such a number is recommended when the inputs are non-segmented rasters, as for the multispectral dataset.

3.2. Identification of Water Surfaces and Coastline

In the geo-monitoring of post-mining processes, the observation of water bodies and also of potential sites that could become floodplains plays an important role. In this study, a vegetation index was used to identify water surfaces: the normalized difference vegetation index - NDVI, which was developed in 1973 by Rouse et al. [110]. This index is based on two spectral bands (Formula 2):

$NDVI=\frac{{\mathsf{\rho }}^{\mathrm{N}\mathrm{I}\mathrm{R}}-{\mathsf{\rho }}^{\mathrm{R}\mathrm{E}\mathrm{D}}}{{\mathsf{\rho }}^{\mathrm{N}\mathrm{I}\mathrm{R}}+{\mathsf{\rho }}^{\mathrm{R}\mathrm{E}\mathrm{D}}}$ (Formula 2)

This indicator makes it possible to observe the state of the vegetation. Kuechly et al. [111] in their paper presented a classification of the indicator, which is shown in Table 3.

Using the above classification, the water surface, according to the NDVI indicator, has values from -1 to 0.1. It is worth noting that soil, sand and snow also show similar spectral band characteristics to water, so they can also be found in the values. Figure 12 shows the results of the search presenting the water surfaces and shorelines of the Weihnachtssee and Pfingstssee reservoirs. Due to the forest remains in the subsidence lakes concerned, there are also tree stumps, which make the surface area of the entire lake not uniform (Figure 11).

3.3. Identification of Flowing Water on the Basis Thermal Orthophoto

An important aspect of post-mining process monitoring is the study of water quality [48,49,113]. Sedano-Cibrián et al. [114] and Koparan et al. [115,116] indicate that water quality can be assessed by, among other things, temperature. Iezzi and Todisco [117] note that water temperature can be influenced by natural and anthropological factors. Various measurement techniques can be used to measure water temperature, primarily based on probing and measuring points [Ibidem] as well as modern methods based on the use of thermal infrared techniques [118] and also unmanned aerial vehicles [114]. A thermal orthophotomap makes it possible to know the temperature of the land surface and water, as any object that has a temperature greater than absolute zero is a source of infrared radiation [119]. Figure 13 shows a thermal orthophotomap of the surface of the Weihnachtssee reservoir.

4. Discussion

The use of drones provides a detailed and reliable source of data for the study of post-mining processes, making it possible to analysis and interpret post-operation phenomena. UAV platforms allow the incorporation of various sensors (e.g. multispectral, hyperspectral, thermal, LiDAR - light detection and ranging), which significantly increases the possibilities of UAVs. The software used in the controllers allows the planning and execution of flight mission parameters, so that results can be obtained with the quality and accuracy assumed before the flight. This paper presents the results of using multispectral and thermal cameras. Drone aircraft with multispectral cameras enable the acquisition of certain spectral bands, associated with vegetation health. By interpreting the processed aerial data, land-use classes can be verified, which are generally available on the Open NRW website. The use of machine learning algorithms makes it possible to classify land use (Figure 10). Therefore, it is possible to identify vegetation biotypes. One of the research methods implemented in this paper was the use of vegetation indicators. These indicators are based on mathematical formulas that take into account at least two spectral bands in their calculation. This makes it possible to study the state of the vegetation in a given area, but also, as shown in Figure 11 and Figure 12, to identify water surfaces and coastlines. Monitoring aquatic environments using optical remote sensing has some limitations due to the need for direct visibility. Vegetation in the vicinity of water bodies and tree crowns can be an obstacle to correctly interpreting the shoreline of water bodies, and it is possible to take images from different angles to solve the problem [120]. However, scheduling a drone flight outside the growing season, as was done in this study, allows the previously mentioned problem to be solved. The resulting imagery can be used to calculate a numerical terrain model (Figure 14), enabling visualization of the terrain contours and derived products, such as:

-

Slope map, which illustrates the terrain’s slope, can be utilized to calculate rainfall-runoff, aiding in the development of flood control programs,

-

Exposure map, facilitating the examination of sunlight impact in specific areas,

-

Visibility map, applicable in constructing observation towers for forestry and tourism, serving as viewpoints.

The result of the use of an unmanned aerial vehicle equipped with a thermal camera is shown in Figure 13. It allows to present the temperature of water reservoirs in the form of a map. When flying a drone with a thermal camera, the position of the sun during the flight must be taken into consideration, as sunlight can cause shadows, which can distort the result. Figure 13 enables to see how the gradient in temperature characterized the SE-NW course. In the case of lakes Weihnachtsee and Pfingstsee, higher temperature were observed in the southern parts of the reservoirs, which may be caused by the rivers Schwarzbach and Elsbach flowing through them. The conducted research showed the monitoring with the use of UAVs makes it possible to provide reliable spatial information regarding a given research area. The analysis of the natural environment’s condition involves observing changes occurring within it and studying the range and temperature of water surfaces. UAV monitoring is one of many methods of geo-monitoring and in order to fully understand the post-mining processes, the obtained results should be verified using other sources and research methods.

5. Conclusions

The aim of this article was to present the possibility of using drone flights as a method of geo-monitoring post-mining processes, using the example of the closed Prosper-Haniel mine in the Ruhr region. Monitoring the environment using unmanned aerial vehicles makes it possible to obtain data with high spatial resolution. The use of various sensors placed on drones, in this case a multispectral camera (DJI Phantom 4 Multispectral) and a thermal camera (DJI Mavic 2 Enterprise Advance Thermal) enables a comprehensive study of the state of the natural environment:

-

State of vitality and changes of vegetation, using vegetation indicators based on data obtained from a multispectral camera

o

Verification and identification of vegetation types using machine learning algorithms - supervised classification

-

Identification of water surfaces and detection coastline of water reservoirs

-

Identification of the temperature of water surfaces and terrain using a thermal camera

-

Creating digital terrain models to visualize the research area

The research methods presented above can be used in long-term monitoring of post-mining processes, which will enable the performance of time-spatial analyzes documenting the course of changes taking place in the environment. It is worth noting that only the integration of all possible data on the research area allows for understanding the phenomena and processes taking place in it.