The Extraction of Roof Feature Lines of Traditional Chinese Village Buildings Based on UAV Dense Matching Point Cloud

1. Introduction

In January 2022, the Chinese government unveiled "the 14th Five-Year Plan" to advance the digital economy by promoting integrated development of new smart cities and digital villages. The most critical man-made feature in this effort is the 3D model of buildings, which plays a vital role in the construction of smart cities and digital villages, making accurate automated 3D reconstruction essential[1]. As a result of the rapid development of dense matching technology and aerial triangulation methods, full-automatic photogrammetric processing of nadir and oblique imagery acquired from low altitude unmanned aerial vehicles (UAVs) has become increasingly precise and intelligent[2]. It plays an instrumental role in various fields including but not limited to 3D reconstruction[3] and point cloud segmentation[4]. Point cloud generated through Multi-view Stereo (MVS) have also become a significant source of 3D geographic information acquisition[5]. In the process of modelling, extracting the roof lines of buildings is crucial for reconstructing a 3D model of a building. However, traditional technical means cannot meet modern development needs, and obtaining roof feature lines in a timely, low-cost, and automated manner has become an urgent problem that demands solving.

In the extraction of feature lines, two primary components are present, namely external outlines lines and internal roof feature lines. From the perspective of data sources, the extraction of edge outlines is typically reliant on high-resolution remote sensing images and airborne LiDAR point cloud data. Moreover, research methods primarily involve a priori knowledge rules and deep learning techniques, where the former focuses mainly on basic building features like spectra, textures, shapes, and spatial relationships to facilitate the extraction process.[6,7,8]. With the development of deep learning in recent years, methods such as Mask R-CNN[9], FCN[10] and PointNet[11] have gained popularity. Meanwhile, the extraction of internal feature lines mostly relies on airborne LiDAR point cloud data and follows steps such as roof point cloud segmentation, plane fitting, contour extraction, and regularization. Various algorithms are used, including Hough transform[12,13], region growing algorithm[14,15], clustering method[16,17], and RANSAC[18,19]. However, these algorithms often rely heavily on normal vectors and curvature thresholds, which may lead to false planes and threshold sensitivity[20]. In addition, the topological relationship between each plane needs to be judged during plane fitting and contour line extraction. Research data mainly comes from publicly available datasets such as ISPRS or airborne LiDAR point cloud data, with the aim of achieving LOD300 specification for 3D reconstruction of urban buildings.

As an important repository of historical and cultural memory for the Chinese people, traditional villages embody production and life wisdom, vernacular art crystallization, and national regional characteristics. They are thus crucial targets of digital preservation in cultural heritage protection efforts. Liu and Deng have proposed digitized protection as a new approach to safeguarding historical and cultural towns and villages[21], but there has been little research on 3D reconstruction of traditional villages. While some scholars have used UAV tilt photography to achieve real-time 3D digital conservation of villages[22], this approach encounters difficulties generating high-precision 3D models given the complex architectural structures and mountainous and hilly terrains common in most traditional villages. Unlike urban and rural buildings, traditional village dwellings in western Hunan are usually low-rise structures with sloping roofs constructed primarily of wood or brick. Few studies have attempted roof line extraction for these buildings. The traditional algorithm is limited and unsatisfactory. In light of this, the current study takes Jingping Village, a typical traditional village in western Hunan, as its focus. Utilizing MVS point cloud generated by UAV tilt photography technology and various algorithms including Visible-band Difference Vegetation Index(VDVI), Cloth Simulation Filter(CSF), and slope segmentation algorithms, building point cloud, roof point cloud, and slope surfaces are isolated and segmented in sequence, and then Canny algorithm and Hough transform are used to acquire roof feature lines. This method can be applied for large-scale village modeling and thereby provide a foundation for digital village construction.

2. Methods

Firstly, high-resolution orthophotos and raw colour point cloud data are generated by processing the raw data from tilt photography using the aerial triangulation method. Secondly, the building point cloud is extracted using the VDVI and CSF algorithms, and the building morphology is classified into types based on the orthophoto. A single example of each building type is selected for the roof feature line extraction experiment, and then the roof point cloud is extracted using the CSF algorithm. Finally, the roof point cloud is converted into regular grid data, and the slope direction of the different faces is calculated followed by reclassification to obtain the final roof slope direction. The final roof feature line is obtained using Canny edge detection and Hough line detection. The technical process is presented in Figure 1.

2.1. Building point cloud extraction

The building point cloud is extracted through a hierarchical approach with stepwise classificatio. After pre-processing the point cloud data, the vegetation and ground point cloud are successively removed using the VDVI and CSF algorithms to obtain the final building point cloud.

2.1.1. VDVI

The VDVI, a vegetation index that utilizes the red, green and blue bands, has exhibited remarkable efficacy in extracting vegetation information from visible spectrum remote sensing images[23]. The distribution of its value range is observed to possess distinct bimodal characteristics. The lowest valley value between the two peaks of the bimodal distribution is designated as the segmentation threshold. Any vegetation indices surpassing the defined threshold are classified as vegetation, while those falling below the threshold are considered non-vegetation. The VDVI calculation formula is as follows:

$$VDVI=\frac{2G-\left(R+B\right)}{2G+(R+B)}$$

(1)

Where R, G and B are the red, green and blue band values respectively.

2.1.2. CSF algorithm

The CSF algorithm, which was first proposed by Zhang et al. in 2016[24], has emerged as a highly effective technique for point cloud filtering across terrains with varying slopes. Unlike traditional methods that require complex parameter settings, the CSF algorithm offers superior advantages in terms of simplicity and ease of use. The underlying principle involves modeling a virtual cloth draped over irregular terrain where gravity causes it to shape and descend, as illustrated in Figure 2. By adjusting the stiffness of the virtual cloth, the final output can be either a DSM or DTM. The position of each mass on the "cloth" is determined by internal forces and gravity resulting in constrained vertical movement, as depicted in Figure 3. In accordance with Figure 3(b), the gravitational force first induces mass descent. Once these masses drop below the ground level, they are denoted as immovable and affixed to their respective positions in Figure 3(c). Subsequently, inter-mass forces align each point by utilizing neighbouring points to exert upward tension, leading to a vertical displacement of the masses, outlined in Figure 3(d). The positional determination of each mass is established definitively through repetitive up-down motions until a target point is achieved. The primary parameters for the CSF algorithm encompasses four elements: namely, grid resolution (GR) for calculating horizontal distance between particles, time iteration increments (DT) for particle motion along the gravity axis during simulation events, softness rigidity value (RI) relative to textile materials, and finally the threshold height (HT) separating ground points from non-ground after sorting operations take place[25].

2.2. Roof point cloud segmentation

2.2.1. Roof point cloud extraction

The CSF algorithm has garnered significant attention as a widely-used filtering algorithm, with numerous scholars attesting to its superior performance. In particular, recent studies have demonstrated the algorithm's efficacy in accurately segmenting point clouds on flat-roofed buildings, as seen in Wang et al.'s work on building extraction from UAV oblique photography point cloud using cloth simulation[26]. Capitalizing on this promising finding, this paper applies the CSF algorithm towards extracting sloped roofs on traditional buildings. Our experiment exclusively employs the CSF algorithm, with adjustments made to its parameters for several rounds of trials.

2.2.2. Slope segmentation

In order to mitigate the influence of traditional algorithms for roof segmentation, which overly rely on normal vectors and curvature thresholds[27], this study proposes a novel approach based on slope direction features. Specifically, the method segments the roof surface by leveraging the distinct slope direction features of building roofs. By projecting the original point cloud onto a regular grid and applying smoothing to eliminate anomalies, the pixel value of each grid represents the mean value of the corresponding point cloud points. To simplify the data structure, the extraction of feature lines is transferred from 3D point cloud processing to 2D image detection. This approach not only adapts well to established image segmentation algorithms but also enhances experimental efficiency.

In this study, the direction of a slope is defined as the angle between the horizontal plane and the projection of the normal to the tangent plane of a point on that surface. The resulting value is reported as the compass direction at that specific location. This measurement is conducted in a clockwise direction with values ranging from 0 (due north) to 360 (still due north), as visualized in Figure 4. In cases where an area is flat and lacks a downhill direction, it is given a value of -1 (representing a horizontal surface). To derive the slope direction for each central unit, a 3 x 3 moving window is employed, as shown in Figure 5. The calculation formula is shown below:

$$D_e=\alpha \tan 2(A,B)$$

(2)

Where D_e denotes the slope direction of the image element e, α denotes the conversion coefficient between radians and degrees, the specific calculation can be taken as 57.29578, A, B are the rate of change of the image element e in the x, y direction, the formula is as follows.

$$A=\left(c+2f+i\right)-\left(a+2d+g\right)$$

(3)

$$B=\left(g+2h+i\right)-\left(a+2b+c\right)$$

(4)

Where a, b, c, d, e, f, g, h and i are the individual image values.

Figure 4. Schematic diagram of slope direction.

Figure 4. Schematic diagram of slope direction.

Figure 5. Moving window for slope direction calculation.

Figure 5. Moving window for slope direction calculation.

Upon calculating the slope direction, a transformation is implemented to convert slope values into compass direction values based on established principles. Given that the roofs of traditional village edifices often exhibit double or four-sloped features, a detailed dissection of individual raster values in this study's experiments is deemed unnecessary. As such, it suffices to congregate raster slope direction values of identical slopes and recalibrate them to yield two or four distinct raster slope directions.

2.3. Feature line extraction

The extraction of feature lines mainly consists of external outlines and internal ridge lines, which serve as the boundaries of each sloping surface on the roof. The Canny algorithm was proposed by John F. Canny in 1986[28] and is one of the most classic and advanced algorithms for image edge detection[29]. Edge detection based on the Canny operator has five main steps, which are Gaussian smoothing denoising, pixel gradient calculation, non-maximum suppression, double thresholding to determine potential edge points and lagged edge point tracking, in that order. In recent years, some scholars have improved the Canny algorithm in many ways, which has improved the accuracy of Canny to a certain extent[30]. However, this paper only applies edge detection to two or four slopes of the roof, and the image content is relatively simple without much detail. Thus, adjusting the size of the Gaussian filter and the high and low thresholds is sufficient to yield better results in experiments.

3. Experimental Results and Analysis

3.1. Experimental data

In Hunan Province, China lies Jingping village - an ancient settlement that boasts a heritage steeped in history and culture. As a proud recipient of National Key Cultural Relics Protection Unit recognition as well as being designated a Chinese Traditional Village of the fourth batch, this village has been well-preserved for over a millennium featuring splendid clusters of Ming and Qing dynasty era structures. With its open spatial arrangement, Jingping village presents a remarkable blend of water, fields, and woodlands - illustrating the transcendent beauty of peaceful coexistence between mankind and nature while highlighting the quintessential features of western Hunan's traditional settlements.

In August 2019, the village MVS point cloud data was meticulously captured by the author through fieldwork employing a Zhonghaida iFly D1 quadcopter UAV integrated with an iCam Q5 mini five-lens tilt camera and a UAV-PPK receiver, equipped with an impressive 120 million effective camera pixel count (24 million x 5). Extensive explorations were carried out in the experimental area, control points were intentionally laid, and routes were strategically planned. Through this effort, orthophotos with a resolution of around 1.5cm were obtained, along with approximately 320 million point cloud data of the village. Following the requisite crop, the main building area measures approximately 0.21km², inclusive of around 180 million building point clouds. The flight parameters utilized for this study are presented in Table 1.

3.2. Building point cloud extraction and typology

Due to the extensive geographical coverage of Jingping village and the voluminous point cloud data, merely three exemplary settings were selected for edifice identification. Notably, when VDVI approaches zero, a set of CSF algorithm parameters was determined through experimental rigor: GR of 0.5, DT of 0.5, RI of 1, and HD of 0.5, as showcased in Figure 6. It is discernible that extirpating vegetation and ground points via VDVI and CSF improved building point cloud accuracy across various scenarios. However, minor disruptions such as fence and extraneous points remain.

In Jingping Village, the architectural configurations can be broadly categorized into four distinctive styles, namely I-shaped, L-shaped (featuring two internal courtyards), U-shaped (comprising three internal courtyards), and "回"-shaped (embodying four internal courtyards), as depicted in Figure 7.

3.3. Roof extraction and slope segmentation

During the process of removing vegetation point cloud, it was discovered that taller trees obscured local information of the roofs due to limitations in the UAV tilt photography technology. To address this issue, one more complete building from each roof type shown in Figure 7 was selected for the outline extraction experiment. The impact of the GR value on the extraction effect of the roofs in the CSF algorithm was examined and found to have a significant influence. As an example, I-shaped roofs were used and changing the GR value while keeping other values constant resulted in the best extraction effect being achieved when GR was set to 0.1, as demonstrated in Figure 8. This approach was then applied to obtain all types of roof point clouds, presented in Figure 9. However, upon closer examination, redundant point cloud was noted at the eaves of different roofs. The reason for this is that elevation information in this area cannot be captured during tilt photography, resulting in absence of clear point cloud data during the aerial triangulation solution. Additionally, the ancestral hall of Jingping village, depicted in Figure 9d, has higher surrounding walls than the roof, leading to partial voids in the area of the walls. Consequently, the application of the CSF algorithm to extract the roof of the building did not guarantee roof integrity.

To mitigate the influence of threshold setting on plane segmentation observed in previous studies, we propose a novel approach whereby point cloud data is projected onto a flat raster and smoothed before slope calculation. In this study, we exclusively experimented with the first three types of roofs mentioned above, as the extraction performance for "回"-shaped roofs was poor and hill walls prevented slope segmentation. Since the traditional village buildings' roof tiles are concave and convex, our slope direction calculation produced eight initial directions, which were subsequently reclassified into a 2-slope or 4-slope map. For example, regarding L-shaped roofs (Figure 10), we partitioned each raster pixel into one of eight slope directions, then divided them into four categories based on morphological classification and Figure 11. Nonetheless, the reclassified raster slope directions may contain a small number of misclassified stray points at edges and intersections, which require additional smoothing and denoising to avoid negatively affecting downstream detection. Our method's segmentation results were compared to those yielded by traditional RANSAC and region segmentation algorithms to verify their effectiveness.

The accuracy of slope segmentation was assessed through the utilization of equations (1)-(3), with the corresponding results illustrated in Figure 12. From the provided visualizations, it is apparent that the precision of the surface slope, which was extracted using a directional segmentation approach, exceeded 99.6%, indicating a remarkably satisfactory outcome.

$$R=\frac{TP}{TP+FN}\ast 100\%$$

(5)

$$P=\frac{TP}{TP+FP}\ast 100\%$$

(6)

$$F=\frac{2RP}{R+P}\ast 100\%$$

(7)

where TP is the number of correctly segmented raster pixels, FN is the number of missed raster pixels, FP is the number of incorrectly segmented raster pixels, R is the recall rate, P is the accuracy rate, and F is the measure.

Table 2. Correct slope segmentation for each type of roof.

Index	I-shaped	L-shaped	U-shaped
TP	30587	6665	26675
FN	0	0	0
FP	126	47	53
R	1	1	1
P	99.59%	99.30%	99.80%
F	99.79%	99.65%	99.90%

Table 2. Correct slope segmentation for each type of roof.

Index	I-shaped	L-shaped	U-shaped
TP	30587	6665	26675
FN	0	0	0
FP	126	47	53
R	1	1	1
P	99.59%	99.30%	99.80%
F	99.79%	99.65%	99.90%

Figure 12. Comparison of different types of roof surface segmentation:(a)RANSAC algorithms; (b)region segmentation algorithm; (c)slope segmentation algorithm; (d)Slope segmentation algorithm+noise removal.

Figure 12. Comparison of different types of roof surface segmentation:(a)RANSAC algorithms; (b)region segmentation algorithm; (c)slope segmentation algorithm; (d)Slope segmentation algorithm+noise removal.

It is noteworthy that, owing to the high precision of the data, the re-categorized slope direction raster might exhibit a marginal number of erroneously classified outlying points at the edges and intersections of two slope directions, namely, inaccurately segmented raster pixels. These artifacts may exert a certain impact on the ensuing detection procedures and thus must be subjected to subsequent smoothing and denoising processes. To ascertain the effectiveness of this approach, Figure 12 and Table 3 were produced by comparing its segmentation outcomes with the conventional RANSAC and region segmentation algorithms.

Based upon the results presented in Figure 12 and Table 3, our proposed method demonstrates significant improvements over both the RANSAC algorithm and the region segmentation algorithm. Firstly, all three algorithms exhibit comparable performance in terms of computational efficiency due to the small size of the input data. Secondly, regarding segmentation accuracy, neither the RANSAC algorithm nor the region segmentation algorithm succeeded in accurately segmenting the facets of simple I-shaped or slightly more complex U-shaped roofs. Instead, these algorithms exhibited varying degrees of under- and over-segmentation on either type of roof. In addition, based on point cloud data analysis, the region segmentation algorithm demonstrated a higher tendency toward under-segmentation when compared to the RANSAC algorithm. For instance, as shown in Figure 12, inner sides of both L-shaped and U-shaped roofs were segmented into single planes and the resulting segmented roofs showed greater noise due to restricted data accuracy. Furthermore, during the experiment, it was observed that the RANSAC algorithm produced inconsistent results under identical parameter settings, indicating significant deviations and instability.

Using the direction of slope, successful segmentation of all slopes was achieved in all three types of roof segmentation, resulting in improved outcomes. Though there were several instances of mis-segmentation, such occurrences did not adversely affect the overall effects compared to the area segmentation algorithm. Furthermore, after denoising, a complete representation of the various roof slopes was obtained. It should be noted that during experimentation, the first two algorithms were directly based on point cloud segmentation, resulting in neater planar edges. In contrast, slope segmentation relied on raster images after point cloud projection, leading to rougher edge contours due to lower raster resolution. However, this issue can be eliminated by regularization in subsequent feature line extraction.

3.4. Feature line extraction and regularisation

The slope direction map was subjected to sequential analysis involving Canny edge detection and Hough line detection. After further refinement and regularization, the resulting feature line extraction map is presented in Figure 13. Our approach offers commendable results in extracting feature lines of various roof configurations, particularly double-sloped and multi-sloped roofs commonly found in low-rise traditional village buildings.

4. Conclusion

Based on point cloud data collected from Jingping village, this paper presents a methodology that integrates the VDVI index, CSF algorithm, and slope segmentation to identify the internal and external feature lines of three common roof types. Experimental results demonstrate that our approach is highly efficient in extracting feature lines for most roofs in traditional villages, highlighting the feasibility of using the CSF algorithm to extract sloped roofs of traditional buildings and the superiority of slope segmentation over RANSAC and area segmentation algorithms in attaining more comprehensive roof surface segmentation and overall better extraction results. By applying Canny detection, Hough transformation, refinement and regularisation techniques, we obtain precise roof feature lines that conform to the LoD300 paradigm for 3D modelling, thus shedding new light on the spatial analysis of traditional architectural spaces and paving the way for digital transformations of rural environments.

In the course of experimentation, several issues emerged due to a limited sample size and constraints in MVS point cloud data. First, it was noted that the CSF algorithm proved efficient only for roofs featuring simple double-slopes and four-slopes in villages, making it challenging to extract roofs shaped like “回”-shaped with hillside walls. Secondly, this paper only tested the applicability of the CSF algorithm in distinct villages across different regions; thus, further validation is required. Additionally, we noticed that the accuracy of the point cloud data significantly affected the surface segmentation of the roof. Therefore, exploring measures to enhance the precision of the point cloud and surface segmentation under prevailing conditions must be considered.