The Extraction of Roof Feature Lines of Traditional Chinese Village Buildings Based on UAV Dense Matching Point Cloud

1. Introduction

2. Methods

2.2. Roof point cloud segmentation

2.2.1. Roof point cloud extraction

The CSF algorithm has garnered significant attention as a widely-used filtering algorithm, with numerous scholars attesting to its superior performance. In particular, recent studies have demonstrated the algorithm's efficacy in accurately segmenting point clouds on flat-roofed buildings, as seen in Wang et al.'s work on building extraction from UAV oblique photography point cloud using cloth simulation[26]. Capitalizing on this promising finding, this paper applies the CSF algorithm towards extracting sloped roofs on traditional buildings. Our experiment exclusively employs the CSF algorithm, with adjustments made to its parameters for several rounds of trials.

2.2.2. Slope segmentation

In order to mitigate the influence of traditional algorithms for roof segmentation, which overly rely on normal vectors and curvature thresholds[27], this study proposes a novel approach based on slope direction features. Specifically, the method segments the roof surface by leveraging the distinct slope direction features of building roofs. By projecting the original point cloud onto a regular grid and applying smoothing to eliminate anomalies, the pixel value of each grid represents the mean value of the corresponding point cloud points. To simplify the data structure, the extraction of feature lines is transferred from 3D point cloud processing to 2D image detection. This approach not only adapts well to established image segmentation algorithms but also enhances experimental efficiency.

In this study, the direction of a slope is defined as the angle between the horizontal plane and the projection of the normal to the tangent plane of a point on that surface. The resulting value is reported as the compass direction at that specific location. This measurement is conducted in a clockwise direction with values ranging from 0 (due north) to 360 (still due north), as visualized in Figure 4. In cases where an area is flat and lacks a downhill direction, it is given a value of -1 (representing a horizontal surface). To derive the slope direction for each central unit, a 3 x 3 moving window is employed, as shown in Figure 5. The calculation formula is shown below:

$$D_e=\alpha \tan 2(A,B)$$

(2)

Where D_e denotes the slope direction of the image element e, α denotes the conversion coefficient between radians and degrees, the specific calculation can be taken as 57.29578, A, B are the rate of change of the image element e in the x, y direction, the formula is as follows.

$$A=\left(c+2f+i\right)-\left(a+2d+g\right)$$

(3)

$$B=\left(g+2h+i\right)-\left(a+2b+c\right)$$

(4)

Where a, b, c, d, e, f, g, h and i are the individual image values.

Figure 4. Schematic diagram of slope direction.

Figure 4. Schematic diagram of slope direction.

Figure 5. Moving window for slope direction calculation.

Figure 5. Moving window for slope direction calculation.

Upon calculating the slope direction, a transformation is implemented to convert slope values into compass direction values based on established principles. Given that the roofs of traditional village edifices often exhibit double or four-sloped features, a detailed dissection of individual raster values in this study's experiments is deemed unnecessary. As such, it suffices to congregate raster slope direction values of identical slopes and recalibrate them to yield two or four distinct raster slope directions.

3. Experimental Results and Analysis

3.3. Roof extraction and slope segmentation

During the process of removing vegetation point cloud, it was discovered that taller trees obscured local information of the roofs due to limitations in the UAV tilt photography technology. To address this issue, one more complete building from each roof type shown in Figure 7 was selected for the outline extraction experiment. The impact of the GR value on the extraction effect of the roofs in the CSF algorithm was examined and found to have a significant influence. As an example, I-shaped roofs were used and changing the GR value while keeping other values constant resulted in the best extraction effect being achieved when GR was set to 0.1, as demonstrated in Figure 8. This approach was then applied to obtain all types of roof point clouds, presented in Figure 9. However, upon closer examination, redundant point cloud was noted at the eaves of different roofs. The reason for this is that elevation information in this area cannot be captured during tilt photography, resulting in absence of clear point cloud data during the aerial triangulation solution. Additionally, the ancestral hall of Jingping village, depicted in Figure 9d, has higher surrounding walls than the roof, leading to partial voids in the area of the walls. Consequently, the application of the CSF algorithm to extract the roof of the building did not guarantee roof integrity.

To mitigate the influence of threshold setting on plane segmentation observed in previous studies, we propose a novel approach whereby point cloud data is projected onto a flat raster and smoothed before slope calculation. In this study, we exclusively experimented with the first three types of roofs mentioned above, as the extraction performance for "回"-shaped roofs was poor and hill walls prevented slope segmentation. Since the traditional village buildings' roof tiles are concave and convex, our slope direction calculation produced eight initial directions, which were subsequently reclassified into a 2-slope or 4-slope map. For example, regarding L-shaped roofs (Figure 10), we partitioned each raster pixel into one of eight slope directions, then divided them into four categories based on morphological classification and Figure 11. Nonetheless, the reclassified raster slope directions may contain a small number of misclassified stray points at edges and intersections, which require additional smoothing and denoising to avoid negatively affecting downstream detection. Our method's segmentation results were compared to those yielded by traditional RANSAC and region segmentation algorithms to verify their effectiveness.

The accuracy of slope segmentation was assessed through the utilization of equations (1)-(3), with the corresponding results illustrated in Figure 12. From the provided visualizations, it is apparent that the precision of the surface slope, which was extracted using a directional segmentation approach, exceeded 99.6%, indicating a remarkably satisfactory outcome.

$$R=\frac{TP}{TP+FN}\ast 100\%$$

(5)

$$P=\frac{TP}{TP+FP}\ast 100\%$$

(6)

$$F=\frac{2RP}{R+P}\ast 100\%$$

(7)

where TP is the number of correctly segmented raster pixels, FN is the number of missed raster pixels, FP is the number of incorrectly segmented raster pixels, R is the recall rate, P is the accuracy rate, and F is the measure.

Table 2. Correct slope segmentation for each type of roof.

Table 2. Correct slope segmentation for each type of roof.

Index	I-shaped	L-shaped	U-shaped
TP	30587	6665	26675
FN	0	0	0
FP	126	47	53
R	1	1	1
P	99.59%	99.30%	99.80%
F	99.79%	99.65%	99.90%

Figure 12. Comparison of different types of roof surface segmentation:(a)RANSAC algorithms; (b)region segmentation algorithm; (c)slope segmentation algorithm; (d)Slope segmentation algorithm+noise removal.

Figure 12. Comparison of different types of roof surface segmentation:(a)RANSAC algorithms; (b)region segmentation algorithm; (c)slope segmentation algorithm; (d)Slope segmentation algorithm+noise removal.

It is noteworthy that, owing to the high precision of the data, the re-categorized slope direction raster might exhibit a marginal number of erroneously classified outlying points at the edges and intersections of two slope directions, namely, inaccurately segmented raster pixels. These artifacts may exert a certain impact on the ensuing detection procedures and thus must be subjected to subsequent smoothing and denoising processes. To ascertain the effectiveness of this approach, Figure 12 and Table 3 were produced by comparing its segmentation outcomes with the conventional RANSAC and region segmentation algorithms.

Based upon the results presented in Figure 12 and Table 3, our proposed method demonstrates significant improvements over both the RANSAC algorithm and the region segmentation algorithm. Firstly, all three algorithms exhibit comparable performance in terms of computational efficiency due to the small size of the input data. Secondly, regarding segmentation accuracy, neither the RANSAC algorithm nor the region segmentation algorithm succeeded in accurately segmenting the facets of simple I-shaped or slightly more complex U-shaped roofs. Instead, these algorithms exhibited varying degrees of under- and over-segmentation on either type of roof. In addition, based on point cloud data analysis, the region segmentation algorithm demonstrated a higher tendency toward under-segmentation when compared to the RANSAC algorithm. For instance, as shown in Figure 12, inner sides of both L-shaped and U-shaped roofs were segmented into single planes and the resulting segmented roofs showed greater noise due to restricted data accuracy. Furthermore, during the experiment, it was observed that the RANSAC algorithm produced inconsistent results under identical parameter settings, indicating significant deviations and instability.

Using the direction of slope, successful segmentation of all slopes was achieved in all three types of roof segmentation, resulting in improved outcomes. Though there were several instances of mis-segmentation, such occurrences did not adversely affect the overall effects compared to the area segmentation algorithm. Furthermore, after denoising, a complete representation of the various roof slopes was obtained. It should be noted that during experimentation, the first two algorithms were directly based on point cloud segmentation, resulting in neater planar edges. In contrast, slope segmentation relied on raster images after point cloud projection, leading to rougher edge contours due to lower raster resolution. However, this issue can be eliminated by regularization in subsequent feature line extraction.

4. Conclusion

References

1. Huo, P.P.; Hou, M.L.; Yang, S.; Hou, Q.; Zhou, Q.; Li, A.; Yang, S. Automatic extraction of building rooftop outlines using airborne LiDAR: A review. Geomatics World. 2019, 26, 1–13. [Google Scholar]
2. Du, J.L.; Chen, D.; Zhang, Z.X.; Zhang, L.Q. Research progress of building reconstruction via airborne point clouds. Journal of Remote Sensing. 2019, 23, 374–391. [Google Scholar] [CrossRef]
3. Panagiotopoulou, A.; Grammatikopoulos, L.; El Saer, A.; Petsa, E.; Charou, E.; Ragia, L.; Karras, G. Super-Resolution Techniques in Photogrammetric 3D Reconstruction from Close-Range UAV Imagery. Heritage. 2023, 6, 2701–2715. [Google Scholar] [CrossRef]
4. Farmakis, I.; Karantanellis, E.; Hutchinson, D.J.; Vlachopoulos, N.; Marinos, V. Superpixel and Supervoxel Segmentation Assessment of Landslides Using UAV-Derived Models. Remote Sens 2022, 14, 5668. [Google Scholar] [CrossRef]
5. Yang, B.S.; Dong, Z. Progress and perspective of point cloud intelligence. Acta Geodaetica et Cartographica Sinica. 2019, 48, 1575–1585. [Google Scholar] [CrossRef]
6. Jabri, S.; Zhang, Y.; Alaeldin, S. Stereo-based building detection in very high resolution satellite imagery using IHScolor system. Geoscience and Remote Sensing Symposium (IGARSS). 2014, 2301–2304. [Google Scholar]
7. Huang, X.; Zhang, L. A Multidirectional and Multiscale Morphological Index for Automatic Building Extraction from Multispectral GeoEye-1 Imagery. Photogrammetric Engineering & Remote Sensing. 2011, 77, 721–732. [Google Scholar]
8. Wu, Y.; Wang, L.Y.; Hu, C.X.; Cheng, L. Extraction of building contours from airborne LiDAR point cloud using variable radius Alpha Shapes method. Journal of Image and Graphics. 2021, 26, 0910–0923. [Google Scholar]
9. Zhu, P.; Li, S.; Zhang, L.; Li, Y. Multitask learning-based building extraction from high-resolution remote sensing images. Journal of Geo-information Science. 2013, 23, 514–523. [Google Scholar]
10. Long, J.; Shelhamer, E.; Darrell, T. Fully Convolutional Networks for Semantic Segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2015, 39, 640–651. [Google Scholar]
11. Charles, R.Q.; Su, H.; Kaichun, M.; Leonidas, J.G. PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation. computer vision and pattern recognition 2017, 77–85. [Google Scholar]
12. Huang, X.; Sohn, G.; Wang, X.; Zhang, F. Roof detection using LiDAR data based on points’normal with weight. Geomatics and Information Science of Wuhan University. 2009, 34, 24–27. [Google Scholar]
13. Gu, Y.F.; Cao, Z.M.; Dong, L.M. A hierarchical energy minimization method for building roof segmentation from airborne LiDAR data. Multimedia Tools and Applications. 2017, 76, 4197–4210. [Google Scholar] [CrossRef]
14. Ma, X.; Luo, W.; Chen, M.; Li, J.; Yan, X.; Zhang, X.; Wei, W. A Fast Point Cloud Segmentation Algorithm Based on Region Growth. International Conference on Optical Communications and Networks. EasyChair. 2019, 1–2. [Google Scholar]
15. Xu, Y.; Yao, W.; Hoegner, L.; Stilla, U. Segmentation of building roofs from airborne LiDAR point clouds using robust voxel-based region growing. Remote Sensing Letters. 2017, 8, 1062–1071. [Google Scholar] [CrossRef]
16. Wang, Y.Q. Research on building extraction and model regularization from LiDAR data. Master’s Thesis, Nanjing Univeristy, Nanjing, China, 2013. [Google Scholar]
17. Saglam, A.; Makineci, H.B.; Baykan, O.; Baykan, N. Clustering-based plane refitting of non-planar patches for voxel-based 3D point cloud segmentation using K-means clustering. International Information and Engineering Technology Association. 2020, 37, 1019–1027. [Google Scholar] [CrossRef]
18. Ywata, Y. Adaptive random sample consensus approach for segmentation of building roof in air-borne laser scanning point cloud. International Journalof Remote Sensing. 2020, 41, 2047–2061. [Google Scholar]
19. Liu, Y.K.; Li, Y.Q.; Liu, H.Y.; Sun, D.; Zhao, S.B. An Improved RANSAC Algorithm for Point Cloud Segmentation of Complex Building Roofs. Journal of Geo-information Science. 2021, 23, 1497–1507. [Google Scholar]
20. Xu, J.; Li, J. An optimal RANSAC method for segmentation of complex building roof planes. Geomatics and Information Science of Wuhan University. 2023, 1–16. [Google Scholar]
21. Liu, P.L.; Deng, Y.Y. Digitized protection: A new way to protect historical and cultural towns and villages. Journal of Peking University( Philosophy and Social Sciences). 2017, 54, 104–110. [Google Scholar]
22. Song, G.T.; Zhao, Y.Q.; Wang, S. Research on protection and application of real 3D digitization of traditional village based on oblique photogrammetry. Construction Science and Technology. 2021, (14), 61–65. [Google Scholar]
23. Bian, X.; Ma, Q.Y.; Liu, C.Y.; Zhao, Y.W. Vegetation coverage calculation based on Low altitude visible spectrum. Bulletin of Soil and Water Conservation. 2017, 37, 270–275. [Google Scholar]
24. Zhang, W.M.; Qi, J.B.; Wan, P.; Wang, H.T.; Yan, G.J. An easy-to-use airborne LiDAR data filtering method based on cloth simulation. Remote Sensing. 2016, 8, 501. [Google Scholar] [CrossRef]
25. Zhang, F.; Li, H.S.; Jiang, T. Digital Elevation Model Generation in LiDAR Point Cloud Based on Cloth Simulation Algorithm. Laser & Optoelectronics Progress. 2020, 57, 114–121. [Google Scholar]
26. Wang, G.; Wang, Q.; Liu, S.T. Method of building extraction from UAV oblique photography point cloud based on cloth simulation. Bulletin of Surveying and Mapping. 2020, (10), 97–100. [Google Scholar]
27. Zhao, C.; Zhang, B.M.; Chen, X.W.; Guo, H.T.; Lu, J. Accurate and Automatic Building Roof Extraction Using Neighborhood Information of Point Clouds. Acta Geodaetica et Cartographica Sinica. 2017, 46, 1123–1134. [Google Scholar]
28. Canny, J. A Computational Approach to Edge Detection. IEEE Transactions on Pattern Analysis and Machine Intelligence. 1986, 8, 679–698. [Google Scholar] [CrossRef]
29. Li, J.; Wang, H.; Yan, K.; Yan, X.D.; Yang, L. Improved canny algorithm for image edge enhancement. Journal of Geomatics Science and Technology. 2021, 38, 398–403. [Google Scholar]
30. Gong, S.J.; Li, G.Q.; Zhang, Y.J.; Li, C.; Yu, L. Application of static gesture segmentation based on an improved canny operator. The Journal of Engineering. 2019, 3, 543–546. [Google Scholar] [CrossRef]