Assessment of SLAM Scanner Accuracy for Outdoor and Indoor Surveying Tasks

1. Introduction

3D Laser Scanners are instruments that capture a vast number of observations with the use of Light Detection And Ranging (LiDAR). LiDAR is an active sensor that sends laser rays that hit surfaces in the environment and reflect (back-scatter) to the sensor to determine the position of the surface in 3D space relative to the sensor to create a cloud of observations commonly known as a point cloud [1]. Panoramic cameras capture images that can be used to better visualize the LiDAR data captured and aid in the processing and final product of the data [2]. These scanners can be used to create digital models, and maps, for deformation monitoring, and for many other applications [3,4].

The accuracy of the LiDAR reflectance can be influenced by factors such as angle of incidence, type of material, and environmental factors [5]. The angle of incidence can affect the amount of backscatter with smoother surfaces having a higher reflectance being more sensitive (the greater the angle the less backscatter) than rough surfaces, with the angle of incidence having little effect on the amount of backscatter [6]. Moreover, the surface type whether the color, how rough/smooth, how shiny/matte, or how transparent can influence the intensity of backscatter. The instruments used to make these types of observations are mounted on tripods and are referred to as static Terrestrial Laser Scanners (TLS). These devices are static while scanning and may require multiple setups to conduct a survey.

A survey-grade static TLS is known to have high precision and can capture a vast number of observations. Each scan produces a point cloud that needs to be registered together with sufficient overlap to create one overall point cloud in post-processing software [7]. Due to these devices being static, some surveys such as indoor environments can be difficult to plan and complete. Indoor environments with short-sight distances, multiple levels with stairs, and other obstacles can be difficult to navigate and can require multiple setups to ensure everything is observed which can increase the time of the survey both in the field and office. A solution to this problem may be the use of a Mobile Laser Scanner (MLS).

MLS is a technology that can take observations while moving. For terrestrial use, these devices can be moved using a vehicle, trolley, or a person. These instruments have three main hardware components: optical sensors, navigations/positioning sensors, and a control synchronization unit [8]. MLS offers advantages such as reduced field and office work due to the absence of multiple setups. These devices are more capable of navigating indoor environments due to their portability and access to space-restricted environments. This paper investigates man-portable Simultaneous Localization And Mapping (SLAM) scanners. This system uses LiDAR and panoramic cameras to capture observations. An Inertial Measurement Unit (IMU) and SLAM are used to discern where the device is. SLAM has an algorithm software that defines the trajectory of the device and the three-dimensional reconstruction of the recorded sensors [8]. This algorithm allows the device to estimate its location concerning the location of landmarks.

Observations are taken at every epoch and SLAM calculates the correlation between the instrument and the environment to update its trajectory equation in conjunction with the (IMU’s) measurements [9]. This enables the device to be used indoors where the Global Navigation Satellite System (GNSS) is ineffective. The accuracy of the SLAM can be affected by drift and tracking errors. Drift errors are the accumulation of minor measurement errors during a survey that can cause the scan data to drift. Furthermore, tracking errors are when the environment does not have enough discernible features and the SLAM algorithm gets confused about where it is [10], e.g., a long homogenous corridor. Both types of errors are amplified in areas poor in the distribution of 3D features [11], outside environments, and areas where objects move [9]. These errors can be mitigated by selecting an appropriate trajectory. An appropriate trajectory is the use of loop closures and not rotating too quickly around corners. Loop closures are the scanning overlap of an area from a different perspective. The user conducts loops around an area, similar to a closed traverse in the surveying practice to ensure this overlap [12]. That is why it is recommended that a survey should start and finish at the same spot as one big loop closure [13]. Also, control points are another method used to also reduce these types of errors by providing alignment adjustments during post-processing [12].

In the same context, speed is another factor that can influence an MLS survey, where the increase in speed can lead to a decrease in the number of observations [14] and can decrease the accuracy of observations [15]. Despite the benefits of an MLS, this type of device is rarely used in the surveying industry due to limited testing, the absence of industry standards for accuracy assessment, and the scarcity of literature on their capabilities. Kaartinen et al. [16] tested the elevation accuracy of the produced point cloud of a road with an MLS. The best results achieved were a planimetric accuracy of 25 mm over a range of 45m. Similar accuracies have been obtained from mapping topography changes and elevation of erosion changes of riverbanks by Vaaja et al. [17] where the obtained RMSE value was between 23 mm and 76 mm. This may suggest that the Navvis VLX could potentially be utilized in such applications where the main intent is to map elevation changes or deformation which could be useful for monitoring surveys.

One of the central challenges in SLAM is data association. This involves linking measurements captured by the sensor at different instances and locations with mapped features to determine if they originate from the same physical place [18]. Unlike photogrammetry, where distinguishing features in each dataset is comparatively straightforward, SLAM depends heavily on recognizing complex features such as changes in wall orientation, window outlines, or building and furniture edges. Thus, the accuracy of this process is pivotal in generating a precise model.

Lauterbach et al. [19] and Chiappini et al. [20] investigated the use of MLS in an outdoor setting in which the focus of Building Information Modelling (BIM) was the main goal. Results indicated that the SLAM scanner can be successfully utilized in an outdoor environment to extract building information of buildings. This is of importance as this component needs to be considered when extracting features in a residential topographic survey. Fassi and Perfetti [21] and Vatandaşlar and Zeybek [22] investigated Digital Terrain Model (DTM) production from the derived data of SLAM in outdoor environments. Both studies indicated with the capture point cloud data, a DTM was successfully produced to the required accuracies. The main differences are that these studies are applied within a natural landscape and did not have features that would be typically found in an urban residential area such as buildings. Further research has indicated that from an application use SLAM scanners in the form of backpack devices are mainly utilized in applications from cultural heritage preservation, environmental monitoring and forestry, and agriculture which mainly covers the main purposes of 3D modeling, erosion change detection, and vegetation inventory [23]. A similar study was conducted by Yiğit et al. [24] where a comparative analysis of MLS and TLS for indoor mapping was conducted. Within the study data was collected from a TLS to an MLS and the Total Station (TS) of a large multistorey building was compared.

SLAM scanners provide 3D LiDAR point clouds, and the accuracy assessment operation may use data classification and geometric feature extraction. Nevertheless, automatic feature extraction is widely used to implement on point clouds to obtain more accurate results on the position of features in a point cloud dataset [25]. Many studies have investigated automatic feature extraction of the use of a RANdom SAmple Consensus (RANSAC) algorithm and Hough transform to obtain roof and building features at high levels of accuracy relative to baseline observations [26,27,28]. Another automatic feature extraction explored by Xiong and Wang [29] and Dey et al. [30] investigated the deep learning framework for BIM construction, finding that the stated method could efficiently and accurately segment the point cloud to list important features.

At this point it is beneficial to highlight the novel contribution of this paper as follows:

• Assessment of the efficiency of a SLAM scanner in the application of detail surveys in an outdoor setting to address the envisaged challenges.
• A comparative study is conducted on residential areas comparing different workflow methods and different reference datasets.
• Assessment of the capabilities of the SLAM scanner for indoor and outdoor data collection.
• Assessment if SLAM can achieve accuracies to a standard fit for conducting surveys.

2. Assessment Design and Used SALM Scanners

The assessment in this paper consists of two sections, outdoor assessment and indoor assessment. In outdoor assessment, first, the Navvis SLAM Scanner data is compared to reference datasets measured by the TS. Second, the BLK2GO SLAM Scanner point cloud is compared to reference TS datasets. Third, edge detection board assessment is carried out using BLK2GO data. Fourth, the TS dataset is used as a reference model for the BLK2GO SLAM Scanner point cloud. Finally, in indoor assessment, the NavVix SLAM scanner data are compared to Z+F static scanner datasets. These experiments allow an understanding of the behaviors of two employed scanners and also compare the efficacity of the two kinds of reference data. Moreover, three types of data comparison will be tested, cloud-to-cloud, cloud-to-mesh, and mesh-to-mesh. However, in indoor data assessment, based on the obtained results of the last experiments, only one test will be realized which is the NavVix SLAM data comparison to static scanner datasets.

For this study, the NavVis VLX as well as BLK2GO from Leica SLAM Scanners were chosen to realize the SLAM accuracy assessment. The first one is a wearable device whereas the second one is portable. At this stage, it is inevitable to present the used SLAM scanners before starting the assessment stage.

2.1. NavVis VLX SLAM Scanner

The NavVis VLX 2nd generation scanner is a wearable and walkable mobile scanner (Figure 1a). It is placed on the user's shoulders as seen in Figure 1a. The instrument has two LiDAR sensors, four 20-megapixel cameras, an IMU, and SLAM software. It offers both indoor and outdoor performance with absolute accuracies of 6 mm at a 68% confidence level, and 15 mm at a 95% confidence level in a dedicated test environment of 500 m² [10]. Given the limited existing research with this device in a residential outdoor environment and the known efficiency of the device to capture point cloud datasets at high speeds whilst moving [10], this device was selected to be used in the data acquisition. NavVis recommends conducting a loop and walking at a normal pace when doing a survey [10] and doing loop when surveying as it reduces SLAM/drift error by providing sufficient overlap.

The LiDAR sensors have a maximum range of 100 m and are positioned with one vertical and the other horizontal as seen in Figure 1a, which allows the device a greater field of view. The cameras have a 360-degree view with the user being stationary when taking photos. The IMU with the SLAM technology allows the instrument to discern where it is and where it has been. The NavVis can geo-reference marks either on the wall or ground with a hook on the bottom of the instrument. The device has a touch screen that provides information and shows an overview map of what the device has scanned [31].

2.2. BLK2GO SLAM Scanner

In 2019, Leica Geosystems introduced the BLK2GO (Figure 1b), a portable mobile mapping device capable of collecting 420,000 points per second with a claimed accuracy of 10 mm within indoor environments [32]. The primary benefit of this scanner in contrast to static models lies in its ability to simultaneously localize itself in an environment while mapping it. This eliminates the requirements for multiple setups with manual localizations to collect data. To capitalize on this benefit, Leica has introduced an app named BLK2GO Live. This app allows users to observe the mapped regions and areas awaiting data collection in real time, thereby enhancing efficiency, and reducing the requirements of mission planning.

The backbone of the BLK2GO system, the LiDAR, utilizes a class 1 laser to gather data. This LiDAR system captures a field of view covering 360° horizontally and 270° vertically, with effective ranges spanning from 0.5 m to 25 m [33]. The laser operates at a wavelength of 830 nm and is accompanied by a stated noise range of 3 mm [32]. The scanner comes with three cameras each with a resolution of 4.8 megapixels. These cameras capture a panoramic image spanning 300° horizontally and 135° vertically. This imagery is then utilized to detect similarities among successive images, aiding in calculating the scanner's movement throughout the environment. Through the utilization of the Grand SLAM Technology, Leica aims to address the challenges encountered in traditional SLAM. In conventional SLAM, where images aren't captured, the precision of maps and real-time instrument positioning relies on how effectively data from various locations can be linked together [34].

3. Outdoor Assessment

As shown in Section 2, four experiments of SLAM scanner measurements will be realized in this section. The next subsection will detail the first assessment which is the Navvis SLAM Scanner and TS datasets.

3.1. Navvis SLAM Scanner and TS Datasets

The goal is to assess the efficiency of a wearable Navvis VLX MLS (Figure 1 a) in the application of a residential contour and detail surveys in an outdoor setting to address these challenges mentioned. A comparative study is conducted on a residential house in Queensland, Australia (Figure 2a), comparing two different workflow methods. Initially, a detailed survey is executed under conventional methods with the use of a TS and GNSS unit for control. In this context, Magnet office was the utilized software to reduce the TS observations to align the coordinates established from the control survey (Figure 2c) as well as a height reduction to make all observed points relative to Australian Height Datum (AHD). Subsequently, that same area was surveyed using the Navvis VLX (Figure 2b) where the measured point cloud is georeferenced using Navvis Ivion software. Both data sets derived from the TS and the point cloud are superimposed together after generating two DTMs (Figure 3). The horizontal and positional accuracies of the produced point cloud from the Navvis VLX are compared to the baseline of the TS with three different data analysis techniques consisting of a cloud-to-cloud, cloud-to-mesh, and mesh-to-mesh comparison. For this purpose, CloudCompare 2.12.4 software is used to carry out this comparison. To analyze the horizontal position (x and y axis) of in the point cloud regarding the baseline TS data, a list of features such as retaining walls, service utilities such as power poles and water meters, fences, concrete driveways, slight sloping elevation and also a house located on the site, are compared with computation of the Root Mean Square Error (RMSE). These features are labelled manually on the LiDAR point cloud, whereas they were assigned during a survey by TS.

3.1.1. Data Assessments

To compare the LiDAR point cloud with the survey data, the point cloud is filtered up by the removal of duplicate points at a tolerance of 5 mm. This tolerance value is considered regarding the mean laser spot radius value (3 mm) added to a tolerance amount. The purpose of the removal of duplicate points in the point cloud is to keep the useful points and make them more manageable to work with, reduce the noise in the point cloud which in turn will enhance the visualization of the cloud [35]. This has also been proven to speed up processing times due to the reduction in point cloud size [36]. After this filtering, the point cloud is compared to the points from the exported Triangulated Irregular Network (TIN) using an Iterative Closest Point (ICP) algorithm [37].

In the second test, utilized for data analysis is a cloud-to-mesh comparison in which the LiDAR point cloud is classified into the ground and off-ground points. Thereafter, a mesh generated by Delaunay 2.5D [38] to be created of the filtered ground points which is compared to the points exported from the TIN produced from the TS observations. The third form of data analysis involved a mesh-to-mesh comparison of the scan data compared to the TS observations. A mesh is produced also using Delaunay 2.5D on these two data sets which is then compared to one another. This chosen test is utilized because it has been shown to ensure the triangles are well-shaped and avoid creating any irregular or elongated triangles in the mesh which can result in some inaccuracy in interpolation [38]. At this stage, it is important to refer that the last three comparisons have been proven to be the main forms of comparison utilized in studies for point cloud comparison [39].

The last conducted assessment is the vertical and horizontal positional comparison between the two data sets of extracted geometric features located on the site. The main points of interest selected for comparison consist of features that would not have much deviation in X and Y coordinates for example the building corners of the house located on the site and for the vertical component features such as finished levels of the buildings and decks located on the site. Comparisons are computed between the differences in X, Y, and Z coordinates. To investigate the standard deviation of the residuals computed, the RMSE is calculated on this data (Equation 1).

$$RMSE=\sqrt{\frac{\sum V^2}{n}}$$

(1)

where v is the deviation, and n is the sample size.

3.1.2. Result and Discussion

The three forms of data analysis for the vertical or Z coordinate comparison between the TS observations and the Navvis VLX scanner data are compared with the three techniques of a cloud-to-cloud, cloud-to-mesh and mesh-to-mesh comparison [39]. To provide a statistical analysis of the two datasets, the Gaussian distribution otherwise known as the normal distribution is utilized as this measure provided information on how far the data was from the mean as well as the standard deviation. The standard deviation from the three conditions tested is the main important value derived from the analysis because it represents the average distance of data points from the mean as well as quantifies the spread of the data. The Gaussian distribution analysis is calculated at three confidence levels 68%, 95%, and 100 %, i.e., 68% confidence level where approximately 68% of the data falls within one standard deviation of the mean. Figure 4a shows the histogram of the cloud-to-cloud comparison with a mean of 35 mm and a standard deviation of 22 mm. Figure 4b shows the histogram of the cloud-to-mesh comparisons with a mean of 110 mm and a standard deviation of 180 mm. Finally, Figure 5 presents the histogram of the mesh-to-mesh comparison with a mean result of 27 mm and a standard deviation of 65 mm.

Concerning the feature comparison, the main selected features from the site are presented in Table 1 and Table 2 where Z coordinate is compared to features measured on the site such as finished floor levels on the front and back deck. This is carried out by calculating the elevation difference between the baseline measurements from the TS compared to the LiDAR point cloud. The RMSE value is 15 mm (Table 1). For the horizontal positional accuracies, the selected features in the two data sets are displayed in Table 2. For this comparison, the X and Y coordinates of the MLS are compared to the baseline of the TS observations. The calculated RMSE for the X coordinate is 28 mm and 63 mm for the Y coordinate. The greatest differences in the horizontal positions are the electrical pole feature with a difference of 63 mm in X and 93 mm in Y.

As presented in Figure 4 and Figure 5, the histograms of the Gaussian distribution of the three comparison tests, the cloud-to-cloud comparison produces the best results based on the lowest standard deviation score of 22 mm. The main reason for testing the three different comparative methods is to define which method would be most accurate for comparing the DTM extracted points from the TS observations as the baseline. The cloud-to-cloud comparison primarily involves calculating distances between similar points of two point-cloud datasets. It is the most efficient and simple to utilize as it does not require another model to be built such as a Digital Elevation Model (DEM), meaning less manipulation and filtering of the point cloud dataset [40]. Indeed, it calculates the nearest neighbor distance, which is the distance between two points in the cloud which is then compared to the two similar points in the other observed cloud, and then their Euclidean distance is calculated [41]. In this paper, the DTM model from the TS data was utilized for comparison as it is one of the main requirements typically provided for design purposes, hence why this dataset was compared to the point cloud. The proven result of a standard deviation of 22 mm suggests that DTM’s would be able to be produced quite accurately with the scanned point cloud data from the Navvis VLX with the tested feature extraction of point picking in the cloud of ground points of features such as surface levels, footpaths and top of bottom of walls to form the ground classification. When comparing this standard deviation result for elevation to other studies with the use of an MLS it is evident that the Navvis fairs quite well.

Concerning the horizontal accuracy, the calculated RMSE for the X and Y coordinates identified some evident variances from the baseline of the TS observations. Certain features compared on a horizontal plane differed in a tolerance which would make it questionable to be utilizing this data to produce an accurate topographic map. This is the case as a certain tolerance must be met when the horizontal position of features is crucial to define in residential contour and detail survey as they are typically relative to cadastral boundaries. The results showed a combined RMSE of 30 mm in the abscissa and 60 mm in the ordinate and when considering these values, it can indicate that these tolerances would be not suitable for residential topographic surveys. These results do however fall upon the method utilized in this study of manual feature extraction from point picking in the cloud and largely rely upon the interpretation of the point cloud data. This provides a suggestion for implementing different methods of feature extraction on the point cloud to investigate if the accuracies are improved on the horizontal comparison. To conduct this experimentation again, the use of automatic feature extraction algorithms as mentioned in Section 1, could be advantageous to obtain more accurate points that represent certain features on site for example building corners of the house which then could be compared to baseline observations.

The Navvis XLV produced a point cloud that was vertically accurate after georeferencing with a standard deviation of 22 mm from the computed cloud-to-cloud comparison. However, the horizontal accuracies, i.e., X and Y coordinates, don’t seem to be as accurate which may be due to an inaccurate feature extraction method and lack of control points. This suggests that further investigation should be realized on feature extraction and the enhanced algorithms are used to improve feature extraction. Finally, from an efficiency and flexibility standpoint, the Navvis VLX produces great results which is reflected in the time to scan the site which was 50 minutes in length compared to the 3 hours it took with the TS.

3.2. NavVix SLAM Data and Static Scanner Dataset

In this section, a new SLAM scanner assessment of the outdoor area (Figure 6a) is carried out using a static scanner to measure a reference point cloud. The equipment used to measure the point clouds included the Z+F 5016 static TLS (Figure 6b) and the NavVix VLX 2 SLAM Scanner (Figure 1a and 1b). The NavVix VLX 2 SLAM scanner was presented in Section 2. Concerning Z+F 5016 static TLS, Table 3 shows the scanner characteristics. In this context, the TLS data is filtered to remove erroneous points such as duplicate points and point cloud noise. Thereafter, depending on the 50 % overlap between neighboring stations as well as the used target points, a registration process between neighboring station clouds is completed successfully with a connection standard deviation of 3.4 mm and a target standard deviation of 0.3 mm results.

To complete the comparison of SLAM and static datasets, a cloud-to-cloud comparison is applied (Figure 7). The outdoor data results accuracy achieved is 3 mm at a 68 % confidence level, with a maximum distance from the TLS of 6 mm at 68%. In comparison to the stated relative accuracy of 8 mm at 68%, and the white paper outdoor building façade scenario accuracy of 6 mm at 68%, the assessment results achieve better accuracy of the outdoor area. The resulting point cloud seen in Figure 7a displays higher distances in red located on the sandstone wall adjacent to the water feature and front entrance. Most of the data can be seen to be within the range of 0 mm and 5 mm in colored Blue. The high accuracy of the obtained result confirms the efficiency of the SLAM scanner regarding the static scanner within the outdoor areas.

3.3. BLK2GO SLAM Scanner Accuracy Assessment

It is unavoidable for data measurement to grasp the potential levels of accuracy attainable from instruments such as the BLK2GO. This section aims to assist in evaluating the accuracy of this instrument and determining whether the instrument can deliver satisfactory levels of accuracy in survey works. It includes the use of an edge detection board as well as a comparison with TS data to assess the SLAM LiDAR accuracy.

3.3.1. Edge Detection Board Assessment

Without any human intervention, the determination of object edges, and subsequently the accuracy of measured objects, relies on the density of the point grid at specific distances. As highlighted in Section 1, the precise identification of objects holds paramount importance in the SLAM process.

Thus, accurate data association plays a critical role in combining datasets and ultimately generating a precise model. Typically, scanning systems calculate measurements by computing the angular and distance differences between successive points. Consequently, targets situated closer to the scanning unit will exhibit a denser concentration of closely spaced points, while targets positioned farther away will display a lower point density [42]. The test apparatus consisted of a 12 mm ply board with eight Medium-Density Fibreboard (MDF) panels of different thicknesses attached to it (Figure 8). A steel machinist ruler and digital callipers are used to directly confirm the dimensions of the MDF pieces attached to the board. Once all the measurements are confirmed multiple scans of the board are executed. This approach enabled the assessment of distance measurements and the determination of object edges concerning the sensor's proximity.

The scans are taken at distances of 2 m, 5 m, 7 m, 10 m, 15 m, and 25 m, each lasting 40 seconds. Running the scan for 40 seconds provided sufficient time for the instrument to establish its position and ensure a comprehensive saturation of points on the board. To assess the collected data, the horizontal and vertical values of each token are measured at varying distances. To assess the depth measurement of the tokens, sections of the edge detection board are considered allowing for an assessment of the depth measurements when using a top view. Figure 9 shows the results of the edge detection board scan where the tokens are 120 mm x 105 mm and varied in depth from 3 mm to 24 mm.

From Figure 9, the instrument consistently delivered high-quality results up to a distance of 7 m. However, the mean standard deviation in the X-axis is 4.7 mm, and in the Y-axis is 4.8 mm. The data obtained from the edge detection board plays a crucial role in evaluating the accuracy of the LiDAR unit and determining the ideal proximity to a subject for obtaining reliable results. The calculated distance throughout the scans provided accurate results. The increase in the standard deviation aligns with the findings of Harrap and Lato [42]. Their research suggests that as an object moves farther from the scanner, it tends to have lower saturation, leading to potentially greater errors in measurements.

From Table 4, The results from the Z-axis are far less impressive with errors of up to 14 mm recorded in the 10 m scan. However, the data obtained from these scans is uninterpretable at distances beyond 10 m. Table 4 shows the difference in definition between the scans taken at 2 m to 10 m of cross-section. The weaker outcomes of in-depth measurement primarily stem from the scanner's stationary position during the scan. Since the scanner is arranged perpendicular to the board, it resulted in the creation of small shadows by the tokens. To address this problem and potentially enhance the results, the instrument should be moved laterally and vertically while maintaining the desired offset. Implementing these adjustments would allow the instrument to perform more effectively by enabling better scanning of the corners.

3.3.2. TS Datasets as Reference

A survey of the site is conducted using TS to provide reference data. The TS used has a stated angular accuracy of 3 seconds of the ark and a distance accuracy of 2 mm +2 ppm when using the reflector less feature. A traverse is executed around the site using the TS following the same trajectory as the data collected from the scanner. Data is collected at three different points: along the base of the walls, halfway up the walls, and at the intersections of the ceiling and walls to enable a three-dimensional model to be constructed. From these approximate positions are recorded for comparison to the scan data.

After filtering the scan data, the cloud-to-cloud comparison strategy is then employed to compare the surveys from the subject area. This analysis visually highlighted any disparities that have emerged between the surveys and provided a statistical summary of the quality of the scan data. Subsequently, a comprehensive investigation is undertaken to ascertain the underlying causes of these discrepancies and to determine their magnitude. Figure 10 shows the results of the comparison between the data collected from the TS and the BLK2GO. It can be noted that the data obtained from the BLK2GO yielded a mean value of 21 mm with a standard deviation of 12 mm. Although, it falls slightly short of the manufacturer's stated accuracy claims, the difference is not dramatic. The mean and standard deviation could potentially be improved by removing less of the point cloud.

Figure 10 shows the result of the comparison between the data collected from the TS and the BLK2GO scanner. The data obtained from the BLK2GO yielded a mean value of 21 mm with a standard deviation of 12 mm. Although it falls slightly short of the manufacturer's stated accuracy claims, and the difference is not dramatic.

4. Indoor Assessment

This section focuses on indoor assessment of SLAM scanner accuracy. This choice has been adopted regarding the wide employment of SLAM scanners for data acquisition in indoor projects. In this context, the NavVix VLX 2 scanner data is compared with reference datasets measured by the Z+F static scanner.

To assess NavVix scanner accuracy, a point cloud measured by the Z+F Imager 5016 scanner is used as reference data because the last scanner is much more accurate than NavVix one [43]. To meet the aim of this research, the Ipswich City Council Art Gallery in Queensland, Australia is chosen to conduct the scan experiment. The test area is set, and data are collected by the TLS and the MLS doing three separate scans: normal walking speed in a loop closure (loop normal), fast walking speed in a loop closure (loop fast), and normal walking speed in a straight path with no loop closure (straight normal) [10]. The results of the loop normal are used to provide the best accuracy against the TLS data and to compare the other MLS scans. The second scan (loop fast) is chosen to test the effects of speed on the device by walking at a fast pace. The third survey (straight normal) is chosen to test the effects of not conducting a loop in the scan. The TLS point cloud will be used as the reference model (ground truth) for the MLS point clouds to compare by carrying out a cloud-to-cloud distance comparison [44].

The analysis is conducted to assess the relative accuracy of each of the MLS scans compared to the TLS scan. Further analysis is done to assess if speed and doing a loop closure with the MLS affects its relative accuracy. Speed is calculated using the metadata of the MLS scans and relative accuracy is calculated. To analyze each of the MLS point clouds; the MLS point cloud is compared to the TLS point cloud using a cloud-to-cloud comparison with a quadratic local model calculated for the TLS to give a better approximation of the test site’s surface. Thereafter, a distribution-fitting statistical test is realized to conduct a Gaussian test; and finally, this data is used to determine the mean, standard deviation, and RMS between the two point-clouds as shown in Figure 11.

Despite the high similarity between the obtained result accuracy, it can be noted that each point cloud aligns better with the loop normal producing a result approximately 5 mm better. Comparing each point cloud results shows that the loop normal point cloud produced the most accurate results as seen in Figure 11 with a mean of 4 mm, standard deviation at 2σ (95% C.I.) of 8 mm, maximum deviation at 2σ (95% C.I.) of 12 mm, and an RMSE of 5 mm. This is shown in Figure 11 with Loop Normal producing the lowest mean, standard deviation, and RMSE. This point cloud is followed by a straight normal, then loops fast in terms of accuracy. The Gaussian histograms for each scan show they have a normal distribution with a high peak, and each is skewed to the left on the lower end of the distance. This indicates some of the higher distance values have caused the mean to shift to the right from where most of the values lie. The high peak of the histograms shows too that most of the cloud-to-cloud distances lie within a small range. The maximum deviation for all points clouds at 2σ (95% C.I.) is 16 mm.

These results show a significant difference in the number of points observed between the loop normal and loop fast (10,371,499 points). The loop normal also produced a more accurate result of a maximum at 95% C.I. of 12 mm. When comparing the number of points observed, the TLS point cloud has over 100 million more points observed than the loop normal which has the biggest number of points for an MLS survey. The TLS survey took approximately 70 minutes from start to finish whereas the loop normal took 2.45 minutes.

5. Conclusion

The key findings from the results and analysis show many factors influence the SLAM scanners' accuracy and capabilities. Speed and flexibility have a greater effect on the accuracy than conducting loop closures and should be taken as a priority factor to consider when using a SLAM scanner in survey work. Surface type and ease with which the instrument can scan these surfaces is a major factor in a SLAM survey of areas with a high angle of incidence which are hard to scan and may not get the backscatter needed to produce an accurate point cloud. The time taken in the field for a SLAM to collect data is significantly quicker than a TLS as well TS. However, the TLS and TS are better suited for high-accuracy scans whereas the SLAM may be better suited for lower-accuracy scans (5 mm indoor scanning and between 10 mm and 60 mm for outdoor scanning), where the difference is insignificant and may be an acceptable compromise in exchange for faster and flexible data acquisition.

The results from the edge detection board highlight the benefits of combining the LiDAR data with the imagery captured. While this did not help with the depth measurements, it provides the user with certainty in what they are measuring. The outcomes derived from the tests performed to evaluate the instrument's precision instil confidence within the surveying community.

With the accuracies proven within this paper, the SLAM scanner shows promise for use in several different outdoor sectors which are tolerated with accuracy viewpoint. Indeed, the underground mining sector and indoor areas are where this instrument could be used to detect movement in the shafts and drives. Due to the ease and flexibility of use, surveys could be conducted at more frequent intervals to potentially detect movement earlier. Further assessment of the effects of elevation change would need to be conducted. Ultimately the results obtained from this research have found that this instrument can capture cadastral survey grade information and potentially in the future there may be a space for this instrument in conducting lease surveys.

Future research with the SLAM scanner can delve deeper into its capability assessment. Environments with homogenous features or surveys over a larger area may show how much of an effect loop closures and the environment have on the drift and tracking errors with SLAM. Speed could be further assessed as to whether going slower than a normal walking pace using the same pathway produces more accurate results that are significant to warrant. A real-world survey such as a building final plan or digital twin could be assessed to see if the SLAM scanner is sufficient to do this type of survey, and how much quicker is the SLAM scanner from start to finish (including the pre-planning, data acquisition, and post-processing) than a TLS.

Authors contributions: Conceptualization, Z.G., K.H., H.G., A.J., and N.R.; Methodology, Z.G., K.H., H.G., A.J., and N.R.; Validation, K.H., H.G., A.J., and N.R; Formal Analysis, K.H., H.G., A.J., and N.R.; Investigation, K.H., H.G., A.J., and N.R.; Data Curation, K.H., H.G., A.J., and N.R.; Writing – Original Draft Preparation, F.T.K.; Writing – Review & Editing, Z.G.; Visualization, F.T.K., K.H., H.G., A.J., and N.R; Supervision, Z.G.; Project Administration, Z.G.; Funding Acquisition, Z.G.