Precision, Inter-rater Reliability, and Accuracy of a Hand-Held Scanner Equipped with a LiDAR Sensor in Measuring Parts of the Body. A Preliminary Validation Study

1. Introduction

Anthropometric measurements play a crucial role in medico-legal activities, commonly conducted during forensic investigations. They serve multiple purposes, including identifying individuals or corpses [1,2,3,4], aiding in cranio-facial reconstructions [5], and providing essential data for assessing traumatic injuries and determining their most probable means of production [6]. The precision and accuracy of these measurements are of utmost importance due to their forensic impact.

To date, anthropometric measurements have been carried out manually using physical metersticks or measuring tapes [7]. However, more recently, digital devices such as three-dimensional computer-aided design (3D/CAD) supported photogrammetry [8] and structured-light 3D scanners [9,10] have been utilized, albeit less frequently due to their lack of portability and high cost in terms of both equipment and training required for personnel.

Over the past few years, the availability of measuring instruments has significantly expanded. Specifically, numerous portable scanners and image acquisition tools, termed hand-held scanners, have been developed for widely-used devices like smartphones and tablets. A recent study [11] tested an Apple® tablet equipped with a LiDAR sensor, generating three-dimensional (3D) reconstructions of ten bodies directly within the autopsy room. The study concluded that this new hand-held scanner could serve as a valuable tool in the forensic field for documenting autopsy findings and capturing high-quality 3D images. However, there is a lack of studies comparing the reliability of such devices to the current medico-legal "gold standard" (i.e., manual measurements).

Therefore, the current study aimed to assess the precision, inter-rater reliability, and accuracy of an iPad Pro equipped with a LiDAR sensor for measuring several parts of the body.

2. Materials and Methods

2.1. Study design

The study was conducted in accordance with local ethical rules and obtained permission from the Public Prosecutor to perform measurements on a body undergoing forensic autopsy at an Institute of Legal Medicine in the North-East region of Italy.

A highly experienced forensic pathologist (GC), with 20 years of expertise, conducted ten sets of manual anthropometric measurements. These measurements comprised 7 longitudinal distances (Figure 1) using a laser level and a meterstick. The same operator utilized the iPad Pro (details provided below) to scan the body ten times, with an average scanning time of approximately 20 seconds per scan. For each body scan and anthropometric measure tested, three independent blind raters conducted digital measurements ten times. These repetitions were performed on different days, with a minimum interval of 24 hours between each session. The blind raters consisted of a forensic pathologist with 15 years of experience (GV), a forensic pathologist specializing in anthropology (PF), and a resident in Forensic Medicine (EC). EC was specifically trained by the senior forensic pathologist (GC) in utilizing the MeshLab software—an open-source tool developed by the Institute of Information Science and Technologies, Consiglio Nazionale delle Ricerche, Rome, Italy. In total, 700 digital measurements (7 distances tested on 10 different scans, each measured ten times) were conducted.

Subsequently, the following parameters were calculated and assessed [12]:

• the precision or intra-rater variability, defined as the degree to which repeated measures performed in different moments produce similar results, was tested comparing, for each anthropometric measure, the measures performed across the ten days;
• the inter-rater reliability, defined as the degree of concordance among different raters who analyse the same parameter independently of each other, was tested by comparing, for each anthropometric measure, the measures performed across the ten days by the 3 involved raters and by comparing the median values obtained by each rater for all the anthropometric measures using the intraclass correlation coefficient (ICC).
• accuracy of the handheld scanner, indicating the closeness between the true value and the value measured with the investigated technique, was evaluated by determining the percentage error between the real (manual) measurement and the median of each digital distance measured by the three raters involved (resulting in 100 measurements for each anthropometric measure).

2.2. Manual measurements

The senior pathologist (GC) conducted the following steps with the body positioned supine on the evisceration table in the autopsy room:

• Identified and marked 6 anthropometric landmarks on the body using an "X" (Figure 1);
• Measured 7 longitudinal distances using a 2-meter rigid meter placed beside the corpse, aligned parallel to its main axis. Additionally, a laser level (Laser Level mode CM-701, Cigman, Essen, Germany) was used to facilitate the orthogonal transposition of the specifically marked landmarks. Each longitudinal distance was measured ten times.

2.3. Body scan

An iPad Pro (2020 - Apple Inc., Cupertino), an 11-inch tablet operating on the iPadOS mobile system was utilized in the study. It was equipped with a LiDAR ("Light Detection and Ranging") sensor, a pulsed laser that records the time taken (at nanosecond speeds) for the signal to return to its source, allowing the generation of 3D reconstructions. The chosen software was Polycam (Polycam Inc. Altadena, California, USA) a paid application designed to create high-quality 3D models from photos and swiftly generate scans of spaces or objects using the LiDAR sensor.

The ten scans were conducted by a forensic pathologist with 20 years of experience (GC). The device was positioned approximately 1.5 meters above the body, and the scan was performed along the midline of the corpse from head to toe. Minor lateral movements were made to ensure better scanning of the body's lateral surfaces. Each scan was processed using the "object" processing option and saved as a .obj file.

2.4. Body scan

The data stored on the hand-held scanner were transferred to a PC through wireless transfer options. Subsequently, utilizing the MeshLab software, the three aforementioned blind raters conducted measurements for each of the seven selected distances ten times over ten different days (one measurement per day).

By selecting "draw XYZ axes in world coordinates," a Cartesian system became visible on the scan (see Figure 2a-b). The "manipulator" tool was used to position the scanned body parallel to the X-axis and perpendicular to the Z-axis (the Z-axis was tangential to the toes).

The “reference scene tool” allowed to select two points, the coordinates of which were identified on the X-axis and Z-axis (Figure 2c-d), in order to calculate the distance between the two landmarks. The measurements were conducted by transposing the coordinates of the two landmarks specifically along the X-axis.

2.5. Statistical analyses

The data were collected in an Excel® database (Microsoft Corporation®, version 2309 Build 16.0.16827.20166) and analyzed using SPSS Statistics® (IBM Corp, version 26.0.0.0). Images were performed using Prism (GraphPad Software, LLC, version 10.0.0.3).

Non-parametric statistics were applied, since a parametric distribution for each set of data could not be proved by scatterplot. Anthropometric measurements obtained with the laser level over the 10-day period were described by median and interquartile range. To compare these measurements, a non-parametric one-way ANOVA for matched or repeated measures (Friedman test) was utilized.

For assessing precision or intra-rater variability, both analog measurements acquired during the autopsy and digital measurements obtained by each rater were compared. The non-parametric one-way ANOVA for matched or repeated measures (Friedman's test) The non-parametric Friedman's test was applied, considering a difference statistically significant with a p-value < 0.05.

Median values of measurements obtained by each operator were utilized for subsequent statistical analyses. Friedman test with multiple comparisons and the intraclass correlation coefficient (ICC) were used to evaluate the inter-rater reliability on the median distances measured by the 3 raters, where the Friedman test is performed separately for each anthropometric measure, while the ICC on the whole set. For the ICC, a two-way mixed model was set, which accounts for “fixed” raters and variable measurements, and by checking for absolute agreement. Based on the 95% confident interval of the ICC estimate, values less than 0.500, between 0.500 and 0.750, between 0.750 and 0.900, and greater than 0.900 were considered indicative of poor, moderate, good, and excellent reliability, respectively [13].

Finally, data from digital measurements were compared to the gold standard value (median of manual measurements) to calculate the percentage of error, evaluating accuracy. Accuracy was assessed for each rater in terms of maximum and median percent error across 10 measurements. Additionally, the median accuracy among the three raters was computed, considering percent errors below 3% as excellent.

3. Results

The results are reported in Table 1 and Table 2.

Measurements obtained with the laser level exhibited negligible intra-rater variability (p=0.0538).

Concerning digital measurements, the Friedman’s test did not find any statistically significant difference among repeated measures performed by the same operator (intra-rater variability), except for the anthropometric measure E (distance between heel and chin) for operator 3 (p=0.0136) (Table 1).

Assessment of inter-rater variability via Friedman's test with multiple comparisons highlighted statistically significant differences in anthropometric measure A (heel-knee, Rater 1 vs Rater 2; p=0.0110), anthropometric measure E (heel-chin, Rater 1 vs Rater 3; p=0.0001) and anthropometric measure F (heel-nasion, Rater 2 vs Rater 3; p=0.0010). A graphical representation is shown in Figure 3. For the other anthropometric measures, inter-rater differences were consistently non-significant (p>0.05). The level of agreement among the measurements, expressed with the ICC, was 1.00, demonstrating an excellent correlation among the measurements across all operators (Table 1).

Median percentage error remained under 1% for all measurements (Table 2, Figure 4). Among the maximum percent errors, anthropometric measures A and B displayed the highest values, and Rater 2 and Rater 3 exhibited slightly lower accuracy compared to Rater 1, although consistently maintaining excellent accuracies.

4. Discussion

In the forensic field, anthropometric measures of living or deceased individuals may play an essential role in multiple situations, such as personal identification (e.g., sex or age estimation) [3,5], reconstruction of shooting crimes [14,15,16], road accidents [17,18], precipitations [19] or any other traumatic injuries [20]. It is strongly recommended to conduct these measurements during forensic autopsies [21,22] and, if feasible, during death scene investigations [23]. Currently, manual measurements using metersticks and callipers represent the “gold standard” in forensic investigations [7].

In this study, the employment of a laser level allowed to orthogonally transpose the body landmarks on the rigid meterstick, granting the measurements to be detected on the same longitudinal axis (Figure 1); This was necessary because manual measurements conducted between two body landmarks can lack precision, as these landmarks might be situated at different levels of the body. The measurement performed matching the meter stick and the laser level exhibited statistically insignificant intra-rater variability.

There are several scanning technologies, which allow the identification and determination of anthropometric parameters. Computed Tomography (CT) is commonly employed in post-mortem investigations [24,25,26,27], and its use for forensic anthropological measurements has recently been tested [28]. Structured Light 3D Scanners are relatively new technologies, mostly employed in clinical settings. It is noticeable that there are plenty of technologies that can produce 3D scans of objects and bodies, many of which being commonly employed during forensic autopsies, above all in Switzerland [10]. However, the above-mentioned technologies are expensive, need specific personnel training, and are of large dimensions (not portable and therefore not usable at DSI).

In the last few years, smaller, portable handheld scanners, such as smartphones and tablets, capable of performing 3D scans of objects and spaces, have been developed and commercialized. The LiDAR technology [29] employs a laser system which can calculate the distance from objects by calculating the time running from the emission of the laser impulse and the reception of the same signal going back to the sensor of the device. This technology is widely employed in seismology, archaeology, geology, architecture, and meteorology.

Recently, Fineschi et al. [11] used and iPad Pro equipped with a LiDAR sensor to elaborate 3D models obtained during ten autopsies in order to evaluate the quality and trustworthiness of these models compared to conventional autopsy photographic records.

In this study, the same type of tablet, paired with the Polycam app, was tested for accuracy and precision in anthropometric measurements compared to the manual measurements performed with a laser level and meterstick.

For digital measurements the Friedman’s test revealed negligible intra-rater variability in digital measurements, except for one instance (E) with operator 3 (a resident in forensic medicine). However, among the measurements conducted by the senior pathologist, no significant differences were observed. These results seem to suggest a lower intra-rater variability for trained operators.

Concerning the inter-rater variability, some anthropometric measures, namely A (heel-knee), E (heel-chin), and F (heel-nasion), exhibited grater inter-rater variability than others. Since the anthropometric landmarks had been previously marked using an “X”, it is unlikely that the disagreements introduced into the measurement process are due to the lack of recognized reference points on the body. Notably, experience in forensic pathology seemed to influence precision, as experienced raters exhibited fewer discrepancies. Despite slight differences among raters, overall agreement remained excellent (ICC = 1), indicating high concordance among the 3 independent blind raters.

Finally, the accuracy of digital measurements showed errors below 2%, with smaller anthropometric measures displaying slightly higher inaccuracies. The greatest inaccuracies were observed in the shortest anthropometric measures (A and B), stemming from errors primarily within a range of a few millimeters. This observation is significant as it highlights that smaller distances entail proportionally more substantial errors. Consequently, it suggests that the method examined here might be less suitable for documenting small injuries or short anthropometric measures.

The preliminary study concluded that LiDAR technology paired with the Polycam app demonstrates high precision, inter-rater agreement, and accuracy, showcasing potential in forensic applications.

As previously indicated [11], optimizing the interface between hardware and software is crucial to achieve optimal performance with the LiDAR sensor. Scans acquired using such type of technology and the Polycam App generated three-dimensional models comprising polygonal meshes, with 3D reconstructions textured faithfully. This fidelity was due to the seamless integration of the camera system, applying photographic textures onto the mesh. Additionally, while conducting scans with the LiDAR system, the senior pathologist (GC) discovered its capability to move across various planes and rotate or oscillate, facilitating the scanning of lateral surfaces of the body without disrupting the scan or texture.

The current study also has certain limitations. First of all, it relies on the senior pathologist, who conducted the 10 body scans and marked the anthropometric landmarks before the raters performed digital measurements. In a real forensic context, if individual raters were tasked with scanning the body and selecting anthropometric landmarks, we might expect greater inter-rater variability and reduced accuracy. Under these circumstances, we anticipate that the raters’ experience in forensic pathology would significantly influence the outcomes (accuracy and precision). Secondly, our study solely considered relatively long distances (always > 40 cm), while the measurement of smaller anthropometric measures might be necessary in forensic pathology to characterize traumatic injuries and their location with respect to landmarks. Lastly, the raters involved had no prior training with Polycam and MeshLab software. However, it is expected that their performance would improve with familiarity and experience gained over a learning period.

More studies on a real forensic setting, especially investigating the impact of a training period and of the forensic experience of different raters (with and without a period of specific training), are needed, in order to assess the true reliability of the proposed method based on a handheld scanner equipped with a LiDAR sensor.

In conclusion, handheld scanners offer promise in forensic anthropology due to their simplicity, affordability, and portability. Developing dedicated scan applications and measurement software tailored for medico-legal purposes (i.e., optimized for small curved objects or small body parts, such as fingers, hands, nose and ears) could significantly enhance their effectiveness in forensic investigations.

Further validation studies under real-world conditions (scanners (e.g., light conditions, dirt skin, absence of landmarks, etc) are necessary to fully establish the reliability and efficiency of handheld scanners in forensic settings. Additionally, optimizing hardware-software interfaces and developing dedicated apps for anatomical measurements are essential steps toward improving their forensic utility.