The Effect of Critical Distance in Digital Levelling

1. Introduction

The total levelling error is composed of elementary errors accumulated in each back-sight and foresight of a station. The main characteristics of accuracy respect this fact and involve the number of levelling stations, sighting distances and the measured height differences as main arguments in their formulas. The paper deals with a levelling error analysis to detect the partial errors that appeared as a sum of elementary errors cumulated on each station to differentiate errors dependent on the sighting distance. Levelling error analysis is based on the basic formulas recommended by the International Association of Geodesy (IAG), which have been subjected to new views on the propagation of random and systematic errors over time. The current error analysis methods respect digital technologies, which brought new errors and changed the ratio of random and systematic error propagation. The well-established method of finding the influence of elementary errors in observed levelling data consists of parameter estimation of a mathematical model created from the components of levelling errors, which are (in)dependent on the sighting distance. The model estimations vary in each station, but the discrepancy in their numerical order should be the same. If not, the suspicion of the influence of systematic error on a levelling station appears. Then the weighted standard deviations were computed for each meter of sighting distances named as sighting groups to find the relationship between the height differences dispersion and the sighting distance to find the possible influence of critical distance for the specific digital level. Critical distance is a well-known term between geodesists and metrologists who know that a high probability of obtaining inaccurate results appears at a certain sighting distance during digital levelling. The existence of this critical distance on specific digital levels has been confirmed by calibrating levelling devices at the vertical comparator. Having values of critical distances of Leica levelling devices published in [5], we can compare the results of field measurements with theoretical values.

2. An overview of levelling error theories

One of the first official theories concerning levelling error propagation was presented by Ch. Lallemand a French geophysicist and the first president of the International Union of Geodesy and Geophysics. He separated random and systematic influence in levelling measurements and defined international valid tolerances for comparing levelling accuracy in other countries (Bulletin géodésique 1930). In 1912, the Union accepted his theory and registered the official formulas for analysing levelling accuracy. Later, G. A. Rune tried to improve Lallemand equations by estimating levelling standard deviation as a quadratic difference between total and random error (Zeitschrift für Vermessungswesen 1930). The head of the French Institut Géographique National in Paris J. Vignal has brought a new view on systematic errors in levelling that influence measurements up to a certain length and after exceeding it, their variable influence is similar to the behaviour of random errors. After some objections from a member of the Finnish Academy of Sciences T. J. Kukkamäki, and J. Vignal modified his theories, which were subsequently accepted by the Geodetic Union in Oslo. J. Bohm and J. Sloboda summarised the existing theories of levelling error propagation in their book [2].

Over time, views on accuracy analysis and levelling error propagation have adapted to technological improvements in levelling instruments and their accessories. L. Sjöberg published a professional paper [10] concerning the influence of systematic and random errors on the precision of one-way levelling routes measured by motorised technique. M. Craymer published a thesis [3] devoted to the analysis of Swiss levelling measurements from the point of systematic effects. P. Vaníček and M. Craymer devoted their research to the rod settlement effect on levelling precision [4]. D. Zilkovski applied a “special adjustment” of levelling data based on using preliminary errors in evaluating levelling results that were published in [17]. Besides field measurements, technological progress and automation affected also the calibration process of levelling systems. Authors H. Woschitz and F. Brunner describe in [15,13,14] calibration facilities and outcomes from testing certain digital levels at the Laboratory of Geodetic Metrology of the Graz University of Technology (TUG). The conception of the TUG vertical comparator inspired the construction of the calibration system at the Stanford Linear Accelerator Center (SLAC). Authors Gassner and Ruland present in [5,6] the hardware and experimental results of the SLAC calibration system used to test levelling invar rods of the levelling devices Leica NA3003, DNA03, and Trimble DiNi12.

3. Basic formulas for an analysis of levelling accuracy

The accuracy analysis of levelling data consists of applying well-known formulas, which have been adapted to new levelling technologies to bring a new view on the representation of the random and systematic components. The basic formulas of classical accuracy theories are based on defining the sources of levelling errors, the representation of the components of random and systematic influences, and the theory of error propagation. Lallemand differentiated the terms random and systematic error in levelling measurement and estimated the variable systematic errors from the cumulative frequency of two levelling routes. The estimation process was based on a hypothesis that assumes an upward or downward tendency of linear regression, possibly caused due to the systematic effect. His theory of accuracy is based on the height differences in bidirectional levelling adjusted to the length of one kilometre of the levelling route. The reliable estimation of the accuracy of a one-kilometre-long levelling route is derived by using the law of error propagation of levelling errors. It is given by the quadratic mean of double levelling, known as unit standard deviation. Lallemand recalculated height differences of two-way levelling on kilometre differences by dividing by root squared of the quantity R, representing the length of the levelling route in km. Then the most used accuracy estimation of a one-kilometre-long levelling route follows from the law of error propagation and is given by the quadratic mean of double levelling. This accuracy characteristic is known as unit standard deviation σ₀ and is given by the formula [2]:

$${\sigma }_0=\frac{1}{2}\sqrt{\frac{1}{n_R}{\displaystyle \sum \frac{{\rho }^2}{R}}},$$

(1)

where ρ is the difference between height differences computed in each direct and reverse levelling section with length R given in kilometres. Over time, the restrictive conditions for the universal use of Lallemand formulas appear. His main opponent was Vignal, whose theory consisted of a determination of a limited length of the levelling route, within which unit standard deviation presented in formula (1) includes only the effect of random errors and short-periodical variable systematic ones. Other levelling lines that exceed the limited length contain also long-periodic systematic errors, which influence levelling observations with the same sign so they need not be eliminated by bidirectional levelling. The Vignal theory for accuracy analysis of precise levelling is based on the following statements:

• Lallemand's deviations of the cumulative height differences of bidirectional levelling are caused not only by systematic errors but also due to random noise.
• The influence of systematic errors defined by Lallemand is valid only within a certain distance (limited length) of the levelling route, beyond which they behave as variable systematic errors dispersed around the mean systematic error.
• Levelling variance is expressed as a root square of the total variance:

  $${\tau }^2={\eta }^2+{\xi }^2.$$

  (2)
  The quantity represents a random and systematic component with variable behaviour in the frame of limit length and beyond, it behaves as a constant. The systematic influence can be calculated according to the following equation [12]:

  $$\xi =\zeta \frac{K}{Z}{R}_m,$$

  (3)

  where K is a constant equal to 2 or 3, Z is the limit length and R_m is the mean distance of the levelling sections. The value ζ can be determined from the levelling closures or using differences in the endpoints of the regression line, estimated from the cumulative differences of the height difference of bidirectional levelling in the levelling sections.
  Assuming n stations in the levelling route, the total variance is equal to the following relationship [2,1,9]:

  $${\sigma }^2=n{\eta }^2+n^2{\xi }^2.$$

  (4)
  Random errors η in the levelling route increase in proportion to the number of stations, and the occurrence of systematic error ξ depends on the square of the number of stations. The behaviour of the random and systematic component varies in the frame of limit length and beyond it behaves as a constant. The systematic components can be determined from the levelling closures or using differences in the endpoints of the regression line, estimated from the cumulative height differences of bidirectional levelling.

4. Analysis of experimental data

The experimental part of the research involved field observations organised in eight cycles over two years. Digital levels of Leica NA3003 and Leica DNA03 have been used, each in four measurement cycles under various atmospheric conditions. Data intended for accuracy analysis were generated from the intermediate sights acquired on each station in two independent 500 m long levelling routes named K1 and K2. The levelling process was organised by the method of differential levelling using 10 benchmarks fixed on the concrete footings of power line poles. The particular points were stabilized by iron pins fixed to the concrete surface and the the sighting distance varied from cca 2 m to about 40 m. The data file consisted of the set of measured height differences obtained at the appropriate sighting distances. The amount of the measured intermediate sights was 1490 in locality K1 and 1509 in locality K2, whereby the number of intermediate sights from one station varied in dependence on the visibility to the particular point. The accuracy analysis adapted to this fact by involving weights in the error model and the apriori standard deviations.

4.1. Identification of the type of elementary errors

The process of finding the influence of critical sighting distance starts with the identification of the levelling errors depending on sighting distance by parameter estimation in the error model and testing the homogeneity of estimated variances by involving statistical hypothesis testing that can help validate the used mathematical model.

Böhm [2] divided levelling errors into four groups. The first group consists only of random influences with different sizes and signs at each station. These random errors can be divided into those that depend on the length of the levelling sight, e.g. reading error, error of spirit level, the inaccurate position of the instrument, etc., and those that depend on the number of stations in a levelling route, e.g. errors arising due to movement of an instrument or levelling staff or scale errors. The second group of levelling errors involves only systematic effects depending on the levelling route direction and is mainly influenced by the oscillation of the Earth’s crust. The influence of external conditions belongs to the third group of levelling errors such as the intensity of sunlight and the gravity of the Moon and Sun act systematically mainly long-period measurements. The fourth group of levelling errors depends on the measured height difference. A typical representative of this group is levelling refraction with its systematic effect [2,16].

The current levelling technologies have brought easier manipulation and a reduction of the subjective errors represented mainly by targetting and reading errors. On the other side, new sources of errors appear resulting from their coding and demodulation principles. Ingensand [8] classifies these errors into four groups according to their impact on levelling:

• Bad illumination caused by various intensities of natural light, inhomogeneous light intensity caused by shadows at the levelling bar.
• Atmospheric influences such as turbulences cause blurred image, and refraction, which causes deviation of the line of sight.
• Mechanical influences such as vibrations (deviation of the line of sight), settlement of the instrument and bar, and bar centring and inclination.
• Instrumental behaviour such as thermal effects (deviation of the line of sight), interference of code element size and pixels (wrong results at certain distances), and bad compensator function.

In addition to the instrumental errors, natural and personal errors appear during levelling such as curvature, refraction, ground settlement and instability, effects of heat on the instrument, parallax, staff out of plumb, etc. Böhm in [2] summarises information about systematic and random error propagation and creates an error model according to the relation of its arguments to the sighting distance:

$$\delta =a+b\sqrt{s}+cs+d{s}^2.$$

(5)

Components of the partial error δ represent the sum of elementary errors on a station, which are independent of the sighting distance (e.g. instrument movement), increases with the root of the sighting distance (e.g. targeting error), increases in proportion to the sighting distance (e.g. non-horizontal sightline), and grows with the square of the sighting distance (e.g. refraction error). Provided these components are random and uncorrelated, the partial variance of a levelling sight is equal to the mean squared error value calculated by the law of error propagation as follows:

$$E({\delta }^2)=a^2+b^{2}s+c^2{s}^2+d^2{s}^4.$$

(6)

In case of the existence of mathematical or physical correlation of the error components, it is recommended to use the general law of propagation of levelling errors:

$$E({\delta }^2)={\displaystyle \sum _{i=1}^4{f}_i^2{\sigma }_i^2}+2{\displaystyle \sum _{i=1}^4{\displaystyle \sum _{j=1}^4{f}_i{f}_j{\sigma }_{ij}}}.$$

(7)

The difference between both formulas (6) and (7) is in the addition of the component with mixed covariances σ_ij generally considered as a measure of the linear relationship of the relevant error components and the corresponding differential components f_i, f_j. In practice, the formulas (6) and (7) are simplified to the following form:

$${\sigma }_{\Delta H}^2={\sigma }_0^{2}R,$$

(8)

where σ_ΔH² represents the accuracy of measured height difference while unit variance σ₀² is computed from Lallemand’s formula (1) and R is the length of a levelling section. If the length of the levelling route is only a few kilometres long, the square root of the variance (8) gives a reliable estimation of the levelling precision.

The estimation process consists of finding out the representation of the types of errors, which can numerically vary from sight to sight, but the discrepancy in their numerical order may indicate a possible occurrence of measurement inhomogeneity caused by internal or external effects. The arguments of the error model (5) were estimated in each observation cycle by applying the least-squares method in the nonlinear regression. The solution of the estimation process consists of a vector of unknown parameters and an appropriate covariance matrix:

$$dx={\left(A^{T}PA\right)}^{-1}{A}^{T}Pl,$$

(9)

with A as the design matrix, l as a vector of input values, and P as a symmetrical weighted matrix with the particular weights laying on the diagonal of the matrix and equal to the number of measured height differences in a group. The covariance matrix was estimated from the Gauss-Markov model by using the law of error propagation

$$D(x)={\sigma }_0^{2}Q\left(x\right)={\sigma }_0^2{\left(A^{T}PA\right)}^{-1},$$

(10)

where σ₀² is an a-posteriori unit variance, and Q(x) is a cofactor matrix of unknown estimations. The estimated coefficients of the error model are shown in Table 1. The discrepancy in the numerical order of estimated parameters led to the verifying and validation of the model by method of variance analysis.

The variances estimated in each sighting group were subjected to statistical hypothesis testing to validate the mathematical model. The null hypothesis assumes the homogeneity of measurements expressed by the equality of the variances. Cochran variance test was used for this purpose to verify the tested value C computed as the ratio between the largest variance and the sum of variances according to the equation:

$$C=\frac{{\sigma }_{\max }^2}{{\displaystyle \sum _{i=1}^N{\sigma }_i^2}}$$

(11)

The critical value C_u was computed for the significance level α=0.05, number of data series N and the number of sighting groups n, according to the formula:

$$C_u={\left(1+\frac{N-1}{F_{\alpha }\left(\alpha /N,(N-1),(n-1)\right)}\right)}^{-1}$$

(12)

Symbol F_α means the quantile of F-distribution with three arguments α /N and the degrees of freedom of used datafiles (N - 1) and sighting groups (n - 1) in a file. The Cochran test is used to identify an outlier file with a variance for which test value C exceeds a critical value C > C_u. The arguments of the hypothesis testing are published in Table 2 and point to confirmation of the null hypothesis, which proves the homogeneity of tested data files and refers to verifying the used mathematical model.

4.2. Detection of the influence of critical sighting distance

Calibrations of certain levelling devices done on a vertical comparator are described in [5,6,13,14,15]. They have brought results concerning the scale determination, detection of the possible errors of height deviations measured in the end sections of the staff, determination of the influence of damaged code elements on the height readings, and the identification of the critical sighting distance of specific digital levels. The scale determination is based on using two separate runs of the levelling staff to detect and compare the edges of all code elements of the staff. The scale value of staff is then determined from a linear regression model. The second published calibration procedure is based on finding the height deviations, which rise by sighting at the end sections of levelling code staffs. It starts with defining the useable area of the staff, which varies from about 2.80 - 2.98 m by 3 m long staff, of both Leica and Trimble systems. According to Woschitz [15], the height reading beyond the useable staff area might be wrong by more than 0.5 mm at the sighting distance of 30 m. The reason might be in an asymmetric pixel image on both ends of a staff, and in the refraction effect, which appears mainly by sighting the lower parts of the levelling staff. Experimental measurements at the Stanford SLAC have brought the results, published by Gassner et al. in [5], which contain formulas for computing the ideal sighting section on the end of the levelling staff to avoid corrupted height differences for Leica DNA03. For the lower end, Gassner suggests the formula:

H_lower [mm]=visible code on the staff+20+7*sighting distance [m]

(13)

and for the upper end of the levelling staff:

H_upper [mm]=visible code on the staff-20-7*sighting distance [m],

(14)

where H_lower and H_upper are the ideal sighting sections on the lower and upper ends of levelling staff equal to 10 millimetres at a sighting distance of 10 meters.

Both equations (13) and (14) are valid for sighting distances up to 15 m. If the sighting distance is less than 3 m, the correct results are only possible in a usable area of 0.078 m – 1.899 m using a two-meter-long levelling staff. The third published calibration procedure concerns damaged code elements' influence on the height readings. Reference measurements realised at the TUG have brought knowledge concerning errors arising from damaged code elements, which depend on the used correlation function, which varies for the different levels. Investigation results of the critical sighting distance obtained on the SLAC vertical comparator described in [5] confirmed the assumptions of Woschitz and Brunner‘s [13] that define the critical distance of Leica NA3000, which is at 15 m. According to the obtained results, the critical distance occurs when the size of code lines, projected onto the CCD array is exactly the size of one pixel or if a multiple of code lines is mapped to a whole number of pixels. It means that for Leica DNA03 one code element of the size 2.025 mm is projected onto the CCD array with the size of one pixel at a distance of 26.7 m and for Trimble DiNi12 the code element with a width of 20 mm corresponds to the critical distance of 10.98 m or its multiply. The described calibration outputs inspire our research based on experimental measurements to find the influence of critical sighting distance of digital levels NA3003 and DNA03. This influence was searched by comparing height differences obtained by certain sighting distances, which were arranged into sighting groups with computed standard deviations, which represent a measure of dispersions of height differences. Standard deviations twice greater than the total standard deviation point out suspicious measurement dispersion, which indicates the existence of gross error possibly caused by critical distance influence. Table 3 shows the measured height differences arranged into sighting groups and the appropriate standard deviations that point to their dispersion in the frame of a sighting group. The total value of standard deviation was computed in the reference system separately for each digital level, and each experimental locality and is displayed at the end of Table 3. The mean standard deviations are graphically displayed separately for the levelling device Leica NA3003 in Figure 1 and for the DNA03 in Figure 2.

7. Discussion

The critical sighting distance of specific digital levels was first demonstrated on a vertical comparator. The experiment's ambition was to determine the influence of the critical distance on the levelling precision by measuring height differences of fixed points from different sighting distances with digital levels Leica NA3000 and Leica DNA03. For this purpose, the standard deviations were calculated in each sighting group created around the average sighting distance and were compared with the total standard deviation that represents tolerance. The double-increased standard deviation appeared in the same sighting group in independent observational localities K1 and K2 can show the probable influence of critical sighting distance. Results in Table 3 and Figure 1 and Figure 2 show the increased standard deviations for Leica NA3003 around the sighting distances 12 m, 22 m, 27 m, and 30 m. Twice higher standard deviations of level Leica DNA03 were at distances 21 m, 29-30 m, and 32 m while the values with sighting distances over 35 meters were not involved in the analysis because of the increased dispersion of measured values (see Figure 1 and Figure 2). The sighting distances with increased values of standard deviations of both digital levels do not correspond to critical ones as published by Gassner and Woschitz. However, the experimental results require further investigation. The uncertainties corresponding to detected sighting distances possibly showed a different systematic source or critical sighting distance varies from instrument to instrument or the research methodology is necessary to revise.

The second part of the experiment consisted of an estimation of particular errors on a station represented by error model arguments to find the ratio between dependent and independent errors on a sighting distance. Though, the representation of the levelling errors varied from station to station the common characteristic in their numerical order can be found in the whole levelling route in both experimental localities. Results from the estimation of the mathematical model (Table 1) that was validated by analysis of variances confirmed the expectations. Digital level NA3003 has about 49.9 % of levelling errors that are independent of sighting distance and DNA03 has about 65 %. Error arguments c and d have insignificant importance.

8. Conclusion

The calibration tests of digital levels Leica NA3003 and DNA03 published by Gassner and Woschitz brought the concrete values of critical sighting distances, which occur when the size of code lines responds to the size of one pixel of the CCD camera or his multiply. The described experiment that consisted of a lot of intermediate sights measured in two independent levelling routes did not confirm the influence of critical distance 15 m of level NA3003 and 26.7 m of level DNA03 on the precision of measured height differences. However, the used digital levels showed systematic influence in other sighting distances that is assumed to be verified by independent methodology in future.