Indeterminacy of Camera Intrinsic Parameters in Structure from Motion Using Images from Constant-Pitch Flight Design

1. Introduction

Unmanned aerial vehicle (UAV) - based photogrammetry has emerged recently as an effective technique to generate high-resolution three-dimensional (3D) models across multiple fields including infrastructure inspection [1,2,3], mining surveying [4,5,6], hazard modeling [7,8,9] and environmental condition monitoring [10,11]. Structure from Motion (SfM) underpins this technique, estimating camera intrinsic and extrinsic parameters. These parameters are then exploited in the Multi-View Stereo (MVS) to produce dense point clouds, enabling the generation of high-resolution deliverables, such as meshes, digital elevation models (DEMs), ortho-photos. Therefore, the precision of camera parameters estimation in SfM has a direct effect on the accuracy and precision of reconstructed models. While the accurate estimation of these parameters relies hardly on the number and distribution of auxiliary information such as Ground Control Points (GCPs), it is cost-intensive and time-consuming. Ensuring accurate parameter estimation without heavy reliance on GCPs remains a primary concern for UAV-based photogrammetry.

Intrinsic parameters - such as focal length (f), principal point coordinates (cx,cy), radial distortion, tangential distortion - can be estimated during the bundle adjustment (BA) stage in a typical SfM pipeline (self-calibration), or alternatively, calculated using imagery captured specially for calibration purposes (pre-calibration). However, in UAV-based photogrammetry, where lightweight and compact drones with small cameras are commonly used, pre-calibrated intrinsic parameters are often considered unstable [12,13,14,15]. This instability can arise from mechanical effects during camera shooting, changes in image sensor temperature [16,17,18] or ambient temperature [19] during drone operations, potentially affecting the sensor geometry. Therefore, self-calibration has become the preferred approach for estimating these intrinsic parameters. By eliminating GCP surveys and dedicated calibration process, self-calibration using image-based SfM significantly reduces fieldwork time and cost, enabling rapid, cost-effective surveys even in remote or hazardous areas.

A widely employed image acquisition method in UAV-based photogrammetry is the constant-pitch (CP) flight [18,20,21,22,23,24], where the optical axes of images within the same strip are parallel. In this flight design (Figure 1), the camera is tilted at a fixed pitch angle along the flight lines, and when drones fly in the opposing direction, the local convergence is achieved, thereby facilitating 3D reconstruction.

SfM using CP-based images has been reported to perform well when the camera positions measured by RTK-GNSS are incorporated in BA [21,23]. However, when camera position measurements are unavailable, referred to as images-based SfM, the performance of SfM becomes notably unstable. Sanz-Ablanedo et al. [18] reported poor images-based SfM performance using CP flight images captured at 25^o pitch angle, highlighting non-linear deformation (dome effect). Their study also noted large mean vertical errors at validation points located at the four corners of the model, which can be caused by inaccuracy in the estimation of focal length (f). Similarly, Kon et al. [25] reported dome errors accompanied with large RMS errors of all validation points, and in some cases, alignment failures, across various SfM analysis settings (1500 trials) for images captured by CP flights at 20^o pitch angle. For these 1500 SfM trials, the standard deviation of f and cy were notably large, 209 pixels and 84 pixels for camera Phantom 4 RTK drone, and 53 pixels and 36 pixels for camera SONY α7 III drone, respectively. Observations of large mean vertical errors at validation points and alignment failures, coupled with the instability in estimations of f and cy, suggest that factors beyond the dome effect may contribute to poor performance of images-based SfM. Despite these observations, the root causes of this instability have not been thoroughly investigated.

It has been mathematically shown that certain combinations of camera motion and scene geometry, known as critical configurations, can introduce indeterminacy into the bundle adjustment with self-calibration [26]. We suspected that the instability of camera parameters observed in images-based SfM under CP flights [21,23,27] stems from such critical configurations, leading to inaccurate or even incorrect model reconstructions.

Therefore, the objectives of this study are: 1) investigate the indeterminacy of certain intrinsic parameters, particularly focal length (f) and principal point vertical coordinate (cy) in images-based SfM under CP flight through numerical experiments under ideal conditions; (2) validate these numerical findings using real datasets acquired at three different ground sampling distances (GSDs); and (3) discuss a practical mitigation strategy to stabilize intrinsic parameter estimation. By bridging the theoretical–practical gap, we aim to provide reliable a UAV-SfM workflow without auxiliary information.

2. Materials and Methods

2.1. Numerical Experiments

The indeterminacy of f and cy was investigated by analyzing RMS reprojection error, which is the objective function (the function to be minimized) at BA stage in SfM. This was achieved by fixing either f or cy to incorrect values in BA and estimating the other parameters along with extrinsic parameters and tie points. If RMS reprojection error remains nearly unchanged regardless of the input values of f or cy, it indicates that these parameters do not influence the objective function, thereby demonstrating their indeterminacy in SfM.

a) Synthetic image acquisition

We used Blender version 2.79b [28], an open-source source computer graphic (CG) software to generate a flat plane (20,000 x 20,000 pixels) and randomly place multiple rectangular parallelepiped objects of various size on it. The textures applied to the plane and objects included local feature suitable for matching in SfM, as shown in Figure 2.

Synthetic image acquisition of the target plane was performed using the image and camera specifications described in Table 1. In CG space, three image-sets were captured using three different flights: CP, CP random and CP-Plus flight. The CP- Plus flight was derived from constant-pitch (CP) flight by adding one image between every two flight lines. The pitch angle was set to 20^o for all three flight designs and the flight altitude was fixed at 73 meters to achieve the Ground Sample Distance (GSD) of 20 mm. Details of flight designs are provided in Table 2.

b) SfM processing

For each simulated image-set created in CG space, SfM was performed using the commercial software Agisoft Metashape version 1.5.5. The SfM workflow (Figure 3) began with “align photos”, followed by bundle adjustment (BA) with the camera parameters effectively fixed to their true values to generate accurate set of tie points. The settings used are detailed in Table 3.

Based on these tie points, a camera intrinsic parameter (either f or cy) was fixed to values higher than true value (v₊) and values lower than true values (v_-), while the other parameter (cy or f) was not fixed. Then, BA was rerun. In this process, BA was performed using the previously generated accurate tie points. This approach allowed us to observe the behavior of RMS reprojection error in relation to the fixed intrinsic parameters (f or cy).

c) Evaluation of the indeterminacy of camera intrinsic parameters in SfM

The variation in RMS reprojection error between BA with fixed true intrinsic parameters (f and cy) and BA with fixed incorrect intrinsic parameters (f or cy) was quantified by the rate of increase in RMS reprojection error. This metric effectively evaluates the impact of inaccurate intrinsic parameters on SfM and helps identify occurrences of their indeterminacy. Additionally, fixing either f or cy to incorrect values while estimating the remaining parameter (cy or f, respectively) provides insight into the correlation between f and cy.

$$R_{increase}={\displaystyle \frac{E_{incorrect\mathrm{ }}-\mathrm{ }{E}_{true\mathrm{ }}}{E_{true\mathrm{ }}}}$$

(1)

Where: R _increase is the rate of increase in RMS reprojection error.

E _incorrect (pixels) is RMS reprojection error when fixing either f or cy to the incorrect values and estimating the remaining parameter in BA.

E _true (pixels) is RMS reprojection error when fixing both f and cy to true values (1824 and 0 pixels, respectively) in BA.

2.2. Real-Data Analysis

a) Study site

The in-situ data in this study utilized the data from Truc et al. [23]. These images captured an area of around 100 x 100 m², located in Tsukubamirai city, Ibaraki prefecture, Japan. This area was constructed for housing purposes. Figure 4 shows the ground condition of target terrain, mostly bare land covered with short vegetation.

b) Image acquisition

Aerial photos were collected using a high-precision 1-inch 20 MP CMOS camera sensor attached to a lightweight quad-copter UAV (DJI Phantom 4 RTK, hereafter named P4RTK) [29]. The assembled camera lens has an approximate focal length of 8.8 mm and a mechanical shutter feature that avoids typical rolling shutter problems. This UAV can measure the camera position at the time of exposure by RTK-GNSS (real-time kinematic global navigation satellite system) positioning. A set of nine validation points (Figure 4) was employed to evaluate the accuracy of the SfM performance using the Aerobo marker [30].

A flight design similar to CP-Plus flight design was adopted for data acquisition. This flight pattern was generated using the 3D photogrammetry preset feature in the P4RTK system to capture images at two-second intervals and record camera coordinates via RTK-GNSS during image capture. In these CP-Plus flights, the camera was tilted at a fixed pitch angle of 20^o along the flight lines and the consecutive images with yaw angle deviation over 10^o from the main flight line, taken during the drone rotation, were referred to as the intermediate short strips. They were removed to simulate three CP flights (Figure 5).

In this study, P4RTK was flown at three altitudes of 73, 55, and 36 m above ground level (at the drone’s home point), which yielded three GSDs of 20, 15, and 10 mm, respectively. All photographs were captured along parallel flight lines with an overlap ratio of 80% forward and 60% side lap when the camera’s optical axis was oriented toward nadir. The details of image-sets used in the real-data analysis are described in Table 4.

c) SfM Processing

For each image set, SfM was performed following the “align photos” in Agisoft Metashape version 1.5.5 software. In this process, camera parameters were estimated by using images only, then the point clouds were georeferenced using projection centers recorded by RTK-GNSS embedded in the images taken by P4RTK. The absolute coordinates of the validation points evaluated by multi-view triangulation using camera parameters estimated in SfM were used to validate the accuracy of the SfM-derived model, in accordance with the nine validation points. It is worth to note that in this study self-calibration was performed using images only, without additional information.

SfM was tested through a combination of SfM analysis settings summarized in Table 5, resulting in 30 settings with 50 trials for each image-set listed in Table 4. This analysis was conducted to observe the variation in the estimation of intrinsic parameters under different SfM settings.

In order to evaluate the variation of intrinsic parameters estimates in one SfM setting, we used a standard SfM setting repeated in 50 times (trials), with the following parameters:

• Size of input images for feature detection: High
• Keypoint limit: a half (½ ) to the average number of detectable keypoints for the input images (35643).
• Intrinsic parameters considered in Brown models: f, cx, cy, k1-k3, p1-p2.

d) Intrinsic parameters estimation and error evaluation

In this study, we quantified the variation of intrinsic parameters estimation (f and cy), in accordance with the accuracy and precision of SfM for each image-set. The accuracy and precision of SfM is evaluated by the root mean square (RMS) error of all nine validation points for each analysis setting, as recommended by James et al. [31]. The validation point error represents the difference between the estimated 3D coordinates of the validation points (calculated through triangulation using camera parameters estimated during SfM process) and those obtained through static GNSS positioning by Aerobo markers. Additionally, we calculate statistics of RMS errors of all validation points for one standard setting (with 50 trials) and 30 settings (with 50 trials for each setting) such as: the maximum value, 75^th percentile (Q3), 50^th percentile (Q2) and interquartile range (IQR).

Moreover, we evaluate the mean vertical error of all nine validation points for each analysis setting. The mean vertical error is defined as the average difference between the estimated elevation of the validation points (derived using camera parameters estimated by SfM) and the measured elevation of corresponding validation points obtained by Aerobo markers.

3. Results

The results are divided into two main sections: numerical experiments and real-data analysis. In the numerical experiments, we present verification of the indeterminacy of intrinsic parameters in image-based SfM for three flight designs, and observation of the RMS reprojection error for each camera in CP-Plus flight. The real data analysis focuses on three points: the estimates of camera intrinsic parameters (f, cy) across multiple repetitions of a single SfM setting and in various SfM settings, and the correlation between the estimates of f and the mean vertical error for all validation points.

3.1. Numerical Experiments

3.1.1. Verification of the Indeterminacy of Intrinsic Parameters in Image-Based SfM for Three Flights

The RMS reprojection error and its rate of increase were analyzed to evaluate the sensitivity of SfM when the intrinsic parameters were fixed to inaccurate values relative to their true values (Table 6, Table 7).

As shown in Table 7, the rate of increase in RMS reprojection error in CP flight and CP random flight was 0, while for CP-Plus flight, it ranged from 5.8 to 7.6. In CP and CP random flight, the RMS reprojection error remained unchanged regardless of values of intrinsic parameters f or cy fixed in BA. However, in CP-Plus flight, the addition of one image between two flight lines results in a significant increase in RMS reprojection error. Moreover, when f was fixed to incorrect values, the estimates of cy derived from BA were strongly inaccurate compared to the true value, and vice versa (Table 6).

These results indicate that both f and cy are indeterminate in image-based SfM. In CP random flight, where camera positions were randomly distributed compared to CP flights, and the camera orientation remained fixed at 20^o forward and backward, RMS reprojection error does not change, whether intrinsic parameters are fixed to incorrect values or the true values. This indicates that altering the shooting positions alone does not resolve the indeterminacy.

In contrast, for CP-Plus flight, the values of f or cy had a substantial impact on RMS reprojection error, with rates of increase in RMS reprojection error exceeding 7 for f and 5.8 for cy (Table 7). This demonstrates that f and cy are no longer indeterminate in SfM when at least one image is added to intermediate short strip.

3.1.2. Observation of the Variation of RMS Reprojection Error for Each Camera in CP-Plus Flight

The CP-Plus flight appears to overcome indeterminacy of f and cy in images-based SfM. To better understand this phenomenon, we analyzed the RMS reprojection error for each camera when the intrinsic parameters f or cy were fixed at incorrect values in SfM.

As shown in Figure 6, the RMS reprojection error for cameras in the intermediate short strips is significantly larger than those in the main flight strips. Additionally, cameras positioned near the intermediate short strips or with a high overlap to images from these strips also exhibit higher RMS reprojection errors. This finding indicates that tiepoints in these images geometrically contradicts with wrong camera parameters, even after BA, thereby preventing the convergence of camera parameters to wrong values in usual BA when f and cy are not fixed. This suggests that adding images into the intermediate short strip of the CP flight design is the key to mitigate the indeterminacy of intrinsic parameters f and cy in image-based SfM.

In summary, numerical experiments showed that the intrinsic parameters f and cy are indeterminate in image-based SfM when using aerial images captured by CP flight design. Adding images to the intermediate short strips effectively resolves the indeterminacy of f and cy.

In the next section, we will evaluate the estimation of intrinsic parameters in image-based SfM using in-situ data captured by CP and CP-Plus flight.

3.2. Real-Data Analysis

3.2.1. Estimates of Camera Intrinsic Parameters (f, cy) Across 50 Trials of a Single SfM Setting

Table 8 presents the SfM results for three image-sets captured at three different GSDs of two flight designs: the CP and CP-Plus. Across 50 trials of a standard SfM setting, the standard deviation in the estimates of intrinsic parameters f and cy was zero for the image-sets taken by CP-Plus flight design. In contrast, for CP flight design, the standard deviation was substantial (0.32 pixels for f and 0.08 pixels for cy). This variation also resulted in a 0.008 m deviation in the interquartile range (IQR) of RMS error of all validation points within the same SfM setting, for image-sets from CP flight.

These results demonstrate that the estimation of f and cy is unstable in SfM when using images from CP flight. However, in CP-Plus flight, where images were added to the intermediate short strips of CP flight, the estimates of the intrinsic parameters f and cy became more stable.

3.2.2. Estimates of Camera Intrinsic Parameters (f and cy) Across 30 SfM Settings

The estimates of f and cy in images-based SfM obtained from 30 different SfM settings, each repeated 50 times, exhibit instability (Table 9). The standard deviation of f ranges from over 1 pixel to 7.3 pixels, while the standard deviation of cy ranges from 0.6 to 3.1 pixels. This instability is particularly pronounced when using images from CP flight, where the standard deviation is about six times higher for f and five times higher for cy (GSD-20 image sets), compared to those taken by CP-Plus flight.

As shown in Table 9, the median RMS error of all validation points is substantially higher for image-sets from CP flight (0.151 m), approximately four times greater than the 0.039 m observed for CP-Plus flight (GSD15). Furthermore, the accuracy and precision of SfM vary considerably across 30 SfM settings for CP flight image-sets. The interquartile range (IQR) of the RMS error of all validation points is 0.105 m for CP flights, compared to 0.047 m for CP-Plus flights (GSD15).

Overall, the estimates of f and cy are significantly less stable for images captured by CP flight than CP-Plus flight, both across multiple trials of a single setting and across different settings.

3.2.3. Correlation Between the Estimates of f and the Mean Vertical Error for all Validation Points

Theoretically, incorrect estimates of f lead to mean vertical error (MEZ) for all validation points in model reconstruction. As illustrated in Figure 7, a negative correlation is observed between the estimates of f and MEZ across 30 different SfM settings (1,500 trials). This relationship can be explained by a simple pinhole camera model: as the estimated f increases, the triangulated ground points move farther from the cameras, causing their estimated vertical coordinates to decrease.

To assess the contribution of vertical error due to incorrect f estimation to the total vertical error, the vertical error ratio was calculated as the proportion of the absolute mean vertical error (MEZ) to the total vertical error (RMSE). As shown in Table 10, the average vertical error ratio for each image set exceeds 80%, indicating that vertical error resulting from incorrect f estimation serves as the primary contributor to the total vertical error, regardless of flight design.

In summary, the indeterminacy of intrinsic parameters, especially f and cy, in images-based SfM has been demonstrated using three flight designs in numerical experiments and its effect has been demonstrated in real-data analysis using three image-sets. In the next section, we will discuss the implications of these findings on existing studies and suggestions for practical application.

4. Discussion

4.1. Indeterminacy of Camera Intrinsic Parameters in Images-Based SfM

To our knowledge, this study provides the first combined numerical and in-situ analysis of the indeterminacy of intrinsic parameters in images-based SfM for CP flight. The numerical experiments show that, when images are captured under a CP flight design, f and cy, are indeterminable in images-based SfM.

Critical configurations - specific combinations of camera motion and scene geometry- are known to induce intrinsic-parameter indeterminacy. Previous studies have extensively documented critical configurations related to linear images (images captured by a pin hole camera without lens distortion) [26,32]. For example, planar motion over general terrain, in which the camera moves on a plane while rotating around an axis normal to that plane, prevents estimation of a single intrinsic parameter [26].

CP flight represents a restrictive form of planar motion in which the camera’s movement is constrained into a plane, and rotation is limited to only two orientations (pitch and yaw) rather than allowing free rotation around the roll axis. Therefore, it is possible that these additional constraints cause more than one intrinsic parameter to become indeterminate, and in fact our study confirmed that two parameters (f and cy) become indeterminate for CP flight.

In three real CP-flight datasets acquired at three different GSDs, we observed significant variations in f and cy across 30 distinct SfM analysis settings and across 50 repeat trials of a single setting. These results demonstrate that the theoretical indeterminacy of intrinsic parameters observed in numerical experiments, manifests in real-world scenarios as pronounced instability.

Our findings suggest that the previously reported poor performance of image-based SfM using CP-flight images [18,25], commonly attributed to the dome effect—is also influenced by inaccurate estimation of f, which causes large mean vertical errors. Images acquired under CP flight, as confirmed in our study, significantly compromise the accuracy and precision of image-based SfM and can even lead to incorrect model reconstruction in MVS. Therefore, practical surveys should avoid employing CP flight design or apply appropriate countermeasure such as augmenting camera coordinates measured by RTK-GNSS or adding many GCPs to ensure stable SfM results when using CP flight.

4.2. Remedy for Indeterminacy of Intrinsic Parameters in Images-Based SfM

Our numerical experiments demonstrate that adjustments to the shooting positions of CP flight, such as CP random flight, do not resolve the indeterminacy of intrinsic parameters (f and cy). However, introducing intermediate short strips between primary CP lines, as the CP-Plus flight, effectively eliminates this indeterminacy in numerical experiments and substantially reduces f and cy instability in real-data analysis.

This finding suggests that practitioners can largely mitigate concerns regarding the significant instability of intrinsic parameters estimated by image-based SfM by incorporating intermediate short strips into CP flights. Some built-in flight missions in flight planning software packages automatically include intermediate short strips. Such strips can be achieved using timed-interval or distance-interval shot modes, as seen in the 3D photogrammetry function of the DJI P4RTK drone or the "area route" option of the DJI Mavic 3E drone, for example.

In contrast, there are other flight missions that do not include intermediate short strips, such as "waypoint" flight, where the camera shutter is triggered at specific pre-defined points. Additionally, some flight planning software, such as Pix4D Capture [18,24,33,34] or Pix4D Capture Pro, does not generate intermediate short strips by default. These choices preserve the CP critical configuration and risk intrinsic parameters unstable. Overall, embedding short cross-strips within CP flight plans prevents theoretical indeterminacy of intrinsic parameters, delivering more stable and accurate SfM reconstructions in practice without reliance on camera coordinates measured by RTK-GNSS or extensive GCP networks.

5. Conclusions

In this study, we investigated how certain camera intrinsic parameters in image-based SfM become indeterminate using aerial photos from CP flights. Our numerical experiments (Section 3.1) confirmed that the CP flight is a critical configuration that causes the indeterminacy of intrinsic parameters, particularly focal length (f) and principal point coordinate (cy), under idealized conditions in CG simulation. Further analysis with real data (Section 3.2), involving three image-sets over 30 different SfM settings, demonstrated that estimates of f and cy are highly unstable when using CP flight imagery.

As a countermeasure, we found that integrating intermediate short strips into the CP flight design eliminates the indeterminacy of intrinsic parameters (f and cy) in numerical experiments and markedly improves the stability of these parameters in real-data analyses.

Our study reveals the potential to improve the accuracy of self-calibration using only aerial images in SfM, without the need for GCPs or RTK-GNSS data. This advancement not only simplifies the data acquisition process but also makes it more accessible and cost-effective for practitioners. By stabilizing SfM calibration, all the subsequent stages - such as MVS, dense point cloud generation - can be effectively enhanced, leading to more accurate 3D reconstruction.