Some properties of the basins of attraction of the Newton’s method for simple nonlinear geodetic systems

: The research presented in this paper concerns the determination of the attraction basins 1 of the Newton’s iterative method which was used to solve the non-linear systems of observational 2 equations associated with the geodetic measurements. The authors considered simple observation 3 systems corresponding to the intersections, or linear and angular resections, used in practice. 4 The main goal was to investigate the properties of the sets of convergent in the initial points 5 of the applied iterative method. An important issue regarding the possibility of automatic and 6 quick selection of such points was also considered. Therefore, the answers to the questions 7 regarding the geometric structure of the basins, their limitations, connectedness or self-similarity 8 were sought. The research also concerned the iterative structures of the basins, i.e. maps of the 9 number of iterations which are necessary to achieve the convergence of the Newton’s method. 10 The determined basins were compared with the areas of convergence that result from theorems 11 on the convergence of the Newton’s method, i.e. the conditions imposed on the eigenvalues and 12 norms of the matrices of the studied iterative systems. One of the essential results of the research 13 is the indication that the obtained basins of attraction contain areas resulting from the theoretical 14 premises and their diameters can be comparable with the sizes of the analyzed geodetic structures. 15 Consequently, in the analyzed cases it is possible to construct methods that enable quick selection 16 of the initial starting points or automation of such selection. The paper also characterizes the global 17 convergence mechanism of the Newton’s method for disconnected basins and, as a consequence, 18 the non-local initial points, i.e. located far from the solution points. 19


22
Determination of the coordinates of the geodetic network nodes based on the results 23 of measurements is related with solving an optimization task. A solution to this task in 24 terms of the least squares method is obtained by means of the iterative Newton-Gauss's 25 method (Ghilani [3,19], Nielsen [14]). An appropriate selection of the initial values of 26 the solution is an important element of the iterative methods [15,19]. Due to the process 27 of linearization of observation equations systems, the initial point of the procedure 28 is usually selected sufficiently close to the determined point. In the case of geodetic 29 networks, approximate values of the network nodes coordinates are computed by means 30 of specialized software on the basis of measurements that define the network [16][17][18]. In 31 this study, the authors dealt with a more general problem of the determination of the 32 initial approximations. The issue in question was: how far can the initial point be from 33 sets of points for which the applied iterative Newton's procedure is convergent for the 48 discussed cases. 49 This paper is organized as follows. Theoretical background contains definitions 50 and theorems used here (Section 2). Our main results that concern the study of the   63 The study of the properties of the nonlinear systems of equations is closely related 64 with the study of convergence of the iterative methods used to solve these systems. The where F is a projection (vector function) consisting of observation equations of the 67 considered geodetic constructions -classic planar intersections or resections [11]. Except 68 for special cases, there are no strict direct methods for solving the systems (1). It is a 69 nontrivial issue analyzed earlier by Traub [10], Ostrowski [9], Ortega and Rheinboldt 70 [7], [8] or Dennis Jr. and Schnabel [2]. The iterative Newton's method based on the 71 linearization of the observation equations is used to solve:

Theoretical background
where the affine approximation of (2) is obtained by truncating the Taylor expansion of 73 F at x k after the linear term; F (x k ) is the Jacobian (Fréchet derivative) of the projection F; 74 x 0 is the initial point, and B(x * ) is the so-called "basin of attraction" for solution x * .

75
According to the definition, the function G : D ⊂ R n → R n is F-differentiable at 76 x ∈ D if for an n × n matrix A: where: V and W -normed vector spaces and A -bounded linear operator equal to the 78 Jacobian matrix G (x). For each solution x * of (2) the basin of attraction B(x * ) for the iterative Newton's 80 dynamical process P (with "discrete time") can be defined as the set of all initial points 81 x 0 : 82 B(x * ) = {x 0 ∈ R n → {x k } = P(x 0 ), lim k→∞ x k = x * } (4) for which a (finite or infinite) {x k } produced by P : x 0 ∈ R n → {x k } ⊂ R n converges 83 to x * . The linearization of (2) induces numerous questions related to the solution's 84 existence and its convergence: 85 1. How many iterations must be performed to acknowledge that the solution is 86 achieved? 87 2.
For what conditions and the initial points the iterations are convergent? 88 3.
Are the sets of such points limited or unlimited? 89 In many cases these questions may be answered only partially. In particular, it 90 concerns the determination of the sets of all initial points, i.e. the basins of attraction, for 91 which the iterative process of the Newton's method is convergent.

92
The system (2) may be replaced by an equivalent fixed-point method system: where the iterative function G(x k ) = x k − F(x k ) −1 F(x k ), G : R n → R n . Here, the 94 point of attraction x * refers to the fixed point x * = G(x * ) of the iteration scheme (2). 95 The fixed-point method is used in numerous proofs of theorems (so-called "contraction-96 mapping theorems" and many of its variants) concerning the convergence of the iterative 97 Newton's method (Ortega and Rheinboldt [8] of F(x) = 0 to be a point of the attraction.

120
In this theorem the existence of x * had to be assumed. Moreover, only local instead is a set of the initial points for which the iterative Newton's method is convergent. 125 Presented research indicates that the neighborhood is usually defined by a small value 126 of the parameter δ and, because of this, it is usually only a part of the basin of attraction. 127 Dennis and Schnabel [2] indicated that the radius of the Newton's method convergence 128 is inversely proportional to the relative nonlinearity of F(x) at x * . They suggested that 129 the relative nonlinearity of a function is the key factor determining the behavior of the 130 iterative algorithms, and all convergence theorems could be restarted and proven in 131 terms of this concept. It was illustrated in details in [2]  Theorem (Ortega and Rheinboldt [8]), it is characteristic that neither the existence of a 147 solution x * , nor a local convergence (x 0 is sufficiently close to x * ), are assumed.

148
Theorem 2 (Ortega and Rheinboldt [8]). Suppose that G : D ⊂ R n → R n maps a closed set 149 S ⊂ D into itself and that there exists p ≥ 1, and a constant λ ∈ (0, 1) such that: Then G has a unique fixed point x * = G(x * ) in S and for any x 0 ∈ S the iterations 151 converge to and satisfy: This theorem can be used to test whether there is any region D such that the points 153 generated by x k = G(x k ) from an x 0 ∈ S will converge to the root of F(x). Furthermore, 154 the theorem indicates that if is found an area D, in which the theorem contraction 155 condition is satisfied, then D is a basin of attraction. Since we do not assume local 156 convergence, a study of convergence for the initial points far from the solution x * may 157 also be considered. There is a certain disparity here, because the Newton's method, due 158 to the expansion into truncated Taylor series, is a local procedure. It may be noticed that 159 theorems of this type theoretically permit wider range of search for basins of attraction. In order to determine the vector of the solution x k+1 for the subsequent iteration 171 step k + 1 using the relations (2), the following system of the linear equations is solved: This approach enables to avoid the inverse Jacobian F (x k ) computation. The criteria 173 of completing the computations are based on the conditions under which the function 174 F(x k ) achieves components values sufficiently close to zero: and sufficiently small differences of distance (norm) between the subsequent vectors of 176 iteration: In practice, the parameters ε F and ε x are related with the uncertainty of measure- x * , the Newton iteration scheme converges at a quadratic rate to the solution: x k → x * 185 (see Kelley [5]). Quadratic convergence means that the distance between the subsequent 186 approximations and the precise solution x * decrease according to the following relation: This property is important because a small number of iterations is required to The case when α ≈ 1, it is unacceptably slow.  For studying the Newton's method convergence the balls of the random numbers 206 are generated by the following relations: (14) where: (x 0 , y 0 )− the coordinates of the ball center.  in the assumed coordinate system are written as follows: where a and b are the measured distances, c is the known length of the intersection base 217 and y is the coordinate of point A.

218
The equations systems (15) and (16) are invariant with respect to translation vector

219
[t x , t y ] and isotropic scaling s (s -scale coefficient), i.e. the replacement of coordinates: 220 x = t x + s · x, y = t y + s · y, c = t y + s · c and measurements: a = s · a, b = s · b. For 221 this reason, the geometric properties of the basins of attraction of the systems subjected to 222 such transformations are the same. An example of a studied subfamily of intersections: belonging to the family defined by the inequality: Figure 2. The exact solution (x * , y * ) of the systems (15) and (16) in the assumed coordinate The expression in the root in (17) is positive, which results from the triangle inequalities: This study is limited to the 229 case of positive solutions: x > 0.
where F (x) −1 is the reciprocal of the Jacobian matrix F (x) of the projection F(x) (15).

234
The matrices F (x) and F (x) −1 are singular in points coinciding with the y axis of the 235 coordinate system ( Figure 1). Expanding (18) and applying y * from (17) we obtain: Relation (19) indicates that G(x) is a function of the coordinate x only. It means that 237 independently of the value of x after the first iteration, the value of y is constant and 238 equal to y * . The solution is hence sought after along a straight line perpendicular to y at 239 y * . After substituting y * with g(x, y * ), after elementary conversion (see Appendix B) we 240 obtain: Including the exact solution x * (17) in (20) we may rewrite it as an iterative function of 242 the root of the quadratic equation Relation (21) indicates that after the first iteration, for any initial point of the coordi-244 nate x 0 > 0, we obtain a point of the coordinate x = g(x 0 ) > x * . This remark is due to 245 the inequality: which for x > 0 is equivalent to the following true inequality 247 This is illustrated in Figure 3 Figure 3(b)). This indicates that the number increases approximately exponentially with This effect can be seen in Figure 4 where at the lower parts 258 of the plots, the contour lines presenting the numbers of iterations are packed together.

259
Determining the derivative g (x): and including the inequality (21) (satisfied already after the first iteration), we may 261 notice that: is the non-zero eigenvalue of matrix G (x).

265
According to (23), λ x = |g (x)| < 1/2, which means that G(x) is a contractive projection 266 in the half-plane: x > 0 which includes the intersected point. For studying the basins of attraction (16), the following iterative Newton's function 279 is used: 5, a fragment of the straight line y * = 1 2 was assumed as the set of the initial points.

334
The performed calculations indicate that the limit λ x = −1 is achieved through 335 values almost equal to λ x ∼ = −1 but greater than λ x > −1. Then, the initial points are In Figure 9 are presented the eigenvalues |λ max | and the norms of the G (x) deriva-342 tive for the linear intersections. Figure 9 also presents the sets of the initial points (lighter The satisfying condition (28) means that the initial points x 0 successively move closer to According to the Contraction-Mapping Theorem 2, relation (28) for the analyzed 368 cases of the linear intersections may be written as: The plots in Figure 10 also show that here we have at least linear convergence 370 (12). The contraction coefficients u j decrease from one iteration step to the next, which 371 means increasing speed of the convergence of the Newton's method when point x * is 372 approached. Generally, the contraction condition of Theorem 2 should be satisfied for 373 every pair (j, m) of points belonging to the ball of the initial points (x 0 j , x 0 m ) ∈ K(x 0 ), i.e.: additional study -which is beyond the scope of this paper.  Newton's method. Unlike the basins of linear symmetric intersections, the asymmetric 388 basins have more complex geometric and iterative structure. The differences grow with 389 the increase of the departure from symmetry (see Figure 11). It may be noticed in Figure   390 11 that the structures of these basins depend on the intersection shape determined by the  Figure 12).

396
The plots in Figure 12 are obtained by increasing the areas (see Figures 11(g), (h),

397
(i)) to which the randomly generated initial points belonged.  In the case of the last plot in Figure 12 the last plot in Figure 16). In general, it may be concluded that the basins of attraction This form of the system is considering the possibility to assume any point belonging 455 to the plane, i.e. x 0 ∈ D ⊂ R 2 for an initial point x 0 of the Newton's iterative procedure.
Relation (32) indicates that for the Jacobian matrix F (x) and its reciprocal F (x) −1 466 the following relations are satisfied: It may be noticed that F (x) and F (x) −1 are singular on the straight line x = 0, in 468 particular in the points determining the base (x = 0, y = 0) and (x = 0, y = c). is not a necessary condition for convergence [7]. Theoretically, existence of such areas The systems in (35) are written in the iterative form (the superscripts are omitted): where after introducing the following denotations: the Jacobian matrices have the form: Equations 35 are invariant with respect to translation and isotropic scaling of 571 coordinates: x = t x + s · x, y = t y + s · y and measurements: a = s · a, b = s · b and 572 c = s · c. Jacobian matrices F 1 and F 2 are singular for points on the straight line x = 0.   are typical for the bifurcation known from the theory of non-linear dynamic systems. 669 We also determine the potential areas of convergence of the Newton's method resulting 670 from the condition applied to the spectral radius: |λ max | < 1 (or norm G (x) < 1). The

671
performed calculations indicate that the Newton's method is convergent for the initial 672 points belonging to such sets. Since the sets are contained in the basins of attraction, the 673 condition |λ max | < 1 ( G (x) < 1) is not necessary (in the case of the considered inter-