Review of the Progress and Process of Satellite Imagery and Road Databases Conflation

: Spatial data conflation plays a fundamental role in in many aspects of modern Geographic Information Systems (GIS) research and development such as geospatial data visualization, incremental updating of databases and disaster evaluation. The primary objective of conflation is to derive valuable information based on the comparison of multiple spatial data sources of homogeneous or heterogeneous nature. This paper reviews the state of art of the concept of conflation, feature matching, the most important progresses of conflation between image and road network data, and the complexity of spatial conflation.


Introduction
The importance of spatial data in conjunction with everyday life and the rapid development of a modern society using suitable, simple and cost-effective geo-databases and geo-referenced raster images is of no doubt.Various aspects of modern GIS research and development, such as spatial data validation and integration, are being used increasingly in road database updates and navigations, visualizations, the process of decision-making, infrastructure planning and disaster evaluation.Above all, the integration of geo-databases and images is an important issue.The homogenous integration of road database data (street maps) and Very High Resolution (VHR) images may serve as a good example to characterize the typical problem of misalignment between homologous features in the road database and images, which may be induced by up to 200 meters displacement [1].The feature misalignments between two sources of datasets are caused by various factors, such as different resolutions or scales, inaccurate camera models, projection errors, different data acquisition times and different data vendors [2].Moreover, the misalignments are not systematic and therefore cannot be corrected by a uniform transformation function.The method of spatial data conflation is therefore essential to combine multiple datasets.Matching including geometrical, topological, and semantic features is the core of conflation [3].When overlaying two inhomogeneous road datasets describing the same region, the homologous road objects may reveal four different forms of offsets: (1) Perfect coincidence.(2) Intersection.(3) Disjunction.(4) Feature inconsistency.
However, there are useful cartographic overlapping problems, which have been slightly displaced for purpose from the original road data.Generally speaking; misalignments may arise as a result of extreme offsets during the zooming of images and a change of resolution.The contents of this review paper are as follows: After some introductory remarks we are presenting an overview for the concept of geo-data conflation.Section 3 briefly summarizes the necessary feature matching problem, followed by an overview of adequate algorithms to overcome this problem (section 4).The concept of deconfliction is discussed in section 5 and section 6 contains a state-of-the-art of spatial conflation.As there are still some problems to solve, we will discuss the complexity of spatial conflation in section 7.

Conflation
Conflation is summarized as the process of integrating geographical datasets, combining multisource data, improving data quality, and updating spatial information.Not only vector datasets, but also raster datasets can be used for conflation.Conflation is normally classified into two categories: horizontal and vertical [4].Horizontal conflation refers to the merging of two adjacent datasets with common edges, and the vertical conflation focuses on overlapping datasets covering the same area.The history of conflation can be traced back to 1985; Saafled [5] implemented a project to join the United States Geological Survey (USGS) and the Bureau of Census vector datasets in the U.S. metropolitan areas.After that, many applications were developed to realize automatic conflation.To manage the automated conflation, apart from distinct steps (e.g.pattern recognition, statistical analysis, graph theory), the basic procedures including positional alignment, matching feature identification, and positional and attribute matching feature the de-confliction process [6].
Since the beginning of the last decade and with continued developments in photogrammetry and remote sensing the automatic detection and extraction of objects from satellite imagery and airborne photography is one the most important challenges in research tasks.Its essential complexity involved in the research questions is far from the most effective solution.Among the large number of existing works, several reviews on road detection algorithms were given by Fortier, Mena, Mayer and Rottensteiner, reported in different time periods [7][8][9][10].
Zhang and Mena discussed topics challenged by the GIS community: updating GIS data needs extensive manual work, whereas the low extraction ratio is the bottleneck of the image processing algorithm, so a combined approach of road network and images seems to be a natural choice to increase the performance.Each dataset offers meaningful prior information for the other dataset [11,12].Zhang reported a map-based road detection method, which included two important applications: updating the GIS data [7,13,14], and validating or increasing the road detection ratio [15][16][17][18].Mena reviewed the updating of GIS data and tried to classify the algorithms for road extraction and identified the challenges involved in the road detection using both vector and raster datasets.

Feature matching
Identifying correspondence entities from source layers is a crucial step in conflation, as the same features could be represented heterogeneously in different datasets.Feature matching aims at seeking for the similarities and establishing relationships between identical entities.Wiemann and Bernard [x] classified the matching according to the method used to match the sequence, and the geometry of input features (Figure 1.1).Geometric matching dominates these matching methods.It includes the similarity comparison of feature length, area, perimeter, angle, Hausdorff distance, etc.Additionally, the combination of geometric, semantic and topologic matching is a trend in geo-data conflation.To identify the correspondence of source datasets, manual and automated matching is optional.Using manual matching is very laborious in areas with many elements [3].Point, line, area based, and mixed are four approaches of automated geometric matching [19].The point based approaches concentrate on the matching of adjacent points, which was first been adapted in 1985 with a relatively long history compared with the other kinds of matching.Chen, et al proposed an approach to integrate information utilizing control points extracted from common vector dataset and imagery [1].A specialized point pattern matching algorithm was developed with the concept of forming a transformation matrix.In the study implemented by Song, et al., they presented a relaxation-based point feature matching approach to accomplish the road intersection matching [20].This algorithm utilizes the relative distances between points and iteratively updates local context.By applying a "buffer growing" algorithm, the line-based method extends the traditional 1:1 matching to the 1:1, 1: n, and n: m pairs storing in a list [21].An alternative solution of line-based feature matching is the detection and matching of counterpart nodes and poly-lines [22].Area based conflation normally computes the property of polygons, for instance, area, center of gravity, number of enclosed lines, percentage of overlap area-to perimeter ratio, and Hausdorff distance [23,24].The combination of point-, line-, and area-based matching is defined as a mixed model.A project carried out by Chen and Walter make the matching between point and line features possible.The merged approaches of point and area feature matching are used for the spatial conflation of non-planar polygons in aerial photography building extraction [25].

Bottom
Between the exchanges of datasets based on different conceptual models, semantic matching represents the similarity comparison of attributes.In the context of semantic matching, a widespread aspect to alleviate semantic heterogeneity is the ontology of GIS.According to Vaccari et al. the ontology in the geospatial field is a logical theory indicating the objective of a formal vocabulary [26].

Feature-based matching algorithms
This section focusses on the matching algorithms for the conflation of image and road network data with the emphasis on featurebased approaches for points and lines.The semantic information embedded in road networks serves as the complementary information for the features in the image.

Geometric Semantic Geometric Semantic
Let S be the reference point set {s 1 , s 2 ,….,sPs } (P s ϵ ℝ ), and M is the target data set { m 1 ,m 2 ,…,m Pm } (P m ϵ ℝ ), and ) is the shortest distance from M to the reference point s i .
Then the algorithm starts as follows: 1) LetT 0 be an initial parameter for the transformation; 2) Setup the correspondence function 3) Compute the new transformation k T that minimizes the mean square error between point s in C. 4) Repeat the steps 2~3 until having met the termination criteria.

Point-based matching
Point features have most frequently been treated in matching methods in a wide range of disciplines as they are relatively easy to be detected from the image.Nevertheless, many reasons make the point-based matching a complicated problem.Based on an analysis of progresses made in point-based matching with the specific focus on the deformation level as well as the 'dynamic' character of the data, Li et al. claimed the primary reasons which strongly influence both: the accuracy and efficiency of the matching.These are the data quality and the distortion level which can be subjected to data distortion, incompleteness and noise from different systems [27].Zitova and Flusser reviewed the point-based matching approaches for the image registration [28].
Zhang regarded point-based matching approaches as a support to a network-matching algorithm [29].Among these works, the ICP algorithm (Besl and McKay) has been reported as the most popular method for point-to-point matching [30].It deals with an iterative procedure that computes the favorable correspondences with the initial matching by the assumption of knowing the prior information e.g. the similarity or transformation parameters.

Algorithm 2.1 ICP-Registration [31]
ICP utilizes the nearest-neighbor relationship to assign the correspondence at each step.It requires a good initial estimation in order to converge to the global minimum.ICP is simple and speedy, therefore suitable for real-time applications.However, it is a difficult task to determine reasonable initial parameters.A fully automated ICP algorithm needs to search among multiple initial conditions which may increase the computing effort.
To find a desirable initial state of ICP, Chui and Myronenko formulated in their methods of TPS-PRM (Thin Plate Spline Robust Point Matching) and Coherent Point Drift (CPD) the point matching as an optimization problem, which usually includes a target function.Unfortunately, the target function is an ill-posed problem, because of the large number of unknowns in the function and it needs regularized terms or constraints to get the optimal results [32,33].Moreover, the physical property of the spatial point is neglected.Apart from this drawback, these approaches are robust and general for different kinds of applications.The point-based matching is a practical approach for point datasets, where the spatial relationship among the points is undefined or difficult to detect.The distance measure is a congenital deficiency of the point-based method.The nearest neighbor metric or its variation has been verified by Veltkampas as a suitable measure for dense point datasets [34].The dense point dataset, however, is an unrealistic requirement for cartographic objects, which are usually irregularly and sporadically distributed within the study area.Another challenge for the point-based matching method is caused by the outliers in the dataset and the noise may dramatically decrease the stability of the matching results, if the signal-to-noise ratio becomes larger in the target dataset.The spatial relationships of the lines are conceptually classified into four groups, i.e. one-to-zero, one-to-one, one-to-many, and many-to-many [29].The matching algorithms have the essential task to search the entire data space, and to detect the spatial relationships between homologous line features based on the established similarity measures.Usually, the determination of the spatial relationships starts from the line elements in the more reliable dataset.The correspondences for the one-to-one relationship may fall into one out of six cases, if we only consider the geometric information.1) Zero correlation (correlatively) Due to different qualities of datasets to be matched, there are certain features without exact correlating pairs in the partner dataset, for instance, the new road could exist only in one of the datasets.It is even worse for the conflation between road networks and images, where some roads are neglected by the detection algorithm.

2) Complete correlation (correlatively)
This is the ideal but a rare situation for the matching approach.It is delivering the most reliable matching result if both lines in the networks reveal exactly the same geometry.

3) Overflow correlation (correlatively)
The target line is longer than the reference line, thus contains both of the end points of the reference line.The overflow correspondence is the counterpart for the containment correlation (correlatively).

4) Extension correlation (correlatively)
If one of the end points of the reference line can get the exact matching result in the target line and the other end point is also verified on the target line, it is the extension correlation (correlatively), which means the target line can be extended to have the same length as the source line.

5) Containment correlation (correlatively)
The reference line is longer than the target line and contains both end points of the target line.It is the counterpart of the overflow correlation (correlatively).6) Partial correlation (correlatively) It is common, that there is no exact relationship as described above, if it only considers the end points of the line, but at least one end point can be registered inside the reference line.Only two lines with a one-to-one relationship are considered in the above discussion.We can extend the discussion to complex situations that deal with spatial relationships among multiple lines.The correspondence (4~6) usually involves more than one single line segment, if it takes the neighbors into account.Therefore, as shown in Figure 1.3, it evolves to a 1-to-N or N-to-1 spatial relationship, where N is a positive integer.More generally, if we continue with the searching step, we may identify all the N-to-M relationships, where M is a positive integer that is not necessarily equal to N. (1) (2) Target Reference

Point Cloud Registrations and Control Point Processing
Automatic registration is the mode of registering a point cloud into a defined reference system without employing manual measurements, thus some errors involved are eliminated without manual interactions.A well-known method for merging pointclouds is the Iterative Closest Point Algorithm (ICP), which supposes linear datum shifts.The exact solution is based on the automatic detection of control points, and then using the Gauss-Helmert model for parameter estimation of the 7 transformation unknowns.In the following the math model used is described in more detail: It is a well-known fact, that most implementations of ICP registration software do not consider at all any precision measures.If we consider the overall 7 parameters transformation, which is non-linear, we get with X the (3x1) u vector of world coordinates of u control points, X o the (3x1) u vector of the 3 translation parameters (X o , Y o , Z o ), µ is the scale, R the (3x3) u rotation matrix depending on the unknown rotation angles α, β, γ and x the (3x1) u vector of the local u control point coordinates.This non-linear transformation is linearized considering only differential changes in the three translations, the rotations and the scale, and therefore replacing (1) by S is the (3x7) u similarity matrix resulting from the linearization process of (1), given as And Representing the seven unknown registration parameters.In order to estimate also the precision of the datum transform, the ICP is embedded in a least-squares Gauss-Helmert model for u≥3 and B:=S leading to 1 st order: Av + Bx + w = 0, and 2 nd order: Solving ||v|| 2 P = min subject to Av+Bx+w=0 the corresponding normal equation system is obtained x 0 00 B (5) Resolving ( 5) wrt λ we obtain ( ) ( ) And resubstitute it again The error propagation problem of any precision parameters is solved by the well-known law of error propagation (of statistical inference): For the (3x1) u vector of residuals we use And for the vector of the estimated unknown datum parameters With D(w)=σ 2 AP -1 A' substituted in (9) the precision of the datum transformation parameters is propagated This general derivation of a least-squares datum transformations using 7 parameters can be reduced to 6 parameters (without scale) in laser scanning (Rigid Body transform, RBT) and is overcome by automatic measurement of artificial targets.
Corresponding points are then assigned having the same annotation (Leica Cyclone).
In the following datum transformations are used to derive accuracy measures.For this reason, about 40 control points are measured using GNSS with an accuracy of about 1-2m.For the Khartoum OpenStreetMap (OSM) road network data the following accuracy has been obtained: σ OSM (x,y)=2,2m.A similar accuracy has been obtained for the road network data of the Sudanese Ministry of Infrastructure (SMI), which is σ SMI (x,y)=2,7m.Although both datasets do not varying in precision very much, the completeness is quite different.In order to derive a value for completeness we compared the line segments of both datasets with each other.
Here just a bit assignment helps the computer to differentiate for the available data: The following measures for completeness have been obtained, using the OSM road data network as reference (100%): Completeness (cps) of the SMI road network data to be 43%.Thus there is a need for spatial conflation to update the official road database of Sudan

Line-Based Matching
The line-based matching approach takes more geometric information into account than the point-based matching method and is widely used for road network matching.The geometric, topologic and semantic characteristics of two road networks are useful clues for the establishment of an exact connection between them.The semantic information serves as auxiliary information while geometric and topologic information is more commonly explored [29].The line-based matching algorithms work on road networks with complete or incomplete topology.The topology bears in mind the most significant information about a road network.A road segment is also an elementary component of the graph.Therefore, line-based matching is mostly supported by graph theory.Conte et al. reviewed the interdisciplinary graph matching approaches with regard to the applications and the algorithms [35].They deemed the graph matching as a general searching problem with the aim to refine the correspondence among the line segments.Different from the graph matching method, the road network matching plays a significant role for the integration of geospatial data and considers more information about the roads and their catchments' areas.Among the related works, two algorithms are quite compelling.Walter proposed the buffer growing algorithm to identify correspondences inside a local region with a specific buffer threshold [36].Multiple correspondences are possible.However, the false correspondence can be minimized or verified as the buffer region moving to the neighboring segments.Zhang presented a line-based matching algorithm which helps to construct delimited strokes from the road segments, and then introduce a buffer around the individual strokes other than the fragmental road segments [29].In further steps, the multiple correspondences are identified and optimized by means of a contextrelated topologic analysis.

Patch-Based Matching
The enclosed meshes or polygons by road segments also provide important clues for the matching algorithm, and we term the approach as patch-based matching.It follows the principle of finding the appropriate shape descriptor or indicator and computing the similarity between two patches.Since road networks contain open lines, patch-based matching can only partly work on closed meshes in road networks.Also Zitova and Flusser review that with general discussion about the patch-based matching method [28].

Deconfliction
With feature matching and feature alignment accomplished, the deconfliction is performed subsequently.By applying vertex insertion, weighted averaging, and feature connections, the source datasets are conflated [37].In their work, the vertices of both datasets are identified, so that the number and distances are equivalent.Then the weighted averaging is calculated to distinguish the accuracy of datasets.Lastly, the conflated vertices are located according to tolerances.Establishing a connection segment with a start or end point of updated feature and unmatched feature connects the matched and unmatched features.Songet al. first employed an approach and named it snakes.It improves the road accuracy from over 100 meters RMS to 3 meters.This model reduces the efforts to split linear feature and transfer attributes of source datasets [38].

State of the art of conflation
This section is devoted to an overview of the most important progresses of conflation between images and road networks.A complementary overview is given by Ruiz et al. [39].Hild and Fritsch introduced a global alignment method between vectors and images for geo-coding according to which the vector features are rasterized to match the extracted polygons from the image [40].Chen et al. presented a conflation approach based on road intersection patterns recognized by using Bayesian classifiers [41].They analyzed the shape of the grayscale histogram and determined clusters on the shape using a size measure with the aim to detect road intersections that reveal certain statistical properties.However, the majority of image regions do not really display the statistical uniformity because of the remaining noise from small objects.To solve the problem, Ruiz et al. developed a nonparametric approach based on texture analysis for the identification and extraction of pixels that belong to road intersections [39].Wu et al. reported an approach to localize the global alignment problem by dividing a large image into regular grids, and then computing the maximum correspondence between the road networks and the features in each image grid.Finally the matched features were transformed by the global TPS transformation function.The correspondence between the image and road network is identified by the peak responses from the corresponding gradient image, moreover, it assumed that there was a dominating orientation in each pre-tiled image.This assumption, however, proves too strict for the road pattern in the image, and is only suitable for certain cities.The approach also computed the confidence factor for the correspondence, thus provided useful information for manually correction [42].Song et al. presented a fully automatic conflation approach based on the road intersections as well as terminations which were extracted from imagery and compared with road networks by means of a relaxation labeling algorithm.The matched points were snapped using a Rubber-Sheeting transformation algorithm.Furthermore, the aligned features were input into the snake models to acquire the refined results.In this approach, the intersections were picked up from the imagery by setting suitable thresholds for the Normalized Difference Vegetation Index (NDVI).The spatial context which assumed the road surface with a similar radiometric behaviour in the test area was considered.However, it was hard to decide the threshold for the image: A small threshold may increase the accuracy, but miss some important intersection; whereas a too loose threshold may increase the number of false intersections in the results [43].These approaches share a common character of avoiding the detection of linear features and using road intersections as the favorable salient features from the image.They have therefore a high computing complexity.Moreover, they ignore the shape of the road and presume the road segments are more or less straight lines, which may lead to inaccurate results after the transformation [44].A combination of road intersection and salient road segments is therefore a more desirable approach, but it requires the extraction of linear features from the image.Zhang provides in his dissertation "A Congruent Hybrid Model for Conflation of Satellite Image and Road Database".The main contributions of the CHM model include: 1) A linear feature extraction approach which consists of an Elastic Circular Mask (ECM) algorithm for feature extraction and a Genetic Algorithm (GA)-based grouping approach for the extracted road candidates; 2) A novel Sparse Matching Algorithm (SMA) approach which is especially useful to handle the problem of multiple correspondences; 3) A performance evaluation of two popular transformation functions -the piecewise Rubber-Sheeting (RUBS) approach and the Thin Plate Splines (TPS) approach.He showed that the experiments with synthesized data as well as the real spatial data have verified the time efficiency of the SMA, which has a stable performance; whereas the required time of ECM and GA is relatively sensitive to the amount of dataset.Comparing to the classical point-based conflation model, CHM can generate better results, especially with regard to the geometry; moreover, the inconsistency in both datasets can be also recognized and rectified when the accurate correspondence is derived from SMA [45].

Non-rigid deformation
The spatial datasets to be merged or conflated are usually collected from different times, different sensors and/or by different vendors, and there is inevitably some inconsistency between them.The same object e.g. the road segment in one dataset appears different in other datasets and has undergone some non-rigid deformation.If we consider all the features at the same time, we have to face an infinite number of unknown transformation parameters because the solutions usually exist in an infinite space [46,47].To make the non-rigid transformation manageable, we may look for discrete solutions instead [48].However, it is difficult to define the exact number for a sufficient estimation of the parameters, moreover, large transformation errors may be introduced if a feature to be matched is outside the convex hull of its matching partner.Both continuous and discrete solutions can be reasonable for spatial conflation of road networks, but their relative efficiency and complexity need to be clarified in transformation applications.It should also be noted, that the deformations in datasets or the misalignments between two road networks are non-systematic and therefore it is unrealistic to transform all features by one global function.Instead, each feature needs an individual treatment in an optimal way.

Incomplete matching measures
The ICP chooses the closest point as the matching measure, but it can be misleading for well-structured spatial features.The distribution patterns of spatial objects with natural or artificial structure are too diverse to be modeled in a single way.No versatile measures exist yet to effectively describe the similarity between the features and preserve their inherent characteristics.To choose or (re)define such measures remain the demanding task for matching approach.
Both the correspondence and the transformation parameters are unknowns of the matching system, but they are interconnected with each other [32].If we know the exact correspondence, the transformation is a simple least-squares problem, and vice versa.However, the high dimensional parameter space for both correspondence and transformation has complicated the handling of the non-rigid feature matching approach.In practice, it is only possible to get approximations by means of additional constraints or regularized parameters.

Outliers (Robustness, Uncertainty)
One of the important preconditions of the matching approach is the equal or similar density of the datasets to be matched.However, it is not always the case for spatial data conflation, especially for the matching between road network data and satellite images.The road features in an image are often disturbed by noise such as shadow, tree etc., which make them difficult to recognize and only fragments of roads are discernible.The performance of the matching algorithm may drop further with the increasing noise or outliers in the dataset.This may lead to unstable and uncertain matching results.Under extreme conditions, for example, if the discrepancy between the two datasets is beyond a reasonable range, no reasonable solution can be achieved [42,43].

Openstreetmap
The OpenStreetMap (OSM) project is an online open source editable map of the world, created by volunteers worldwide through the collection and contribution of geographic data.Anyone can contribute to the map by simply registering on the OSM website.The data is completely free with an open content licence, it is current as it constantly being updated by the subscribed users who can also add points of interest import to them.OSM has the potential to establish volunteers from all over world including less developed regions, where obtaining data in such places can be difficult for most commercial mapping companies.

Ministry of Infrastructure in Khartoum State
The Ministry of Infrastructure in Khartoum state is responsible of public infrastructure such as transport infrastructure, road planning, urban development and other civil services.It plays as important role in creating maps that can be used by different governmental sectors in Khartoum state.

Experimental Data
The table below show the study area waypoints and their attributes:

Figure 1 . 2
Figure 1.2 Six variants of one-to-one spatial relationship between two lines.

Figure 1 . 3 :
Figure 1.3: Examples of one-to-many spatial relationship for lines.