Conceptual Neighborhood Graphs of Discrete Time Intervals

1. Introduction

Qualitative topological spatial and temporal reasoning is one of the core pieces of the research agenda of spatial information science (Figure 1) [1]. For four decades, research has been conducted to provide the theoretical tools to software developers to enhance the infrastructure of powerful spatial information systems, including formal models [2,3,4,5,6,7,8],relation sets [9,10,11,12,13,14], data structures [15,16,17,18], analysis techniques [19,20,21,22,23,24], artificial intelligence applications [25,26], and many other aspects. While the ecosystem for qualitative topological reasoning is diverse and prolific, not all of the important theoretical innovations over the past few decades have made their way into modern software. At the dawn of an age of geographic artificial intelligence [25], it is essential to integrate all available spatial and temporal reasoning insights into AI systems that will result in interpretable, verifiable conclusions in terms that humans naturally understand. One fundamental aspect of that is of course language [26,27,28,29].

Geographic concepts are traditionally analyzed in static terms, but this approach overlooks the temporal dimension that is fundamental to understanding spatial phenomena [30]. Fundamentally, geographic information science is a direct application of a broader concept: spatial information science [1,31,32,33]. Time represents an added dimension to our physical world, mathematically applied symbolically as a cross-product of an existing space and a temporal dimension [14,34] (Figure 2). On a technical level, such a concept motivates the spatio-temporal stack data structure [18], or theoretically referred to as the space-time cube. Time is fundamentally a language of change, and to fully comprehend a spatial event, in many cases temporal relations are pivotal, yet GIS solutions minimally implement qualitative temporal reasoning directly.

To realize a spatial-temporal decision support system, it is imperative to understand the concepts behind how humans understand their environments [38]. One crucial aspect of this is found in topology [27,28,29]. Topology serves as the backbone for a large set of decisions, manifest in our spoken and written languages through prepositions [39]. Prepositions apply to both space (such as inside) and time (such as during). These terms differ only in their semantics: inside implies a container object, while during implies a containing interval. When we consider intervals as objects, these two concepts are one and the same. These prepositions are what we might consider topological primitives, namely that they represent a mathematically explicit concept endowed in an image schemata [40]. Other prepositions behave differently, for example the spatial term along [26,28]. This term takes certain members from various topological relations (in this case, disjoint, meet, and overlap) and combines them into a specific term. Such a term is difficult for an information system to derive because of the humanistic nature of the categorization itself. Other terms such as within [41] are direct aggregates of topological primitives. Terms of the latter two types benefit from aggregating similar relations, and to do that effectively, it is important to define similarity of concepts, in essence, constructing an ontology of spatial relations [42].

The conceptual neighborhood graph is a tool that has been developed to construct models of similarity among qualitative relation sets [19]. Conceptual neighborhood graphs organize the members of a qualitative relation set into a network where the nodes of the network represent relations and the edges represent a topological change under minimal units of modification. Conceptual neighborhood graphs have received a lot of attention over the last 30 years, with conceptual neighborhood graphs appearing in either the same article as the relation set (e.g., [43]) or as separate articles (e.g., [10,24]). On top of this, researchers have considered new structures of conceptual neighborhood graphs, including matrix difference neighborhoods [44], aggregates of multi-granular neighborhoods [45], intersection and union neighborhoods [46], and non-homeomorphic deformation neighborhoods [47]. While conceptual neighborhood graph research may seem archaic, the topic has seen a revolution with respect to the big data world and the growth of spatio-temporal artificial intelligence [25,26]. This research specifically acknowledges the variety inherent in big data, and that variety leads to a definitive need for data integration, which may need to invoke discretization neighborhoods [24,26,48] and may also need to translate topological primitives into other terms and vice versa [26,28].

The spatio-temporal stack specifically creates an environment in which time is discretized [18]. To work with this reality, a set of discretized temporal relations were constructed [14]. This relation set consists of a sizable collection of 74 relations. While this may be larger than practically needed, this result was a combinatorics exercise: can this circumstance exist? To further refine that theory, conceptual neighborhood graphs provide the mechanism for determining neighboring concepts. As such, it is imperative to engage in the work of determining the conceptual neighborhood graph for this relation space. Prior work regarding spatio-temporal stack architectures did not embark on providing a query processing strategy against the structure itself that would leverage the temporal dimension [35], thus this work starts to fill a missing link. While the tools exist to ask a question such as “find all objects that overlap one another,” [41] it is a different question altogether to pose “find all objects that overlap one another which start an event where a specific object contains another specific object.” To answer this question, knowledge of both spatial [4,12] and temporal relations [9,14] are needed, and given that a human is asking the question, precisely what is meant by start [26] is also fundamental.

It is this last piece of the query (what is meant by start) where this paper provides its merit. In this paper, the conceptual neighborhood graphs for translation, anisotropic scaling, and isotropic scaling are derived from simulation data upon a pair of discretized lines embedded within ${\mathbb{Z}}^1$. Using SQL queries upon the output data, neighboring concepts are identified based on the appropriate criteria, inducing the query through the criteria necessary to identify the appropriate change in the dataset.

The remainder of the paper is structured as follows. Section 2 introduces the set of 74 discretized temporal relations [14]. Section 3 introduces the literature pertaining to conceptual neighborhood graphs [19]. Section 4 describes the simulation protocol and query procedure leveraged to construct the dataset (adapted from [24,48]). Section 5 presents the three conceptual neighborhood graphs for the discretized temporal relations. Section 6 analyzes the output conceptual neighborhood graphs with specific attention paid to their relationship to the neighborhood graphs of the Allen interval algebra [9,19]. Section 7 provides discussion, conclusion, and calls for future work.

2. Discretized Temporal Relations

Temporal relations represent some of the earliest (and also most recent) work within the geographic information science community. As with the history of most formalisms for object relation sets, the vectorized sets came first, then followed by discretized versions (e.g., [9] and [10,12,14,49], and [13,43]).

The Allen interval algebra serves as the foundation for studies of temporal relations in an abstract sense [9]. Allen’s work specified intervals by their starting and ending points in ${\mathbb{R}}^1$. Allen explored the 25 possible sequences of boundary points and determined that 13 of those relations were realizable given basic constraints about boundary points simultaneously occupying either the starting or ending point of the other boundary (eliminating two candidates) and that the ending point of an interval must naturally come after its starting point (eliminating a further ten candidates). The resulting relations are shown in Table 1.

The relations in Table 1 have a definitive relationship to one another. Relations such as before and after are converse relations. Converse relations consider the objects in the opposite order. If a is before b, then it is directly implied that b is after a. This set of relations is comprised of six converse pairs and an additional relation that is self-converse (equal), as shown in Table 2 [9].

The Allen interval algebra became the springboard by which to consider minimum bounding rectangle relations [6], direction relations [7,8,51], but more importantly for this paper discretized temporal intervals [14]. To maintain starting and ending points that are part of the accessible space, the digital Jordan curve [3] was chosen to represent these intervals in ${\mathbb{Z}}^1$. Given this definition, it is possible for an object to consist of only boundary, a similar result to discretized region relations in ${\mathbb{Z}}^2$ [12]. With the assertion that an interval object must have distinct starting and ending points, a series of constraints were applied in a constraint sieve to isolate verifiable relations between intervals in ${\mathbb{Z}}^1$. The constraint sieve identified 34 relations (with an additional eight relations present if considering touching relations) in the 9-intersection [10,14]. By exploiting the converse table (Table 2), the ordered interval relations turned into 74 total possible relations in the 9+-intersection [52], shown in Table 3.

Table 3 shows each symmetric pair of relations. It is clear that there is an organizing principle connecting the relations together. This set of relational images, identified by 9+-intersection matrices [52], forms the vocabulary for the remainder of this paper. Relations in subsequent sections of the paper will be identified based on their names from [14].

3. Conceptual Neighborhood Graphs

Relation sets (such as in Section 2) form the vocabulary of spatial and temporal relations between objects. These features become critical features of human-centric query systems [43,53]. The vocabulary of a relation set, however, is not enough. There is an implicit set of relationships between the relations themselves insofar as when an object is deformed in a certain way, there are predictable behaviors exhibited in the relations that describe the configuration. The research outcome that describes this deformational pathway is called the conceptual neighborhood graph [19].

Freksa [19] first utilized the conceptual neighborhood graph principle to explain the relationships between the Allen interval algebra relations under three specific conditions (Figure 3):

• anisotropic scaling, namely one of the objects has a boundary point moved, while the other three boundary points in the scene remain fixed (called the A neighborhood),
• translation, namely one of the objects moves without changing its duration while the other is unaltered (called the B neighborhood), and
• isotropic scaling, namely one of the objects grows (or shrinks) the same amount at each boundary point while the other remains unaltered (called the C neighborhood).

Figure 3. Three homeomorphic deformations that can be applied to temporal intervals: (a) anisotropic scaling; (b) translation; and (c) isotropic scaling [54]. For intervals, anisotropic scaling is a monodirectional stretching or contraction of the object, translation is simply movement, and isotropic scaling is proportional growth/decline in both directions simultaneously.

Figure 3. Three homeomorphic deformations that can be applied to temporal intervals: (a) anisotropic scaling; (b) translation; and (c) isotropic scaling [54]. For intervals, anisotropic scaling is a monodirectional stretching or contraction of the object, translation is simply movement, and isotropic scaling is proportional growth/decline in both directions simultaneously.

In essence, whenever a change in relation is encountered between the prior state and the modified state, an edge is constructed in the conceptual neighborhood graph. The three conceptual neighborhood graphs are shown in Figure 4.

It is important to recognize that these three distinct conceptual neighborhood graphs are necessarily different. The differences in the conceptual neighborhood graphs are the direct result of the type of allowable deformation. To highlight what is occurring, consider the relation equal. If two objects that start as equal have one of the four boundary points move (without loss of generality, one of A’s boundary points), the direction of movement will change the relation in each case to something different. One pair of boundary points will however remain fused together. This changes equal to either starts, started by, finishes, or finished by. This is shown in Figure 4(a). If that same equal configuration undergoes translation, if object A moves left, it now overlaps object B. Similarly, if it moves right, it now is overlapped by object B. This is shown in Figure 4(b). Finally, if an equal object grows in both directions, it will now contain object B. Similarly, if it shrinks, it will now be during object B. This is shown in Figure 4(c).

Several other ways of envisioning conceptual neighborhood graphs have been found in the literature over the last 30 years. Examples include accounting for changes to the topological structure of an object by adding holes or separations [47], least matrix difference [44], intersections and unions of conceptual neighborhood graphs to reflect various purposes [46], mixed granularity neighborhoods [45], discretization neighborhoods [24,48], and association rules mining neighborhood graphs [24,48]. Independent of the type of conceptual neighborhood graph, rules exist to define what constitutes an edge in the graph.

As this paper progresses, the relations from Table 3 will be placed into conceptual neighborhood graphs mirroring Figure 4.

4. Simulation Protocol

Recent advances in conceptual neighborhood graphs involve discretized relations. Hall and colleagues [24,48] constructed conceptual neighborhood graphs in ${\mathbb{Z}}^2$ using a simulation protocol that involved fixing a discretized object in ${\mathbb{Z}}^2$ of a fixed size and systematically moving another object of fixed size pixel by pixel throughout an embedding grid. After the configuration was finished, the embedding was reset, with one of the objects changing its dimensions, and the process continued. This process continued until all possible configurations were completed between discretized rectangles. This simulation protocol is summarized in Figure 5 as applied to a subspace of ${\mathbb{Z}}^1$ large enough to bound a particular object on either side of another ground object A.

This dataset records the duration of each object, the position of the moving object, and the topological relation between the two interval objects. With this data, an SQLite database was queried to extract relations symbolic of each relevant homeomorphic deformation based on information in the rows of the dataset, consistent with Hall and colleagues [24].

Similar to the prior protocol, when defining each conceptual neighborhood graph, two possible types of neighbors could exist: persistent neighbors and coincidental neighbors. Persistent neighbors are neighbors such that when both relations can exist at a particular pair of durations of their constituent objects, they are always neighbors. A good example of this would be relations such as before and meets from the Allen interval algebra. Coincidental neighbors are just that: they happen by chance, but they are not always neighbors under all configurations where those relations are available. An example of that phenomenon would be overlapMeet and overlapCovers from the region-region relations in ${\mathbb{Z}}^2$ [24,48]. These are only neighbors where the size of at least one of the objects is insufficient to support overlap, the appropriate intermediary relation that would be expected from continuous space [43,55]. We are interested predominantly in persistent neighbors in this paper.

5. Results

The SQL queries from Section 4 can be visualized as graphical outputs. Section 5 shows the outcomes of these graphs. Each graphic will be broken into types of relations, similar to how the relations were defined in Dube [14]. The results for isotropic scaling, translation, and anisotropic scaling are shown in Figure 6, Figure 7 and Figure 8.

To get to temporal relations, we can simply append the graphs together such that the relations are temporally symmetric. The temporally symmetric relations are equal, contains, inside, containsTotal, insideTotal,minimalInside, minimalContains, minimalInsideTotal, minimalContainsTotal, and equalNoInterior. The temporal neighborhood graphs are shown in Figure 9, Figure 10 and Figure 11.

6. Comparison of Discretized Temporal Conceptual Neighborhood Graphs to Continuous Temporal Conceptual Neighborhood Graphs

Dube [14] demonstrated that the discretized temporal interval relations fit into categories that reflect a corresponding Allen interval relation [9]. As such, it is hypothesized that the conceptual neighbors of each relation within a specified conceptual neighborhood graph are either of the same Allen family (such as in Table 3), or neighbor a relation in an Allen family that is a conceptual neighbor in the corresponding conceptual neighborhood graphs of the Allen interval relations (such as in Figure 4). Such a result would be further evidence of the simulation method working. Hall and Dube [24] demonstrated the discretized region-region relation conceptual neighborhood graphs followed this exact behavior. Given that these objects are of co-dimension 0 to their embedding space (just as in the region-region case), the same behavior is expected. The outcomes are shown in Figure 12, Figure 13 and Figure 14.

7. Discussion, Conclusions, and Future Work

In this paper, the set of discretized temporal relations [14] were placed into three distinct conceptual neighborhood graphs, each representing a different homeomorphic deformation of a single discretized temporal interval. It was shown that the discretized temporal interval relations organized into these neighborhood graphs in a predictable manner from the corresponding continuous temporal interval relations [9,19], a behavior similar to the discretized region-region relations [12,24].

The results of this work provide a critical piece of infrastructure for spatio-temporal knowledge discovery. Knowledge discovery predominantly requires the detection of events, in this case spatio-temporal events. To determine events, it is imperative to be able to map human language onto computational data sources as ultimately humans will ask the questions or an artificially intelligent agent will need to translate the results back into human language itself [25,26,27,28,29,40,43,56]. While the relations themselves represent the primitive space for that mapping, they do not represent fully the construction of human language which might abstract away less critical details that are unnecessary for the circumstance (such as the within operator in a modern GIS). Conceptual neighborhood graphs provide the easiest possible way to model that behavior because humans naturally group relations that share fundamental similarity. An easy example of this is shown in Figure 15. Dube and Egenhofer [28] demonstrated that this concept is not only manifest in reasonably topological spatial terms, but also showed that those terms held specific mathematical properties within the conceptual neighborhood graphs themselves, specifically convexity. Convex subgraphs maintain all possible shortest paths between their vertices from within the graph overall. Most qualitative spatial reasoning work has focused on conquering the layers of the spatio-temporal stack; this work focuses on conquering the stacking dimension.

Not only is this concept fundamental to defining terms; it is also a matter of pragmatism in our data forward world with respect to data filtration [26]. Decision support systems function differently than do information systems; information systems determine what fits sufficient criteria only; decision support systems are interested in ranking solutions. The conceptual neighborhood graph provides the framework by which a decision support system can support “next best” results. Furthermore, a conceptual neighborhood graph architecture (if endowed in the decision support system’s query logic or within the programming of an artificially intelligent geospatial agent) allows for a user to specify the exact nature of the specificity of their query. Because discretized representations of data are fundamentally uncertain with respect to a continuous space that they in effect model, it is possible that particular edges of the conceptual neighborhood graph may lead to relevant neighbors in a contextual application, while others would not. Without conceptual neighborhood graphs as a guiding paradigm, this problem is not simple; with a conceptual neighborhood graph, this process has simple guideposts to direct the refinement of the query itself. These innovations lead to smarter data systems overall.

While there are many conceptual neighborhood graphs that have been developed (almost as numerous as relation sets themselves), there is still much work to be done. While most conceptual neighborhood graphs focus on homeomorphic deformations, there are other very crucial transformations that fundamentally alter objects structurally. Most notably for temporal intervals is the concept of the conversion between discretization and vectorization. This paper demonstrates the linkage between these dimensions. The future work that this leads to is a conceptual neighborhood graph that fundamentally links these concepts, effectively a layered conceptual neighborhood graph. An example of this concept is shown in Figure 16. This work has been suggested both here and in its region-region counterpart [24].

To fully realize this line of research, two fundamental concepts remain. The first is an appropriate definition of lines in ${\mathbb{Z}}^2$ and their corresponding relation sets and conceptual neighborhood graphs. The second is to consider yet another type of conceptual neighborhood graph: a generalization neighborhood. Cartographic generalization is a critical component of geographic information science [56,57,58,59]. In our big data world, we must cope with variety of data representations. To appropriately do that, we must come to a fundamental understanding and synergy of continuous and discretized spaces. By considering both a discretization/vectorization and cartographic generalization neighborhood, we will be able to leverage more data resources toward our worthy tasks, whatever they are [26]. For example, consider the term outside. There are corresponding versions of separated in all relational sets, which is not surprising because it is one of the core parts of the Natural Semantic Metalanguage [60]. Though objects may not structurally be the same, the same root concept is involved in this characterization. More importantly, because we generalize objects on the fly [61,62], two representations may actually be discussing the same object with that object existing in different cartographic forms. Both of these scenarios demonstrate the necessity to understand this particular concept. The basic infrastructure to consider a generalization neighborhood exists within continuous spaces [55,63,64], but it does not within discretized spaces yet due to the definition of lines in ${\mathbb{Z}}^2$.

Of course, the overall goal for all of this research agenda is the creation of a fully endowed qualitative spatio-temporal decision support system [65] and eventually geospatial AI with similar capacity [25,26]. To fully realize this, it is more than just a technical exercise of data structures and theoretical reasoning formalisms; this process involves the intrinsic knowledge of how humans interact with a system and making that usable on those terms. The qualitative spatial reasoning and spatial cognition community need to come together to figure out the structure appropriate for a query language designed with these principles in mind. Theoretical formalisms and data structures guide this process, but there is much work with human subjects to be done to get the linguistic concepts correct, a basic concept that made a language like SQL successful [66].