From Where to What: The Geo-Intervention Modelling Framework

1. Introduction

Over a million deaths occur every year from preventable causes, such as traffic collisions [1] and homicide [2]. Interventions implemented in geographic space, such as hotspot policing [3] and traffic camera monitoring [4], have had success in greatly reducing the number of preventable deaths. In this paper, these interventions are referred to as geospatial interventions (geo-interventions), which are actions applied to geographic spaces that are intended to alter target outcomes [5,6]. Geo-interventions result in benefits not only for the vulnerable, but also for people living in the surrounding area. A few examples include: reducing the patient load on hospitals [7], improving traffic flow [8], and increasing access to education [9] in relation to reducing traffic and crime related deaths. Geo-interventions are often designed within the constraints of available resources, such as expertise, data, and budget, which are difficult to optimally balance and are heavily reliant on domain expertise [10,11,12]. With high performance computing hardware and the abundance of geospatial data today, analyzing thousands of geo-intervention scenarios within practical periods of time for decision making has become possible [13]. Decision makers are then able to simulate the effects of potential interventions and evaluate outcomes before heavy investments in testing or implementing interventions are spent. This also helps discover more evidence-informed interventions with the available data. Currently, many studies related to geo-interventions only consider a small range of alternative intervention scenarios, constraints, and models predetermined by experts based on experience, research, and experimentation [11,14,15,16,17]. A literature review of over 3000 articles indicated wide use of multi-criteria decision analysis for environmental applications (e.g. waste management, site selection), but revealed heavy dependency on pre-determined scenarios or criteria to evaluate alternative geospatial outcomes [18]. Another similar review of 300 articles demonstrated that decision makers must provide criteria and preference information supplied by experts or stakeholders, rather than systems producing them autonomously [19]. The reliance on experts create difficulty in comparing results and approaches in a standardized manner – leading to issues in optimizing interventions, while leveraging all available data and models. In addition, these studies often identified where to implement geo-interventions with computer models, but to the best of our knowledge, seldom identify what these geo-interventions are [20,21,22,23,24,25]. A recent review of 58 articles [26] and a past review of over 300 articles [11] on spatial analysis for health research highlighted that geospatial approaches primarily focus on identification of spatial patterns, but seldom focus on prescriptive actions or interventions. For example, spatial clustering techniques are used to locate road traffic collision hotspots, but what actions can be taken to decrease road traffic collisions at these hotspots (e.g., deploying traffic enforcement devices, reducing road widths) are often left to decision makers and experts to identify prior or post analysis. Lastly, our past review of 136 articles on spatial decision support systems found that studies leveraging Geographic Information Systems (GIS) were mostly theoretical or experimental, lacking evidence of usefulness to indicate practical success [20]. Thus, we emphasize the importance of not only modelling spatial phenomena, but rather how these phenomena can be altered to influence the real world.

This paper aims to provide a generalized modeling framework for automatically generating geo-interventions using geospatial data for a target outcome, without defining these geo-interventions prior or post analyses. Specifically, the following research questions are answered: (Q1) how do we model geo-interventions? (Q2) how do we generate geo-interventions? and (Q3) how do we apply the geo-intervention generation framework using available models and data? These research questions are answered with the following research contributions respectively: (C1) A framework component for using predictive models to generate geo-interventions for a target outcome, (C2) A framework component for optimizing geo-interventions based on a target outcome, and (C3) A case study of generating geo-interventions to reduce traffic collision deaths and serious injuries in Toronto, Ontario, Canada.

2. Materials and Methods

To answer research questions Q1-3, a three-step approach was taken involving: (1) problem statement, (2) framework specification, and (3) case study. This three-step approach was adapted from a consensus of approaches from [27]. The first step, problem formulation, defined concepts and relationships of actions, locations, and outcomes as they relate to geo-interventions. Research questions Q1-2 were structured into quantifiable optimization problems based on spatial data containing actions and locations. The second step, framework specification, created conceptual components and sub-components based on the definitions of actions, locations, outcomes, and geo-interventions in the problem statement. The definitions of actions and locations were captured as a spatial data component, while the definitions of outcomes and geo-interventions were captured in the outcome modelling and geo-intervention generation components, producing research contributions C1-2 to answer questions Q1-2. These three components formed the geo-intervention modelling framework, which structured the conceptual process of research question Q3. In the third step, a case study in Toronto, Ontario, Canada was designed to produce research contribution C3 by demonstrating use of the framework for a real-world problem, where geo-interventions were generated as locatable infrastructure changes to reduce the number of Killed or Seriously Injured (KSI) individuals, using the geo-intervention modelling framework. Figure 1 visualizes the three-step approach, while Section 2.1 to Section 2.3 detail each of the three steps taken.

2.1. Problem Formulation

This section defines the concept of geo-interventions based on the sub-concepts of actions, locations, and outcomes, while further structuring these defined concepts into quantifiable optimization problems. Geo-interventions are actions that change particular locations or areas in an attempt to produce desirable outcomes in the context of specific decision-making problems, consisting of three concepts seen in Figure 2 [28]:

• Actions: the actions that change entities or phenomena at certain locations or in certain areas (e.g., addition of traffic lights and reduction in road widths)
• Locations: the target locations or areas that actions in are applied to for producing a desirable outcome in the context of decision-making (e.g., neighborhoods with high traffic collisions and streets with high-speed limits)
• Outcomes: the outcomes from applying actions to change the locations or areas in 2. (e.g., reduced traffic collisions and improved traffic flow)

The problem of generating geo-interventions then becomes searching for actions at target locations that influence outcomes in a desired manner. For example, reducing road widths (the action) on streets with high traffic collision rates (the locations) to decrease road traffic injuries (the desired outcome) [29,30,31,32]. Section 2.1.1 to Section 2.1.4 structure the problem of generating geo-interventions by quantifying the three concepts (actions, locations, and outcomes), and their interactions into optimization functions representing desired outcomes.

2.1.1. Actions

Entities, phenomena, and their characteristics in space can be represented as a set of variables containing adjustable values. Actions are then the increase or decrease in these values, indicating changes applied to the entities, phenomena, or their characteristics. For example, a road contains a width and length as two variables $\left\{x_1,x_2\right\}$ with two values $x_1= 25$ and $x_2=100$ (and indices $j=1$ and $j=2$ respectively) representing the characteristics of a road entity. A change in road width and length values $\Delta x_1=-5$ and $\Delta x_2=+25$ represent the actions. Thus, a series of actions $A$ for variable $j$ can be defined as changes $\left\{\Delta x_1\dots \Delta x_n\right\}$ for a series of variables $\left\{x_1\dots x_n\right\}$, representing entities, phenomena, or their characteristics:

$$A_j=\left\{\Delta x_1\dots \Delta x_n\right\}\in ℝ$$

(1)

2.1.2. Locations

Each series of actions has a unique location, which is referred to with an index value referencing a set of paired values containing coordinates in space. These coordinates form spatial objects (point, line, polygon) in space [33], representing unique locations. Points consist of a pair of coordinates, lines consist of an ordered set of connected points, and polygons consist of an ordered set of connected points with the starting and ending points connected together to form an area [34]. A point $P$ is represented as a pair of ordered coordinates $\left(a, b\right)$ (e.g., (43.65, -79.38)):

$$P=\left(a,b\right)$$

(2)

A series of ordered points $\left(P_1\dots P_n\right)$ are then used to represent an $i$th unique location $L$ described as either a point, line, or polygon (e.g., (43.65, -79.38), (44, -78), (45, -77)):

$$L_i=\left(P_1\dots P_n\right)$$

(3)

Thus, each series of actions $A$ has a unique location $L$ referenced by index $i$, with its own series of changes per location $\left\{\Delta x_{i,1}\dots \Delta x_{i,n}\right\}$:

$$A_{i,j}=\left\{\Delta x_{i,1}\dots \Delta x_{i,n}\right\}\in ℝ$$

(4)

2.1.3. Outcomes

Each outcome is a target entity, phenomenon, or characteristic of interest with a unique location. These outcomes are expressed as a variable with a single value at each location and can be affected by a series of actions at the same location. For example, a series of actions at a location containing road width and length changes can change an outcome representing the number of road traffic collisions at the same location. For simplicity, we assume any observed series of actions $A$ influence the outcome $Y$at location $i$, where outcome $Y_i$ changes depending on actions $A_{i,j}$:

$$Y_i= f\left(A_{i,j}\right)$$

(5)

Under this assumption, we then build a model of outcomes by finding a function $f$ that most accurately produces outcomes $Y_i$ based on actions $A_{i,j}$. This outcome model predicts actual outcome $Y$ as $\widehat{Y}$ at locations $i$ based on a series of variables $X_{i,j}$:

$${\widehat{Y}}_i= f(X_{i,j})$$

(6)

For simplicity, we say that the outcome model seeks to minimize the error $e$ between the absolute value of the actual outcome value $Y$ and estimated outcome value $\widehat{Y}$ at locations $i$:

$$minimize e_i =\left|Y_i - {\widehat{Y}}_i\right|\mathrm{given}{\widehat{Y}}_i= f(X_{i,j})$$

(7)

2.1.4. Geo-Interventions

Given that outcomes $Y_i$ can be estimated with acceptable error $e_i$ using the outcome model, we can predict these outcomes as ${\widehat{Y}}_i$ when actions $A_{i,j}$ on variables $X_i$ are applied:

$${\widehat{Y}}_i= f\left(A_{i,j}\right)$$

(8)

The actions $A_{i,j}$ represent the generated geo-interventions, while the predicted outcomes of those geo-interventions correspond to ${\widehat{Y}}_i$. The next step then becomes finding the actions $A_{i,j}$ that lead to the most desirable predicted outcomes ${\widehat{Y}}_i$, where the choice of minimizing or maximizing ${\widehat{Y}}_i$ is selected based on the context of the decision-making problem (e.g., it is desirable to minimize road traffic collisions and maximize traffic flow):

$$minimize/maximize {\widehat{Y}}_i=f\left(A_{i,j}\right)$$

(9)

Equation (9) is a local optimization problem that seeks to optimize the predicted outcomes ${\widehat{Y}}_i$ for each individual location $i$ [35]. However, most studies consist of more than one location, and a global optimization problem is more appropriate [36]. Thus, we include an aggregation metric $m$, which considers all predicted outcomes ${\widehat{Y}}_i$ together:

$$minimize/maximize m\left({\widehat{Y}}_i\right)=f\left(A_{i,j}\right)$$

(10)

The aggregation metric $m$ is dependent on the context of the decision-making problem. For example, we can sum the number of traffic collisions at each location $i$ in the study area using the aggregation function $m=\sum {\widehat{Y}}_i$ and minimize the result of this value, if the intention was to reduce the number of traffic collisions across the study area. Since the number of locations $i$ and actions $A$ can be large, it may be infeasible to comprehensively try all values of actions $A_{i,j}$ [37,38]. Instead, we may reduce the number of values to try by limiting the locations $i$ and actions $j$ to subsets $\widehat{i}$ and $\widehat{j}$ respectively, where these subsets are selected based on the decision-making problem (e.g., locations approved for change by policy makers, actions that are possible according to engineers):

$${\widehat{A}}_{\widehat{i},\widehat{j}} such that \widehat{i}\subset i, \widehat{j}\subset j$$

(11)

Furthermore, we may also reduce the range of values for each action $A_{i,j}$ to be between sensible values $\left\{A_{i,j}^{min} \mathrm{...} A_{i,j}^{max}\right\}$, determined by the decision-making problem (e.g., 3-5 meter road width reductions following guidelines, 50-100 meter road length increase due to budget):

$$A_{i,j}\in \left\{A_{i,j}^{min} \mathrm{...} A_{i,j}^{max}\right\}$$

(12)

2.2. Framework Specification

Based on the problem formulation in Section 2.1, this section specifies conceptual components and their sub-components for the concepts of actions, locations, outcomes, and geo-interventions. Recall research questions Q1-2. For Q1, we produce research contribution C1 as the outcome modelling component, conceptualizing the modelling of geo-interventions by predicting outcomes as a function of variables and actions from spatial data. For Q2, we produce research contribution C2 as the geo-intervention generation component, conceptualizing the generation of geo-interventions produced by the outcome models through the application of optimization algorithms and prior knowledge to limit the range of actions to optimize for. Section 2.2.1 to Section 2.2.3 detail the specifications of the spatial data, outcome modelling, and geo-intervention generation framework components, which are summarized in Table 1.

2.2.1. Spatial Data Specifications

Spatial data, commonly used in GIS, capture entities, phenomena, and characteristics as variables $\left\{x_{i,1}\dots x_{i,n}\right\}$, tying these variables to the real-world by locating them in geographic space as described in Section 2.1.2 [39]. By having records of these variables and their locations through spatial data, we can model these variables and their changes as actions $A_{i,j}$ in relation to an outcome $Y_i$ for the $i$^th location, which solves the problem of representing actions, locations, and outcomes in geo-interventions [40]. Thus, we required that the spatial data component contained variables $\left\{x_{i,1}\dots x_{i,n}\right\}$ and their locations $L_i$, where sub-components were organized into vector data, namely point, line, and polygon type spatial data. We focused on vector data to be the spatial data component as it enabled more flexibility and level of detail in geometric shapes (point, line, and polygon) than raster data, which required consistent square or rectangular pixels with level of detail dependent on the image resolution [41,42]. However, we also required an optional sub-component for incorporating raster data, such that raster data be converted to vector data, as is possible with various GIS data processing methods (e.g., polygon grids [42], edge-tracing [43], object detection [44]). For improving interpretability of the locations for geo-interventions, we also required the inclusion of an optional spatial feature engineering sub-component that creates, modifies, and removes variables based on the available variables and geometry data from the vector sub-components [45,46]. In addition, this sub-component can help standardize points, lines, and polygons into common spatial units of interest (e.g., census boundaries for public health interventions [47], regularized grids for simulating land use change [48]) by creating, modifying, and updating variable values inside each spatial unit [49,50]. This spatial feature engineering process enables the possibility to create additional variables that capture the spatial effects among entities, phenomena, and characteristics by aggregating each record and their variables through spatial proximity and relationships [51,52,53].

2.2.2. Outcome Modelling Specifications

Outcomes $Y_i$ can be seen as one of the variables $\left\{x_{i,1}\dots x_{i,n}\right\}$ captured in the spatial data, where one of the variables $\left\{x_{i,1}\dots x_{i,n}\right\}$ are removed and set as the outcome $Y_i$ for modelling. Thus, the outcome modelling component attempts to solve the problem of finding a function that predicts the outcome given the variables from the spatial data ${\widehat{Y}}_i= f(X_{i,j})$, such that the difference between the predicted outcome ${\widehat{Y}}_i$ and actual outcome $Y_i$ is minimized as seen in Equation (7) from Section 2.1.3. Similar to concepts of predictive modelling [54] (e.g., machine learning [55], statistical modelling [56]), the outcome modelling component conceptualized how to solve this problem by framing outcome models as a function that processes variables $X_{i,j}$ and model parameters $\left\{p_1\dots p_n\right\}$ into predicted outcomes ${\widehat{Y}}_i$, which we organized into the four sub-components of variables, parameters, models, and predicted outcomes. The variables sub-component was required to hold records from the output of the spatial data component, where each record contained a point, line, or polygon with one or more variables. The parameters sub-component was required to contain a set of optional values that altered the behavior of models to gain potential benefits, such as improved model performance, interpretability, or efficiency [57,58,59]. The models sub-component was required to process the variables and parameters, discover patterns among these variables that had predictive relationships with a selected outcome, alter its behaviors based on the parameters if applicable, and produce predicted outcomes based on the discovered patterns. The predicted outcomes sub-component was required to hold records consisting of points, lines, and polygons with predicted outcome values from the model. We note that the minimization of predicted and actual outcome differences is a simplification of predictive modelling, as models tend to not generalize well to new data, performing well on training data, but poorly on new unseen data [60]. There is also not a single model that works well for every problem, and the exploration of multiple models can result in improved predictive performance [61]. Thus, we added three additional sub-components of model selection (optional), model metrics, and best model, that held algorithms executing strategies (e.g., cross validation [62]) to evaluate and select the best performing model out of a test of multiple models, using a defined model metric on the predicted outcomes.

2.2.3. Geo-Intervention Generation Specifications

As noted in Equation (8) of Section 2.1.4, generated geo-interventions are actions $A_{i,j}$ that can predict outcomes ${\widehat{Y}}_i$ given a model $f$ with some error $e_i$, and can be expressed as ${\widehat{Y}}_i=f\left(A_{i,j}\right)$. Thus, the geo-intervention generation component ideally generates geo-interventions that contain desirable predicted outcomes, where desirability is measured by some aggregation metric $m$ with the goal of maximizing or minimizing this metric $m\left({\widehat{Y}}_i\right)=f\left(A_{i,j}\right)$. The geo-intervention generation component was organized into five required sub-components of actions, outcome metric, optimization algorithm, best outcomes, and best geo-interventions. The actions sub-component was required to hold values that represented a change in variables $A_{i,j}$, available in the best model sub-component from the outcome modelling component, at different locations $i$. The outcome metric sub-component was an aggregation metric $m$ that was required to be selected or defined by the user, representing a measure of the desirable predicted outcome. The optimization algorithm sub-component required that a strategy was selected or defined to find the most desirable outcome, using the best model sub-component from the outcome modelling component, and the actions and outcome metric sub-components. The best outcomes sub-component represented the most desirable predicted outcomes found by the optimization algorithm sub-component, while the best geo-interventions are the associated actions that produced the best outcomes using the best model. In Equation (11) and (12), a set of constraints on the actions and locations reduce the number of actions and locations to more appropriate values based on domain knowledge, saving computation time and likely discovering better geo-interventions [63,64]. Thus, we also included an optional constraint sub-component that limited the range of values in the action sub-component for the optimization algorithm to evaluate. In addition, an optional variable metrics sub-component was included to quantify the association of variables in the best model in relation to the predicted outcomes. These metrics provide information on the strength of association (e.g., variable importance [65], correlation coefficient [66]) to the predicted outcomes, which helps set constraints on actions (e.g., top-n actions with strongest association) and verify domain knowledge assumptions in relation to the actions and best model.

2.3. Case Study Design

Road traffic injuries are among the leading cause of death for Canadians 15 to 49 years of age in 2019, ~8% of all deaths for populations aged 15 to 49 years in Canada [67]. Toronto is one of the largest cities in Canada with over 2 million people in 2021 [68] and approximately 55 fatal traffic collisions per year from 2006 to 2021 [69]. Since 2017, the City of Toronto has initiated the Vision Zero Road Safety strategy to reduce road traffic fatalities, with the goal of reaching zero road injury related fatalities through a safe systems approach [70,71]. The City of Toronto and Toronto Police Services collect and distribute detailed geodata on traffic collisions in Toronto through the City of Toronto Open Data Portal [72] and the Toronto Police Service Public Safety Data Portal [73]. In addition to traffic collision data, the City of Toronto Open Data Portal also provides government curated geodata (e.g., infrastructure and sociodemographic entities), which may be linked to the traffic collision data.

We applied the geo-intervention modelling framework to a case study in Toronto, Ontario, Canada, producing research contribution C3 to answer research question Q3. For the case study, we defined our outcome as the number of Motor Vehicle Collisions (MVC), while our geo-interventions as the changes in city infrastructure. Our objective was to reduce the number of MVCs across the city by identifying modifiable infrastructure-related geo-interventions. We organized the case study around the three components of the geo-intervention modelling framework, namely spatial data, outcome modelling, and geo-intervention generation. Section 2.3.1 to Section 2.3.3 detail the approaches taken for the case study based on these three components.

2.3.1. Spatial Data Approach

Spatial data for the outcome of motor vehicle collisions was recorded by the Toronto Police Services, available on their Public Safety Data Portal [73]. Spatial data for city infrastructure geo-interventions was available on the City of Toronto’s Open Data Portal [72]. We also included crime from the Public Safety Data Portal, and amenities and other locations of interest (e.g., cultural hotspots, places of worship) from City of Toronto’s Open Data Portal, as additional variables to include in the outcome modelling component. A total of 21 datasets with 518 variables (columns) and 1,488,148 spatial entities (rows) were used for the case study as seen in Table 2, where all datasets were of point geometry, except for centrelines.

We spatially aggregated the 21 datasets into regularized vector grids of 10 by 10, 40 by 40, and 80 by 80 cells to improve interpretability and provide simpler visualization of the results. Inside of each cell, spatial entities from each dataset had all variables (columns) aggregated by the following statistics: sum, mean, median, min, max, variance, skew, standard deviation, standard error of the mean, and mean absolute deviation. Depending on the geometry of the dataset and data type of variables inside each dataset, we also applied different additional aggregation processes seen in Table 3.

2.3.2. Outcome Modelling Approach

For the case study, we applied Automated Machine Learning (AutoML) for the outcome modelling component as it automates several sub-components related to improving model performance, including the selection of variables, parameters, and models [95]. We tested two python packages for AutoML: auto-sklearn and tpot. The package auto-sklearn uses Bayesian optimization [96], meta-learning [97], and ensemble construction [98] to automatically select the best model from a test of several models and parameters [99,100]. The package tpot uses genetic programming [101] to automatically optimize and design machine learning pipelines through pipeline operators, including the operations of feature engineering, model selection, and parameter tuning [102]. For the variables sub-component, the output of the spatial feature engineering sub-component from the spatial data component was used, while the models sub-component contained both auto-sklearn and tpot models, which automated the parameters and model selection sub-components given a minute to build models each. For the metrics sub-component, we used Mean Absolute Error (MAE) [103], an aggregate measure of the errors between the actual and predicted outcomes, representing the MVCs. Thus, the best model sub-component was determined by the auto-sklearn and tpot models, which automate parameter tuning, model selection, and variable selection/creation based on the MAE metric.

2.3.3. Geo-Intervention Generation Approach

After the best model for predicting MVCs was found using the outcome modelling component, it was used for the generation of geo-interventions based on a change of variable values, representing actions, used in the best model. Thus, the actions sub-component represented increases or decreases to the variable values, containing changes to infrastructure, amenities, crime, and other locations of interest, for the best model. The variable metrics sub-component contained variable importance metrics [65] from the best model, which provides a measure that can be used to rank the most to least influential variables corresponding to the predicted MVCs. We then used Bayesian optimization [96] as the optimization algorithm sub-component to find the best outcomes and geo-interventions, based on the constraints and largest reduction in predicted MVCs. The best outcomes sub-component represented the most optimal reduction in MVCs, while the best geo-interventions component represented the associated increases or decreases in infrastructure-related variables (actions) that led to the best outcomes. For the constraints sub-component, we limited the number of optimization iterations to 1000, along with 1 initial iteration containing the actual variable values from the data, and 100 random iterations to diversify the optimization search space, for a total of 1101 iterations. We also limited the variables for the optimization algorithm to the top three most important modifiable infrastructure variables (determined by the variable importance metric), which was the top three most important variables of the automated speed enforcement cameras, watch your speed devices, red light camera, police facilities, ambulance stations, bicycle parking, transit shelters, wayfinding structures, schools, and childcare centers datasets. In addition, we also included location constraints for two scenarios: (1) areas where the top traffic-related variable had higher than average values, and (2) areas where the top three infrastructure-related variables had lower than average values. In scenario one, we limited optimization to areas with higher-than-average levels of traffic, while in scenario two, we limited optimization to areas with scarcity of important infrastructure for reducing MVCs. We used these scenarios to demonstrate the importance of domain knowledge (e.g., knowing which areas to apply geo-interventions), and its effects on finding the best outcomes and geo-interventions.

3. Results

After applying the three-step approach described in Section 2, we developed the geo-intervention modelling framework and conducted a case study to demonstrate the framework. The geo-intervention modelling framework answered research questions Q1-Q2. For Q1, the framework structured a data modelling process to create prediction models for a target outcome associated with potential geo-interventions, while for Q2, the framework transformed the problem of generating geo-interventions into a global optimization task. Finally, the case study applied the framework to reducing MVCs in Toronto, Canada, with infrastructure-related geo-interventions, demonstrating real-world use of the framework to answer research question Q3. Section 3.1 details the geo-intervention modelling framework and Section 3.2 reports the results of the case study.

3.1. Geo-Intervention Modelling Framework

The geo-intervention modelling framework consisted of three components: (1) spatial data, (2) outcome modelling, and (3) geo-intervention generation. The spatial data component received vector spatial data and standardized it into a set of variables that represented phenomena, entities, or their characteristics. The relevant raster sub-component was optionally included as raster data can be processed into vector structured data with various GIS approaches (e.g., polygon grids [42], edge-tracing [43], object detection [44]). The set of variables from the spatial data component were then used to build models that predicted a target outcome in the outcome modelling component. Here, a model’s accuracy was measured by a metric on the predicted outcomes. This component was divided into two main sub-component processes of models and model selection. Optionally, parameters can be included in each model to adjust model behaviors, with the goal of improving model performance. The models sub-component’s objective was to discover patterns in the variables that can predict an outcome variable with performance measured by the model metric sub-component. If more than one model was built, the model selection sub-component must be defined with a selection strategy or algorithm (e.g., cross validation [62], selecting the model with the best model metric value) based on the predicted model outcomes and model metrics. This model selection process results in a best model being selected to generate geo-interventions in the geo-intervention generation component. This component consisted of sub-components that formed an optimization problem, where the actions sub-component was optimized based on the constraints and outcome metric sub-components. The optimization algorithm process searched for the most optimal outcome metric value given the actions and optional constraints sub-components. The actions sub-component contained changes to the best model’s variables to produce a set of predicted outcomes associated with those changes, while the constraints sub-component contained limitations on the possible actions to optimize for based on prior knowledge. The outcome metrics sub-component sets the objective for the optimization algorithm sub-component, in which a measure is defined based on the predicted outcomes, and a minimization or maximization objective is chosen for the algorithm based on this measure. The optimization algorithm sub-component produced the best outcomes and geo-interventions sub-components based on optimizing for the selected outcome metric, which represents the generated geo-interventions and its associated predicted outcomes. These geo-interventions represented the most optimal actions for each of the locations to produce the optimal predicted outcomes, given the constraints set. Appendix A illustrates minimal examples of the three framework components for better understanding, while Appendix B provides common considerations when defining and applying framework components.

Figure 3. Geo-intervention modelling framework.

Figure 3. Geo-intervention modelling framework.

3.2. Case Study

Regression-based AutoML models auto-sklearn and tpot were built for each of the three set of standardized grids, which produced six models (two for each 10 by 10, 40 by 40, and 80 by 80 grid). Out of the six models, the 80 by 80 grid autosklearn model had the best performance (lowest MAE=82.95), while the 10 by 10 grid autosklearn model had the worst performance (highest MAE=1241.93). The performances improved with higher cell counts from 40 by 40 to 80 by 80 grids, with the greatest improvement seen in 10 by 10 to 40 by 40 grid models (-1039 and -951 MAE for autosklearn and tpot respectively). The 80 by 80 grid autosklearn model was selected as the best model for the geo-intervention generation component. The performance of the six models by MAE is shown in Figure 5. An overview of the case study process, applying the geo-intervention modelling framework, is visualized in Figure A1 of Appendix C.

With the 80 by 80 grid autosklearn model as the best model (lowest MAE=82.95), permutation variable importance was calculated for 1408 variables used in the model [104]. The top 25 most important variables from the best model are shown in Figure 6. The most important variable was the number of arterial roads, followed by the number of red light cameras, transit shelters, and crime occurrences involving pointing a firearm. The rest of the variables with variable importance lower than 0.005 were mostly related to traffic volumes and crime occurrences, with a few related to infrastructure (wayfinding, fire hydrants, public art locations, litter receptacles), and one related to the curvature (sinuosity) of road centrelines. The most important traffic volume related variable was traffic_eb_cars_l_sum, the total volume of left-turning eastbound cars. This variable was selected for defining the higher than average traffic areas constraint for scenario one.

The top 25 most important infrastructure-related variables are shown in Figure 7. Asides from the number of red light cameras and transit shelters being the top variables, the number of wayfinding structures, fire hydrants, public art locations, and litter receptacles were among the top most important infrastructure-related variables. Other variables with variable importance less than 0.002 were related to bicycle, Watch Your Speed (WYS) program, school, and cultural hotspot infrastructure. The top three infrastructure variables selected as the actions for scenarios one and two were the number of red light cameras, the number of transit shelters, and the number of wayfinding structures according to their variable importance.

Geo-interventions were generated using the 80 by 80 grid autosklearn model, 1101 iterations of Bayesian Optimization, and the top three most important infrastructure variables, namely the red-light camera, transit shelter, and wayfinding structure counts. Recall that in scenario one, Bayesian Optimization was constrained to grid cells with higher-than-average traffic, while in scenario two, optimization was constrained to grid cells with lower than average red light camera, transit shelter, and wayfinding structure counts. In scenario one, the best geo-intervention occurred with a predicted 554,584 MVC, a reduction of 33,606 (5.7%) compared to the initial 588,190 MVC, while in scenario two, none of the iterations resulted in a reduction in predicted MVC. See Appendix C for details on the iterations to find optimized geo-interventions for each scenario. In scenario one, changes in red light cameras were spread out away from the downtown Toronto area, changes in transit shelters were more concentrated in the downtown area, and changes in wayfinding structures were mostly concentrated in the downtown area and along the center and lower portions (Figure 8). Similarly, the changes in predicted MVC were more concentrated in the downtown area and more spread out away from the downtown core. However, predicted reductions in MVC were seen more in the western than eastern portion of the city.

4. Discussion

This section discusses the considerations, advantages, disadvantages, limitations, and future research directions relevant to the geo-intervention modelling framework and the case study from the results in Section 3. Section 4.1 highlights several advantages from incorporating the geo-intervention modelling framework as a foundation for discovering data-driven geo-interventions in research and practice. The geo-intervention modelling framework provides adaptability and generalizability across disciplines, accelerating the transformation of research into practice by improving human trust through user guided experimentation and design. Section 4.2 discusses the disadvantages of applying the geo-intervention modelling framework related to time and knowledge. The major disadvantages of the geo-intervention modelling framework lie in the omission of very recent advances in modelling approaches and big data, which have been still developing, but rapidly emerging throughout the fields of artificial intelligence and GIS. Section 4.3 discusses the major limitations of the geo-intervention modelling framework relative to validating generated geo-interventions, and lack of empirical evidence for the utility of the framework. The limitations reveal the need for interdisciplinary research of geo-interventions, encouraging the collection of geo-intervention verification data and promoting more studies focused on geo-interventions to further strengthen the framework in the future. Finally, Section 4.4 highlights research opportunities to improve the geo-intervention modelling framework, and long-term visions relative to developments in the framework. These opportunities follow recent shifts towards data-driven decision-making and the incorporation of interdisciplinary expertise into research, both enabled by the rapid advances in artificial intelligence and large-scale computing infrastructure available today.

4.1. Advantages

4.1.1. Bridging Research and Practice

Common GIS modelling and AutoML techniques focus on quantitative approaches based on theory [20], but often do not consider the importance of practitioner knowledge [105,106]. The geo-intervention modelling framework bridges the gap of research and practice through the incorporation of past knowledge from experience or literature into technical modelling processes. For example, applying knowledge from experience about the outcome and its variables to constrain the geo-interventions to actions and areas from practice known to influence the outcome. As most models do not hold knowledge or reasoning of feasibility in relation to the decision-making problem, knowledge from practitioners and experts is invaluable in adjusting the geo-interventions for feasibility [107,108]. Thus, it is not the most optimal solution that is often desired, but the most feasible one, given available resources, that is adequate for implementation in the real world [109]. Results from GIS research are also made more feasible and easier to adopt in the real-world through knowledge from practitioners, encouraging collaboration between academia and industry practice, notably seen in fields like implementation science [110]. Implementing interventions in case studies may require large investments and commitments, with the risk of undesirable or minimal results [111]. Predicting the effects of geo-interventions under feasibility considerations, without directly implementing them, reduces risk in investment and enables exploration of a larger range of alternatives for practice, including innovations in research that have yet to reach practice.

4.1.2. Adaptability and Generalizability

The geo-intervention modelling framework is built as a system of three modular components, spatial data, outcome modelling, and geo-intervention generation, such that each component may be used individually if the input is satisfied. Thus, these components may be used in other modelling frameworks or systems, while being applied to various domains when spatial data is available. For example, we may apply the framework into public health intervention studies to consider spatial data and geo-intervention generation, incorporating more precise and locatable health interventions [112]. Another example is to modify existing GIS frameworks, where we integrate the third geo-intervention generation component to frameworks where the scope of the research is to predict outcomes or events (e.g., traffic event detection [113], predicting urban traffic air pollution [114]). Here, the predictive model can be used as the outcome model for geo-intervention generation, incorporating expert knowledge to examine feasibility through the simulation of changes to input variables and predicted outcomes in different areas. In addition, each framework component may be defined to either sensible defaults or changed to adapt to situations involving differences in resources, user requirements, and expertise [115]. As the concept of geo-interventions is broad and solely requires the transformation of spatial data into outcome models, the framework can be generalized to various domains (e.g., public health [116], environmental science [117], crime [17,118]) and updated for new models or situations, such that the decision-making problem follows the concepts of geo-intervention related actions, locations, and outcomes.

4.1.3. Improving Human Trust

Human guidance of the geo-intervention generation component is crucial in discovering feasible geo-interventions. Major barriers in adopting spatial decision support systems and machine learning are user mistrust for decision-making and lack of technical expertise [105,119]. Both barriers are tied to improve understanding and reducing complexity with engagement and transparency to increase trust for decision-making [119,120]. The geo-intervention modelling framework helps improve understanding by simplifying the modelling of geo-interventions to three major components that are generalizable to any domain, enabling non-experts with practical knowledge to be involved in the process. For example, non-experts can collaborate with engineers or developers to evaluate what data is used for the spatial data component, what considerations should be incorporated into the metrics of the outcome modelling component, and what actions and geo-intervention areas are likely to be effective or feasible. Additionally, non-experts can experiment with different scenarios, as done in this paper’s case study, to simulate the effects of different geo-interventions based on their knowledge. This improves trust by introducing user through external validation based on reasoning and past knowledge, supplementing pure quantitative approaches seen in research, to observe if the model behaves as expected with changes to the input variables [121,122]. Furthermore, a standard framework helps users compare different solutions, allowing for evaluation of different intervention ideas among stakeholders, decision-makers, studies, and analyses, which is not commonly addressed due to inconsistencies in intervention design [112]. For geo-interventions to be implemented or adopted, they must not only be feasible and accurate but also trusted by decision-makers. The geo-intervention modelling framework addresses this issue by enabling human feedback and guidance to shape the feasibility of geo-interventions, while leveraging the precision of data models for simulating impacts geo-interventions, removing need for expensive implementation.

4.2. Disadvantages

4.2.1. Temporal and Real-Time Data

The geo-intervention modelling framework did not consider temporal data and their variables, which may be important for modelling datasets that capture temporal phenomena over multiple years or decades. Temporal geodata differ from static geodata, as a dependency on time exists in the data structure and implications [123]. For example, temporal geodata has three dimensions, containing records in space (first and second dimensions) that are also sequentially dependent on time (third dimension), where each record occurs before, during, or after another record. This data can capture the temporal effects of variables and outcomes, as time-based models have been well studied and used for analyzing public health interventions [124,125,126]. For example, it is common to analyze when and how long interventions are expected to be implemented and effective. In addition, temporal geodata is often continuously collected with new records added over time, particularly real-time geodata [127,128,129]. The framework does not consider updates in data or models, and instead relies on re-processing when enough new data is available. This may result in models missing new variables and patterns due to changes over time (e.g. context drift [130]). Integrating approaches for temporal geodata into the geo-intervention modelling framework will enable the modelling of geo-interventions over time and more efficient methods for updating spatial data and outcome models on demand.

4.2.2. Knowledge Transfer

The transfer of knowledge is important in decision-making and modelling to encourage reuse, reproducibility, and collaboration that leads to desirable interventions and outcomes [131,132]. However, the geo-intervention modelling framework does not consider the transfer of knowledge from previous or existing processes and models despite a focus on modularity. Incorporating a standardized manner of knowledge transfer in the framework can promote reuse, reproduction, and sharing of models and related processes [133,134], enabling savings in resources (e.g. computational expenses required to train models [135]), wider adoption to different domains (e.g. easier reuse by various domain experts [136]), and refinement of models and processes that lead to more effective geo-interventions or accurate models (e.g. improvements of adopted models or processes to similar problems in contrast to past studies [137]). Although the framework encourages the application of knowledge to the geo-intervention modelling process, users of the framework may benefit from explicitly defined components or processes for knowledge application and transfer as seen in incremental or transfer learning [138,139] .

4.2.3. Contextual Reasoning

With the recent emergence of Large Language Models (LLMs), computers can generate natural language text similar to human-level reasoning [140,141,142,143]. This has allowed LLMs to produce working programming scripts [144], answer online store inquiries [145], analyze financial market trends [146], and suggest medical diagnoses [147]. As human guidance and knowledge greatly influences the generated geo-intervention quality, there is potential for the integration of LLMs to generate this guidance and knowledge, and perform functions based on it. For example, the LLM may provide common knowledge on what variables influence the outcome, followed by what variables are considered feasible actions, and then finally select them for generating geo-interventions. In this example, the LLM may provide known knowledge on what areas are the most effective and feasible to target from the actions, and limit the generation to these areas. However, it is noted that LLMs are still relatively new and perform similar to the wisdom of the crowds [148], but does not currently reason or operate at the human level for all tasks despite improvements [149,150] (e.g. chain-of-thought [151]), failing simple logic and reasoning tests [152,153]). Nonetheless, the framework does not consider models that offer contextual reasoning, which can be beneficial in ensuring that geo-interventions are feasible at the level of common knowledge amongst experts in the decision-making problem.

4.3. Limitations

4.3.1. Geo-intervention Verification

Due to limited availability of precise intervention data (e.g. historical changes to road design affecting intersection safety, policy shifts applied at certain locations), it was not possible at the time of study to verify that the generated geo-interventions were feasible and effective in the real world. Impact evaluations for interventions often lack data (e.g. erroneous and missing data [154], compatible standards [155], or ethical issues [156]) or sufficiently detailed data (e.g. inadequate time intervals or spatial resolution [157]), while study design alone is not adequate to indicate a successful intervention [112]. This makes it difficult to conclude interventions are effective in studies, as it introduces bias that omits external factors, unrelated to the interventions, that potentially cause beneficial effects [158]. It was also impractical to implement case studies due to the scale and expenses of the unverified geo-interventions, where most interventions are often complex and costly to implement at scale [111]. The difficulties and complexities in sourcing verification data and implementing experimental geo-interventions reveal a need for intervention related datasets, which help verify that models perform to the level of existing interventions.

4.3.2. Empirical Evidence

As this paper’s scope was to introduce the geo-intervention modelling framework, it did not yet have adequate empirical evidence of its use in the real-world, asides from the conducted case study. Evidence of adoption and utility of the framework is crucial to prove its effectiveness in variety of fields, in practice, and for different audiences. When frameworks are increasingly used in research and practice, hidden issues and limitations are further uncovered from various perspectives and impact evaluations [159], allowing for innovative improvements that create better frameworks [160]. However, this is a long and complex process, involving collaborative work in uptake and dissemination (e.g. development of software, advertising, guidance materials, partnerships) [161]. Nonetheless, frameworks are generally introduced, before empirical evidence follow, which presents a common limitation of newly introduced methodological frameworks.

4.4. Opportunities

4.4.1. Framework Extensions

Research in extending the geo-intervention modelling framework with additional components aids in establishing a standard of understanding with the base components, while allowing for work into addressing the framework’s disadvantages and limitations. For example, temporal sub-components (e.g. feature engineering temporal variables, time-series models, sequential generation of geo-interventions) may be added to all components to enable time-relevant geo-interventions. Another example involves applying the framework to different domain-specific problems. This helps identify component additions and modifications that create sub-frameworks specific to problems in particular domains (e.g. public health, transportation engineering, planning), assisting in better knowledge transfer. Incorporating LLMs as optional sub-components for knowledge support will also mediate the lack of contextual reasoning (e.g. ranking feasibility of actions from variables, evaluating practicality of geo-interventions, suggesting approaches for outcome modelling and spatial feature engineering). Work towards creating framework extensions will address noted issues in the framework related to temporality, knowledge transfer, and contextual reasoning, while offering a standardized manner for improving the framework when future issues arise.

4.4.2. Precision Geo-Interventions

The geo-intervention modelling framework provides a foundation towards precision geo-interventions, where interventions are precisely located with quantifiable actions and outcomes. Big data has enabled more granular interventions in health, allowing the field of precision public health to target smaller subpopulations among larger populations with sociodemographic information [162]. Similarly, precision geo-interventions enable granular interventions, but across space and with the distinction of generalizing to location, which is applicable to any domain that utilizes spatial data. Thus, practitioners and experts across different domains may use a consistent framework for examining not only interventions in their own domains, but the effects of related or combined interventions outside their domains, which are often not considered [163,164,165]. Examining different interventions on multiple outcomes, rather than a single outcome, is made possible to evaluate their effects, not only for a targeted outcome, but for adverse effects on non-targeted outcomes as well [166]. With recent advancements in artificial intelligence and computing power, geo-interventions may emerge as an important research topic and development in the field of Geographic Information Science (GIScience), particularly as more decision makers incorporate spatial data in their processes, strategies, and overall systems.

4.4.3. Universal Geo-Intervention Platform

The geo-interventions framework provides a standardized framework that can be used across various disciplines, which establishes the opportunity for a universal geo-intervention platform. The effectiveness of interventions vary across different populations locations, and time [167,168], while they are difficult to compare in finer spatial and temporal as intervention data are often stored and disseminated in research papers and reports without interoperable data for standardized comparison [169,170]. Hence, there is much data for developing and monitoring interventions, but not much data about interventions (e.g. resulting outcomes, relevant effects/risks on populations, and actions taken from applying interventions). Thus, it is difficult to directly compare, reuse, reproduce, and share interventions at the quantitative level across different fields of study. With recent developments in geo-foundation models (e.g. AlphaEarth [171,172], population dynamics foundation model [173]), integration, standardization, and searchability of diverse spatial data has improved drastically [174,175,176]. We envision that this will be possible for geo-intervention data as well, providing an open universal technology platform that promotes collaboration among researchers and practitioners to accelerate innovation and application of research to practice.

5. Conclusions

A generalized modeling framework for generating geo-interventions from spatial data is described in this paper with details on how to model and generate geo-interventions while applying the framework. We provided specifications on three major framework components (spatial data, outcome modelling, and geo-intervention generation) that form the framework, based on the concept of geo-interventions defined by the interactions between actions, locations, and outcomes. In addition, we demonstrated an application of the framework for generating infrastructure-related geo-interventions on reducing road collisions in Toronto, Canada. Advantages, disadvantages, and limitations regarding the geo-intervention modelling framework were discussed. Notably, the framework presented advantages in bridging research and practice through flexibility of application across disciplines for better collaboration, and integration of prior knowledge to improve human trust and understanding. However, recently rapid advances in artificial intelligence and large-scale computing have not yet been fully developed and widely adopted. This led to several disadvantages in the framework, where components for temporal data processing, reuse of prior models, and automated contextual reasoning were omitted. Perhaps the largest barriers to adopting the framework lied in the lack of data on geo-intervention and their outcomes to date, along with the application of the framework in more studies, presenting important limitations of this paper. Although the framework was introduced in this paper, it may be improved in the future with initiatives in data collection and collaboration amongst the research and practicing community. Thus, we foresee opportunities to extend the framework by incorporating additional components to address disadvantages, and eventually the development of a universal geo-intervention platform that enables researchers and practitioners to share, extend, reuse, and precisely apply geo-interventions across various disciplines. As innovations shift towards the application of standardized data and interdisciplinary knowledge to decision-making and GIS, future outcomes from proposed actions can eventually be predicted precisely at scale, accelerating discovery of effective interventions that potentially prevent the loss of lives and improve the quality of life across the world.

Appendix A.1. Spatial Data Component Example

Table A1 shows a simple example of tabular data representing vector sub-components and processed variables from the spatial feature engineering sub-component. Variables Xi,1…Xi,2 represent the original variables for each location Li, while Xi,1 + Xi,2, Nearesti, Length(Li), and Type(Li) are additional variables from spatial feature engineering, representing the sum of Xi,1 and Xi,2, the index i of the nearest location, the length of the location entity, and the type of the location entity respectively. Here, each location has a set of variables, and the spatial feature engineering process created additional variables to capture spatial relationships and properties.

Table A1. Example spatial data for the geo-intervention modelling framework.

i	Xi,1	Xi,2	Xi,3	Xi,1 + Xi,2	Nearesti	Length(Li)	Type(Li)	Li
1	1.25	2	4	3.25	2	0	Point	(43, -79)
2	2.55	4	0	6.55	1	1.41	Line	(45, -79) (44, -78)
3	5.75	6	16	11.75	2	11.09	Polygon	(46, -77) (47, -80) (48, -75)

Table A1. Example spatial data for the geo-intervention modelling framework.

i	Xi,1	Xi,2	Xi,3	Xi,1 + Xi,2	Nearesti	Length(Li)	Type(Li)	Li
1	1.25	2	4	3.25	2	0	Point	(43, -79)
2	2.55	4	0	6.55	1	1.41	Line	(45, -79) (44, -78)
3	5.75	6	16	11.75	2	11.09	Polygon	(46, -77) (47, -80) (48, -75)

Appendix A.2. Outcome Modelling Component Example

Table A2 expands on Table A1, showing the outcome model as a function of the spatial data provided. A model is built for outcomes Y_i as a function of variables X_i,1, X_i,2, X_i,1 + X_i,2, Nearest_i, Length(L_i), and Type(L_i) to produce predicted outcomes ${\widehat{Y}}_i$ with errors e_i as the model metrics. Here, the predicted outcomes was selected as variable X_i,3, and excluded as one of the model’s input variables, while $i$ was ignored as the index for each unique location.

Table A2. Example outcome modelling data for the geo-intervention modelling framework.

i	Xi,1	Xi,2	Xi,1 + Xi,2	Nearesti	Length(Li)	Type(Li)	Yi = Xi,3	${\widehat{Y}}_i$= f(X1 … Type(Li))	ei = |Yi -${\widehat{Y}}_i$|
1	1.25	2	3.25	2	0	Point	4	4.59	0.59
2	2.55	4	6.55	1	1.41	Line	0	0	0
3	5.75	6	11.75	2	11.09	Polygon	16	16.62	0.62

Table A2. Example outcome modelling data for the geo-intervention modelling framework.

i	Xi,1	Xi,2	Xi,1 + Xi,2	Nearesti	Length(Li)	Type(Li)	Yi = Xi,3	${\widehat{Y}}_i$= f(X1 … Type(Li))	ei = |Yi -${\widehat{Y}}_i$|
1	1.25	2	3.25	2	0	Point	4	4.59	0.59
2	2.55	4	6.55	1	1.41	Line	0	0	0
3	5.75	6	11.75	2	11.09	Polygon	16	16.62	0.62

Appendix A.3. Geo-intervention Generation Component Example

Finally, Table A3, expanding on Table A2, shows a simple example of one iteration of an optimization algorithm, where a set of action values A_i,4 and A_i,5 was applied to variables X_{i,1 +} X_i,2 and Length(L_i) to produce estimated variables ${\widehat{X}}_{i,4}$ and ${\widehat{X}}_{i,5}$. Here, constraints limited the optimization to only modify variables X_{i,1 +} X_i,2 and Length(L_i) for locations 1 and 2, while A_i,1…A_i,3 would be set to zero (no change) as they were non-modifiable according to the set constraints. The estimated variables were used as input for the best model, producing the associated set of predicted outcomes ${\widehat{Y}}_i$, which can be evaluated with an outcome metric. For example, given the sum of the predicted outcomes${\displaystyle \sum }_{i=1}^3{\widehat{Y}}_i$, and the objective of minimizing this sum, the optimization algorithm attempts to adjust actions A_i,4 and A_i,5 to produce the lowest sum given a set number of iterations or when all possible values are explored. In this example situation, the actions A_i,1…A_i,5 that produced the lowest sum would be the best geo-interventions, while its predicted outcomes ${\widehat{Y}}_i$ would be the best outcomes.

Table A3. Example geo-intervention generation data for one iteration of the optimization algorithm.

i	Xi,1 + Xi,2	Length(Li)	Ai,4	Ai,5	${\widehat{X}}_{i,4}$= (Xi,1 + Xi,2) + Ai,4	${\widehat{X}}_{i,5}$= Length(Li) + Ai,5	${\widehat{Y}}_i$= f(${\widehat{X}}_{i,1}$ …${\widehat{X}}_{i,5}$)
1	3.25	0	+7	+2	10.25	2	4.25
2	6.55	1.41	-3	-1	3.55	0.41	7.1

Table A3. Example geo-intervention generation data for one iteration of the optimization algorithm.

i	Xi,1 + Xi,2	Length(Li)	Ai,4	Ai,5	${\widehat{X}}_{i,4}$= (Xi,1 + Xi,2) + Ai,4	${\widehat{X}}_{i,5}$= Length(Li) + Ai,5	${\widehat{Y}}_i$= f(${\widehat{X}}_{i,1}$ …${\widehat{X}}_{i,5}$)
1	3.25	0	+7	+2	10.25	2	4.25
2	6.55	1.41	-3	-1	3.55	0.41	7.1

Appendix B.1. Spatial Feature Engineering Considerations

Depending on the input spatial data structures (e.g. lines, points, polygons), various spatial feature engineering methods may be used to extract more informative features, or remove them, for modelling. For example, we may create additional features that capture spatial information, using spatial relationships (e.g. nearest neighbor, distance decay), geometry (e.g. line curvature, polygon area), or spatial aggregation [49,178] (e.g. units per square kilometer, sum of nearest neighbors). Another common example is to remove any input variables that are directly related to the outcome (e.g. variables aggregated using the outcome variable). The decision to create and remove these spatial features may rely on computational resources (e.g. memory limits, storage, parallel processing, expenses [179,180,181]), past knowledge (e.g. literature, experience, expertise [182]), or an algorithmic process (e.g. feature selection [183]). Thus, the efficiency of data processing for spatial features needs to be considered using past knowledge and expertise. Although spatial features are often not included directly in datasets, they enable outcome models to account for spatial effects that are not present in non-spatial data, which can potentially improve model performance or uncover important hidden variables for inference.

Appendix B.2. Outcome Modelling and Evaluation Considerations

The choice of outcome models and evaluation metrics determine how accurate predicted outcomes from geo-interventions are to the input data, and if the outcome is represented adequately according to the decision maker’s perspective. Models are often selected based on prior knowledge of the input variables and outcome (e.g. statistical modelling), but recent advances in machine learning enabled the automatic selection of models, without prior knowledge, as a tradeoff between processing time and performance (e.g. AutoML) [61,184,185,186]. While selecting models based on prior knowledge generally offers improved interpretability and computational efficiency on smaller datasets, automatic selection approaches may provide better performance and test a larger variety of models to provide empirical evidence [187,188]. Ideally, a hybrid approach may be used, where trusted models from prior knowledge can be used as baseline comparisons to more automated approaches to produce stronger empirical evidence for the selection of models. In addition, the choice of metrics needs to reflect the decision-making problem, related risks, and the characteristics of the input data [189,190]. Considering an evaluation metric involves identifying the outcome, how to quantify that outcome to be interpretable for the decision-making problem, and whether an increase or decrease in those quantities are desired. Data characteristics and related risks influence the validity of the metric for practical use, as imbalances in the outcome variable and side effects from related risks greatly influence whether the model and its predictions are valid in the real world. For example, the F1 score is used to address issues where models overfit to one or a small number of categories, providing a false interpretation of high performance [191]. Another example involves incorporating outcome effects on vulnerable populations (e.g. disabled, low-income families) and equity (e.g. distribution of benefits or damages from outcome across areas and sub-populations) in evaluation metrics to account for risks [192,193,194]. In this example, we may also consider multiple outcome sub-models with a unifying metric to address the target outcome (e.g. road traffic injuries), and their outcome effects (e.g. vulnerable populations and equity). With adequate computational resources, efforts on the choice of metrics may provide more benefit than efforts spent on the choice of models as automated modelling approaches develop, and more non-expert users are able to apply these models.

Appendix B.3. Optimization Algorithm and Guidance Considerations

After creating an outcome model to generate geo-interventions, an optimization algorithm is required to approximate the best geo-intervention from a constrained set of alternative geo-interventions. Selecting and guiding this algorithm generally helps in finding more desirable outcomes, as it limits the extremely large search space (set of alternative geo-interventions), where it is impractical to explore all possible options [195,196]. First, an optimization algorithm must be selected. Common choices are sequential [197], grid [198], random [199], Bayesian [200], evolutionary [201], gradient descent [202], and particle-swarm search [203], where these algorithms outperform brute-force and search under similar time limits. Ideally, experimenting with a combination of different optimization algorithms under reasonable time limits, before committing more resources, is desirable, as it provides empirical evidence for selecting an algorithm given the model, dataset, and various constraints. For example, grid, random, Bayesian, and evolutionary algorithms may be run in parallel for two hours, while the algorithm that yields the best geo-intervention is chosen to be run for several days. Here, computational resources, potential for parallelization, and hybrid algorithms may also be considered. Computational resources and parallelization may allow certain algorithms to explore more options (e.g. parallelized grid or random search [204]), and disadvantage others (e.g. dependency on previously explored options in Bayesian search [96]). There is also possibility in combining algorithms, such as applying random search to initialize better prior options for Bayesian search [199]. Second, guiding the selected algorithm has perhaps the largest influence on finding more optimal geo-interventions, as it defines constraints that drastically both reduce search spaces and improve understanding, as these constraints often represent applied knowledge from expertise in the outcome’s behavior. In the case study, a common relationship is known between traffic volumes and collisions, and thus, two scenarios focused on defining constraints on high and low traffic areas to reduce collisions. In scenario two, low traffic areas had a much larger search space and did not result in any geo-interventions that would reduce collisions. In scenario one, high traffic areas had lowered the search space, and resulted in several geo-interventions that lowered collisions, which is consistent with past knowledge on traffic collisions [205,206,207]. Additionally, the number of variables were limited to the top three most important infrastructure variables, further limiting the search space to represent modifiable phenomena in the real world. Here, the search space was limited by reducing the number of variables and the areas in which geo-interventions can be applied. Careful consideration is needed in limiting the search space to modifiable variables and spatial areas relative to the decision-making context. Hence, the decision maker’s knowledge is crucial in limiting the search space to guide the optimization algorithm towards generating both more practical and effective geo-interventions that can be applied in the real world.