Structured Decision-Making Overview
Structured decision making (SDM or decision analysis) is a formal framework for aiding decision makers and other interested parties (e.g., cooperators and / or rightsholders) in working through a series of steps to identify a best course of action for a decision problem (Gregory et al. 2012). The SDM process is value-based, in that it enables the decision maker and associated working group to first identify the problem and the things that they care about and want to achieve, prior to identifying actions (Keeney 1992). In this way, SDM is a decision-aiding tool that allows the decision maker and any other participants to better understand how well particular actions will achieve their specific objectives (values) and how to tradeoff differential levels of achievement.
Structured decision making is often implemented using the “PrOACT” process, which stands for problem, objectives, alternatives, consequences, and tradeoffs (Hammond et al. 1999, Hemming et al. 2022;
Figure 1).
Problem−In step 1, the group identified the decision that needed to be made. Problem definition is often one of the most difficult steps of the process for groups, because the group must agree on the decision at hand, as well as identify the spatial and temporal scales of the decision, who should be involved in or would be affected by the decision process, the reasons for bringing the problem to the table, the uncertainties that hinder decision making, as well as other issues that may be affecting the group’s ability to make the decision.
Objectives−In step 2, the values of the decision maker and stakeholders are identified and turned into a set of objectives that they would like to achieve in solving the problem. In this step, measurable attributes, which are the metrics by which objective achievement is measured, are also identified.
Alternatives−In step 3 (alternatives), the group is able to identify a suite of alternatives or actions that could be implemented to achieve the objectives. The group is often asked to be as creative as possible during this step, leaving measures of feasibility for later steps. Asking decision makers to consider objectives up front and letting those determine the alternatives to be considered is known as “value-focused thinking” and is a hallmark of SDM (Keeney 1992).
Consequences−In step 4, each of the alternatives is evaluated relative to achievement of each objective. In natural resources decision problems, this step often involves predictive modeling to simulate populations of animals or other organisms into the future under different actions. Yet, this can also be accomplished through other types of qualitative or quantitative models, elicitation of expert knowledge, survey instruments or focus group conversations, or other means that may depend on the measurable attribute.
Tradeoffs−In step 5, the relative importance of each of the objectives must then be evaluated. We often determine that there is not one action that can best achieve all objectives, so the group must consider how to tradeoff among objectives given this differential achievement across actions. The information gained from the consequences step, along with decision maker and other stakeholders and rightsholders preferences, is used to place weight on the different objectives. This process can allow the decision maker to see how well different actions might perform at achieving different objectives, integrate their values and those of stakeholders and rightsholders into the decision, and identify an action or set of actions that are optimal or preferred. The SDM process is also iterative, such that previous steps can be revisited throughout the decision process as new information becomes available or changes need to be made.
We used the PrOACT framework to walk through the steps of the SDM process for developing a communications strategy for the CRU Program during the course of a week-long workshop.
Problem
The problem framing step accounts for not only the decision that must be made, but all aspects of the scope of the problem. We focused our discussions on the considerations outlined above in detail. Based on these conversations and the concerns articulated by the group, the problem statement was defined as,
“The CRU Communications Team needs to develop a cohesive, scalable, and adaptable communications strategy at the programmatic level to increase visibility and amplify the relevance of the CRU Program mission locally, regionally, and nationally to ensure the sustainability of the CRU Program and its tripartite mission.”
Objectives
Objectives are developed to describe what the decision maker, stakeholders, and rightsholders ultimately want to achieve. All elicited objectives are included in this step, regardless of whether some objectives are at odds with others or whether some working group members do not value a particular objective. During this step, we identified the full list of objectives and categorized them to develop an objectives hierarchy. This hierarchy included fundamental objectives, which are the objectives that the group fundamentally cares about, means objectives, which describe how to achieve the fundamental objectives, and strategic objectives, which describe objectives that would apply to all decisions made by the CRU Communications Team. The final objectives hierarchy that resulted from the SDM workshop is shown (
Figure 2). Ultimately, this hierarchy of objectives consisted of strategic and fundamental objectives, although initially we identified means objectives (see
Appendix I, Figure A1), these were ultimately elevated to fundamental objectives.
In addition to the objectives for this decision problem, the group identified objectives that are inherently part of any strategy, as well as objectives to consider as part of a linked decision related to implementation at the unit level. As part of any strategy, the Team wanted to maximize proactiveness in communications, which included considerations for having to reach out to unit scientists to gather information, creating new documents on the fly, time spent discussing communications, and creating scalable communications products, as well as having the ability to update the plan and knowing when to do so. After strategy development, the group identified a need to maximize understanding of the chosen strategy by Unit scientists and the management team, as well as their participation in science communication efforts.
Alternatives
The goal of the alternatives step is to describe a set of actions that could be implemented to achieve the objectives. The facilitators asked the group to be creative to identify actions and their timing, first creating a list prior to consideration of feasibility. This list was then used to identify a set of actions that could be evaluated in the consequences step. The group first discussed the “status quo” for communications to identify the types of actions currently being taken and as a jumping off point for what could be changed, omitted, or added (
Table 2). The status quo for communications included creating summaries of activities in different levels of detail (e.g., an abstract book that is produced every five years, a year in review document for cooperators that highlights all three aspects of the CRU mission, multiple fact sheets for congressional liaisons, prospective students, cooperators, and the Coop Catch-up newsletter), developing directories and summaries of unit scientist expertise for cooperators, a monthly article in WMI’s Outdoor News Bulletin by unit scientists, and two websites (Watermark populated and USGS CRU). In addition, the status quo included some social media presence, occasional articles in wildlife and fisheries magazines, tables at conferences, and responses to inquiries from other parts of USGS and the DOI, as necessary. The group then developed a list of potential communications actions and strategies that could be implemented, including actions like having a more selective and targeted strategy for socials and tables at conferences, a fact sheet on the value of the CRU Program, a more holistic and deliberate social media strategy, overhaul of the USGS CRU website, and a photo contest. The full list of alternatives described by the group is provided (Appendix 1). A subset of this list was evaluated during the workshop based on existing capacity of the CRU Communication Team and group discussion (
Table 3).
In addition to the individual actions that were evaluated, the group decided to more fully describe four different communications tactics as a first step towards creating a more finalized set of actions to consider. The four tactics were: 1) social media, website, and newsletter (“Social Media”), 2) Alumni Campaign, 3) Who Are We? Campaign, and 4) Fact Sheets. Each of these tactics is still under development, but we describe the first draft and discussions of the tactics below.
The social media, website, and newsletter tactic focused on communicating CRU’s tripartite mission with numerous audiences. The social media aspect included considerations of a goal of building awareness of the CRU Program by increasing the number of followers and designing effective campaigns. There would be a need for a consistent frequency of posting engaging content and consistent campaigns, as well as monitoring the apps and pivoting as needed, all of which would require an investment. The website aspect was focused on updating the USGS CRU website (
https://www.usgs.gov/programs/cooperative-research-units). The suggested update for the website would be comprehensive and would require developing a tactic for rewriting existing content and creating new content, developing a communication plan including what messaging is to be conveyed (i.e., who we are and why, and what do we want to convey on graduate education, applied research, and technical assistance and how we can help). The group also described creating a monthly newsletter to highlight the three parts of the CRU Program’s mission, perhaps by choosing information from the revamped website, highlighting work from Units, among other tactics.
The goal of the alumni campaign would be to leverage Unit trainees (past and present) to form an alumni organization to communicate the tripartite mission and CRU Program activities. This organization could develop alumni and student awards and find creative ways to get the word out about the research, education, and technical assistance conducted by the Units. Potential audiences could range from prospective students to members of Congress.
The “Who Are We?” campaign was developed with the goal of ensuring that those who support and fund the CRU Program (e.g., AFWA members, Congress and their staff) understand the function of the CRU Program. The group identified potential actions within the campaign of producing videos and creating story maps (e.g., CRU Program’s tripartite mission, the history of the CRU Program, how we achieve the mission, how can we serve you better). The Fact Sheets tactic was developed for a similar set of audiences named above but would provide at-a-glance information to these groups. These would be focused on describing the value of the CRU Program.
Consequences and Tradeoffs
The goal of the consequences step is to project the potential outcomes of implementing each action in terms of the measurable attributes for each objective. Different methods, including quantitative modeling, surveys, readily available data, and expert elicitation, can be used to project the consequences. In the case of the CRU Program communications decision process, the consequences for most objectives were developed through a combination of expert knowledge and evaluation of available data. During the workshop, the group developed a set of measurable attributes that could be easily scored for the SDM team’s prototype decision framework.
After determining the consequences of each action or tactic in terms of each objective, the tradeoffs step allowed the decision maker or working group to place weights on the objectives to better reflect how much they valued each objective considering the potential for achievement with the suite of alternatives available. There are formalized methods for developing the weights for the objectives, but at its core, this is a value-based part of the process. Therefore, a goal for these methods is to be transparent about the way in which the weights were identified.
The group (all participants, except the facilitators) decided to first evaluate individual alternatives that were either part of the status quo portfolio strategy or that could be implemented in the future, based on the list of alternatives (
Appendix I). This would then allow for the group to take advantage of the iterative nature to then devise and describe tactics that represent combinations of individual actions that scored well or that individually achieved objectives. We present the results for a combination of some individual actions and the four campaigns that emerged from the consequences step (
Table 3).
As a first step at this stage, the group predicted consequences of each alternative for each objective (
Table 4). Predictions were based on simple scales and each prediction represented the expected value (EV) of the alternative for the objective. For example, for the objective of communicating CRU Program applied research, the group entered an EV of “1” for “yes” and “0” for “no” to predict how each alternative communication action or tactic would meet this objective. As another example, for the objective of minimizing costs in terms of time, the group entered an EV of “1” for the most costly, “2” for moderately costly, and “3” for least costly.
After predicting consequences, we normalized the EV (from
Table 4) into a consistent 0 – 1 scale (
Table 5) using the formula:
These calculations allowed for easier comparison across objectives and began to clearly reveal which alternatives do the best or worst job at meeting the 11 objectives.
Our approach to this point (
Table 5) assumed that all objectives were of equal importance, so as a next step, we elicited importance weights on objectives from the group (
Table 6). To complete this step, the group first ranked objectives in order of importance (1 = most important) and added scores of the relative importance of meeting each objective, compared to the top ranked objective’s score of 100. We then calculated the relative weights of objectives by dividing each score by the sum of scores. This step revealed that maximizing meaningful engagement was the most important objective (14% of weights), followed closely by the 3 objectives of communicating CRU Program graduate education, applied research, and technical assistance (13% of weights on each), and then minimizing costs in terms of time (12% of weight).
In the final step, we calculated weighted, normalized scores (
Table 7) by multiplying the predicted consequence (in
Table 5) by the respective weights on objectives (in
Table 6). We then summed each column for each alternative to reveal the final scores (
Figure 3). Alternatives with the highest scores are those that best meet objectives, although tradeoffs are apparent in the scores for each objective.
This approach revealed that the top performing alternative, Fact Sheets (final score = 0.88), performs best on meeting numerous objectives (
Table 7). However, it comes with tradeoffs, as it is predicted to perform only moderately at leveraging cooperators and minimizing costs in terms of dollars and time. The next best alternatives are the Who are we? campaign (score = 0.86) and Alumni campaign (score = 0.84). Both scored best on numerous objectives. However, the Who are we? campaign entails tradeoffs of worst performance on the objectives of maximizing responsiveness and minimizing costs (dollars) and performs moderately at minimizing costs (time). In contrast, the Alumni campaign was predicted to perform worst on the two objectives related to targeting audience tiers, and on objectives for maximizing responsiveness and minimizing the risk of losing the narrative; it performed moderately at minimizing costs (time). At-a-Glance and Website Overhaul were dominated by the Fact Sheets alternative; each performed equal or worse on each objective. Another observation was that the objective of communicating science was redundant in that it was equally scored for each alternative.