AGCI Session II: Characterizing and Communicating Scientific Uncertainty

Session Chairs: Dr. Richard H. Moss and Dr. Stephen H. Schneider

July 31 to August 8, 1996

---

Treatment of Uncertainty in Integrated Assessments

John Weyant

Stanford University

Stanford, California

---

Weyant focused on how uncertainty is handled in integrated assessment models (IAMs). Beginning with the general issue of uncertainty in modeling and policy analysis, Weyant points out that the formal discipline of decision analysis recognizes that uncertainty should not be used as a reason for indecision; no change in the status quo is actually a significant decision. He also distinguishes between long range projections and those useful for decision making, saying that what is important are the implications for each time frame. Conceptually, we only have to decide what to do today; over time, we can revise projections as well as decisions. Looking at the history of modeling analysis, Weyant says that the insights provided by such analysis are often a more important product than the numerical projections.

Focusing for a moment on the Energy Modeling Forum (EMF), Weyant says its primary goal has been to prove the use and usefulness of analysis. In the EMF process they have looked at why different results come from different modeling systems for the same questions, what information and insights from the collection of tools available are useful to decision makers, and what the gaps are in the information and analysis base that need to be filled.

Does explicitly adding uncertainty into models lead to better insights? Not always, Weyant says. Sometimes, simple deterministic models give very good answers. Does factoring in uncertainty lead to better numerical predictions? Weyant says it probably helps in deciding what and how to communicate to policymakers about the numerical predictions. He also points out that while we say insights are more important than numbers, in the policy process, numbers do matter. Decision makers want and need to hear at least order of magnitude trade offs when asked to make policy.

Uncertainty should not be used as a reason for indecision; no change in the status quo is actually a significant decision. ... Conceptually, we only have to decide what to do today; over time, we can revise projections as well as decisions.

In the EMF studies, the procedure is to use standardized model comparisons to take that source of variation out of the process. Also, because model variability might depend on what scenario is used, EMF uses a standard reference case and then alternative scenarios with different variables. Then, due largely to how the results were being used, they began also doing "modelers' reference cases" in which modelers can designate preferred values for the key factors: population, economic growth, some minimal technology assumptions, and fuel prices. Doing these modelers' references cases led to a spreading out of the predictions of carbon emissions in the year 2100, almost doubling the range from the standardized reference case.

Because uncertainty is such an important issue, EMF formed an uncertainty study group to work with models that were specifically designed to explore the implications of uncertainty, rather than using the full deterministic integrated assessment models to explore these issues.

Weyant says education is the main purpose of integrated assessment modeling. First, analysts need to become educated, and then they need to communicate to policymakers and the public. Modeling is primarily useful for gaining insights, not specific numbers. It can be used to identify smart and ill-advised policies and to identify previously unrecognized interactions and feedbacks. However, he reiterates, it must be recognized that numbers do matter to policymakers, whether or not they should. Models can help with numerical predictions, but we should be careful of overstating their ability in this regard. They can help to project rough trade-offs between competing objectives, using sensitivity and uncertainty analyses to check for robustness of conclusions.

The authors of Working Group III, Chapter 10 recognized that there are two distinct types of deterministic models. The first group are top -down, bottom line economic models, such as Nordhaus' DICE model, which include a simple carbon cycle, climate sensitivity and climate damages. They short circuit much of the action of a full-scale IAM regarding ecosystems, atmospheric composition, etc. The authors of Chapter 10 chose to call these "Policy Optimization Models" as they seek to balance costs and benefits (e. g., in determining the appropriate level of a carbon tax), aggregating everything to a simple dollar metric.

Education is the main purpose of integrated assessment modeling. It can be used to identify smart and ill-advised policies and to identify previously unrecognized interactions and feedbacks.

Models belonging to the second group, called "Policy Evaluation Models," include more physical properties and grow more from the physical climate modeling tradition. These models include human activities as well as more detailed representations of emissions, atmospheric composition, and climate (not incorporating full-scale GCMs, but generally two-dimensional simulations). Each modeling group specializes in a few main impact areas, some including economic evaluation and some not. These models do not attempt to aggregate all damages to the nearest dollar metric. Weyant had hoped to find a method of aggregating these two major approaches but found that deep philosophical differences between the two made that unfeasible at present.

Each of these two modeling approaches is critiqued by practitioners of the other in various ways. For example, the more complex physical modelers would say that a simple cost-benefit model tells us that a 3°C temperature rise will lead to a 2 percent reduction in global GDP, but it does not tell us whether this loss means 100,000 people drowned in Bangladesh or less sunny beach days in California. More detail is needed for policymakers to find such models useful, the argument goes.

From the other point of view, economists say that particular threshold limits suggested by the more complex models, such as the suggestion that we contain emissions enough to ensure no more than one category of change within each vegetation biome type, include no consideration of what this would cost and that it could turn out to be more than it's worth, in economic terms. Much can be gained, Weyant suggests, by going back and forth between the two approaches, using the most detailed and robust representations from each to strengthen the validity of the overall conclusions.

The desire to include uncertainty has led to two additional sets of models Weyant calls Partial and Full Stochastic Simulation Models which generate composite probability distributions. This last group, called "Decision Making Under Uncertainty Models" includes the model described in this report by Schlesinger.

The more complex physical modelers would say that a simple cost-benefit model tells us that a 3°C temperature rise will lead to a 2 percent reduction in global GDP, but it does not tell us whether this loss means 100,000 people drowned in Bangladesh or less sunny beach days in California.

Preliminary insights from Policy Optimization Models include the value of timing flexibility, which seems to suggest the value of more emissions control in the middle term and long term and perhaps more research and development in the short run, and the value of emissions trading, even if the entire world is not involved in the program.

From the Policy Evaluation Models, preliminary insights include the likelihood of sulfur aerosol policy complications, e. g., China could greatly reduce its sulfur emissions for acid rain control and other reasons only to find that doing so exacerbates climate warming. Another issue raised by these models is the land use competition implications of large - scale biomass energy development.

From the Stochastic Decision Making Under Uncertainty Models, insights include the value of hedging against bad outcomes and the robustness of research and development policies. R&D on new technologies is the one thing policymakers can't take back after the fact; there is potential for OECD countries to sponsor technologies that are salable domestically as well as transferable to the developing world.

Dealing with Uncertainty in IAMs

Weyant briefly listed approaches for dealing with uncertainty in IAMs. First, under the Stochastic Optimization approach, he mentions stochastic math programming and decision analysis, hedging calculations, partial stochastic sensitivity analysis, and value of information calculations. Under Stochastic Simulation Models he includes simulation and dynamic programming, computing probabilities of meeting stated objectives, full stochastic sensitivity analysis, and richer learning and knowledge accumulation representations. He also mentions option value approaches.

Weyant then described a simple experiment which compared a "clairvoyant case" with Sequential Decision Making Under Uncertainty (7 models could do this experiment). The numbers used are based on Nordhaus' quantification of a combination of the Morgan/Keith and Nordhaus surveys of climate experts (see Session Synthesis Essay in this report). Assuming uncertainty regarding the optimal level of emissions is resolved in 2020, and using a 5 percent discount rate, Figure 2.19 shows carbon emissions through the year 2100 based on various policies, and Figure 2.20 shows the incremental cost/value of carbon emissions through 2100. This analysis suggests that hedging is a good strategy. Doing more than would be done by following a mere cost-effectiveness test is at the core of all of their suggestions, Weyant says.

This analysis suggests that hedging is a good strategy. Doing more than would be done by following a mere cost-effectiveness test is at the core of all of their suggestions, Weyant says.

Conclusions

Many integrated assessment insights are available without much consideration of uncertainty and these can help put numbers in perspective, at least suggesting order of magnitude trade-offs. Treatment of uncertainty in integrated assessment is still in its infancy. This work should improve and expand upon available insights. Uncertainty analyses can put the numbers in perspective. More work is needed on the impact of uncertainty on economic agents. New work is also needed on multi-party negotiations and coalition formation, Weyant says. In addition, culture theory, strategic scenarios, and attempting to account for changing preferences are examples of promising approaches that are difficult to incorporate into models but could broaden their perspective.