AGCI Session II: Characterizing and Communicating Scientific Uncertainty

Session Chairs: Dr. Richard H. Moss and Dr. Stephen H. Schneider

July 31 to August 8, 1996

---

Expressing Uncertainty and Subjective Probabilities

Stephen Schneider

Stanford University

Stanford, California

See also: The Role of the Scientist-Advocate

---

Schneider tackled a number of challenging issues: How can we take experts' subjective probabilities about climate change into account in a consistent manner? What is our level of confidence in various conclusions? Are our assessments objective or subjective? If they're subjective, can we have a consistent set of criteria and an explicit set of values apply? How can we consistently say precisely what our own uncertainties mean? How do these questions relate to the climate change debate and political context? Do we all mean the same thing when we say 1.5 to 4.5°C warming as the climate sensitivity range?

Schneider traced the history of this 1.5 to 4.5°C range projected for climate sensitivity from its inception in the Charney report, National Academy of Sciences (NAS), 1979. Before 1979, General Circulation Models (GCMs) projected ranges from 2 to 3.5°C. The authors of the Charney report decided to allow 0.5°C as an additional margin of error on the low side (since there was no evidence of negative cloud feedbacks) and 1°C on the high side (due to possible positive cloud feedbacks). Thus the range of 1.5 to 4.5°C was born, and became an "anchor" which is now widely accepted and difficult to change. Such anchoring devices can be helpful in coalition building (see Shackley in this report).

How can we bring subjective probabilities into all phases of assessment? And how can we get the media and political establishment to understand what subjective probabilities mean?

Then, at the second Villach conference in 1987, Jager, Dickenson and Clark presented a chart of projected mean global temperature rise that included uncertainties not only in physical factors but in social and biological factors (e. g., what energy technologies would be used in the future, and biogeochemical feedback processes) and such factors expanded the range greatly to 0.5 to 6-8°C. At this point, the word "scenario" began to be used for such projections, to distinguish them from predictions. In a sense, these were early, mental, partially integrated assessments. Despite the wider ranges emerging from analyses that took a wider array of uncertainties into account, the accepted range of 1.5 to 4.5°C for climate sensitivity remained in place, as it does today. Though some models may indicate slightly higher or lower values, Schneider believes this range to be a reasonable representation of the scientific consensus with regard to likely (but not low probability extreme) outcomes. The stability of this range has arisen largely because there have been no compelling data or theories to modify it especially since it was somewhat arbitrarily defined in the first place.

Many scientists now believe it is their responsibility to give their best subjective views on climate issues and indicate their levels of confidence in a way that policymakers and the public can understand.

The Political Context

A report by Accu-Weather, Inc., prepared for the Global Climate Coalition (GCC), purported to be a comprehensive, global climate analysis, but actually used data from just three cities, Schneider reported. Tom Karl, on the other hand, used data from several thousand stations in his comprehensive analysis of global climate change. However, the GCC's public relations effort to get their side of the story into the media led to their press conference receiving much more attention than Tom Karl's vastly superior scientific efforts. This relative attention is due, Schneider quips, to a "one fax-one vote syndrome." We must realize that this is the context in which we live and work, he says. Scientists tend to shy away from the media, but we must find ways of bringing what scientists know to a wider audience in terms they can fathom.

Scientists have generally been uneasy about offering their subjective probabilities for use in the political arena. There has, however, been a social change in the receptivity of many in the scientific community over the past couple of decades to the need to perform subjective assessments and many scientists now believe it is their responsibility to give their best subjective views on climate issues and indicate their levels of confidence in a way that policymakers and the public can understand.

Two studies were briefly discussed that use elicitations of subjective expert judgments with regard to climate change: the Morgan and Keith study, and the Titus and Narayanan EPA study on sea level rise estimates (see also, the Morgan and Titus essays, in this report). With regard to such studies, key questions are: who are the experts, what is the process for choosing them, what questions should be asked, and how should the answers be aggregated? Schneider points out that in studies of this kind, there is always one "contrarian" included on purpose and that usually over-represents that point of view. How, he asks, can we bring subjective probabilities into all phases of assessment? And how can we get the media and political establishment to understand what subjective probabilities mean?

Conveying Confidence Levels

A report of the conclusions of a 1987 NAS climate workshop used a set of terms to convey levels of confidence in various findings. Large stratospheric cooling is called "virtually certain," global mean surface warming is called "very probable," etc. In the 1990 IPCC assessment, a 5-star system is used to rank confidence levels: five stars indicate virtual certainties, one star indicates low confidence. It is explicitly stated that these degrees of confidence were "determined subjectively from the amount of agreement between models, our understanding of the model results and our confidence in the representation of the relevant process in the model."

Discussing confidence levels inevitably leads to the question: confidence for what? The level of confidence needed to make global policy decisions is not the same as the level of confidence needed for other purposes, e. g., establishing statistical confidence in certain associations.

Discussing confidence levels inevitably leads to the question: confidence for what? The level of confidence needed to make global policy decisions is not the same as the level of confidence needed for other purposes.

Is the Scientist/Advocate an Oxymoron?

Schneider next turned to a discussion of the role of the scientist in responsible advocacy. Advocacy can be personal interest advocacy for a theory, model, measurement, crony, institution, nation, or epistemology. There is also world view advocacy for such things as entrepreneurial rights transcending commons protection; that the present is worth more than the future; that commons protection justifies global-scale rulemaking; risk aversion; or the idea that other species can be as valuable as human pursuits.

Schneider believes that scientists can practice responsible advocacy. Everyone has value positions (consciously and unconsciously); the keys to being a responsible advocate are:

a. making your value positions conscious,

b. making them explicit,

c. not letting your value positions distort your "subjective priors" (see Berk in this report) on issues of fact (a community is needed to do this well because no one can do this perfectly alone), and

d. defending your value positions separately from debates over probabilities and consequences.

Some scientists think it is impossible to do these things, and instead say they will be neutral, but no one is, Schneider says. Among the positions one can take is being an advocate of science itself, in which one argues for a rational world view and has faith that science has something constructive to contribute to decision making for the future. Even that is a value position, even if many who adhere to this position don't see it as such.

The public, media and politicians misread normal scientific contention as a lack of consensus, which leads to confusion and policy gridlock.

On the role of scientists in popularization/advocacy of scientific issues, Schneider says the context is that:

a. Scientists are contentious, but people listen.

b. The media love a fight and often set up stories as "dueling scientists."

c. The public, media and politicians misread normal scientific contention as a lack of consensus, which leads to confusion and policy gridlock.

d. Communication requires simplification but audiences deserve respect; therefore, use familiar metaphors (e. g. , cards, dice, insurance) that don't do violence to the truth. These metaphors cannot completely define the problem but the alternative is to not be in the game (or think you're not and allow journalists or the opposition to define you).

e. Sound bites are selective information transfer and present a double ethical bind. A story will go unreported if there are no sound bites, so scientists should learn to craft good ones that convey both urgency and uncertainty, if that is what the issues represent (e. g., global change).

Some scientists oppose popularization/advocacy no matter how credibly it is done. Skepticism is appropriate but some opposition is pathological and may take the form of:

1. Elitism "Simplification is vulgar and we do not play this game. It is not possible to communicate simply what we know to the public so we shouldn't even try."

2. Jealousy "A scientist's reputation comes from toiling in the lab, not from press coverage." A way to minimize this problem is to discuss colleagues' work and attribute it to them (but unfortunately, journalists generally attribute it to you anyway).

3. Special Interests These ideological opposites/hired guns will try to discredit the science (usually proclaiming to be defending sanctity of science), or discredit the scientist (redefine terms of debate from the science to the scientist or the assessment process, find a trivial error and expand, etc.). They count on the public and media not to check up on the credibility of their attacks, i.e., they exploit the media's "balance" doctrine.

f. Policy analysis (i. e., probability/consequences) is professional science and is amenable to expert judgments.

g. Policy choice (advocacy) is about personal values.

Sound bites are selective information transfer and present a double ethical bind. A story will go unreported if there are no sound bites, so scientists should learn to craft good ones that convey both urgency and uncertainty, if that is what the issues represent.

Schneider's Rules for Popularizing

Schneider's reaction to this context is to suggest that scientists popularize their own work or have someone else do it. He suggests these rules:

a. Use familiar metaphors that convey both the urgency (to get in press) and the essence of the uncertainties surrounding the issue (e. g., "loading dice" conveys urgency and uncertainty).

b. Back up sound bites with articles and other products ( e. g., full length books, so one's full views are on record for those few who want to check up).

c. Separate expertise (probabilities/consequences) from values.

d. Enjoy the process but don't be naive about the risks.

e. Don't counterpunch with ad hominem attacks and polemics, despite what the opposing advocates do.

f. Remember that long term success is built on establishing the credibility of the scientific community over time.

Journalists in the group pointed out the social legitimization function the elite press plays in science. For example, if a paper published in the New England Journal of Medicine is reported on in The New York Times, it is cited twice as often in the medical community as a New England Journal paper which is not reported on in The Times.

Schneider believes that it is best to talk to the press often or not at all, as there are risks involved in only one or two stories which by chance could be excellent or terrible. Not all scientists should be popularizers or advocates. We should let those who are good at it do it without disdain from their peers if they tell the story straight. Scientists can serve as a resource to raise reporters' level of knowledge on important scientific issues.

If scientists fail to convey probabilities of particular outcomes, they fail to convey the basic information that the public needs (e. g., what is the probability of significant climate change versus the probability of an invasion by aliens from space).

Schneider says that if scientists fail to convey probabilities of particular outcomes, they fail to convey the basic information that the public needs (e. g., what is the probability of significant climate change versus the probability of an invasion by aliens from space) to make any kind of decision (health, safety or economic). On the other hand, when one gives a subjective assessment with probabilities, there is always the danger that others can grab the tails of the distribution and run with them to suit their own agenda. A community effort is needed to overcome the "dueling scientists" problem, as well as tackle the other issues presented here.

On the other hand, when one gives a subjective assessment with probabilities, there is always the danger that others can grab the tails of the distribution and run with them to suit their own agenda.