Cross-Level Inference in Political Science

Detlef F. Sprinz

Potsdam Institute for Climate Impact Research

Potsdam, Germany

The Problem of Cross-Level Inference

Social scientists have a long tradition in modeling across various levels of analysis or levels of aggregation. While economists often distinguish microeconomics ( e. g., the study of individual firms, individual consumers, etc.) and macroeconomics ( e. g., whole economies as part of the world economy), sociologists distinguish, inter alia, individuals, groups and states. Similarly, political scientists, especially specialists in international relations, may focus on at least three levels of analysis such as the individual decision-maker, country aggregates ( e. g., country positions in international environmental negotiations), and the international system of countries ( e. g., the occurrence of international war on the global level). However, the relationships across the various levels of analysis do not necessarily receive adequate attention, in particular if findings at lower levels of aggregation hold or do not hold at a higher level of aggregation. Furthermore, researchers are often interested in micro-level relationships but only have aggregate data at hand. The problem of cross-level inference deals with both types of problems: relating findings at lower levels of aggregation to higher levels of analysis and suggesting ways to recover individual-level relationships when only aggregate-level data are available.

Most of the literature in the social sciences seems to be influenced by the research findings of William Robinson in 1950 who showed that states with more foreign-born residents tended to have more residents literate in English. A scholar using only aggregate data would have concluded that the foreign-born were unusually literate in English. However, the individual-level census data showed just the reverse was true: foreign-born residents were less literate in English than native-born Americans. Thus even the sign of the aggregate-level relationship was wrong. Robinson concluded that we should avoid using aggregate-level correlations to draw conclusions about otherwise unobservable individual-level relationships; he called this the "fallacy of ecological correlation."

The problem of cross-level inference deals with relating findings at lower levels of aggregation to higher levels of analysis and suggesting ways to recover individual-level relationships when only aggregate-level data are available.

Since data from both levels of analysis are available, the problem could fortunately be resolved: In this specific case, the ecological fallacy occurs because particularly strong groups of foreign-born persons were clustering in the same geographical units with highly literate groups of persons, and while the individual-level relationship between being foreign-born and literacy is negative, the clustering of foreign-born groups with highly literate persons in cities resulted in an positive aggregate-level relationship. However, detection of the error in inference rests on the additional information available from individual level data - which do not always exist.

Examples of Aggregation Bias

Students of comparative politics have been advancing the various merits of different election schemes for decades, e. g., the simple majority rule versus proportional representation for allocating Parliamentary seats, as well as single versus multiple member districts. Not intending to review this discussion, in the following, Sprinz simply demonstrates the aggregation bias introduced by relative majority electoral laws as compared to proportional representation.

The recent British Parliamentary elections were held on the basis of single-member districts, i. e., the candidate receiving the relative majority of votes will represent his or her voting district in the House of Commons (see Table 1.36).

The table contrasts the actual percentage of seats received in Parliament with those based on proportional representation. For example, with 45 percent of the popular vote, the British Labour party received two-thirds of the seats in Parliament. In order to easily detect the degree of bias (in percentage points), Table 1.36 computes the absolute difference between the percentage of seats expected by way of proportional representation as opposed to the actual percentage of seats allocated by way of a relative majority system (plurality). The results show that the governing party has benefited substantially from the single member, relative majority electoral system as opposed to proportional representation laws. In fact, from the perspective of proportional representation laws, the opposition parties have "won" the election!

Aggregation bias is introduced by relative majority electoral laws as compared to proportional representation.

While such aggregate results from election outcomes are well-known to comparative political scientists, they also have implications for cross-level inference. National governments negotiate international environmental agreements and implement them by way of domestic laws. Since we often find sufficient differences in the preferences of national governments to negotiate such agreements, the aggregation bias introduced by electoral laws may be quite relevant: National policies may lead to international obligations which lack parallel support at the level of voting districts or even the country at large - which may generate serious problems at the stage of complying with the provisions of international environmental agreements.

An Overview of Major Approaches to Cross-Level Inference

As James S. Coleman reminds us, a first observation is that good social history makes the transitions between micro and macro levels successfully. Sociologists and political scientists have advanced the use of so-called hierarchical (linear) statistical modeling techniques which allow for nesting across levels of analysis. The general concept is that the behavior of individuals is influenced by the social contexts to which they belong and that the properties of a social group are influenced by the individuals who make up that group.

Due to aggregation bias, national policies may lead to international obligations which lack parallel support at the level of voting districts or even the country at large, generating compliance problems.

This allows, for example, individual voting behavior to be explained not only by individual-level variables, but also by variables operating at a higher level of aggregation, such as the macroeconomic variables at the level of the voting district ( e. g., rates of unemployment or growth rate of the gross regional product). When voting data are made available only at the district level, this grouping of the individual-level data may lead to problems in the estimation procedure because, e. g., the borders of voting districts may be drawn on partisan grounds. As a result, the vector of independent variables may now be correlated with the error term at the aggregate level - leading to a violation of regression assumptions which is not present at the individual level. In essence, model estimation becomes possible as error terms are carefully modeled across levels of analysis in order to avoid violation of statistical estimation procedures. As Langbein and Lichtman eloquently summarize:

Aggregate models used in lieu of individual data may be comprised both of variables which are theoretically relevant at the individual level and of variables which are added in order to remove the bias by grouping, i. e., variables that reflect the grouping process itself. (Langbein and Lichtman, 1978, 11)

Since the 1950s, two methods of cross-level inference have been advanced to make inferences on the (normally unobserved) individual level from (observed) aggregate level data: the "method of bounds" and Goodman "ecological regression." In essence, the "method of bounds" rests on accounting identities. Regrettably, this method often does not generate sufficiently narrow intervals for the proportions under consideration to yield interesting results by themselves, and they cannot be statistically estimated due to underidentification problems (more unknowns than equations).

The latter drawback has attracted many researchers to Goodman "ecological regression ," which overcomes the problem of underidentification with the help of strict assumptions. While this allows conventional regression models to estimate the parameters, there is a major practical drawback: Parameter estimates often fall out of the logically permissible range, namely below 0 and above 1! As Achen and Shively suggest from a review of voter transition studies:

Logically impossible estimates in ecological regression are not flukes. They are encountered perhaps half the time, and more often as the statistical fit improves. Ecological regression fails, not occasionally, but chronically. (Achen and Shively 1995, 75).

More recently, King (1997) has devised a way to fruitfully combine the deterministic results from the method of bounds with statistical estimation techniques in order to overcome the problems posed by the two methods of ecological inference introduced above. His method of ecological inference has been verified with the help of examples where individual-level data are known. Using information from the deterministic method of bounds in combination with advanced statistical methods may generate plausible (and in the case of available individual-level data) verifiable results for the quantities of interest.

Ecological inference undoubtedly seeks to yield important substantive insights by filling cells left previously empty, but it does not generate results at a lower unit of analysis (or aggregation).

In its simplest version, ecological inference seeks to fill the cells of a two-by-two table (or more elaborate tables) when only the marginals are known. However, since the summation of cells (either horizontally or vertically) generates the marginals (either as counts or proportions), it is the same unit of analysis which is involved within a geographic unit, both in the cells or the margins. Thus, ecological inference undoubtedly seeks to yield important substantive insights by filling cells left previously empty, but it does not generate results at a lower unit of analysis (or aggregation). In King's solution, it even "borrows strength" from the bivariate distribution which is estimated across voting districts - thereby creatively using information outside the original domain of primary interest. Thus, there might be an important difference with many of the natural sciences where downscaling refers to using information available at higher levels of geographic aggregation to create representations of data at smaller geographic units.

The Effectiveness of International Environmental Regimes:

A Multiple Level Measurement Concept

Although cross-level modeling is challenging, it also enables the development of new concepts. Following the pioneering work by Putnam (1988), which relates the positions taken by governments in international negotiations to properties on the subnational level (e. g., influential interest groups, the electorate, etc.), scholars have developed two-level game-theoretical models to formalize these relationships across levels of analysis in order to deduce new hypotheses. For example, Wolinsky (1994) develops a sequential, two-level game which relates governmental policies on signing international environmental agreements to electoral control; i. e., the electorate takes cues from a government's decision to sign or abstain from international environmental agreements in order to conclude if they are presently ruled by an effective or ineffective government. Other scholars have advanced the notion of the domestic prerequisites for international environmental negotiations or suggested a domestic-level argument about characteristics which induce countries to strive for international environmental agreements (e. g. , Sprinz and Vaahtoranta, 1994).

Perhaps the most intuitive link between domestic political properties and international environmental performance can be demonstrated with the help of a concept for measuring the effectiveness of international environmental regimes. In effect, this measurement concept avoids the problems posed by aggregation bias, and since its most interesting result is on the aggregate level (namely the degree of regime effectiveness), it does not face problems posed by the ecological fallacy. In the following, the concept for the case of transboundary pollution problems is briefly presented and illustrated with select findings from a research project on long-range transboundary air pollution in Europe - an environmental problem which has been regulated under the auspices of the United Nations Economic Commission for Europe.

For international environmental cooperation to emerge, some weight must be given to the damages created abroad by polluting activities taking place within one's own jurisdiction.

Underlying the concept of the effectiveness of international regimes presented here is a cost-benefit calculus generally found in environmental economics which has also been applied to political science. The central assumption is that some unitary actor (such as a federal government) is determining the level of effort to protect the environment based on the profitability to its own country ("non-cooperative or counterfactual solution"). In the presence of negative externalities (such as transboundary pollution), environmental damages created outside the jurisdiction are neglected (e. g., lake acidification in Norway is partially caused by air pollution emissions in the UK), and result in transboundary environmental problems. For the group of all countries, this is unlikely to be the optimal solution, since some reciprocal emission reductions may benefit some or even all countries. However, for international environmental cooperation to emerge, some weight must be given to the damages created abroad by polluting activities taking place within one's own jurisdiction ("cooperative or collectively optimal solution"). Under any circumstances, any rational actor will normally choose an "actual policy" between the non-cooperative solution (minimum) and the cooperative solution (maximum), and the relative positioning of the actual policy in between the minimum and maximum represents the score of international regime effectiveness.

The empirical derivation of the results requires the following modules:

• a transboundary pollution matrix,
• knowledge of economic abatement costs, and
• knowledge of economic damage costs.

In the case of transboundary air pollution in Europe, data of sufficient quality and precision are available for the first two categories, whereas the economic damage costs remain unknown. Therefore, the economic damage cost function has been replaced by a political damage cost function which takes into account the power of domestic political actors, the salience of the topic, and the degree to which a country faces the specific environmental problem (i. e. , an indicator of the distance between the current ecological vulnerability and when this value approaches zero). The calculus employed permits the derivation of country-level results as well as an aggregate results. For reasons of presentation, only the results for three countries are shown for the Helsinki Sulfur Protocol (which mandated a 30 percent reduction of sulfur emissions for signatories between 1980 and 1993) (see Table 1.37).

The results indicate that the group of all countries substantially exceeds the non-cooperative solution and falls far short of the cooperative solution, however, the international environmental treaty regime does have effects on countries. In some cases, the cooperative solution prescribes a lower emission reduction than for the non-cooperative solution. This applies especially to countries who already undertake high emission reduction policies in the non-cooperative solution, and given the cooperative solution of other countries, their cooperative emission reduction is lower than their non-cooperative solution (e. g., Norway, in Table 1.37). In such cases, the score "1*" was awarded if a country did not fall below the (higher) non-cooperative solution for its actual policy.

The aggregate-level results show that the effectiveness score is substantially larger than zero and clearly lower than 1. This suggests that (macro) international regimes are not able to carry countries away from their (micro) domestic political basis which constrains domestic and international environmental policy; however, the regime has some discernible effect which should not be underestimated in the empirical domain presented here.

The group of all countries substantially exceeds the non-cooperative solution and falls far short of the cooperative solution.

Notes: Reductions are expressed in percentage for the periods 1980-93. Those countries where actual reductions are higher than the cooperative reductions have been assigned the score 1*, indicating that they have done more than would have been required in the optimal cooperative solution. The results for "all countries" are based on the weighted average of all countries within the relevant airshed not just the results for the select countries shown here. Source: Sprinz and Helm (1996).

Conclusions

Sprinz provided a brief overview of aggregation and disaggregation problems encountered by political scientists and the major methods used to solve such problems. In addition, the multi-level measurement concept of the effectiveness of international environmental regimes was introduced. While this summary may be comforting to the reader, advances in carefully disaggregating data would be of major interest to students of international environmental policy and politicians alike. In particular, it would be helpful to develop gridded maps of the likely capability of regions to implement international environmental agreements and to use these results in negotiating international environmental agreements. While agreeing on such a map may be politically contentious and will be hard to realize internationally, it would more realistically examine the potential for protecting the international environment .

References

Achen, Christopher H. and W. Phillips Shively (1995). Cross-Level Inference, Chicago, IL: The University of Chicago Press.

King, Gary (1997): A Solution to the Ecological Inference Problem: Reconstructing Individual Behavior from Aggregate Data, Princeton, N.J.: Princeton University Press.

Langbein, Laura Irwin and Allan J. Lichtman (1978): Ecological Inference, Newbury Park, CA: Sage Publications.

Putnam, Robert D. (1988): Diplomacy And Domestic Politics: The Logic of Two-Level Games, in: International Organization, 42 (3):427-460.

Advances in carefully disaggregating data would be of major interest to students of international environmental policy and politicians alike.

Sprinz, Detlef and Carsten Helm (1996). International Governmental Organizations and National Participation in Environmental Regimes: The Organizational Components of the Acidification Regime (EU Contract EV5V-CT94-0390), Potsdam: Potsdam Institute for Climate Impact Research.

Sprinz, Detlef and Tapani Vaahtoranta (1994): The Interest-Based Explanation of International Environmental Policy, in: International Organization, 48(1):77-105.

Wolinsky, Yael (1994): International Bargaining Under the Shadow of the Electorate, Ph.D. dissertation, Department of Political Science: The University of Chicago, Chicago, IL.

It would be helpful to develop gridded maps of the likely capability of regions to implement international environmental agreements.

---

Forward to Next Section//Back to Table of Contents//AGCI Homepage//Comments: agcimail@agci.org