Three inter-related “methods problems” of importance in policy analysis and management

1. The problem of prediction

Start with mess

Mess has never been far away in my own profession of policy analysis and public management, which is full of wicked policy problems, muddling through, incrementalism, groping along, suboptimization, bounded rationality, garbage can processes, second-best solutions, policy fiascos, fatal remedies, rotten compromises, coping agencies, crisis management, groupthink, and that deep wellspring of miserabilism called, simply, implementation.

The more mess there is, the more reliability decisionmakers want; but the more reliable we try to be, the more mess produced. The more decisionmakers try to design their way out of policy messes, the messier actual policy implementation gets; but the messier the operations at the micro level, the more decisionmakers seek solutions at the macro level. This metastasizing feedback cycle does not augur well for the future and predicting that future becomes much of the mess we are now in.

What then is predictably unimaginable?

In answer, turn to an insight of literary critic, Christopher Ricks, drawn from the Oxford English Dictionary (OED):

‘Many adjectives in -ABLE suffix have negative counterparts in UN- prefix, and some of these are attested much earlier than their positive counterparts, the chronological difference being especially great in the case of UNTHINKABLE.’ The OED at this point withholds the dates, but here they are: unthinkable, c. 1430; thinkable, 1805.

Christopher Ricks (2021). Along Heroic Lines. Oxford University Press, Oxford, p. 240

The notion that some humans started with “unthinkable” is suggestive: We first confront unthinkable disasters and then think our way to making them more or less imaginable.

Current practice is we start with the worst-ever floods and earthquakes in the US and then argue that the Magnitude 9 earthquake off of the Pacific Northwest will be unimaginably worse. In this way, we end up with disproportionate contingencies and aftermaths about which we have no real causal understanding.

Let’s suppose, however, we started with disasters so indescribably catastrophic that we need to narrow our focus to something like a M9 earthquake in order to even think about the worse-ever floods and earthquakes that have happened. Here we can end up with possibilities, instead of contingencies, and impacts instead of aftermaths, about which we have some knowledge. In this way, we approach a “predictably unimaginable” that is not oxymoronic.

But you have to remember that imagining is not predicting, and both are downstream of the case-specific granularity

–-Consider the following example:

Once an artificial island, the ancient site of Soline was discovered in 2021 by archaeologist Mate Parica of the University of Zadar in Croatia while he was analyzing satellite images of the water area around Korčula [Island].

After spotting something he thought might be human-made on the ocean floor, Parica and a colleague dove to investigate.

At a depth of 4 to 5 meters (13 to 16 feet) in the Mediterranean’s Adriatic Sea, they found stone walls that may have once been part of an ancient settlement. The landmass it was built upon was separated from the main island by a narrow strip of land. . . .

Through radiocarbon analysis of preserved wood, the entire settlement was estimated to date back to approximately 4,900 BCE.

“People walked on this [road] almost 7,000 years ago,” the University of Zadar said in a Facebook statement on its most recent discovery. . .”Neolithic artifacts such as cream blades, stone [axes] and fragments of sacrifice were found at the site,” the University of Zadar adds.

accessed online at https://www.sciencealert.com/road-built-7000-years-ago-found-at-the-bottom-of-the-mediterranean-sea)

This discovery is also part of an on-going installation work by German filmmaker and moving image artist, Hito Steyerl, and described in a recent article as:

In The Artificial Island, the work traces a submerged Neolithic site off the coast of Korčula, discovered in 2021 by archaeologist Mate Parica. The site, originally connected to the mainland by an ancient road, now lies four to five metres beneath the Adriatic Sea, submerged by rising waters that speak both to geological deep time and contemporary climate upheaval.

accessed online at https://aestheticamagazine.com/flooded-worlds-parallel-realities/

After being primed by the two texts, take another look at the photo. You can see the submerged island, see its causeway to surface land, and imagine how the still-rising waters will submerge even more settlements ahead in the climate emergency.

–-The problem here arises when the preceding “imagine” becomes a prediction about what is to happen, now and ahead.

I wager that no reader primed as above asks first: “What about the presettlement template displaced by the Neolithic roadway and settlement?” Or from the other direction, “What about what’s been preserved from having been submerged for so long? What does this tell us about how the retreat from rising sea level was managed?”

That is, no one, I wager, reads the above text and looks at the photo and immediately asks: “What happens next here?” I mean that literally: “What happens next at and around these submerged sites? Are they to be protected (that is, why these sites and not other worthy candidates for protection in the face of the climate emergency)?”

More formally, you may think the quoted example predicts the need to do something with respect to the climate emergency elsewhere and over the longer haul. I am instead suggesting that really-existing accomplishments that happen next and at that site go to reframe the pertinent issues. People already understand what are case-specific accomplishments in ways that broader progress and success are understood by others only later on.


2. The problem of crisis scenarios

“We are living our whole lives in a state of emergency” (Denise Levertov, poet, 1967)

But which is our #1 global crisis? I could pick recent articles from any well-known media outlet. That I choose the online Guardian is not intended to single it out as worse. It’s just that these four competing examples came at around the same time there:

With so many #1 crises competing for attention, which to choose? The fact is that there is no choice to be made.

Not one of the four is at a level of granularity with which to assess whether this #1 is more hazardous than the other #1’s and under what conditions. This is particularly true as the hazards are in terms of uncertainties (i.e., their respective unknown knowns, known unknowns and unknown unknowns.)

Crises of course can be reported in short- or long-form media articles. But in the same way that no decisionmaker should want to take a policy decision on the basis of one piece of information only, so too should decisionmakers treat “this is our number 1 crisis” as one piece of the needed. Real-time action is taken for more granular, context-based reasons.

For example, the creators of AI may have a variety of crisis scenarios about the downside of AI. But when they say that the AI scenarios pose threat-equivalents to nuclear war or worse, you need to remind yourself that they have no equivalent level of detail for the latter as they do the former. In fact, they might be the last people on Earth you’d ask for nuclear war scenarios.

So what? So: Stop fossil fuel; stop cutting down trees; stop using plastics; stop this defense expansiion; stop imperialism; stop techno-solutionism; be small-d democratic, small-p participatory and big-T transformative; dismantle capitalism, racism and the rest; renew cities; save biodiversity; never forget class, gender, race, inequality, religion, bad faith, identity. . .and. . .and

Keep going on and it sounds like crisis kitsch.


3. The problem with economists

I

When rolling electrical blackouts take place, we policy analysts ask our friends, the economists, why.

After a blackout, one tells us it was because of all that underinvestment in the transmission grid you get when treating the grid as a public good. During a blackout, another assures us that having to shed load reflects the negative externalities associated with prices not reflecting electricity’s full cost to consumers, who “thus” over-consume and overload power lines.

Before a blackout, a different economist says energy deregulation will guarantee the reliability we want because it reflects the Efficient Market Hypothesis–remember, the idea that won a Nobel Prize–where nothing can be better than market prices in reflecting what is known about energy supply and demand, like our willingness to pay for transmission.

To which still another adds: Whether or not there is a blackout at all, rational expectations theory–remember, the idea that won another Nobel Prize–tells us that policy interventions are ineffective anyway.

If we aren’t sufficiently convinced and press our colleagues about what we should do to prevent blackouts altogether, they tell us not to worry—as long as electricity services are in market equilibrium, with reserve margins optimal, everything is okay.

II

Not quite granular enough for reliability professionals–but, hey, why ask economists at all? They’ll tell you no one listens to them anyway.

“Energy deregulation was never really tried,” they insist. Society never reallyReally adopted thorough-going cost-benefit analysis, economics in law, market designs engineered for efficiency, and far greater use of randomized controlled trials.

And when an economic market design is adopted whole hog, we have the 2021 Texas power and grid debacle: In the view of the Harvard economist who designed it, the energy market “worked as designed.” “It’s not convenient,” he added, “It’s not nice. It’s necessary.”

All of which is a bit like wanting to believe the Cultural Revolution would have succeeded if only Madame Mao and the Gang of Four were really given the chance. This is not what policy analysts who are reliability-seeking aspire to.

Leave a comment