Optimal ignorance

I

When I started out in rural development in the early 1970s, one challenge was to manage for optimal ignorance: Professionals should manage to the point where what they are learning is not worth knowing. Learning need take them only to where what they don’t know doesn’t add or subtract value for their acting now. Managing for optimal ignorance and its variants got a good deal of press from a range of writers at that time, notably social scientists Warren Ilchmann and Norman Uphoff, the development scholar Robert Chambers, and Peter Berger the sociologist.

The appeal of optimal ignorance waned when I implemented projects that I had helped plan. On those occasions, I’d find myself mulling over what my first boss, the district commissioner, told me when I arrived in rural Botswana: “A piece of advice, my dear boy. Either stay in the kitchen all the time or never go in.” Nothing major gets implemented as planned, and only by staying in implementation (later, management) did I appreciate how little I knew with my formal education in public policy analysis. 

My view is that “optimize” should be banned, as a term, from policymaking and management. Like the dog returning to its vomit, optimality criteria are never satisfied in the imperfection of circumstance. But I didn’t fully understand that until later when I started researching large critical infrastructures, their control rooms and control operators. These large sociotechnical systems are so complex that their managers cannot really “know” what are inevitably unstudied conditions and their real-time inexperience and difficulties are permanent reminders of this. Optimizers with whom I’ve worked, on the other hand, seemed to think it’s better to burn the building down to save the rest of us the trouble of repairing it.

Yes, of course, studying and adapting to unknown unknowns are important and that’s why the idea of “chipping away at ignorance” is not all just hubris. But control room operators are attuned to stay out of unstudied conditions not because some things are not worth knowing but for the opposite reason: No way can these professionals afford to be in prolonged ignorance when the safe and continuous provision of critical services, like water and electricity, is paramount. “[I]f the grid fails and there are blackouts, people die,” one control room executive told us. Control rooms put up with uncertainties they can live with in order to avoid unknown unknowns they can’t or mustn’t tolerate.

II

But you press: What could be more respectful of complexity than managing and learning adaptively? Change course as uncertainties are reduced and more is learned. No one can be against learning, right?

That may be true as far as it goes, but even then it doesn’t go far enough. Here’s a story from my time as an advisor in Kenya. I had oversight responsibilities for a handful of integrated rural development projects in Kenya’s arid and semi-arid districts. One of the worst projects, in my judgment, was fixed around soil and water conservation measures. You asked villagers there what their three most important development priorities were and they’d say: water, water, water. Water for drinking, water for cooking, water for their livestock, water for everything that mattered in their daily lives. Here instead the donor was spending a fortune on ditches and bunds to prevent soil erosion on the hillsides primarily for crop purposes, without any direct increase in water supply for the households.

Unsurprisingly, villagers just wouldn’t “participate” in the project: Food-for-work schemes didn’t work, giving them hoes or such didn’t work, nothing worked. Later on, I tracked down one of the project’s designer and asked: “Why ever was the project designed that way? Absolutely no one there was for soil and water conservation.” It was like he’d been waiting years for someone to ask him that question. He leaned forward, “But who can be against soil and water conservation?”

So too for managing adaptively: Who, really, can be against it? Why, that would be like arguing against norms of rationality, the scientific method or evidence-based policymaking, worse yet trial and error learning.

III

Yet, as with soil and water conservation and other projects, we must ask: managing adaptively for what? And here too, what is often desired is its own version of high reliability water, water, water—reliable water for urban use, for agricultural use, for ecosystem rehabilitation and the environment; for ports, for shipping lanes, for recreation, for hydropower, for…you name it, water is needed for it. And a very great deal of that provision depends on large-scale water supplies, electricity supplies and other infrastructures—which is why I keep coming back to their importance.

Obviously, control room operators of large infrastructures (and not all critical infrastructures have control rooms) are from time to time pushed into the unknown unknowns by contingent events. (In case it needs saying, unknown unknowns have been contingent in the myriad ways past “unobservables” have been, e.g., Epicurean atomism, phlogiston, and aether.) It turns out that to ask really-existing control room operators—“What if the unimaginable happened?”—is not to ask something new or unusual in systemwide control centers. The fact of the matter is that all manner of people all across the world are having to imagine the unimaginable and think the unthinkable, and acting upon that without recourse to faith-based “Innovation will get us out of the mess we’re in.”

But it’s still too easy to confuse being pushed into ignorance unwillingly with a much valorized albeit half-blind trial and error learning that places a premium on “testing” unstudied conditions intentionally. (Not to worry when being “fully creative, imaginative, and inspired” means being bereft of vigilance and self-reflection at the same time!)

For control operators, real time is too important to experiment in when their first error ends up being our final system trial. The last thing we want is our airplane pilots “to embrace failure” mid-flight, notwithstanding all those anodyne business and management articles on the virtues of error, failure and unstudied conditions. Too much of that privileging borders on modern-day priestcraft, miracle-mongering, and the criminal.

Uncertain superlatives

I

Certainty has such a strong place in politics not just because it serves as the preferred foundation/platform from which to act, but also because certainty supports and drives the belief that any such choice to act can be superlative, i.e., serve as the best or superior or optimal course of action.

A key part of the challenge of a politics of uncertainty is to insist superior and superlative are still achievable, and not in a diminished sense of the economist’s “second best.” The challenge is to show, with and through examples, where superlative and best are not only really-existing in the midst uncertainty, but also how uncertainty’s superlative and best are better than so many of certainty’s counterparts.

II

Each of us probably has our own examples. Three additional pathways ahead are, I feel, under-acknowledged and deserve further consideration:

  • The first is to underscore how certainty can truly mislead, whatever your starting point, as in: Francois Jacob, Nobel Laureate, reflecting that “Our breakthrough was the result of ‘night science’: a stumbling, wandering exploration of the natural world that relies on intuition as much as it does on the cold, orderly logic of ‘day science’”. As in: Nothing quite smacks of certainty as do habits, inhibitions and defense mechanisms. As in: We all know of revered ideals that ended in irrelevance. As in: Humans are never fully in the present; we are ourselves now, but reserve other of our intermittent selves for later action. As in: When in doubt, make the puzzle bigger.
  • The second pathway is to recognize the impossible is never perforce a bar to action in the face of uncertainty. Here is Richard Falk writing on the critic and Palestinian activist, Edward Said: “To dedicate action to achieve the impossible should never be a matter of optimistic false consciousness. It is rather a recognition that there is no way for the rational mind, in light of present circumstances, to figure out a solution that accords with the postulates of a just peace. Yet at the same time there are present moral and political imperatives of carrying on the struggle to reach such a solution, because the future is unknowable and the present circumstance of occupation, oppression, dispossession, and dispersal intolerable.” The insight, I take it, is that we might well be in a position to do something but not know it until we start trying (a position also associated with economist, A.O. Hirschman)..
  • A third way ahead is to insist that uncertainty is real and unavoidable and that this “certainty of uncertainty” looks nothing like the certainties offered up by the political class and deskoid pundits. Lines from one of Norman MacCaig’s poems make the point:

Who owns this landscape? –
The millionaire who bought it or
the poacher staggering downhill in the early morning
with a deer on his back?

Who possesses this landscape? –
The man who bought it or
I who am possessed by it?

False questions, for
this landscape is
masterless
and intractable in any terms
that are human.