Recasting this paradox of high reliability

When colleagues and I started to study the reliability of large critical infrastructures—that socio-technical anchorage of the modern—a core paradox confronted us: How can the infrastructures be as reliable as they were with so much that could go so wrong at any second? How could they be managed reliably and safely with so many moving parts and potential interactions?

It’s important to understand that this paradox has been with us from early on. Here is Frederick S. Williams on the British railways from his Our Iron Roads (1883):

This immense number of passengers and enormous bulk of goods are drawn by engines of the most complicated mechanism, held together with millions of rivets, each engine—containing an intricate network of tubes, numerous cranks, and other delicate pieces of workmanship, and the engines and vehicles are connected by chains and couplings. In every separate item of all these innumerable parts lurk elements of danger, and the slightest fracture might produce disaster. All this is done, and with what result? That there is no safer place in the world, as Professor De Morgan said some years ago, and it is still true, than a railway train.

https://archive.org/stream/ourironroadsthei00willrich/ourironroadsthei00willrich_djvu.txt

My own answer to the paradox starts by focusing on the safety and and reliability track records alluded to in the quote.

But it is necessary not to miss the performative, self-referential nature of labelling the initial question “a paradox.” In doing so, I and others demonstrate our own cognitive limits of understanding the systems we study.

It’s this—and not “next is worse”—we should be alert for

I

Many ecologists and environmentalists I’ve read and with whom I’ve worked insist that, when it comes to ecosystems and the environment, more things can go straight-out, hair-raisingly wrong than right.

It is easier to mismanage an ecosystem than it is to manage it. Ecosystem collapse is more certain than ecosystem sustainability. Negative externalities in the environment are to be expected, positive ones not. Probabilities of large system failure and cascades are primed to flip to 1.0 in no time flat.

We must manage the planet’s resources better, but no one should expect technology to help. Economic growth is never a sufficient condition for improving the environment, while economic growth’s impacts on the environment are always sufficient for precaution. So much is uncertain that anything is possible, which means everything must be at risk.

Let’s call this standpoint, next-is-worse.

II

Of course, there are cases where next is worse. In March 1999, a colleague and I interviewed a well-known ecologist who insisted the Delta smelt would not “go extinct even if we try to wipe them out”. Then came the articles with titles like “the collapse of pelagic fishes in the upper San Francisco Bay-Delta”.

That said, this—realism, manifold anxiety, existential panic, dog-whistle alarmism—describes a world not made to my colleagues’ specification.

Nor is there a scintilla of recognition that my colleagues’ specifications to get us to do the right thing by way of the environment—namely, Stop!—pale and wither before the persisting record: Really-existing humans with real problems in real time routinely do not follow orders, even in the most totalitarian of regimes (as we now know to have been the case in communist East Germany and China).

Nor is there a scintilla of recognition that the major feature of their disaster scenarios aren’t the proleptic ruins, but the massive lack of attention to the multiple ways necessary to triangulate and increase our confidence about how specifically to respond. We’d be fools to expect geo-engineers to provide the descriptive specifics for their disaster scenarios. That can only come from others who have far more detailed counter-arguments to insist upon.

III

So, it’s no surprise when next-is-worse fails to create anything like a shared, collective dread pushing and pulling us to manage better.

Where so, how then can it be denial on our part to insist that all existential disasters pose difficulties, highlight inexperience, and emphasize the strategic interludes of surprise, both good and bad?

What can we be doing instead?

IV

Start by accentuating the contradictions.

Let’s agree today’s rotten core is modernity—international capital, fossil fuel, global urbanization, the Enlightenment project—while in the same breadth insist all this is best understood in the very terms of that modernity: Anything and everything is at risk; all thinkable risks are warnings; any could be catastrophic.

In putting the paradox this way, we are like those trying to predict a poet’s next poem from their current body of work.

A more productive approach might well be to ask: What are we getting from this habituation to it-always-could-get-very-very-much-worse?

One answer: Doing so saves us all the trouble and worry of having to figure out the details. Anyway, who wants more research to sort all this one way or another when foundations and government agencies suspect, if not already know, that “It turns out we’re not even asking the right questions. . .”

A more obvious reason for habituated next-is-worse-ism is the trained incapacities of fatalism. Repeatedly said: Post-apocalyptic novels—doomer literature generally—nail home that we don’t need widespread fear and dread of COLLAPSE to provoke remedy and recovery, because, well, far too many no longer believe in or see chances of either. Less repeatedly said: But that means you are still here, reading those very words.

V

But what’s so special about “still being here,” when those around us are saying next-is-worse?

A familiar example: COP26, the 26th United Nations Climate Change Conference, was for many (me included) a clear failure to do the needful in limiting temperature rise.

Even then, the crux is not: “Thus, alternative voices were left out and alternative politics side-lined.” You can no more essentialize those voices and politics than you can essentialize the conference. For it first has to be asked: Which COP26 failed?

Such a conference is never altogether there, if only because those attending in Glasgow are being themselves in one venue while being other selves in other venues there. COP26 is and was riddled with this intermittence and who’s to say the earlier or later versions around and in between October 31 and November 13 2021 are not its upside? Next-is-worse is just one venue. What about the others?

Not to see intermittence at work and its importance is like the actor playing Hamlet, who rushed through the bedroom scene with Gertrude by forgetting to kill Polonius.

VI

The fact that we are different selves at different times and occasions would be banal were it not for its imperative: having to listen more carefully to what these selves are saying in real time and across time. I have in mind listening to and for two positive sets of statements from policy types, namely, their references

  • “with respect to,” “under what conditions,” “this is a case of”. For example, it’s risks and uncertainties with respect to these failure scenarios and not those that we should be worried about. It’s under those conditions and not these that we take action. What we are talking about is something different, it being a case of . . .
  • “Here’s our track record…,” “Here are our measures of success…or failure,” “We’d be ok with these mixed results…”. Does what actually happened match what was originally proposed? Or, how does what happened compare to the success record of others in like situations? Or, what would have happened even had not the policy been implemented?

These statements (and variants) reduce to versions of “yes, but” or “yes, and,” and in so doing indicate the willingness and the ability of the speakers to identify differences that matter for policy and management, right now.

VII

Accordingly, am I the only one who trembles when officials, experts, and advocates say of a particularly tricky state of affairs, “We need to clear the table and make a fresh start!” Dangerous dumbing down is occurring when you hear this and the like from policy types:

  • It’s a win-win, so who can be against it?” (when everyone within hearing distance knows winners rarely if ever compensate losers), “We just need the political will” (when obviously we’ve had too much political will in committing to any and everything), “If implemented as planned” (when the entire point is you cannot assume any such thing); “We need to take back control…,” (when we have to manage precisely because we can’t control and never have been);  and
  • It’ll pay for itself” (when costs, let alone benefits, can’t be measured, aren’t evenly distributed, nor collectively borne), “We must do this at all costs” (when what the policy types are really doing is refusing to tell you what they already know about the likely costs, including loss of lives), and “Failure is not an option” (when failure is always a very real possibility in complex situations having mixed results at best).

VIII

So what?

I remember a tense meeting in the midst of the 2001 California electricity crisis. A senior executive at state grid transmission center told the group: “My view is that we are jumping under the table and the earthquake is happening and what we have to do is to hope this isn’t a nuclear attack and that the rubble will settle and when it does, we get up and be the only ones around who know what to do.”

Now that is something we should be listening for and to.

Recasting national policies for pastoralist development

I propose to categorize policies according to their intended goal into a three-fold typology: (i) compensation policies aim to buffer the negative effects of technological change ex-post to cope with the danger of frictional unemployment, (ii) investment policies aim to prepare and upskill workers ex-ante to cope with structural changes at the workplace and to match the skill and task demands of new technologies, [and] (iii) steering policies treat technological change not simply as an exogenous market force and aim to actively steer the pace and direction of technological change by shaping employment, investment, and innovation decisions of firms.

R. Bürgisser (2023), Policy Responses to Technological Change in the Workplace, European
Commission, Seville, JRC130830 (accessed online at https://retobuergisser.com/publication/ecjrc_policy/ECJRC_policy.pdf)

This epigraph focuses specifically on the how to think about policies that better respond to effects of automation on displacing workers.

Please re-read the epigraph and then undertake the following thought experiment.

I

Imagine it is pastoralists who are being displaced from their usual herding workplaces, in this case by land encroachment, sedentarization, climate change, mining, or other largely exogenous factors.

The question then becomes what are the compensation, investment and steering policies of government, among others, to address this displacement. That is, where are the policies to: (1) compensate herders for loss of productive livelihoods, (2) upskill herders in the face of eventually losing their current employment, and (3) efforts to steer the herding economies and markets in ways that do not lose out if and where new displacement occurs?

The answer? With the odd exception that proves the rule, no such national policies exist.

II

Yes, yes, of course there are the NGO, donor project, and local department trying to work along these lines. But one has to ask at this point in development history whether their existence is the excuse government uses for avoiding having to undertake such policies, regionally or nationally.

III

A more productive exercise might be to ask: How would various pro-pastoralist interventions be classified: as compensatory, as investment, and/or as steering?

It seems to me that many of the pro-pastoralist interventions fall under the rubric of “steering policies”. The aim is to keep pastoralists who are already there, there–and better off in some regards. Better veterinary measures, paravets and mobile teachers that travel with the herding households, real-time marketing support, mobile health clinics, restocking programs as and when needed, better water point management and participation, and the like are offered up as ways to improve herding livelihoods in the arid and semi-arid lands.

IV

Fair enough, but clearly not far enough, right?

For where are the corresponding compensation and investment policies?

Where, for example, are the policy interventions for improving and capitalizing on re-entry of remittance-sending members back into pastoralism once they return home? Where are the national policies to compensate farmers for not encroaching further on pastoralist lands, e.g., by increasing investments on the agricultural land they already have? Where are the national (and international) policies that recognize keeping the ecological footprint of pastoralist systems is far less expensive than that of urban and peri-urban infrastructures?

V

So what?

This missing government policies would function much along the lines government support of various “new green initiatives” are meant to: They seek to derisk dryland development by enlisting the private capital of pastoralists, agro-pastoralists and farmers via adjusting the risk/returns on their private investments in local infrastructure, such as markets and transportation. Obviously this is easier said than done and would have distributional impacts if done.

Source

On the derisking state, see https://osf.io/preprints/socarxiv/hpbj2/

Another must-read for policy and management types

Environmental and Natural Resource Economics and Systemic Racism (short report from Resources for the Future)
By Amy W. Ando, Titus O. Awokuse, Nathan W. Chan, Jimena González-Ramírez, Sumeet Gulati, Matthew G. Interis, Sarah Jacobson, Dale T. Manning, and Samuel Stolper

https://www.rff.org/publications/working-papers/environmental-and-natural-resource-economics-and-systemic-racism/

Mercifully well-written and short, this working paper summarizes in one place all that is wrong with environmental and natural resource economics as applied to real people with real problems making real decisions.

As frequently said of meditation, it’s the nature of the mind to wander, again and again

–If, as novelist Henry James put it, what is real is what remains, then the playwright Samuel Beckett’s “nothing” in “Nothing is more real than nothing,” is what remains—before or after—everything has been ironized?

–What’s to be criticized when the positives are those that make room for knowing more by knowing less, that at times what clarifies is blur, that good enough is to be naïve enough as an adult to see anew, that recasting categories of living and acting happens at the limits of cognition, and that thinking the hitherto unthinkable is an everyday extraordinary?

–It should be scant surprise that a species constantly recasting its past finds it difficult to predict its future.

–“Collect all the facts that can be collected about the life of Racine and you will never learn from them the art of his verse. All criticism is dominated by the outworn theory that the man is the cause of the work as in the eyes of the law the criminal is the cause of the crime. Far rather are they both the effects.” Paul Valery

–The more you have to lose, the less you can take for granted. We are left somewhere between “Though to/hold on in any case means taking less and less/for granted…” and “to lose/again and again is to have more/and more to lose…” (Amy Clampitt from her “A Hermit Thrush” and Mark Strand from his “To Begin”). What to do? Elizabeth Bishop suggests: “Then practice losing farther, losing faster”.

–As one critic points out, reversion to the mean is not reproduction of the same.

–We’d like to believe that an idea isn’t responsible for those who believe in it, but that misses professed ideas reflect intentions, and intentions can be part of action.

–There is that sheer delight in turning catastrophism against the catastrophists. The delicious part of an otherwise dispiriting meeting on one-more-crisis comes when I get to add “. . .and of course there are the other things to worry about.” Heads look up, eyes dart, you can almost hear them thinking. If someone does ask—“What other things?”—I offer nothing explicit. We, well, can’t quite put our finger on what’s going wrong, this unease. . .

–They hanker after the old language, that of Baroque music or Mozart, and keep asking why we can’t have more of that now. Yet it’s not only that the language has changed, that we can’t go back, and that new language is needed for meanings pushed further. They also want more Bach because that way they don’t have to think about the new, let alone the changes in between.

–The idea was that critique would ensure an imagination that was ahead of history, or in our case, ensure change is in the race with inertia.

–As the law has no eyes (said Xenophon’s Cyrus), so too macro-design.

–They read less as crisis scenarios in need of details than grudges passed off as threats.

–You’d think that “radical” in “radical uncertainty” would require responses other than the same-old same-old. Yet in his book on the 2008 financial crisis, Mervyn King, former head of the Bank of England, ends up recommending the conventional: Radical uncertainty–King’s term–needs to be better incorporated in economic and financial theories and practices. In this way, “radical” is dumbed down at the exact moment when needed most.

–If, as they say, need connects everything, then rarely have we been as connected as we were when isolated from each other during the pandemic.

–It’s an odd kind of a-historicism to deny utopian possibilities because we live in an endless present that forecloses on anything like a future.

–Overdetermination: too much wind-up for the pitch thrown. Resilience: the play in a steering wheel. Progress: watching Sovietology fade away. Solipsism: the last stage of society’s polarization.

–To “see” the unknown unknowns means sensing ignorance through surprise and contingency. The opacity of ignorance leaves these traces and traces mean ignorance of unknown unknowns is never “placeless.”

–How can you have “proper pricing of risk,” if you don’t know the system to be managed and the reliability standard to which the system as a whole is to be managed? Only then, can you ask: How are the risks entailed by subscribing to that standard for this system to be managed?

–Utopians are, in my view, too often criticized for their optimism. Utopians, after all, are the ones more likely to ask: What kind of society designs its critical infrastructures in such a way that not managing them reliably is more costly than having to manage them reliably, all the time 24/7/365? How long can our reliability professionals provide just far enough and just soon enough what is just enough? No wonder utopians without answers want something better!

Frederick Scott Oliver and politics

I

I first came across Frederick Scott Oliver,in a passage that still resonates from a 1955 lecture of T.S. Eliot:

But it should also be obvious to everyone from his personal experience, that there is no formula for infallible prediction; that everything we do will have some unforeseen consequences; that often our best justified ventures end in disaster, and that sometimes our most irrational blunders have the most happy results; that every reform leads to new abuses which could not have been predicted but which do not necessarily justify us in saying that the reform should not have been carried out; that we must constantly adapt ourselves to the new and unexpected; and that we move always, if not in the dark, in a twilight, with imperfect vision, constantly mistaking one object for another, imagining distant obstacles where none exist, and unaware of some fatal menace close at hand.

Does this all mean futility or what we called at that time, the quietism of despair? Not so, Eliot immediately adds: “This is Frederick Scott Oliver’s Endless Adventure.

II

Even though his language is dated and some opinions rebarbative, Oliver’s The Endless Adventure, published as three volumes in the 1930s, is so contemporary as to make one weep. The book is out of print, so here’s a sampler of quotes::

  • “Without bringing all the Christian virtues into this discussion, it is enough to say that a positive and strict veracity is impossible for the politician. For truthfulness even forbids you to allow the person you are dealing with to deceive himself.”
  • “…like water can’t be kept out of most things, so too morals can’t be kept out of human affairs. But it is an external factor not inward gyroscope.”
  • “For surprising accidents and sudden changes are the rule of politics. It is not often that the circumstances of the world will let a statesman have his head. The situation into which he comes so confident of victory may be transformed in a single revolution of the globe. Thereupon all the schemes that he has framed so carefully for the service of his country will vanish hurriedly like ghosts at cock-crow. He will be forced at once to devise a new plan fit for the occasion, and he will be lucky if he produces one that does not involve a sacrifice of this consistency.”
  • “The wisest government must make mistakes; nay, sometimes when it has acted with most wisdom it affords the easiest target for plausible misconstruction.”
  • “The student of politics will not make a beginning till he has realized that in this art there are antinomies everywhere, and that it is no shame to a politician, or to the man who writes about him, if the opinions he utters are often in conflict one with another. The politician or the writer who succeeds in proving his life-long consistency is less an object of admiration than of derision.”
  • “Phenomena of this sort, phenomena in a continual flux, will not submit themselves to the methods of a land surveyor.”
  • “Politicians, like soldiers, are often obliged to guess at the motives, intentions, and movements of the enemy. As they often guess wrongly, their own tactics are apt to appear purposeless and foolish, or altogether evil and malevolent, to a later generation which looks wonderingly, after ‘the fog of war’ has lifted, at the hooks and bends of an ancient controversy. . . .If actors themselves saw less clearly than we do, it is partly because there are now far fewer things to be seen. Much has long ago fallen through the sieves of memory and written records, while the historian, of set purpose, has eliminated much of what remained.”

III

Oliver defends politics specifically against university specialists (another contemporary resonance), and his comments about economics and economists ring true to this reader. Oliver does not defend all politicians: “It was in plain truth only a quack cure-all at which a cabinet of ignorant shirkers had snatched in its perplexity,” he writes.

Fortunately, Oliver takes us further than that. He also talks about political imponderables and chance, as well as inexperience and difficulty, as if they were the churn in that watercourse—the thalweg—at the lowest part of the valley along which we cross and whose waters we ford.


ChatGPT and high reliability

In the Guardian’s online Bookmarks today, the science fiction author, M. John Harrison, answers the following question:

How do you feel about the emergence of AI?
I’d separate the thing itself from the boosterism around it. We’re at a familiar point on the curve when it comes to the overenthusiastic selling of new scientific ideas, where one discovery or tech variant is going to solve all our problems. I’d say wait and see. Meanwhile I’ll be plotting to outwrite it; I want to be the first human being to imitate ChatGPT perfectly. I bet you it’s already got mimickable traits.

https://www.theguardian.com/books/2023/may/20/m-john-harrison-i-want-to-be-the-first-human-to-imitate-chatgpt-wish-i-was-here?utm_term=6469c17386bac9427580944744a8948a&utm_campaign=Bookmarks&utm_source=esp&utm_medium=Email&CMP=bookmarks_email

I very much like the last part of that answer. A neat turning of the tables while cleverly not classic oneupmanship.

–But there is that narrative discrepancy, namely: to mimic ChatGPT perfectly. Current research indicates large language models, including ChatGPT, produce unexpected mistakes: “LLMs exhibit unpredictable failure modes,” a recent comparison study phrased it.

Now, to err is human and we commit all manner of mistakes. So in that sense a mistaken ChatGPT is already mimicking and mimicked by humans

–But it’s Harrison’s “perfectly” that should stop us.

Perfectly with respect to what? Perfectly with respect to how a black box large data algorithm gets the answer wrong? Well, that can’t be for at least two reasons. First, all the stuff about our brain having 86 billion neurons and being vastly more complex than said algorithm. Second, black box means black box when it matters, right now: We just don’t know how the current mistake came about–albeit LLM developers have the same remedy: more training of the LLMs.

–Harrison, though, is onto something very important, I think: What’s better than humans mimicking ChatGPT perfectly?

It would be humans managing (not just designing) a ChatGPT that reliably avoids mistakes it can’t otherwise prevent. That is, ChatGPT is managed in real time so as to correct unavoidable mistakes of its own making before answer delivery. The effect of management is that it, the LLM, does this correction just in time even with the current level of training. Now that wouldn’t be science fiction!

Source


https://doi.org/10.48550/arXiv.2305.00050

Paul Valéry, 1931

Every habitable part of the earth, in our time, has been discovered, surveyed, and divided up among nations. . .The age of the finite world has begun. . . .

The effects are already immense. An entirely new, excessive, and immediate interdependence between regions and events is the already perceptible consequence of this great fact. Henceforth we must see all political phenomena in the light of this new situation in the world; every one of them occurs either in obedience or in resistance to the effects of this definitive limitation and ever closer mutual dependence of human actions. The habits, ambitions, and loyalties formed in the course of earlier history do not cease to exist but being insensibly transferred into quite differently constructed surroundings, they there lose their meaning and become causes of error and fruitless striving. . . .

Henceforward every action will be re-echoed by many unforeseen interests on all sides; it will produce a chain of immediate events confused reverberations in a closed space. The effects of effects, which were formerly imperceptible or negligible in relation to the length of a human life and to the radius of action of any human power, are now felt almost instantly at any distance; they return immediately to their causes, and only die away in the unpredictable. The expectations of the predictor are always disappointed, and that in a matter of months or a very few years.

In a few weeks, the most remote circumstances can change friend into foe, foe into ally, victory into defeat. No economic reasoning is possible. The greatest experts are wrong; paradox reigns.

There is no prudence, wisdom, or genius that is not quickly baffled by such complexity, for there is no more duration, continuity, or recognizable causality in this universe of multiple relations and contacts. Prudence, wisdom, and genius can be identified only by a series of successes; once accident and disorder are predominant, an expert or inspired game is in no way different from a game of chance; the finest gifts miscarry.

Hence the new politics are to the old what the short-term calculations of a stock market gambler the nervous spurts of speculation on the floor of the exchange, the sudden fluctuations and reverses, the uncertain profits and losses are to the old patriarchal economy, the slow, careful accumulation of a patrimony. . . . The long-pursued schemes and profound thought of a Machiavelli or a Richelieu would today have no more reliability and value than a “stock market tip.”

https://archive.org/stream/outlookforintell013551mbp/outlookforintell013551mbp_djvu.txt

Our challenge is to push Valéry’s truth further by finding the positive in the nine decades of complexity since then and now ahead.