for Robert Chambers
For reasons that will be obvious, no names or references are given in what follows. The numbers, however, remain roughly as first identified.
–Researchers estimated the annual probability of a major stretch of island levees failing ranged somewhere between 4% to 24% due to a slope failure. (Slope instability in this scenario would be caused by flooding behind the levee as well as high water levels on its water side.)
Our estimates were considerably higher than the official one, in large part because the research project relied on better validated methodologies for accommodating uncertainties.
–We presented the findings to the island’s management board. Their first and really only question was whether our estimates would be revealed to island insurers.
–We had a “hotwash” afterwards to figure out their—how to put it?—underwhelming response:
- Didn’t they understand the upper range, 24% per annum, implied a levee breach was nigh inevitable with respect to our slope instability scenario? Or to put the question to our side, in what ways did the 24% per annum estimate fall far, far short of being a failure probability of 1.0?
- But if as high as 24% per annum, why hadn’t there been a levee breach over the many decades since the last major one there?
- What about the other islands nearby? Assuming even only a few of these had a similar upper range, why weren’t levee failures happening more often in the same watershed and variable flooding conditions there?
- The 4% – 24% range was with respect to annual levee failure due to slope instability only. If you add in all the levee failure modes possible there (e.g., due to seepage rather than overtopping and flooding), the total probability of levee failure would have to be higher. So, even if all levee fragilities were at the lower end of each failure mode’s range, this was scarcely reassuring. (But then again, what are the conditions under which the more ways there are to fail, the more likely failure is?)
- One answer to why levee failure there hadn’t happened—yet—was there hadn’t been a long enough period to observe levee breaches so as to form the distribution from which the 24% could be established empirically. But these levees, and worse ones on nearby islands, had been in place for decades and decades; some had been improved in fact. The burden of proof, in other words, was on us, the team of levee experts, to explain why this wasn’t “long enough” or what that long-enough would have to be.
- The absence of actual levee failures at this island specifically could be more complicated than we first thought. The levee stretch in question could be “failing to fail.” It might be that this stretch had not undergone events that loaded them to capacity and worse. (But that again the question begging: How much worse would the conditions have to be in our expert view? Just what is a probability of failing to fail?)
- Even though our team had used multiple methods to triangulate on levee fragility and even though the others methods enabled us to incorporate uncertainties better, these methods still may have fallen well short of the needful. This suggests spending more resources on reducing the uncertainties imported into risk estimates or otherwise ignored.
- Or to put the preceding point differently, was this levee stretch on that island more diverse and more resilient (say, in the way biodiverse ecosystems are said to be more resilient) than current methods capture but which islanders there better understand and manage?
–But our most significant observation was the one none of us saw need to voice: How could we accuse the management board and islanders of being short-sighted or worse, with so much else going on challenging us, the team, to make sense of such estimates?
In the profoundest sense possible, their not-fleeing the island was our problem, not theirs. Let John Kay, the economist, have a last word:
There are no 99 per cent probabilities in the real world. Very high and very low probabilities are artifices of models, and the probability that any model perfectly describes the world is much less than one. Once you compound the probabilities delivered by the model with the unknown but large probability of model failure, the reassurance you crave disappears. . . .
–This demand to assess the possible rather than assume the probable is far from that of Reverend Bayes in updating probabilities and closer to that of economist, G.L.S. Shackle, with his notion of focusing on the counter-expected or countered-desired as possibility. Those serious about mess and reliability should be very worried at the increasing sanctification of Bayesian analysis in decision theory.