Triangulating complexity for policy and management

I

–A key lesson to take away from a system being highly complex—i.e., having a high number of elements, different functions for those elements, and interconnections between elements and functions—is the centrality of triangulation in decisionmaking.

You need as many different approaches to analysis and strategy as you can—qualitative, quantitative, reductionistic, holistic, positivist, post-positivist, more (computational, bargaining, judgment, inspiration, more)—in the hope that you converge from diametrically different directions on a common factor to consider. Such triangulation is most successful when “Whatever the direction you look at this issue, you get to this same point. . .” Familiar examples are the importance in development of women and of the middle class(es).

–Triangulation here is the use of multiple methods, databases, theories, disciplines and/or analysts to converge on what to do about the complex issue. The goal is for analysts to increase their confidence–and that of their policy audiences–that no matter what position they take, they are led to the same problem definition, alternative, recommendation, or other desideratum.

In doing so, the analyst accommodates unexpected changes in positions later on. If your analysis leads you to the same conclusion regardless of initial positions already highly divergent, then the fact you must adjust that position later on matters less because you have sought to take into account utterly different views from the get-go.

–Everyone triangulates, ranging from the everyday cross-checking of sources to more formal use of varied methods, strategies and theories for convergence on a shared point of departure or conclusion. A popular form of triangulation is the use of multiple—the “tri” doesn’t mean three only—methods. Methodological triangulation figures prominently in applied fields (though not all!), e.g., practicing policy analysis, marketing, investigative journalism, and participatory rural appraisal, to name several.

–Triangulation is thought to be especially helpful in identifying and compensating for biases and limitations in any single approach. Obtaining a second (and third. . .) opinion or soliciting the input of the range of stakeholders or ensuring you interview key informants with divergent backgrounds are three common examples. Detecting bias is fundamental, because reducing, or correcting and adjusting for bias is one thing analysts—better yet, human beings—can actually do.

Triangulating on a common point is in no way guaranteed a priori just as canceling out biases—be they cognitive, statistical, cultural, other—cannot be assumed to have occurred as a result of triangulation. (Anyway, it remains an open question which biases are most important–material interests, cultural beliefs or built-in cognitive biases, among many other candidates.) To the extent that bias remains an open question for the case at hand, it must not be assumed that increasing one’s confidence automatically increases certainty, reduces complexity, or gets one closer to the truth of the matter.

–That said, failure to triangulate also provides useful information. When findings do not converge across multiple orthogonal metrics or measures (populations, landscapes, times and scales…), the search by the analysts becomes one of identifying specific, localized or idiographic factors at work. What you are studying may in reality be non-generalizable—that is, it may be a case it its own right—and failing to triangulate is one way to help confirm that.

Triangulation is consuming and expensive. Limitations on time, money and other resources make it infeasible to employ multiple interviewers, multiple methods, and/or multiple databases as much as one would like. A second problem is that inexpungible bias. No matter how many cross-checking questions in the survey, they cannot correct for the fact that the interviewer is white, male, middle-class and asks questions in English only. On the other hand, while triangulation is time consuming and expensive, so too taking positions in its absence can prove costly.

II

–Much of the above should not be news. What is new, I think, is the importance of mixed methods and approaches for better honing bias.

To return to our starting point: The approaches in triangulation are chosen because they are, in a formal sense, orthogonal. The aim is not to select the “best” from each approach and then combine these elements into a composite that you think better fits or explains the case at hand.

Why? Because the arguments, policies and narratives for complex policy and management already come to us as composites. Current issue understandings have been overwritten, obscured, effaced and reassembled over time by myriad interventions. To my mind, a great virtue of triangulation is to make their “composite/palimpsest” nature clearer from the outset.

–Not only is triangulation not about assembling a “seamless” explanation from parts of the pre-existing frameworks or methods you value; it in fact asks you to undertake a kind of analysis that runs against the grain of assuming coherence and seamlessness. To triangulate asks what, if anything, has persisted or survived in the multiple interpretations and reinterpretations that the issue has undergone over time up to the point of analysis. As we just saw, finding fewer and fewer resemblances across the family portraits in a complex policy is a very important point for more and more case-by-case analysis.

–There is irony in this. It turns out that another way to build up the policy audiences’ confidence that you do know what you’re talking about—at least, compared to them—is to undertake a policy analysis as moments of further questioning and reflection: Just what is the problem or problems, just what constitutes the evidence we need to consider, just what are criteria to evaluate options, and so on. (Just what is “development,” if not case by case?) Increasing the sense of social construction of these matters turns out to lay the basis for triangulating as well—though it should go without saying there are no guarantees in this.