Friday, May 1st 2020

“It is better to be roughly right than precisely wrong.”
— John Maynard Keynes

To be logical and to be rational are not synonymous (at least not in any real-world, practical sense). Below is a rambling mix of lazily segway’ed quotes, which I’ve collected on why.


“…as a scientist who scientifically studies rationality in human reasoning…it is often surprising to me how little of the science of rationality advocates of rationality make use of…the idea that rationality is just the logical manipulation of propositions is something we should question…” — John Vervaeke, Awakening From the Meaning Crisis


Like most of what I write here this has not been written to be easily read by others (though, you’re welcome to try). It’s just an exercise in clearing my own mind of a topic I can’t stop thinking of. So below is very stream of consciousness-style, intended more as ‘notes to self’ for me personally, but probably not structured or written well enough to be coherent for others.

But if you want to try, have at it.


Below lists all books/articles that went into this, so that you don’t have to parse it one-by-one from the quotes:

I think I ended up removing the quotes from some of them. But anyway, I’ll leave them above.

Logical Nonsense

Logical analysis (despite perhaps appearing ‘sound’), often just results in nonsense in the real-world:

Physics is, by far, the simplest science. It doesn’t seem that way, because we know so much about it, and the required knowledge often seems esoteric and technical. But it is blessed by this amazing feature: we can very often make ludicrous simplifications — frictionless surfaces, perfectly spherical bodies — ignoring all manner of ancillary effects, and nevertheless get results that are unreasonably good. For most interesting problems in other sciences, from biology to psychology to economics, if you modelled one tiny aspects of a system while pretending all the others didn’t exist, you would just end up getting nonsense. (Which doesn’t stop people from trying.)”
Sean Carroll in The Big Picture

Good thinking (or at least, good science) is predictive making propositions that actually, consistently come true. But when logicistic models require modelling one part of a complex system in isolation…

Just as logicistic notions of objectivity hold that the aspiring good thinker must remove himself from the act of thinking, so logicistic notions of abstraction maintain that the object of thought must be detached from its environment.
Kerry S. Walters: Beyond Logicism in Critical Thinking

This approach has been described as ‘logicism’:

By “logicism,” I mean the unwarranted assumption that good thinking is reducible to logical thinking…best describe as “analytic reductionism.” This method teaches how to break arguments down into their simplest constituents—premises and conclusions—and then investigate whether or not the latter are logical inferences, either deductive or inductive, from the former.”
Kerry S. Walters: mixed quotes from Re-Thinking Reason

…which often leads to very logical-“sounding” conclusions which are, nonetheless, complete nonsense:

We can define analysis as the process of trying to understand a problem by breaking it down into its components and then performing logical and/or mathematical operations on these components. Analytical methods such as deductive logic will help us arrive at sensible conclusions. One of the risks of relying upon analysis, though, is that we’ll distort the problem when we deconstruct it so that it won’t make sense when we try to put the pieces back together.
Gary Klien in The Power of Intuition

This has led to formal models in all sorts of disciplines that ‘look good’, but don’t map to reality. They’re what I mean when I say ‘logical nonsense’. The easiest area to pick on is economics:

Macroeconomics…is like a science that has not only stalled for three decades, but has actually gone backwards in its ability to understand reality.
Paul Romer quoted in The Guardian

…but it goes well beyond the easy-target of economics:

Formal models have been usefully employed to solve problems in such relatively undemanding areas as inventory control and logistics. They have generally failed to yield effective results in the more complex, less clearly defined problems of business management, housing policy, or criminal justice…Formal modellers have responded to this unpleasant discovery in several different ways…But for the most part, the use of formal models has proceeded as though it had a life of its own. Driven by the evolving questions of theory and technique, formal modelling has become increasingly divergent from the real-world problems of practice.
Donald A. Schon in The Reflective Practitioner

…culturally, it’s as if we’re playing a game where signalling the appearence of being ‘logical’ pays more than actual truth-seeking does:

…there might be a “general failure mode” in any discipline that becomes over-reliant on maths. Basically, the kudos goes to people at the cutting edge of designing mathematical models, not to those whose models match reality.
Paul Romer quoted in The Guardian

…this, unfortunately, extend beyond academia and into everyday life and could be called the…:

“Trivial Pursuit” theory of knowledge. …[it tends] to treat the substantive knowledge and information for critical thinking as though they typically consist of facts that are relatively simple and discrete. Like “Trivial Pursuit,” knowledge is assumed to be the kind of thing that can be fitted into one-sentence questions, with one-sentence answers…The operative strategy seems to be: first, you get the relevant facts, and, as in quiz shows, this step is assumed to be relatively straightforward, then you use these various skills to derive clever solutions or arguments
John E. McPeck: Critical Thinking and the “Trivial Pursuit” Theory of Knowledge

…in the real-world, we should care that logicism rarely actually works:

In real life, however, this assumption about knowledge cannot be maintained…This complex processing of knowledge is always involved in real-life problems that require critical thought, and it is the norm not the exception…It is only by denigrating or ignoring the complexities of substantive knowledge and information that the specific skills approach can be made to even sound initially plausible.”
John E. McPeck: Critical Thinking and the “Trivial Pursuit” Theory of Knowledge

…it’s one (of many) places we could learn something from Lee Kuan Yew…:

…we were sufficiently practical and pragmatic enough not to be cluttered up and inhibited by theories. If a thing works, let us work it, and that eventually evolved into the kind of economy that we have today. Our test was: does it work?
Lee Kuan Yew in Farnham Street

It’s troubling that “appearing logical” seems to be more culturally important than actual, authentic truth-seeking.

A “Let’s be logical about this…”-style deconstruction conversation about complex problems is not just considered acceptable today, but admirable…whether you have the concrete knowledge to it back up or not.

It is considered the rational approach by most people. So long as you appear to apply sequential logic, that makes you ‘rational’…and despite its poor results, it’s heretical to denounce…:

It is striking that the dominant model of professional knowledge seems to its proponents to require very little justification. How comes it that in the second half of the twentieth century we find in our universities, embedded not only in men’s minds but in the institutions themselves, a dominant view of professional knowledge as the application of scientific theory and technique to the instrumental problems of practice?”
Donald A. Schon in The Reflective Practitioner

…it is important to understand that this is ideological, and there’s a history:

Why Do We Believe This?

Why do believe in a logicistic model that so consistently…doesn’t actually work? Well, …:

The answer to this question lies in the last three hundred years of the history of Western ideas and institutions. Technical Rationality is the heritage of Positivism, the powerful philosophical doctrine that grew up in the nineteenth century as an account of the rise of science and technology and as a social movement aimed at applying the achievements of science and technology to the well-being of mankind. Technical Rationality is the Positivist epistemology of practice. It became institutionalized in the modern university, founded in the late nineteenth century when Positivism was at its height, and in the professional schools which secured their place in the university in the early decades of the twentieth century.”
Meaningful propositions were held to be of two kinds, either the analytic and essentially tautological propositions of logic and mathematics, or the empirical propositions which express knowledge of the world. The truth of the former was to be founded in the fact that their negation implies a self-contradiction; the truth of the latter, in some relevant empirical observation. The only significant statements about the world were those based on empirical observation, and all disagreements about the world could be resolved, in principle, by reference to observable facts. Propositions which were neither analytically nor empirically testable were held to have no meaning at all. They were dismissed as motive utterance, poetry, or mere nonsense.
Donald A. Schon in The Reflective Practitioner

This remains today, by and large, the standard by which we judge ‘good thinking’. So long as you present inline with the above, you can claim to be a good and logical thinker.

And it’s not wrong, exactly…if we can reduce a problem to a very limited, discrete, and objective scope it’ll work just fine. But the issue is, in every practical sense, we can’t isolate the overwhelming majority of real-world problems this way.

Yet, we carry on pretending as if we can — to return a shortened quote:

Physics is, by far, the simplest science…it is blessed by this amazing feature: we can very often make ludicrous simplifications… For most interesting problems in other sciences…if you modelled one tiny aspects of a system while pretending all the others didn’t exist, you would just end up getting nonsense. (Which doesn’t stop people from trying.)”
Sean Carroll in The Big Picture

We get nonsense when logicism, which is suited to Tame Problems, is applied to Wicked Problems instead…

Wicked Problems

What causes wicked problems? In a word, complexity. You have to take into account more stuff. There are more inter-relations, more people involved, more disagreement, and less reliable information. You know the drill because these are the kinds of problems you deal with or think about everyday. Systems thinking has so much promise and is so popular because it offers us hope that we may be able to solve a few of these wicked problems.
Systems Thinking Made Simple

Tame problems (unlike wicked problems) assume discrete, easily definable inputs which lend well to sequential-inference:

…the process of inference defined by the principles of analytic reductionism must be sequential. Premise A must lead immediately to premise B, premise B to premise C, and so on until the entire set of premises inevitably points to the conclusion.
Kerry S. Walters: Beyond Logicism in Critical Thinking

…but with Wicked Problems, you cannot reduce to sequential-inference without severely distorting reality (and rendering the conclusions nonsensical):

Increasingly we have become aware of the importance to actual practice of phenomena — complexity, uncertainty, instability, uniqueness, and value-conflict — which do not fit the model of Technical Rationality. Now, in the light of the Positivist origins of Technical Rationality, we can more readily see why these phenomena are so troublesome.
From the perspective of Technical Rationality, professional practice is a process of problem solving. Problems of choice or decision are solved through the selection, from available means, of the one best suited to established ends. But with this emphasis on problem solving, we ignore problem setting, the process by which we define the decision to be made, the ends to be achieved, the means which may be chosen. In real-world practice, problems do not present themselves to the practitioner as givens. They must be constructed from the materials of problematic situations which are puzzling, troubling, and uncertain. In order to convert a problematic situation to a problem, a practitioner must do a certain kind of work. He must make sense of an uncertain situation that initially makes no sense. When professionals consider what road to build, for example, they deal usually with a complex and ill-defined situation in which geographic, topological, financial, economic, and political issues are all mixed up together. Once they have somehow decide what road to build and go on to consider how best to build it, they may have a problem they can solve by the application of available techniques; but when the road they have built leads unexpectedly to the destruction of a neighbourhood, they may find themselves again in a situation of uncertainty.”
Donald A. Schon in The Reflective Practitioner

The more ‘wicked’ a problem the more disconnected from reality reductionist premises become, and the less logicism works:

Most ordinary inquiries and arguments are about empirical facts. They involve premises that, even when we accept them as true, are less than certain and admit of exceptions. A conclusion derived from such premises inherits their precariousness. A syllogism with premises that are not strictly true is not a strict proof.
The Enigma of Reason

How Wicked?

We don’t all face Wicked Problems in our work…and some seem to prefer fields with less uncertainty and more amenable, Tame Problems:

This dilemma of “rigour or relevance” arises more acutely in some areas of practice than in others. In the varied topography of professional practice, there is a high, hard ground where practitioners can make effective use of research-based theory and technique, and there is a swampy lowland where situations are confusing “messes” incapable of technical solution.
There are those who choose the swampy lowlands. They deliberately involve themselves in messy but crucially important problems and, when asked to describe their methods of inquiry, they speak of experience, trial and error, intuition, and muddling through.”
Other professionals opt for the high ground. Hungry for technical rigour, devoted to an image of solid professional competence, or fearful of entering a world in which they feel they do not know what they are doing, they choose to confine themselves to a narrowly technical practice.”
Donald A. Schon in The Reflective Practitioner

…not to say that Wicked Problems can’t (eventually) be reduced to natural science, but simply to say that doing so is often far beyond practical reach, and we do no favours deluding ourselves with pretense of appearing logical, while actually producing nonsense.

Logicism serves those dealing with discrete, Tame Problems Problems but just produces nonsense in the swamp of Wicked Problems:

“…an effective thinker in one area is not necessarily an effective thinker in all other areas…[logic, fallacy, etc.] skills account for such a small portion of the total reasoning required vis-à-vis the complex of cognitive demands posed by different problems, that they are far from sufficient for regarding a person as a critical thinker in all (or even most) domains.
John E. McPeck: Critical Thinking and the “Trivial Pursuit” Theory of Knowledge

Non-Linear Thinking & Wicked Problems

The more wicked the problem, the more difficult it can be to explain the thinking that underlies it. For example, below, an only semi-coherent explanation, but from someone few could deny as extremely knowledgable:

His rambling is not because he does not understand what he is talking about. It is because…

Intuition eludes the grasp of linear thinking, with its exclusive emphasis on cause and effect that are close in time and space. The result is that most of our intuitions don’t make sense—that is, they can’t be explained in terms of linear logic.
Peter M. Senge, The Fifth Discipline

They’re tough to talk about. That’s the messy world of wicked problems.

Holistic Reasoning in Wicked Problems

Most real-world problems are wicked problems, where holistic understanding blends with analysis (two parts of a single whole):

…[moving past logicism] is not to suggest that imaginative paradigms subsequently may not be analyzed according to the logical rules of the calculus of justification and accordingly strengthened, modified, or rejected, but only that their initial discovery is based upon a holistic imaging rather than a reductionistic computation.
Kerry S. Walters: Beyond Logicism in Critical Thinking

Meaning systems thinking, other holistic models > logicism:

Neither mystics nor mechanists, we hold that all available evidence is consistent with the view that on the functional level the mind/brain underlying expertise is a holistic system.
Mind Over Machine

It’s not a question of intuition vs. analysis

The hoary old split between the mystical and the analytic will not do…for neither pole of that often misleading dualism names the ordinary, non-mystical intuition that we believe is the core of human intelligence and skill…analysis and intuition work together in the human mind. Although intuition is the final fruit…analytic thinking is necessary for beginners learning a new skill. It is also useful at the highest levels of expertise, where it can sharpen and clarify intuitive insights….Detached deliberation and intuition need not be viewed as [opposite] alternatives, as is all too often [the case] in simplistic treatments.
Mind Over Machine

It’s calibrated intuition with analysis. See Conditions for Intuitive Expertise: A Failure to Disagree by Daniel Kahneman and Gary Klein..

The question of gut feeling vs. analysis is framed wrong. These are not independent.
Gut feel that does not rely on analysis as a sanity check and verification, is likely to be very arbitrary and very wrong. Analysis that is not answering questions that are raised by somebody’s intuitive judgement that’s based on thousands of exposures to individuals’ thoughts and observations, is a sterile analysis.
So the best of these things is a synergy between intuition — and intuition is not arbitrary, intuition is a summation of many many experiences — a synergy between intuition and analysis, and that synergy is better than either intuition or analysis.
Andy Grove, Business God

When we speak of going beyond logicism, of course analysis is not discarded. Simply, good thinking does not reduce only to what seems ‘sound’ logic:

Good thinking necessarily implies the ability to manipulate the analytical procedures of informal and formal logic. Second wave proponents of critical thinking are in unanimous agreement on this point. They are not irrationalists. But they do contend that the logicistic reduction of good thinking to logical thinking legitimizes a theoretical model and pedagogical tone that are both problematic.
Kerry S. Walters: Beyond Logicism in Critical Thinking

Intuition and analysis go hand-in-hand, synthesized — one the source, the other the arbiter:

“The synthesis between intuition and analysis that seems most effective is when we put intuition in the driver’s seat so that it directs our analysis of our circumstances. This way, intuition helps us recognize situations and helps us decide how to react, and analysis verifies our intuitions to make sure they aren’t misleading us.
Gary Klien in The Power of Intuition

But…not everyone’s intuition carries equal weight

Intuitions’ Merit

In the past I used the model of ‘ideas vs. opinions’ to describe this idea, which I took from here:

“If you haven’t talked to users and you haven’t looked at data, you don’t get to have an opinion about the product. I’m sorry. The person who’s actually done the work gets to have the opinion. You can have ideas, but they get to make the call.”
Emmett Shear, CEO Twitch

But I think the spectrum of Believability from Ray Dalio is a better model than the binary ideas/opinion model I use to try and use:

a. Think about people’s believability in order to asses the likelihood that their opinions are good. While it pays to be open-minded, you also have to be discerning…The best way to make great decisions is to know how to triangulate with other, more knowledgeable people. So be discerning about whom you triangulate with and skilled in the way you do it…”
b. Remember that believable opinions are most likely to come from people 1) who have successfully accomplished the thing in question at least three times, and 2) who have great explanations of the cause-effect relationships that lead them to their conclusions. Treat those who have neither as not believable, those who have one as somewhat believable, and those who have both as the most believable. Be especially wary of those who comment from the stands without having played on the field themselves and who don’t have good logic, as they are dangerous to themselves and others.
Ray Dalio, Principles

‘Good logic’ again, is not bad. But except in the simplest of cases the credibility of where an opinion comes from matters, just as does its logic. What’s dangerous is logicism that is pretense, opining on complex problems with little real knowledge underneath, …or “the unwarranted assumption that good thinking is reducible to logical thinking.”

Ironically (and complicating matters further) it’s especially important to critically assess causal-explanations from experts in Wicked Problems. For more see Chris Argyris’ Flawed Advice and the Management Trap. But that’s another story…

Anyway, it is meritocratic synthesis we want — not raw logicism alone, nor raw intuition alone. From those with deep knowledge, who know how to speak to it:

“…such people have both the disposition (or propensity) and the relevant knowledge and skills to engage in an activity with reflective scepticism. That is, not only are they prone to question things, but they have the relevant knowledge and understanding to help them do so productively.
John E. McPeck: Critical Thinking and the “Trivial Pursuit” Theory of Knowledge

Logicism without adequate knowledge more often becomes sophistry, than it remains authentic truth-seeking…:

…players dislocate claims and arguments from their broader contexts in order to manipulate them in accordance with the mechanics of logical analysis…critical thinking instruction that focuses on decontextualized arguments schools students in sophistry rather than good thinking.”
Kerry S. Walters: Beyond Logicism in Critical Thinking

This should go without saying, but again, actual knowledge matters:

“…when there is a discussion or argument about some public issue, be it the war in El Salvador, disarmament, Reaganomics, or what have you, who is usually able to make the more useful contribution to such a dispute? Is it the person who possesses the relevant knowledge and information or is it the person who has been trained in [logic, fallacies, etc.]? If your experience has been anything like mine, it is the person who has the relevant knowledge…We should notice this is not a matter of skill, but again, a matter of knowledge.
John E. McPeck: Critical Thinking and the “Trivial Pursuit” Theory of Knowledge

Unfortunately, when pretense of being logical is valued more than authentic truth-seeking, as it seems it often is, this cannot go without saying…

So, How to Reason?

The answer (from every source I trust…) is consistently just…do the work:

The work is the hard part, that’s why people avoid it. You have to do the reading. You have to talk to anyone competent you can find and listen to their arguments. You have to think about the key variables and how they interact. You have to listen and chase down arguments that run counter to your views. You have to think about how you might be fooling yourself. You have to see the issue through multiple lenses. You need to become your most intelligent critic and have the intellectual honesty to kill some of your best-loved ideas.”
Shane Parrish, Farnam Street

There are no shortcuts, quick hacks, or magic bullets:

There isn’t a single formula. You need to know a lot about business and human nature and the numbers…It is unreasonable to expect that there is a magic system that will do it for you.”
Charlie Munger

Logical analysis can (and should) augment your thinking, but only if put on top a foundation of hard-earned, multi-disciplinary learning:

What is elementary, worldly wisdom? Well, the first rule is that you can’t really know anything if you just remember isolated facts and try and bang ’em back. If the facts don’t hang together on a latticework of theory, you don’t have them in a usable form. You’ve got to have models in your head. And you’ve got to array your experience both vicarious and direct on this latticework of models. You may have noticed students who just try to remember and pound back what is remembered. Well, they fail in school and in life. You’ve got to hang experience on a latticework of models in your head.
Charlie Munger

The work, again, comes first…

In their attempt to develop critical thinking, they have the order of cause and effect reversed. They believe that if you train students in certain logical skills (e.g., the fallacies) the result will be a general improvement in each of the other disciplines or qualities of mind. Whereas I contend that if we improve the quality of understanding through the disciplines (which may have little to do with “logic” directly) you will then get a concomitant improvement in critical thinking capacity.
John E. McPeck: Critical Thinking and the “Trivial Pursuit” Theory of Knowledge

It seems to say a lot that:

In my whole life, I have known no wise people (over a broad subject matter area) who didn’t read all the time – none, zero. You’d be amazed at how much Warren reads–and at how much I read. My children laugh at me. They think I’m a book with a couple of legs sticking out.
— Charlie Munger

I’m a mungerite when it comes to this sort of thing. But I believe that we are living in a culture which values the “appearance” of being logical more than actual knowledge, or authentic truth-seeking.

Signalling yourself as “logical” is culturally-speaking, often enough.

That’s foolish. Do the work and earn your opinions.