The Danger of Catapulting To Conclusions and The Power of Alternative Hypotheses


“So what’s going to happen next?” The room was full of physics grad students and nobody wanted to answer. You could hear a pin drop….

Dr Richard Berg ran the physics lecture demonstration group at University of Maryland. He designed and provided demos that professors and TAs could use to make their classes more engaging. My personal favorite was the electromagnetic can smasher (for obvious reasons). Every year, first year graduate students in physics were required to attend a seminar where the various research groups presented their work. It was an opportunity to learn about what was going on in the department, to help us shop for future advisors. But, traditionally, the first presentation was given by Dick Berg. He would present simple table-top experiments that inevitably defied intuition; so much so that a room full of kids with 4 years of physics study under their belts would squirm in their seats. He’d egg us on, “You tell me what’s supposed to happen next. We’ve got a room full of future physics professors here.”

One of the things you learn as a career scientist is that the world rarely conforms to common sense and it certainly doesn’t care what we want it to be. Demonstrations like Dr Berg’s are not exceptions. They are the rule. There is a huge chasm between taking a  finite collection of observations and drawing a systematic conclusion from them. I often find myself floored by the way that many non-scientists (and sometimes even scientists) hurl themselves to conclusions that cannot be drawn from the very limited number of facts they have available. For many people, it is enough that a conclusion “just makes sense”, especially if that conclusion reaffirms what they want to believe. But, “making sense” can be an illusion and a dangerous one at that. In popular culture there is a concept of not “jumping to conclusions” prematurely. I think this is an understatement. When we talk about complex subject matters (whether they be the dynamics of Supernovas or curing cancer or tax policy or solving urban poverty), opinionated people can often be seen, not merely “jumping” to premature conclusions, but catapulting to them.

print-galleryCognitive Illusions

The human brain is an excellent pattern finding machine. In fact, it is too good at it! Many people are familiar with optical illusions. But, few people are aware of cognitive illusions: the brain’s ability to see patterns in things that aren’t really there. Given a few facts, our brains quickly connect the dots. I hope to discuss many of these in future posts. In the mean time, wikipedia provides an excellent list of cognitive biases here. Take the time and look at some of these.

It’s More Complicated Than I Think

The most important lesson I wish more people would take away is that in complicated, multifactorial problems it is much harder to come to a solid conclusion or understanding than you may think it is. It’s a humbling and liberating realization.

Above all, I wish that people would realize that this principle applies to all aspects of our human experience: not just science. I would like to live in a world where even our political lives were informed with the awareness that problems are unlikely to fit into our tidy, ideological boxes.

I would suggest that the realization of the world’s complexity offers a “third way”, a middle path in approaching truth. Rather than accepting the false choice between the relativist approach to truth (There is no absolute truth; it’s all relative) and a sort of chauvinistic approach (Of course there is an absolute truth and it just so happens to agree with my beliefs. What a coincidence!) it is possible to take on the realist view that there is an objective truth to the matter, but it is difficult if not impossible to fully understand it. Just remind yourself, “It’s probably more complicated than I think“. One needs to be disciplined to resist falling for intellectual mirages.

The Power of “The Alternative Hypothesis”

In my own experience as a scientist, no skill is more exciting, more important, and yet overwhelming than the ability to generate alternative hypotheses.

People make educated guesses all the time, to fill in the gaps in our understanding. These guesses can make a lot of sense. They can be really clever. And, if our intuition is well-trained, they might just be right. But, a major trick that the brain can pull on us is to underestimate either the complexity or the range of freedom available to the system we’re observing. One way we can shake ourselves out of this complacency, is to force ourselves to think of alternative explanations, beyond the first hypothesis to pop into our heads. In fact, it is best for us to come up with as many alternative hypotheses as possible. With practice and experience, one will get increasingly better at this exercise. Often, you learn it the hard way: testing your hypotheses and finding out they were wrong (again and again and again)…By seeing a more full scope and range of possible explanations that all make reasonable sense but imply very different conclusions, you can better figure out how to design tests to narrow down the possibilities, and you can open your mind enough to accept facts and observations that run counter to what you actually expect. Once you’ve gone through the exercise of listing as many possible hypotheses as you can think of, feel free to pick a pet hypothesis. The important thing is not letting yourself too hastily come to the belief that your hypothesis must be right or that it provides “the only possible explanation”.

Another important realization: even after ranking by which guesses “make the most sense”, observation can surprise you. There is not rule that nature has to conform to what makes sense to us. And, the very idea of something “making sense” can easily become suspect. Hypotheses are starting points not ends unto themselves.

Mixing Alternative Hypotheses and Politics: Be Careful

Scientists don’t just think up alternative hypotheses for ourselves. It is a part of our culture to offer suggested alternative explanations to our colleagues. This is sort of being like a friendly devil’s advocate. It is so important to the early process of developing a new theory or experiment. Because of this habit, we often tend to offer alternative hypotheses to non-scientists, when we hear them making really strong first guesses about an observation. In political contexts, I often find myself pushing back really hard as a devils advocate. The result is that non-scientists interpret my questioning as disagreement. My liberal friends end up convincing themselves that I’m an arch conservative and my conservative friends will think I’m a socialist. If offered patiently and politely, such thoughts can be very helpful. Just be careful; It can end poorly.

Modesty And Curiosity

Through the course of this post I presented a strong case for why we can’t easily draw conclusions from our observations of complicated systems. But, I do not mean for this to imply futility or nihilism. On the contrary, the beauty of the scientific enterprise is that it shows us the extent to which the world can be known. Appreciating how little we know opens us to knew ideas, to seeing the world through a different lens. It opens us up to our most important intellectual driver: curiosity. You cannot be curious if you’ve fooled yourself into thinking that the matter is closed and you hold all the answers. But, if you allow yourself to embrace that curiosity, if you approach problems with an intellectual modesty and a thirst for new knowledge, then you will be inspired to do the necessary work to systematically understand the given problem. Ultimately, the pursuit itself will be far more interesting and rewarding than whether or not you ever reach a fully complete or satisfying conclusion.



  1. Amir Zadaka

    Great stuff, Matt. I agree with approaching problems with an intellectual curiosity and humbleness. I had a similar discussion with my friend Jorian and want to share what he sent me.

    “lost in my rational discussion of the gemara is the strong belief that we know very little about the way the world works, how prayer works, and miracles might happen, and so much more about human nature, and creation, and how time and space might be twisted and warped…. all of this gives validation to the beautiful folk theories of the Chassidic tradition, which are the perfect antidote to scientific hubris (because we forget how radically Newton and Einstein, etc. changed everything, and how everything will be changed again”

    • Matt

      Hey Amir,

      Thanks for commenting! And thanks for sharing Joriam’s thoughts. I’d like to clarify a couple of things:

      While my point in this article is to say that understanding the world is difficult, it is *not* necessarily impossible. Science’s contribution to the world is a set of evidence-based tools and methods that allow us to overcome at least some of our cognitive illusions. The scientific method is flawed and imperfect (as the New Yorker article goes to show). But, it remains unsurpassed among intellectual systems as a method of chipping away at complicated problems and achieving systematic understanding. It has also been remarkably successful.

      “because we forget how radically Newton and Einstein, etc changed everything, and how everything will change again.”

      Yes and no. Science is constantly changing and progressing. In many cases, mistakes are found and whole ideas can completely reverse themselves. Nothing in science is sacred, and we as scientists are willing to consider that any theory or observation could be rejected or superseded. However, one should not miss the signal in the noise. With time, with more experiments, and with more data our scientific understanding of the world generally tends to *converge*. It does not just fluctuate randomly back and forth like last year’s fashion. New theories, like Einstein’s, build onto and add to old ones. But, they still have to explain all of the observed phenomena of the old theories. In short: there are many solid, robust findings in science that seem highly unlikely to reverse themselves.

      “all of this gives validation to the beautiful folk theories of the Chassidic tradition, which are the perfect antidote to scientific hubris ”

      The folk theories of the Chassid tradition are indeed beautiful and speak to some deep truths. But, there is a danger in using scientific fallibility to validate one’s religious beliefs. I don’t think Joriam necessarily intended it this way, but there are religious groups that use the impermanence and imperfection of science as a pretense for avoiding face-to-face contact with those scientific observations that seem to contradict their beliefs. Such an approach can represent a hubris of its own and certainly impedes curiosity (why be curious about something that is unknowable or is only possible due to divine fiat?).

      I do agree with Joriam that Judaism embraces the intellectual modesty of which we speak. One of my favorite lines from Pirkei Avos describes a wise man as one, who “when asked of a matter he has not heard about says ‘I have not heard’.” Another passage in Avos says “We are not expected to complete the task, but we are not freed from trying.” Imperfect as our human efforts to understand the world may be, we are not excused from trying. We must ask questions and we must be willing to engage and struggle evidence, even if it may be impermanent and even if it is inconvenient to us.

      • Amir Zadaka

        Well said, I agree with everything you wrote. I don’t think Jorian meant his comments on scientific hubris to validate his religious beliefs, but can see how it may be interpreted that way. Rather, I think he was referring more to the idea that science is not perfect.

  2. Brian Powell

    Good stuff Matt. This line of thought is especially relevant to today’s disagreement over climate change and the economy, where some of the loudest and most certain voices come from the most clueless among us. These are notoriously complicated problems that are nonetheless critically important to our lives. This forces *everybody* to have an opinion on the matter, and measured and humble views are hard to come by. One of the conveniences of ideology — and why they tend to be so pertinacious and ingrained in people who generally have no idea what they’re talking about — is that they give one a prefabricated viewpoint that allows them to sidestep ever having to know anything about the issues, while somehow also giving them validity. Today’s discourses on climate change and economics are greatly polluted by this kind of nonsense.

    • Matt

      Hey Bri, Thanks! I’m glad you liked the post, and I appreciate the feedback.

      I totally agree with your point. Especially in economics, I don’t see how anyone can be so confident – certainly among non-economists. That is one irreducibly complex mofo of a topic.

      On climate change, I think there’s actually some funny cognitive dissonance. Many of the same people who have infinite confidence in their “arm-chair theories” also say things like “the climate is too complicated to be understood”. So which is it? On the topic of “gut science”, you might get a kick out of this video:

      The guy who does it, Peter Hadfield, is great. I will be referencing his videos more.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s