Coherence between many lines of evidence: Part II

In the last post we discussed one of the key hallmarks of good scientific methodology: reproducible results across many different and complementary measurement techniques. In this post, we will discuss the ways in which phony scientific skeptics can spin these coherent results in order to make a particular measurement or conclusion look weak.

100 methods, 100 problems (Argument by Anecdote)

The more different methods corroborate a result, the more confidence we can place on that result: especially if the strengths of each different method can cover for each other’s weaknesses. Ironically, in the eye of a phony skeptic: the more techniques there are, the easier it is to make a given finding look weak. His equation is simple: more methods = more weaknesses.

No measurement technique is perfect.  For a finding supported by many different lines of evidence, a phony skeptic can produce a laundry list of the weaknesses of each method without ever putting them together to form a bigger, coherent picture. If a hundred different methods show the same conclusion, he will ignore or gloss-over the common conclusion and simply discuss all of the particular failings with each technique, taken by itself. This sort of laundry list is what I call “argument by anecdote”. And, for such a faux critic: numbers are all that matter. The more doubts he can throw your way, the more likely he hopes one of them will stick.

The thing that is inevitably missing is context. And, this is the distinguishing characteristic of true scientific skepticism. When a scientist tries to make a judgement on the strength of scientific evidence, context is her top priority. Those who wish to contradict science tend to value lists over context.

Asking Rhetorical Questions

Nothing is more frustrating to a scientist than an insincere rhetorical question. We work hard at what we do. We love tough questions, but we want them to be asked out of genuine curiosity. Unfortunately, rhetorical questions are an effective technique for casting doubt on scientific findings.

A common approach is the use of rhetorical questions to imply oversight on the part of scientists. People ask questions with a tone as if the scientific community did not even think to ask said questions. More often than not, these questions have been both asked and answered throughout the scientific literature. But, the kind of person who asks these questions typically has no real interest in finding out. Better to leave the question open-ended. Better to rely on innuendo.

My doctoral dissertation was based on research that I spent 6 years working on (along with 10 other colleagues). I will talk more about it in future posts. Suffice to say, I spent all of every day thinking about, quantifying, and testing all of the ways my measurement could go wrong. Way more time was spent on the error analysis than the measurement itself. This is typical in my field. It is certainly true that sometimes obvious questions are overlooked even by the experts. But, more often than not, if a question about a particular scientific finding seems pretty important, odds are high that the people who did the work already thought about it. I don’t expect or even want you to take that on faith. Next time someone asks an open-ended and accusatory question about a scientific finding, check for yourself. Email an expert. Search the literature. You will see that the people doing the research are thoughtful, thorough, and skeptical (it’s what we’re hired to do) and you will see that the people asking the accusatory questions often have a transparent agenda.

Purposely testing a technique under circumstances where you know it won’t work.

In the previous post I presented the example of a bathroom scale that works perfectly well in earth’s gravity. Given the assumption of earth gravity, the scale would give incorrect results if it were used on the moon. Now imagine someone purposely used the scale on the moon in order to discredit the reliability of the scale. Pretty dishonest, huh? Yet, this sort of approach is common place among those who have an axe to grind with certain scientific ideas.

Take the case of carbon dating: Carbon dating is based on the premise that a living organism incorporates C-14 from the atmosphere into is body (through plants breathing CO2 and animals eating the plants, for example). When that animal dies, the C14 starts to decay. How much of it has already decayed tells us how long ago it died. There are two significant preconditions for carbon dating to work: (1) It needs to have carbon and (2) it needs to have been primarily in direct contact with the atmosphere. Aquatic life, for example, is in contact with “old carbon” stored deep in sea. Thus, it is known to produce anomalous carbon dating results. Many people who spin doubt regarding radiometric dating will come up with countless lists of stories wherein samples-that-shouldn’t-be-carbon-dated for known reasons are dated and give nonsense results. This approach is a combination of my argument by anecdote (lists and doubts) and the asking of rhetorical questions (since the person listing these anecdotes does not mention that these samples were of a pedigree known to fail carbon dating).

In this article we focused on some aspects of false skepticism. As promised in my introduction to this blog, I hope to also provide a rubric for how to exercise legitimate scientific skepticism. Looking forward to your thoughts and comments!

 

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s