Anatomy of an Internet Rumor

June 18, 2013

A friend of mine alerted me to this linked article and wondered if there is any weight to the story. I thought this would be a fun chance to work my blogging chops and sort of live-blog how I would go about dissecting and validating this story.

The article describes the “independent verification” of a potential “cold” nuclear process developed by Italian industrial tycoon, Andrea Rossi. Cold fusion is the name given to the idea that one can produce sustained nuclear fusion reactions at low temperatures. Conventional fusion reactions only occur at very high energies. Cold fusion has a very uncredible past history, starting with its claimed observation by Pons and Fleischman and followed by a very public and embarrassing discrediting by the larger scientific community. The story provides an excellent cautionary tale for what can happen when researchers fail to adhere to proper standards of scientific method (for example, see Ref [2]). Since then, cold fusion has essentially been viewed as quack science by most physicists and the public at large. But, as is often the case, it has attained a certain cult status among a small but very passionate community of conspiracy theorists and pseudo-scientists.

What I did not know prior to researching this article is that there is a new, scientifically more credible idea [3] bouncing around for what are called Low Energy Nuclear Reactions (LENR). The reactions are not the same as cold fusion, but do potentially amount to low-temperature, nuclear energy generating mechanisms. It is on this basis of this new idea about LENR, that Andrea Rossi comes in.

Andrea Rossi has, for several years now, claimed to have developed a LENR technique he calls “ECat” that yields energies far beyond chemical processes such as the burning of fossil fuels. He has been extremely secretive and cryptic, but now people are claiming that his technique has been independently verified by a group of scientists who have posted an article on arXiv [4], a commonly used repository for physics pre-prints (unpublished drafts of papers). It is worth noting that arXiv allows anyone to post articles, and the documents on it are not necessarily peer-reviewed. This pre-print, in particular, is not yet reviewed or published. Frankly, it doesn’t even look ready to be submitted for a review in a real journal, as we shall discuss. Nonetheless, this event has sufficed to set the blogosphere abuzz. It even has attracted the attention of some big-name papers such as Forbes [5]. But is there any punch behind this story?

Step 1: Is this source reliable?

One of the first questions one should always ask is whether the source of the report is trustable? This is not a simple question. Some sources are reliable on some subjects, but not others. Moreover, just because a source is a major media outlet, that does not at all automatically mean it’s reliable. It is worth noting: Most formal news sources are pretty unreliable on scientific subject matters.

No magazine, paper, news show, or blog should get a free pass. Everything should be fact-checked. If you haven’t ever fact checked your news sources, I highly recommend it. And, once you’ve tested a source enough times, it is possible for said source to earn your trust. But, make it earn your trust.

In this case, I would say that I have no experience with ExtremeTech.com, so I will approach the article with appropriate skepticism.

Step 2: Make note of “red flags”

The first thing any skeptic will do is look for red flags: potential indications that the claim is bogus. The presence of red flags does not necessarily mean that the claim is bogus. They simply bring attention to the possibility that it could be. Here are a few things that jump out at me:

The cold fusion device being tested has…1,000 times the power density of gasoline. Even allowing for a massively conservative margin of error, the scientists say that the cold fusion device they tested is 10 times more powerful than gasoline….

So the power is between 10 and 1,000 times greater than gasoline? This is a pretty huge margin of error. When I read a scientific finding, I generally expect them to get the “order-of-magnitude” right. The above quote is equivalent to saying: “The nearest gas station is several kilometers away. Even allowing for a conservative margin of error, the gas station is 10 meters away!” If an experiment cannot distinguish between a factor of 10 and a factor of 1,000, maybe it isn’t time to break out the bottle of champagne.

Then there are these quotes:

Rossi, it would seem, has discovered a secret sauce that significantly reduces the amount of energy required to start the reaction. As for what the secret sauce is, no one knows — in the research paper, the independent scientists simply refer to it as “unknown additives.”

While Rossi hasn’t provided much in the way of details — he’s a very secretive man, it seems…”

One need not be a scientist to be skeptical of claims involving a “secret sauce”. And, of course, there is Rossi himself. Rossi’s business history [6] is very much a cause for red-flags. Should I trust a man who has a history of fraud charges? Well, let’s not off-handedly dismiss him, but let’s not give him a free ride either.

Step 3: Find the “linch pins” of the article

What are the key points on which the article hangs its credibility? Every article or blog post will identify one or two key points that the author thinks will legitimize his claims. Even questionable claims can have a grain of truth to them. Our goal is to identify these key points and establish: (1) are they true, and (2) if they are true, do they actually support the full thesis of the article?

This article is about Rossi’s great discovery. Due, in part to his “secretive” nature, the author cannot really legitimize Rossi’s claims on their own terms. Instead, he needs to look to independent sources. Who are these sources? Are they legit? And do they validate Rossi’s claims?

It seems the credibility of Rossi’s claims is hung on two points:

(1) the idea that the claimed mechanisms for the “ECat” device resemble some LENR research by scientists at NASA [7],[8]

(2) the notion that his device has now been “independently verified” in a “paper” on the arXiv [4].

Let’s examine these claims, more in depth.

Step 4: Dig Deep, aka Google is our friend

Just googling the name of the key NASA researcher, Joe Zawodny along with keywords like “Ecat” and “Rossi”. I found this blog posting. Here’s what he has to say:

There have been many attempts to twist the release of this video into NASA’s support for LENR or as proof that Rossi’s e-cat really works. Many extraordinary claims have been made in 2010. In my scientific opinion, extraordinary claims require extraordinary evidence. I find a distinct absence of the latter. So let me be very clear here. While I personally find sufficient demonstration that LENR effects warrant further investigation, I remain skeptical. Furthermore, I am unaware of any clear and convincing demonstrations of any viable commercial device producing useful amounts of net energy. [9]

So attempts to tie Rossi’s work to Zawodny are tenuous, at best. Zawodny, as an expert in the field, is skeptical of Rossi’s claims or even the possibility of his own research leading towards any near-term applications.

Next, I looked at the arXiv article, itself. My first impression was shock at how amateurish the pre-print looked. In terms of formatting and clarity of plots, it did not look anywhere near publishable. Not a good sign, but not a substantive problem either. Then I began to read the paper. Even the content seemed a little sketchy. A big red flag for me is that the output of the reaction was primarily measured using an infrared (IR) camera. It seems an odd way to measure the heat output. And, I was not convinced by the text that even the researchers had a solid handle on their technique. The biggest problem for me is the massively obvious thing they didn’t do: They didn’t put up any radiation sensors, as far as I can tell. This is a huge no-brainer to me, far more important than the heat output. If one is going to claim a nuclear reaction then one would expect to see radiation But, I’m cheating and relying on my physics experience to evaluate this paper. What can a person do if they don’t have such knowledge?

Even if one does not have the scientific background to smell something fishy from a direct read of the pre-print, there is still much one can do by searching around. One can certainly try to find science blogs that deal with the topic. For example, I found an excellent technical discussion of the arXiv article on the blog Starts With A Bang [10]. As with everything, one should be skeptical of even science blogs. However, posts like these can at least help one to get a sense of some of the scientific objections.

This article in The New Energy Times even calls into question the “independent” nature of the of the study on arXiv [11]. It seems that the researchers were invited in at Rossi’s request, the setup was built by Rossi, and the researchers were not allowed much access to the setup. They certainly could not see inside. If these claims are true, the whole matter seems suspect. More importantly, it weakens the very point of all of the buzz: that Rossi’s device had been independently tested! Personally, I feel this needs further follow up, maybe even by other researchers. I am fine with Rossi keeping aspects secret, but the researchers need more complete access to the device. Check out the letters-to-the-editor beneath the main article on New Energy Times. They are informative and even outline some of the ways that Rossi could have pulled some smoke-and-mirrors stunts. There are a lot of questions that must be addressed before I would consider this testing to be credible.

Step 5: Where matters are unresolved, wait and see…but don’t hold your breath

My experience and intuition makes me extremely skeptical of the claims in this article. I would be willing to wager a good amount of money that it is nonsense. That said, unfortunately, we do not have a “Scooby-do” ending, where fraud is completely and unambiguously unmasked.

One need not let this bother you. The purpose of skepticism is to filter out good knowledge from amid noise. Some people mistake open mindedness with a liberality in accepting new ideas. But, unfiltered acceptance of new claims just leads to clutter. True open mindedness, the skeptical approach is to entertain the possibility of new ideas while only accepting them if they are well supported. I would say that, though there is no reason to absolutely reject these claims, there is no reason to accept them either. If Rossi is sitting on the real deal and he’s just waiting to secure his patent, then the world will radically change in the next 10 years and we’ll know it. I, for one, would be glad to be wrong. Otherwise (and more likely) we’ll never hear of this again. If you are really interested, check back in a year and see if anything has changed. I bet it won’t have.

Step 6: So what’s the point?

This all may seem, at first glance, to be an exercise in futility. An article that sounds “too good to be true” often is too good to be true. Is it a waste of time to find this out? I offer an emphatic no! It isn’t a waste of time to debunk or take a claim “down a notch”. On the contrary, the process of skeptically investigating claims is a tremendously useful learning exercise. The more you do it, the better and the faster you will get at doing it. And, you will be surprised to discover that even your most trusted blogs and news outlets are not as reliable as you believed them to be.

More than just learning how to be skeptical (which is valuable in itself), the exercise is also a fun way of encountering new science. As I mentioned, every rumor has a grain of truth. This grain of truth can be interesting and educational, even if everything else in the rumor is wrong. In researching this article, I first wanted to dismiss the whole thing simply because I automatically associated “cold fusion” with quackery. I was intrigued to discover the work of scientists like Joseph Zowadny and theorists Widom and Larsen,  providing LENR with theoretical and experimental basis for renewed (albeit tentative) credibility in the scientific community. I’ve also learned about an interesting new news source: TheNew Energy Times, which I am impressed does an excellent job of promoting this new scientific direction without allowing itself to fall prey to charlatans.

Conclusion

To be fair to ExtremeTech: the article in question is somewhat cautious, and it ends with an acknowledgement that we’ll have to “say tuned”. My main nitpick is that I think they could have been more skeptical, and more cautious. In the end, there is much worse pseudo-scientific hearsay on the web. It abounds on Facebook. So be warned: Don’t take claims for granted. This is especially true if those claims reinforce what you want to hear. A repeated theme in this blog will be the urge for self-skepticism. The easiest bias that one can fall prey to is the “confirmation bias”: researching until you find something that affirms your belief and then just stopping. Real skepticism and real growth comes when one starts to fact check one’s own favorite sources [12].

For another fun discussion about fact-checking and chasing down rumors, I highly recommend this video by Peter Hadfield [13].

Bibliography

[1]  ExtremeTech- Cold fusion reactor independently verified, has 10,000 times the energy density of gas

[2]  Berkeley, Understanding Science- Cold Fusion: A Case Study for Scientific Behavior

[3]  Widom and Larsen-  Eur. Phy. J. C46, 107 (2006); arXiv: cond-mat/050502. 

[4]  Levi, Foschi, et al; arXiv preprint- Indication of anomalous heat energy production in a reactor device

[5]  Forbes- Finally, independent testing of Rossi’s Ecat cold fusion device

[6]  New Energy Times- Rossi’s profitable career in science

[7]  ExtremeTech- NASA’s cold fusion tech could put a reactor in every basement

[8]  NASA, Climate Change News- The nuclear reactor in your basement

[9]  Blog of NASA researcher, Joseph Zawodny

[10]  Wired- Cold fusion gets hot and aims for EU

[10] Starts With a Bang – The Ecat is back and people are still falling for it

[11] New Energy Times- Rossi Manipulates Academics to Create Illusion of Independent Test

[12] Saturday Morning Breakfast Cereal – All of your biases have at some point been confirmed by anecdote

[13] Why  the media screw up science, Part I- Peter Hadfield

Written by Matt Posted in fact checking