Saturday, May 24, 2008

The GIGO fallacy

Here, at humbugonline, from time to time, we find the need to either coin or promote a name for a new type of humbug. Not necessarily because we have made an original discovery of a hitherto unidentified error in reasoning; but because there seems to be a niche for a new term. I.e., people have previously recognised the flawed reasoning, but have not yet given the flaw a name or the name is in not yet in common use. This could happen because the “new” form of dodgy reasoning has only recently gained prevalence and has not been dealt with before, or simply because an old term doesn’t encapsulate the error as well (in our view) as a new term. For example, I think I am right in saying we can claim originality to LAME claim (Look At Me Everybody), WTF? Fallacy, Argument by Artifice, Burden of Solution, and False Attribution, and promoting Appeal to Celebrity. There is a varying level of merit with these new fallacies. Some are more for humour than critical analysis. However, they do seem to “work”.

With that in mind, I was recently reminded of a concept that, though not yet explicitly recognised as a fallacy, has been used as if it is a fallacy. Given a google search does not produce a substantial result for “GIGO fallacy” I think I can claim some “originality” (for whatever that’s worth when you simply re-contextualise someone else’s idea). The closest I've seen it being used explicitly as a fallacy is by Gary Curtis.

GIGO is actually a reasonably well known (if you mix in my circle anyway) principle: Garbage In, Garbage Out. See this Wikipedia page for the history of GIGO.

The way I like to put it is Garbage In = Garbage Out.

My working definition for GIGO is:

If the data, evidence or underlying theory used as the basis of a claim is flawed, then the claim and all conclusions based on it should be treated with great skepticism. (The claim and any conclusions may or may not be true; however there is simply no reasonable evidence either way.)
From this principle we can derive the "GIGO fallacy". In this case GIGO stands for:

Garbage In = Gospel Out.

(As with the original meaning of the second “G” in GIGO, I don’t claim originality to second “G” in this second version.)

The GIGO fallacy – Description

The advocate is certain his or her belief is true, even though the data, evidence or underlying theory used as the basis of the belief is demonstrably flawed or unsubstantiated. Another way of putting it is when the advocate treats conclusions leading from some poorly controlled or flawed data, unsubstantiated evidence or theory, as gospel.

Example

Rose Well is having an online discussion at AboveTopSecret.com, explaining to her friend Joan Mack why she believes there is “life out there”.

“The Drake equation proves it”, she says. “It states that: N = R* x Fp x ne x f x fi x fc x L, where N is the number of civilizations in our galaxy with which we might hope to be able to communicate; and R* is the average rate of star formation in our galaxy, fp is the fraction of those stars that have planets, ne is the average number of planets that can potentially support life per star that has planets, f is the fraction of the above that actually go on to develop life at some point, fi is the fraction of the above that actually go on to develop intelligent life, fc is the fraction of civilizations that develop a technology that releases detectable signs of their existence into space, L is the length of time such civilizations release detectable signals into space.

When you substitute all the right values into it, you get an answer of 5000! That means there are at least 5000 civilisations in our galaxy that we can communicate with. Imagine how many more there are in the entire universe!”

Joan Mack is a well respected psychologist and as such has an understanding of fallacies. She types back:

“Look, you can get whatever answer you want with the Drake equation – so long as you pick the input values that give you the answer you want. GIGO! I am happy to accept the equation itself as valid. But this doesn’t mean we can get any useful information from it. For example, we can calculate the strength of the Earth’s gravitational field (g) at the surface, using the equation: g = GM/r2. Where G = the universal gravitational constant, M = the mass of the Earth, and r = the radius of the Earth.

If we didn’t know the actual values for G, M and r, we could just choose numbers that “feel” right (i.e., based on our best guess). If enough people have enough guesses, we might even chance upon the right answer (9.8 N/kg), because the equation works (though this would be extremely unlikely in itself). However, we’d never know, because we were just guessing at the values we assigned to the input variables. Even if the equation is right, it is useless without the correct input data.

The problem with the Drake equation, why it falls into the GIGO category, lies with the parameters. There is no way to tell if the values we assign to the input parameters are garbage or not. Given the impossibility of assigning justifiable values to them, we can treat them all as garbage (though we can argue about which values stink more). All probabilities found using the Drake equation are therefore, to some extent, invalid (even if you accidentally guessed the right values). Some are more reasonable than others, but all the answers the Drake equations spits out still suffer from GIGO.”

Rose replies: “So, are you saying you don’t believe?”

“Of course not”, replies Joan. “Just that the Drake equation is not a good argument. However, in my practice as a psychologist I have specialised in treating patients who have been abducted and fiddled by aliens. Now that’s conclusive proof. Unfortunately the rules of doctor-patient confidentially prohibit me from backing up this claim with any specific evidence – other than the cash money I’ll no doubt make when my book comes out. I’ve also been very successful in treating my patients with my anti-alien mind control helmet.

“Cool.” Types Rose. “Where can I order mine?”

More examples of GIGO: