GIGO (Garbage In Gospel Out)

Other terms and/or related concepts

False premise; hidden premise; circular reasoning; begging the question; simple-minded certitude

Description

The advocate is certain their belief is true, even though the data, evidence or underlying theory or assumption on which the belief is based is demonstrably flawed or unsubstantiated. Or rather, the advocate treats conclusions leading from some flawed data, unsubstantiated evidence, unfounded assumption or baseless theory, as gospel.

Example

Roz Well is having an online discussion at www.locoinel_coco.com, explaining to her friend Joan Mack why she knows there is “life out there”.

She says:

The Drake equation proves it. It states that: N = R* fp ne fl fi fc L, where N is the number of civilizations in our galaxy with which we might hope to be able to communicate; and R* is the average rate of star formation in our galaxy, fp is the fraction of those stars that have planets, ne is the average number of planets that can potentially support life per star that has planets, fl is the fraction of the above that actually go on to develop life at some point, fi is the fraction of the above that actually go on to develop intelligent life, fc is the fraction of civilizations that develop a technology that releases detectable signs of their existence into space, L is the length of time such civilizations release detectable signals into space.

When you substitute all the right values into it, you get an answer of 5000! That means there are at least 5000 civilisations in our galaxy that we can communicate with. Imagine how many more there are in the entire universe!

Joan Mack is a well respected psychologist and as such has an understanding of fallacies. She types back:

Look, you can get whatever answer you want with the Drake equation – so long as you pick the input values that give you the answer you want. GIGO! I am happy to accept the equation itself as valid. But this doesn't mean we can get any useful information from it.

Roz replies: "So, are you saying you don’t believe?"

“Of course not", replies Joan. "Just that the Drake equation is not a good argument." She goes on to explain:

In my practice as a psychologist I have specialised in treating patients who have been abducted and fiddled with by aliens. Now that's conclusive proof. Unfortunately the rules of doctor-patient confidentially prohibit me from backing up this claim with any specific evidence – other than the cash-money I'll no doubt make when my book comes out. I’ve also been very successful in treating my patients with my anti-alien mind control helmet. It’s made from til foil!

"Cool.” Types Roz. “Where can I order mine?"

Comment

GIGO is a reasonably well known principle: Garbage In, Garbage Out. It was originally coined in a computer science environment to point out that an algorithm will process all data, no matter the quality, and give an answer. Thus, if junk data is fed into an algorithm, a junk answer will be the result. (The quality of one’s evidence is only as good as the weakest bit).

From this principle we can derive the “GIGO fallacy”. In this case GIGO stands for: Garbage In = Gospel Out. Again, in the computer science context, it refers to the blind belief in the answer obtained from computers. The use here is intended to be broader, and refers to the blind acceptance of a result of a process being a fact because of a simple-minded belief in the process. This can be for obvious examples such as the Drake equation above (which should be viewed as a fun thought experiment) to more important processes.

In the example above, Joan is correct in her overall dismissal of any value that is determined by the Drake equation. The logic of the equation is sound enough. But if the input values are essentially guesses, the multiplication of uncertainties becomes so large that it makes any calculation worthless.

Let's consider another example. We can calculate the strength of the Earth’s gravitational field (g) at the surface, using the equation: g = GM/r2. Where G = the universal gravitational constant, M = the mass of the Earth, and r = the radius of the Earth.

If we didn’t know the actual values for G, M and r, we could just choose numbers that "feel" right (i.e., based on our best guess). If enough people have enough guesses, we might even chance upon the right answer (9.8 N/kg), because the equation works (though this would be extremely unlikely in itself). However, we’d never know, because we were just guessing at the values we assigned to the input variables. Even if the equation is right, it is useless without the correct input data.

The problem with the Drake equation, why it falls into the GIGO category, lies with the parameters. There is no way to tell if the values we assign to the input parameters are garbage or not. Given the impossibility of assigning justifiable values to them, we can treat them all as garbage (though we can argue about which values stink more). All probabilities found using the Drake equation are therefore, to some extent, invalid (even if you accidentally guessed the right values). Some are more reasonable than others, but all the answers the Drake equation spits out still suffer from GIGO.

More generally, if the data, evidence or underlying theory used as the basis of a claim is flawed, then the claim and all conclusions based on it should be treated with great skepticism. (The claim and any conclusions may or may not be true; however there is simply no reasonable evidence either way.)

Statements such as "Tried and convicted by a jury of peers", or “It’s been peer reviewed”, could potentially be GIGO. If the legal process in a court case allows for flawed evidence or biased jury selection then we might not have confidence in the jury's verdict. If the peer review was by "peers" in a journal whose editorial standards poor, we might not have confidence in the conclusions of the papers in the journal (c.f. the Sokal affair and the journal Social Text).

Whilst we should be alert for GIGO, we need to also note the potential for a Red Flag Faux Pas. Using the examples in the preceding paragraph again, as a rule of thumb the criminal justice system and peer review can be trusted. Therefore if one wants to claim GIGO legitimately, one needs to provide evidence for it. Otherwise a claim of GIGO might simply be a "conversation ender".

University GIGO

An article in the Australian, Ratting on the Ratings, gives an excellent example of the GIGO Fallacy. David Woodhouse is the executive director of the Australian Universities Quality Agency, and he discusses the flaws of the University ranking system employed by Shanghai University:

THE central criticism of whole-of-institution rankings relates to the methodology that addresses quality in a superficial way but projects a complex image. Most rankings rely on two types of data: information from institutions that may not be validated and data obtained from opinion polls in the name of "expert opinion". With both components providing shaky foundations, the use of complex formulas with weights and indicators helps to project a pseudo-scientific image to outcomes that may be statistically irrelevant.

Put simply: GIGO. That is, even though a complex calculation is applied to the data to come up with a rank order of universities, given the input data may not be validated and is based on opinion; the output should be treated with skepticism. It does not matter how powerful the analytic tools being applied are, if the raw data being supplied is suspect, any conclusion based on this data is suspect too. Woodhouse goes on:

Rankings rely on quantification, indicators and weights, assigning weights to indicators and using the weighted scores to rank the institutions. This forces the multidimensional quality aspect into a linear scale. In this process, aspects of the institution that cannot be measured with weightings and numbers get distorted… Furthermore, they are sensitive to relatively small changes in the weightings used. Small changes in the weightings of indicators alter the results from year to year without any tangible change between institutions.

Not only is the input data suspect, the data is weighted and it only takes a minor change in the weighting to alter the results. This adds to the GIGO.

Now, this is not to say such a system in inherently useless. It gives us information which we can use to inform judgments about universities. However, it is to say one needs to be aware of the limitations of the system and that one should not take its results as “Gospel”.

The GIGO fallacy

Here, at humbugonline, from time to time, we find the need to either coin or promote a name for a new type of humbug. Not necessarily because we have made an original discovery of a hitherto unidentified error in reasoning; but because there seems to be a niche for a new term. I.e., people have previously recognised the flawed reasoning, but have not yet given the flaw a name or the name is in not yet in common use. This could happen because the “new” form of dodgy reasoning has only recently gained prevalence and has not been dealt with before, or simply because an old term doesn’t encapsulate the error as well (in our view) as a new term. For example, I think I am right in saying we can claim originality to LAME claim (Look At Me Everybody), WTF? Fallacy, Argument by Artifice, Burden of Solution, and False Attribution, and promoting Appeal to Celebrity. There is a varying level of merit with these new fallacies. Some are more for humour than critical analysis. However, they do seem to “work”.

With that in mind, I was recently reminded of a concept that, though not yet explicitly recognised as a fallacy, has been used as if it is a fallacy. Given a google search does not produce a substantial result for “GIGO fallacy” I think I can claim some “originality” (for whatever that’s worth when you simply re-contextualise someone else’s idea). The closest I've seen it being used explicitly as a fallacy is by Gary Curtis.

GIGO is actually a reasonably well known (if you mix in my circle anyway) principle: Garbage In, Garbage Out. See this Wikipedia page for the history of GIGO.

The way I like to put it is Garbage In = Garbage Out.

My working definition for GIGO is:

If the data, evidence or underlying theory used as the basis of a claim is flawed, then the claim and all conclusions based on it should be treated with great skepticism. (The claim and any conclusions may or may not be true; however there is simply no reasonable evidence either way.)

From this principle we can derive the "GIGO fallacy". In this case GIGO stands for:

Garbage In = Gospel Out.

(As with the original meaning of the second “G” in GIGO, I don’t claim originality to second “G” in this second version.)

The GIGO fallacy – Description

The advocate is certain his or her belief is true, even though the data, evidence or underlying theory used as the basis of the belief is demonstrably flawed or unsubstantiated. Another way of putting it is when the advocate treats conclusions leading from some poorly controlled or flawed data, unsubstantiated evidence or theory, as gospel.

Example

Rose Well is having an online discussion at AboveTopSecret.com, explaining to her friend Joan Mack why she believes there is “life out there”.

“The Drake equation proves it”, she says. “It states that: N = R* x Fp x ne x fℓ x fi x fc x L, where N is the number of civilizations in our galaxy with which we might hope to be able to communicate; and R* is the average rate of star formation in our galaxy, fp is the fraction of those stars that have planets, ne is the average number of planets that can potentially support life per star that has planets, fℓ is the fraction of the above that actually go on to develop life at some point, fi is the fraction of the above that actually go on to develop intelligent life, fc is the fraction of civilizations that develop a technology that releases detectable signs of their existence into space, L is the length of time such civilizations release detectable signals into space.

When you substitute all the right values into it, you get an answer of 5000! That means there are at least 5000 civilisations in our galaxy that we can communicate with. Imagine how many more there are in the entire universe!”

Joan Mack is a well respected psychologist and as such has an understanding of fallacies. She types back:

“Look, you can get whatever answer you want with the Drake equation – so long as you pick the input values that give you the answer you want. GIGO! I am happy to accept the equation itself as valid. But this doesn’t mean we can get any useful information from it. For example, we can calculate the strength of the Earth’s gravitational field (g) at the surface, using the equation: g = GM/r2. Where G = the universal gravitational constant, M = the mass of the Earth, and r = the radius of the Earth.

If we didn’t know the actual values for G, M and r, we could just choose numbers that “feel” right (i.e., based on our best guess). If enough people have enough guesses, we might even chance upon the right answer (9.8 N/kg), because the equation works (though this would be extremely unlikely in itself). However, we’d never know, because we were just guessing at the values we assigned to the input variables. Even if the equation is right, it is useless without the correct input data.

The problem with the Drake equation, why it falls into the GIGO category, lies with the parameters. There is no way to tell if the values we assign to the input parameters are garbage or not. Given the impossibility of assigning justifiable values to them, we can treat them all as garbage (though we can argue about which values stink more). All probabilities found using the Drake equation are therefore, to some extent, invalid (even if you accidentally guessed the right values). Some are more reasonable than others, but all the answers the Drake equations spits out still suffer from GIGO.”

Rose replies: “So, are you saying you don’t believe?”

“Of course not”, replies Joan. “Just that the Drake equation is not a good argument. However, in my practice as a psychologist I have specialised in treating patients who have been abducted and fiddled by aliens. Now that’s conclusive proof. Unfortunately the rules of doctor-patient confidentially prohibit me from backing up this claim with any specific evidence – other than the cash money I’ll no doubt make when my book comes out. I’ve also been very successful in treating my patients with my anti-alien mind control helmet.”

“Cool.” Types Rose. “Where can I order mine?”

More examples of GIGO:

• University GIGO
• WTF? Fallacy of the year

WTF? Fallacy of the year

If there's a stronger case than this for creating the WTF? Fallacy, then I'd like to hear of it. This has been getting much attention from bloggers of late. Supposedly this "experiment" demonstrates that a jet-fuelled fire couldn't have caused the world trade centre towers to collapse.

Spooked911, the poster, though an ingenious experimentalist, is not quite so proficient WRT the scientific report genre (though he claims he's a molecular biologist!). I'll help him out by using his text but following a format loosely based on what I get 12 year olds to do:

Aim

I set up the following experiment using steel rabbit fencing as the steel structure supporting a heavy cement block. If kerosene/jet fuel/hydrocarbon fires can indeed cause steel structures to collapse, it should be quite simple to show this in an experiment-- right?

Hypothesis

That a fairly flimsy steel structure will not distort and bend and collapse very easily from a simple hydrocarbon fire.

Method

1. Note, this fencing is easily bendable, has no significant rigidity, and was not reinforced in any way.
2. The fencing was bent into an outer square and an inner rectangle (the core)…
3. Then I damaged the "columns" by cutting them with wire cutters.
4. Just inside where the gash was made in the outer wall, I placed a cup of kerosene (jet fuel), and there was newspaper around the bottom on the structure.
5. Then I put a heavy cement block on top, weighing about 15 pounds.
6. I don't think the wire structure would hold more than three of these blocks, so the "safety factor" was not particularly high.
7. Then I tipped over the cup and lit the kerosene.

Results

Then fire burned for about twenty minutes, and toward the end, I put my foot on the structure to see if it would extra weight. It still did. The structure held up fine after the fire died…. After the fire was hot, the "columns" were not hot at all… [see picture - he can touch it].

Discussion and Conclusion

In a second experiment, I used the same wire fence and block set up, but increased the amount of "airplane damage", added in newspaper all around the inside of the structure, and soaked everything thoroughly with kerosene. In this expt, the fire was more intense and lasted significantly longer, but... the structure held up just fine.
What I conclude is that a fairly flimsy steel structure does not distort and bend and collapse very easily from a simple hydrocarbon fire. And thus, it is not clear why the much stronger steel columns in the WTC towers weakened so much from fires that the towers underwent global collapse.

Spooked911 - Response to criticism

It's a simple model to shed light on a complex system. Scientists do this sort of thing all the time. It's their bread and butter. I did not draw any elaborate conclusions. I did not say this proved anything. Moreover, I invited people to try their own experiment, and as far as I can tell, no one has. I think it's sad that this has become the subject of such ridicule. Moreover, note I said nothing political here, nothing about Bush. It's just an experiment that I thought was interesting and worth sharing. I am indeed a biomedical scientist-- a molecular biologist, not an engineer. But I have friend who is an engineer who thought what I did was cool.

You sure your "friend" isn't your mum (that's "mom" for you "seppos")? He goes on to defend himself against accusations that he is a "crankpot" [the spawn of Mr Crank and Miss Crackpot!]:

…what's funny to me is how they point to this experiment as primary evidence that I'm a kook or crankpot-- but not to posts where I say the 2nd hit videos were faked, no plane hit the WTC. Why does THIS post amuse them so? It is just because it is easy to look at the pictures as opposed to, say, reading and thinking critically?

I fell off my chair when I read that.

Overall Grade* = C-

Update - see how I "prove" the moon landings were faked using similar experimental techniques.

Update 2 - I didn't know it at the time (as I hadn't thought of it yet), but Spooked911 gives us an excellent example of the GIGO fallacy. He makes this conclusion based on his garbage experiment:

What I conclude is that a fairly flimsy steel structure does not distort and bend and collapse very easily from a simple hydrocarbon fire. And thus, it is not clear why the much stronger steel columns in the WTC towers weakened so much from fires that the towers underwent global collapse.
____________
*I'm hardly going to fail him am I! I'd like to avoid his irate and "eccentric" parents.

Tagged - Conspiracy Theory, 9-11, Fallacy, Skepticism.