Other Terms and/or Related Concepts
Selective observations; counting the hits and ignoring the misses; searching for confirming instances; observer bias; publication bias.
Description
This error occurs along a broad spectrum, from individuals forming their own views on a subject of very little importance, to research into sensitive and complex social issues, and even "hard sciences" such as physics. At either end of the spectrum, observations are made by people and people have unconscious (and sometimes conscious) biases. These biases influence the observations people make.When research is carried out by advocates of a particular viewpoint, the researcher has definite convictions about the importance of particular variables and for this reason, they may consciously or unconsciously tend to seek confirmation of their views in the data and ignore contradictory evidence. The advocate "observes", but only pays attention to information which seems to support their existing convictions.
Example
Jenny Frame is being interviewed on the national radio program Social Issues. The subject of the interview is her PhD research on "recovered memories of childhood sexual abuse". In the course of the interview, she states: "My research is designed to demonstrate that child sexual abuse is very widespread and that most victims repress their memories of abuse throughout their adult lives."Comment
Jenny's description of her research indicates that she is seeking to "prove" a hypothesis, rather than to test it. Her approach to research is clearly partisan and biased. Policy-makers, lawyers or practitioners in the field of child protection could not safely rely upon Jenny's conclusions. She would be far more credible if she described her research in the following terms: "I am seeking to establish the nature and extent of child sexual abuse, and whether and to what extent the victims repress their memories of abuse throughout their adult lives."This improved description of Jenny's research intentions is certainly more even-handed than the original description, but of course there is still no guarantee that her research will prove to be unbiased. Ultimately the credibility of her research can only be assessed by closely examining her methodology.
Observational selection can often be found in research programs. PhD candidates, for example, naturally frame a topic which accords with their current interests and convictions. They try to find a supervisor whose mind-set is compatible with their own. The thesis is also probably examined by academics who are sympathetic to the topic, methodology and findings. This is not to say that these lines of research are therefore flawed. Each would need to be treated on its own merits. (For a notorious case history of observational selection of this kind, the reader is encouraged to retrieve and read articles and/or books by Derek Freeman and others on Margaret Mead's anthropological fantasies in Samoa.)
Public enquiries about social issues conducted by panels of self-selected advocates are usually profoundly biased and hopelessly compromised. Campaigners for human rights are arguably the least suitable panellists for a disinterested, truth-seeking enquiry on human rights; judges are arguably the least suitable panellists for a truth-seeking enquiry on judicial powers; parliamentarians are arguably the least suitable panellists for a truth-seeking enquiry on parliamentary superannuation; developers are arguably the least suitable panellists for a truth-seeking enquiry on local government rezoning powers.
Observational Selection is not confined to flawed methodologies in formal research programs or public enquiries. Few individuals can even read a newspaper article without selectively attending to information which confirms their own biases. Casual readers of magazine horoscopes often see accurate predictions where none exist. Two bystanders present at a brawl between police and protesters will sincerely ascribe blame for the incident to different protagonists.
In any contentious situation, the seeker after truth will at least recognise the potential for biased observation, and will be wary about possible distortions of the truth, overstatement, exaggeration or outright fabrication by partisan advocates.
