Confirmation Bias and Sampling Errors

In a recent discussion I had, I was arguing against the random drug testing of welfare recipients. I noted, by linking studies, that welfare recipients are no more likely than non-welfare recipients to use drugs. Someone disagreeing with me noted that he/she has worked in liquor stores and grocery stores and has seen many people who appear to be addicted to drugs and people talking about illegal substance use with Access cards in their wallets. Because of this, she reasons, welfare recipients are more likely to use drugs than non-welfare recipients.

This line of reasoning commits mainly two errors including confirmation bias and a sampling error. Confirmation bias is the tendency to notice and favor information that accords with their pre-established beliefs and ignore contrary evidence. Michael Shermer also calls this "noticing the 'hits' and missing the 'misses' when we remember certain information." The commenter who worked in grocery stores and liquor stores may have seen some welfare recipients who seem to be addicted to drugs, but she has certainly seen many people who were addicted to drugs who were not welfare recipients and many welfare recipients who were not addicted to drugs. The commenter, though, does not mention this and instead uses a very small sample to justify her conclusion.

Another error the commenter makes is applying a conclusion formed about a very small group of people to a larger whole. If it really is the case that welfare recipients are more likely to use drugs than those who are not welfare recipients in this area, this does not entail that this is the case for the entire population of welfare recipients.

It is very important to consider all data instead of a select group that you remember that accords with your pre-established belief. Confirmation bias also occurs with psychics. Through cold reading, the process of making general statements about a 'client' in which the person receives feedback to make more specific statements, people will make 'accurate statements' about the client and the clients remember these accurate statements while forgetting about the inaccurate statements. Psychics will say many statements and eventually something will stick, but clients will often selectively remember the sticking statements.

Instead of trying to look for information that supports our already held beliefs, we should draw conclusions from a larger group of data, consider opposing arguments, and not draw conclusions about an entire group of people based on a small set of data.

For more on drug testing for welfare recipients, visit the following two links...but this discussion is beyond the scope of this post:

Rationally Speaking Blog

ACLU