Tuesday, November 18, 2008

Thus Saith Science

The New York Times has an interesting piece today about controversy surrounding research that allegedly accurately measures racial bias. Check it out:

"Last year, a team of researchers at Harvard made headlines with an experiment testing unconscious bias at hospitals. Doctors were shown the picture of a 50-year-old man — sometimes black, sometimes white — and asked how they would treat him if he arrived at the emergency room with chest pains indicating a possible heart attack. Then the doctors took a computer test intended to reveal unconscious racial bias.

The doctors who scored higher on the bias test were less likely than the other doctors to give clot-busting drugs to the black patients, according to the researchers, who suggested addressing the problem by encouraging doctors to test themselves for unconscious bias. The results were hailed by other psychologists as some of the strongest evidence that unconscious bias leads to harmful discrimination.

But then two other researchers, Neal Dawson and Hal Arkes, pointed out a curious pattern in the data. Even though most of the doctors registered some antiblack bias, as defined by the researchers, on the whole doctors ended up prescribing the clot-busting drugs to blacks just as often as to whites. The doctors scoring low on bias had a pronounced preference for giving the drugs to blacks, while high-scoring doctors had a relatively small preference for giving the drugs to whites — meaning that the more “biased” doctors actually treated blacks and whites more equally.

Does this result really prove dangerous bias in the emergency room? Or, as critics suggest, does it illustrate problems with the way researchers have been using split-second reactions on a computer test to diagnose an epidemic of racial bias?

In a series of scathing critiques, some psychologists have argued that this computerized tool, the Implicit Association Test, or I.A.T., has methodological problems and uses arbitrary classifications of bias. If Barack Obama’s victory seemed surprising, these critics say, it’s partly because social scientists helped create the false impression that three-quarters of whites are unconsciously biased against blacks." (Read the whole thing here)

One of the things that I've been thinking about as a social scientist in training is the way that social science seems to be viewed in the popular imagination. A researcher or researchers investigate a possible relationship between variables through various measures and statistical analysis which determine the percentage of confidence one can have that an apparent relationship is not simply due to chance. At it best, this is essentially all that research (quantitative research anyway) can tell us about phenomena. Researchers and those of us who are consumers of research must then make sense of what the findings mean, which is essentially an interpretive exercise. From beginning to end this process is a human process which means it is imperfect. However, my sense is that social science research findings are treated in the popular imagination as if they were inscribed on stone tables and delivered from a mountaintop. There is a "faith" if you will in the "truth" of research findings from studies such as the one described in the Times article that borders on the kind of uncritical acceptance commonly associated with religiosity. What is true in matters of the spirit is just as true regarding social science:

"God has given man the eye of investigation by which he may see and recognize truth. He has endowed man with ears that he may hear the message of reality and conferred upon him the gift of reason by which he may discover things for himself. This is his endowment and equipment for the investigation of reality. Man is not intended to see through the eyes of another, hear through another's ears nor comprehend with another's brain. Each human creature has individual endowment, power and responsibility in the creative plan of God. Therefore, depend upon your own reason and judgment and adhere to the outcome of your own investigation; otherwise, you will be utterly submerged in the sea of ignorance..."
(Abdu'l-Baha, The Promulgation of Universal Peace, p. 293)