NPR’s Shankar Vedantam and his producers explored why findings in scientific studies may fail to hold up when other researchers try to reproduce them. The issue was spotlighted in 2015 when University of Virginia psychologist Brian Nosek and a consortium of colleagues reported they had been able to reproduce the original results in fewer than half of 100 published psychology studies. Vedantam looked more closely at efforts to replicate one study on the effects of gender and ethnic stereotyping on performance by Asian women in math tests. In the study, volunteers who were reminded they were women did worse on the tests. Others, selected at random, who were reminded instead about their Asian heritage, did better. There were two efforts to replicate the study, which was conducted in the Boston area by a female researcher. One follow-up in Georgia ─ also conducted by a woman ─ was successful. Another in California, conducted by both male and female facilitators, was not. Variables such as the geographic location of the studies, the sex of the experimenters, the strength of the stereotypes in the populations under study, can affect the outcome of such studies. “Any individual study is just that, an individual study,” Vedantam told his listeners. “It isn’t the truth.” While there certainly are studies that are poorly designed, shoddily carried out and even occasionally fraudulent, Vedantam said most are well-considered efforts by researchers to slowly accumulate evidence about the workings of the world and the people in it. Scientists and journal editors are looking for ways to publish more reproductions of earlier work, including results that are mixed or confusing. The results may be more nuanced than popular notions of how science proceeds, says Vedantam, who cautioned his listeners: “If you want answers that never change, definitive conclusions and final truths, odds are you don’t want to ask a scientist.” Naomi Starobin, a veteran radio producer, said Vedantam “brings sophistication to the coverage and yet keeps things at an approachable level. His writing and voicing displays his enthusiasm and confidence about his subject matter.” In a statement, Vedantam and his colleagues said: “Many people turn to science for answers. But as our podcast says, science is more in the question business than the answer business.”
