Scientific Uncertainty

Science is not as certain as some of us may think: that is the message of an article in last week's New Yorker, by Jonah Lehrer: "The Truth Wears Off: Is there something wrong with the scientific method?" (sub. required).  This brings both Karl Popper and Zhunagzi to mind.

Popper is not nearly as skeptical as Zhuangzi, and Lehrer notes that Popper "... imagined falsification occurring with a single, definitive experiment...", which turns out to be a tad too simplistic.  But the Viennese thinker gave us the notion of science as conjecture, which can, and perhaps should, encourage a certain humility regarding the empirical standing of any statement. 

In any event, the thrust of Lehrer's article is that there is a tendency for the results of scientific research to weaken as particular studies are replicated, which he calls the "decline effect."  In other words, when what appear to be established "truths" are subject to repeated experimental scrutiny their evidentiary support erodes.  And so, he writes:

"But now all sorts of well-established, multiply confirmed findings have started to look increasingly uncertain.  It's as if our facts were losing their truth: claims that have been enshrined in textbooks are suddenly unprovable. This phenomenon doesn't yet have an official name, but it's occurring across a wide range of fields, from psychology to ecology..."

Our facts are losing their truth - enter Zhuangzi.  He might say our facts never had the truth we ascribed to them.  The distinctions we draw to create theories that invest facts with meaning are, to Zhuangzi, artificial devices that obscure as much, if not more, than they elucidate: "Those who divide things cannot see."  And the language we employ to describe and explain fails to capture the fullness and complexity of the realities that surround us:

The spoken isn't just bits of wind.  In the spoken, something is spoken.  But what it is never stays fixed and constant.  So is something spoken, or has nothing ever been spoken?  People think we're different from baby birds cheeping, but are we saying any more than they are?

Zhuangzi would never have accepted scientifically established theories to be fixed and constant, which is just about where Lehrer is coming down:

"Such anomalies demonstrate the slipperiness of empiricism.  Although many scientific ideas generate conflicting results and suffer from falling effect sizes, they continue to get cited in the textbooks and drive standard medical practice.  Why?  Because these ideas seem true.  Because they make sense.  Because we can't bear to let them go.  And this is why the decline effect is so troubling.  Not because it reveals the human fallibility of science, in which data are tweaked and beliefs shape our perceptions. (Such shortcomings aren't surprising, at least for scientists.)  And not because it reveals that many of our most exciting theories are fleeting fads and will soon be rejected. (That idea has been around since Thomas Kuhn)  The decline effect is troubling because it reminds us how difficult it is to prove anything.  We like to pretend that our experiments define truths for us.  But that's often not the case.  Just because an idea is true doesn't mean it can be proved.  And just because an idea can be proved doesn't mean it's true.  When the experiments are done, we still have to choose what to believe."

I can hear Zhuangzi now: "You mean you're only now realizing that there are no answers?" (105)

 

Comments

[Ahistoricality]

I'm sorry, but the "decline effect" in medical research - a notoriously complex field to begin with, compounded with immense financial pressures and egos - is not grounds for radical skepticism of the scientific method itself, nor a rejection of the idea that science studies something real about which true things can be said.

[Manyul Im]

Yeah, I was just reading this, Sam. I thought it started interesting then it seemed like the problems were less with method than with practice -- i.e. the practices of publishing and reporting findings. Is that really encompassed by "scientific method"? Maybe in a broad sense. This is like criticizing the method though by invoking the sociology of the scientific community, which might reveal problematic flows of information, but doesn't really seem to me to touch on the method.

(I actually thought the article was going to end up with some new "new problem of induction" from the scientists' side, not from the analytic philosophers'. Alas, not.)

[Marc]

Just a comment on Popper. Saying that he equated science with conjecture isn't exactly accurate, I think. He pointed out the limits of what is provable, which gave science a more solid footing, and did say that if we see a series of points on a graph that form a smooth curve, we assume that we didn't happen to miss all the wildly divergent points, but his conclusion wasn't that science was therefore conjecture. The lesson was: collect as much data as you can.

[Sam]

Thanks for the comments.
Manyul,
I, too, was expecting a stronger argument in the article, based on the title. But I think that the piece does point to a fairly significant methodological problem: the vagaries of replicability. I was most taken by the section that described an attempt to carry out the exact same experimental procedure in three different locations simultaneously. Even with rather strict scientific controls there were significant variations across locations, suggesting that any of a myriad of small chance events can disrupt the findings of any particular experiment. And without replicability, it is difficult to know what is random and what is systematic.
Ahistoricality,
I am not as radical as Zhuangzi himself. But I think skepticism is usually a good thing. The key point here, I believe, is not that it is impossible to say that true things cannot be said, but that it is rather hard to know when a true thing is said, and that is especially important to keep in mind when talking about medicine.
Marc,
You may be right, but I am remembering certain passages from The Logic of Scientific Discovery; like this:

"We can never know, of course, whether a supposed law is a genuine law or whether it only looks like a law but depends, in fact, upon certain special initial conditions prevailing in our region of the universe... We cannot, therefore ever find out of any given non-logical statement that it is in fact naturally necessary: the conjecture that it is remains a conjecture for ever (not merely because we cannot search our whole world in order to ensure that no counter instance exists, but for the even stronger reason that we cannot search all worlds that differ from ours with respect to initial conditions)...." (454)

So, perhaps it would be better to say that the product of science is conjecture.

[Ahistoricality]

No, the product of science is data and theories (sometimes in the form of equations) that both fit the data and predict future outcomes. "Theory" doesn't mean "conjecture" or "guess": it means a coherent, discrete explanation that fits the data and predicts future outcomes. For a much more detailed discussion of Lehrer's article, Orac's your man.

[Sam]

Thanks for the link. Good piece. But I think Orac would agree that "theory" is closer to "conjecture" than it is to "truth." He says:
"Indeed. I would argue that there is really no such thing as scientific "truth.""
I agree.

[TFF]

"Truth" is not attainable in this world, as we know it.

[Read this Sam Crane (Name: See)]

I can't email you because of Apple's email program.

I just thought you might want to check out Chinese Poetry by David Hinton, but more like an Anthology of Classical Chinese Poetry. Just thought you might be interested. Here is the Google site to check it out. Link:

http://www.google.com/#hl=en&expIds=17259,17291,24283,27615&sugexp=ldymls&xhr=t&q=david+hinton+chinese+poetry&cp=17&qe=ZGF2aWQgaGludG9uIGNoaW4&qesig=c3QTcodxEpuxFu6e3mPkDw&pkc=AFgZ2tl5Cw9DTc0J5fhLh760lmcw_ZUyWu3VTfnovafvwTge4jM4C0ZZO2y2TmNE6oivsx4fE1bKRbWW2vNYNHzr0DFP6QpoQw&pf=p&sclient=psy&aq=0&aqi=&aql=&oq=david+hinton+chin&gs_rfai=&pbx=1&fp=e64ee6e4e8056c32