The Truth Wears Off

[jaydelott]

I would like to direct discussion to a recent article at the New Yorker which investigates the interface between scientific research, human perception, and peer review, and its implications for the validity of published scientific findings.

[MrAmsterdam]

Conclusion;

http://www.newyorker.com/reporting/2010 ... z1jYBQQxZ2
Such anomalies demonstrate the slipperiness of empiricism. Although many scientific ideas generate conflicting results and suffer from falling effect sizes, they continue to get cited in the textbooks and drive standard medical practice. Why? Because these ideas seem true. Because they make sense. Because we can’t bear to let them go. And this is why the decline effect is so troubling. Not because it reveals the human fallibility of science, in which data are tweaked and beliefs shape perceptions. (Such shortcomings aren’t surprising, at least for scientists.) And not because it reveals that many of our most exciting theories are fleeting fads and will soon be rejected. (That idea has been around since Thomas Kuhn.) The decline effect is troubling because it reminds us how difficult it is to prove anything. We like to pretend that our experiments define the truth for us. But that’s often not the case. Just because an idea is true doesn’t mean it can be proved. And just because an idea can be proved doesn’t mean it’s true. When the experiments are done, we still have to choose what to believe.

One of the researchers mentioned in the article published the following;

http://www.plosmedicine.org/article/inf ... ed.0020124

Why Most Published Research Findings Are False

John Ioannidis, an epidemiologist at Stanford University

Summary
There is increasing concern that most current published research findings are false. The probability that a research claim is true may depend on study power and bias, the number of other studies on the same question, and, importantly, the ratio of true to no relationships among the relationships probed in each scientific field. In this framework, a research finding is less likely to be true when the studies conducted in a field are smaller; when effect sizes are smaller; when there is a greater number and lesser preselection of tested relationships; where there is greater flexibility in designs, definitions, outcomes, and analytical modes; when there is greater financial and other interest and prejudice; and when more teams are involved in a scientific field in chase of statistical significance. Simulations show that for most study designs and settings, it is more likely for a research claim to be false than true. Moreover, for many current scientific fields, claimed research findings may often be simply accurate measures of the prevailing bias. In this essay, I discuss the implications of these problems for the conduct and interpretation of research.

[Sparky]

The decline effect is troubling because it reminds us how difficult it is to prove anything. We like to pretend that our experiments define the truth for us. But that’s often not the case. Just because an idea is true doesn’t mean it can be proved. And just because an idea can be proved doesn’t mean it’s true.When the experiments are done, we still have to choose what to believe.

For way too many people in this world, "proving something" is just a matter of having someone or some group agree with one's conclusions. The ideological extremists have come to the conclusion that a certain "source" is reliable, if not Omniscient, and dismiss any evidence contrary to what they have from their "source."

The world seems to have fractured into huge cults, driven by superstitions that serve certain emotional needs. They have no ability to see what nonsense they espouse, nor the obvious direct contradictory evidence. Even those with some knowledge of history ignore what history has shown to not work, while the most ignorant of history cherry pick a few examples that may or may not be true to support their ideology.

Proving something obvious to a religious or political ideologue is extremely difficult, if not impossible. They believe that they have chosen what to believe for very rational reasons. They are unaware that genetic predisposition and environmental prejudices have molded them into what they are, be it a socially acceptable or unacceptable monster.

[jetstove]

Fractal of Lies

Scientific dogma is the bane of us all. A student must run the gauntlet of the senior researchers in order to get to the top when they can finally apply for all that lovely research money. By that time it is too late, for the student has been fully and carefully indoctrinated into the world view of his mentors.

Often the search for truth in science pushes way past the boundaries of knowledge and into the whimsical and fantasy.
For instance, the big bang theory is, apparently, no longer a theory but a certitude in the general population and scientific community. They push past this assumed fact and add a theory to the existing theory and a theory onto that.

What do you get? A giant fractal of theories twisted and bent to conform to the initial supposition. This is not science! If the initial assumption is wrong then all the others are unsupported and come crashing down. The weight of failed theory is enormous. Whole careers have been made paving the pathway to the dead end street. Do you think they will just stop and say "My life and the lives of my colleagues have been wasted in searching for the wrong thing"? What about "We thought we knew what we were doing and had the mathematical models to prove it"?

These scenarios are never going to happen and we will have to rely on Planck's second law: "Science progresses one funeral at a time."

[phyllotaxis]

The decline effect is troubling because it reminds us how difficult it is to prove anything. We like to pretend that our experiments define the truth for us. But that’s often not the case. Just because an idea is true doesn’t mean it can be proved. And just because an idea can be proved doesn’t mean it’s true.When the experiments are done, we still have to choose what to believe.

For way too many people in this world, "proving something" is just a matter of having someone or some group agree with one's conclusions. The ideological extremists have come to the conclusion that a certain "source" is reliable, if not Omniscient, and dismiss any evidence contrary to what they have from their "source."

The world seems to have fractured into huge cults, driven by superstitions that serve certain emotional needs. They have no ability to see what nonsense they espouse, nor the obvious direct contradictory evidence. Even those with some knowledge of history ignore what history has shown to not work, while the most ignorant of history cherry pick a few examples that may or may not be true to support their ideology.

Proving something obvious to a religious or political ideologue is extremely difficult, if not impossible. They believe that they have chosen what to believe for very rational reasons. They are unaware that genetic predisposition and environmental prejudices have molded them into what they are, be it a socially acceptable or unacceptable monster.

I heartily agree with you Sparky...sad as it is that i must...