bearing blog

Quotable Ace on scientific overconfidence.

Here's Ace of Spades writing about the latest study to show, in the face of the establishment, that low-carb dieters burn more calories:

When evidence and data are thin, Narrative and Theory rushes in to take their place. Nature abhors a vacuum, and so too does natural science. Science -- or, I should say, scientists -- human beings with human flaws which science isn't actually burdened with -- do not like saying "I don't know." Even when "I don't know" is actually the proper, most scientifically-valid answer.

"I know" is much more satisfying. The funny thing is that we only hear "the science is settled" in the precise cases where it's not settled. No one ever says "the science is settled" as to whether the earth revolves around the sun. It is settled, of course; so no one has to attempt the Appeal to Authority to establish it.

A growing consensus says they know, you know.

So, for 50 years now, the medical establishment and the government have been telling fat people to do the exact opposite thing they should be doing.

Ace is also drawing a comparison to the overconfidence of much of the man-made artificial global warming crowd; it doesn't much matter the specific policies, though, as his point stands whatever the appeal to authority. It is a common logical flaw (or a very shrewd tactic, as it works so well on the gullible who don't see their own weakness), and the point is well put.

Every once in a while, when I write against... oh, what shall I call it... "scientism," perhaps... that belief that scientists are people with special moral authority to set policy... I wonder if I look like I suffer from sour grapes.

I trained for, and used to aspire to working in, academic scientific research (well, engineering, but to most folks it looks pretty much the same). I don't anymore, mostly because of a series of choices our family made to secure our overall happiness. So you could say that membership in the Authoritative Class was within my grasp, and I let it slip away, and now I have sort of a grumpy habit of poking at Oz's curtain.

I like to think that the training I did have prepared me to recognize the argument from authority in many of its guises, but perhaps that thought is just the argument-from-authority in another form.

Maybe if I were a working scientist instead of a nonworking one, I would be just as vulnerable to belief in the superrational powers of my own authority (at least in my own field) as journalists often are to belief in the powers of people whose work they don't understand.

 

Scientists Say

Mark was leafing through last week's issue of Science at the breakfast table.  "This is a weird magazine."

I said, "It's really a journal, even though it looks like a trade mag.  It's fairly prestigious to be published in there." 

"Yeah, it looks like one in the back."  He pushed it across the table and pointed to the Letters page (link requires paid subscription).  "Is it prestigious to get your retraction published?"

The report "Defective transcription-couped repair of oxidative base damage in Cockayne syndrome patients from XP group G" (1) is retracted.  An ad hoc investigatory committee ... has found that the last author (S. A. L.) of the paper "fabricated and falsified research findings"... The first three authors of the paper were not cognizant of any irregularities and were not involved in any wrongdoing.  The fourth author (S. A. L.) declined to sign this retraction.

I don't know how often papers are formally retracted because fake data is uncovered.  Retraction because mistakes are discovered is a normal part of the process, or at least it should be, because mistakes happen all the time.

A year out of graduate school, I suspect fakery is very, very common, especially if you include the omission of data along with fabrication and falsification.  I suspect that it falls on a spectrum:  at one end, the deliberate creation of data that never existed, reports from experiments that were not done or that turned out quite differently; at the other end, decisions to omit certain data and report others that are rooted in unconscious bias.  Somewhere in there, too, obfuscation or exaggeration in the write-ups.

A lot of people are suspicious of university research that is funded by corporations with a stake in the findings.  Supposedly this creates an incentive for the investigator to skew the data in favor of the funding corporation.  Maybe that's true at the level of the primary investigator, i.e. the supervising professor. 

In my experience, it's not true for the graduate students who do most of the research, at least not in physical science and engineering.  What do they care who funds them as long as they get their stipend? 

No, with graduate students the incentive is all about time.  I've been here seven years and I don't have anything to show for it.  If this keeps up they'll kick me out.  As soon as I finish this up and get something---anything---that will look good to the thesis committee, I'll be making real money. 

Or maybe it goes like this:  Everyone else in my group finishes within five years.  I'm starting to look bad.  My advisor is pressuring me to finish.

My own thesis was theoretical; the "data" came from one computer program I wrote myself, and later from another computer program that a visiting professor, who is much better at programming than I am, wrote after I set up the equations.  (Yes, I properly attributed the work to the professor.)  Mine has nothing but my own reasoning to stand on.  That's one thing I like about theoretical investigation.

UPDATE:  Here's a link to a news article on the retraction.  Sounds like the alleged faker has issues that go beyond research.