05 December 2007

The sad tale of Jacques Benveniste

The following was a comment that was intended to go on this post at Marchway Memoirs. It got a bit big, so I posted it here instead.

Oh yeah, Jacques Benveniste. Here we go. Benveniste was a kook bloke who was a scientist. And, of course, like most scientists, he had his human side...

I have the impression you think scientists are derived from some Vulcan-like race of purely logical beings. We’re human, with human faults.

...and his scientist side. And let’s face it; because scientists are human, they are prone to error. And emotion, unlike Vulcans.

On the whole, scientists are disciplined enough at work to let the cold, hard facts get in the way of petty human emotion. But in rare instances, they do it the other way round,..

Scientists are human. I’ve never met [James] Randi, and have no wish to, because I'm human and one of the things humans do is take a dislike to people.

...and when they do, it can really blind the practitioner to the facts. Particularly if the practitioner is displaying very strong emotions such as love, hunger and greed.

Or dislike.

James Randi is one who certainly inspires very strong emotions from both scientist and non-scientist alike. He's not a scientist. But his knowledge of the scientific method appears to be beyond question.

Until now.

And here's the question: Is it possible that dislike of the man could cause one to temporarily take leave of their senses outright and completely miss the point of a particular story?

This blogger thinks so. But let's be honest here - some of the criticism of Randi makes that very allegation against the man himself...

It's not always purely on logical grounds, although he has done some absolutely disgraceful things in my opinion. One example – do you remember the ‘Memory of Water’ hullabaloo? In the interests of fair play, here’s a page against and a page for the idea.

...and I suspect that this is going to be one of those times.

So it can become a bit of a pissing match, where someone stands accused of losing sight of the facts in their fury over the form of the tactics being employed. On the other hand, we have a contention, possibly with merit, possibly not, that those very tactics are being used in a mean-spirited fashion, and their application can skew the very data where they’re being used.

So what can we do when faced with something like this? I would firstly suggest looking at the claim itself

Jacques Benveniste, a talented scientist, reported in Nature that water was capable of retaining a ‘memory’ of something that had been dissolved in it, even after dilution to the point where none of the compound could be left.

…to see if there’s anything there. Jacques Benveniste was a chemist who thought that he was on to something when he made the claim that water’s chemical structure somehow altered, allowing it to ‘remember’ substances that had been dissolved in it, even if the substance had been removed through successive dilutions. And this, of course, was why homeopathic remedies worked.

The problem with Benveniste’s claim, was that by explaining how the ‘memory’ of water was supposed to work, he had completely bypassed the bit that hadn’t been established about whether homeopathy actually even worked in the first place.

And if you think that’s bad, Nature published it!

Now to be fair, Nature is one of the world’s most read science journals, and gets tonnes of submissions each year. And as we pointed out before, scientists are human, and prone to error.

Nature is peer-reviewed, but because human error occurs, stuff does slip through. Some celebrated slip-ups include the following:


Benveniste’s investigation was underpinned by an assumption that homeopathy worked – which, when you think about it, should have set off alarm bells from here to Timbuktu.

Nature’s alarm bells didn’t go off, unfortunately, until the complaints started rolling in, and they ended up having to send a bunch of investigators round to Benveniste’s lab to investigate.

“And why not?”…

Well, Nature arranged to send a delegation to the good doctor’s laboratory. He agreed to this test because he expected his method to work. It had worked many times before. The idea was simple: they prepared three flasks of water, one of which had been treated as described in the method, the other two were just ordinary water. That’s a ‘blind’ experiment – Benveniste, and indeed most of the 'testers', had no idea which flask contained which sample – and it’s the ideal way to carry out any experiment, wherever possible. If Benveniste’s experiment was right, if he had made no mistakes, if there had been no contamination of his samples, then he should be able to find the treated one.

…said Benveniste.

Benveniste and his team were found to have made some errors that were staggering in their ineptitude. These included the following:

  1. Benveniste’s experiments were "statistically ill-controlled", and the lab displayed unfamiliarity with the concept of sampling error. The method of taking control values was not reliable, and "no substantial effort has been made to exclude systematic error, including observer bias"
  2. "interpretation has been clouded by the exclusion of measurements in conflict with the claim". In particular, blood that failed to degranulate was "recorded but not included in analyses prepared for publication". In addition, the experiment sometimes completely failed to work for "periods of several months".
  3. There was insufficient "avoidance of contamination", and, to a large extent, "the source of blood for the experiments is not controlled".
  4. The study had not disclosed that "the salaries of two of Dr Benveniste's coauthors of the published article are paid for under a contract between INSERM 200 and the French company Boiron et Cie."
  5. "The phenomenon described is not reproducible". "We believe that experimental data have been uncritically assessed and their imperfections inadequately reported."

(Source: Wikipedia)

It's relevant at this point to discuss the reviewers engaged by Nature to investigate this and report back:


Of these, Randi appears to suffer the most vitriole from Benveniste supporters. Randi is not a professional scientist...

You’d think that a group of scientists and a representative from Nature would be enough to deal with this, but they took Randi along.

The information on which flask contained which sample was in a sealed envelope. Sealed before Benveniste could have seen it, so he couldn’t possibly cheat. Enough?

...yet he was in a position almost straight away, along with the other investigators, to see that the process was flawed. Although Benveniste was not in a position to see the contents of the sealed envelope containing the details of which flask contained which sample, other people in the lab whom the tester [Benveniste] and the test apparatus came into contact with were.

So they double-blinded the test. Randi is an ex-stage magician. In a combination of flashy showmanship...

No, Randi insisted on taping the envelope to the ceiling. A serious study turned into a circus. Surely it would have been enough to keep the envelope in someone’s pocket? Or did he think scientists are capable of picking his pocket, steaming open the envelope, then resealing it without leaving a trace and putting it back?

...and a wish to ensure that the lab stayed in good spirits, Randi taped the envelope to the ceiling. This also had the effect of ensuring that any attempts to sabotage the study would also be minimised - it is unlikely that Randi would have assumed that he would be the only person in the lab with a practised sleight of hand.

Supporters of Benveniste like to seize upon this stunt of Randi's...

Randi had gone in there not with the premise that Benveniste might have made a mistake, but that he was a deliberate fraud and was likely to tamper with that envelope, given half a chance. Further, he didn’t trust the other scientists, or even the Nature guy, with that envelope either. It was an unnecessary, childish, and grossly insulting act.

...as evidence that Randi had some kind of axe to grind and that he was out to deliberately discredit Benveniste and his team.

Or even that Randi did this without the support of Maddox and Stewart. This is a questionable claim that isn't supported by the account that they subsequently published in Nature.

None of this is relevant.

At the end of the day, all that matters is that Benveniste was brought undone by sheer incompetence and some extraordinarily bad record-keeping.

Benveniste's reputation was ruined by this. But he never gave up on his idea.

Despite being the junior party in the review panel, most supporters of Benveniste see Randi as being the ringleader in Benveniste's "debunking"...

In the event, Benveniste’s experiment failed. So Randi chalked up another ‘debunking’. Benveniste’s career and reputation took a beating (to be fair, he really should have had someone else look at that data before sending in the first paper, but that’s hindsight for you).

...

I don’t like the way he works. It’s more in tune with the stage than the laboratory, and that’s something I really want to stay clear of.

...despite the fact that at the end of the day he was able to contribute towards an important step towards ensuring that yet another erroneous piece didn't contaminate the human knowledge bank.

Benveniste never forgave Randi. Neither did a whole bunch of Randi critics who saw this as yet another excuse to hate the guy.

In his later years, Benveniste lost the plot entirely and was convinced that the "memory" that he was so convinced that water had, could be transmitted over the internet.

He was awarded an Ig Nobel Prize in 1998 for that.

11 comments:

Romulus Crowe said...

Hi Dikkii

He didn't really double-blind it by that act. If nobody knew what was in the envelope, it was already double-blind. Benveniste would have failed anyway.

Benveniste messed up. Possibly under pressure from a vested interest, but I don't think he intended fraud. He thought he was right. He wasn't.

There have been worse. Real, absolute frauds have made it to the top of their fields before being caught. I'm thinking about a recent one involving stem cells...

I bet there are some as yet uncaught. Personally, I think they should be shot. If anyone wants to take a pop at science, these guys are handing out the ammunition.

I'll look up the poltergeist investigation where Mr. R. turned up uninvited. That one dents his scientific-method claim. It won't be quick, the book pile has grown to a considerable size over the years, but I'll find it.

Dikkii said...

G'day Romulus,

I wasn't intending to convey that particular impression. The "taping to the ceiling" incident was merely the icing on the cake - Randi, Maddox and Stewart had already taken steps to ensure that the test was double-blinded.

Benveniste messed up. Possibly under pressure from a vested interest, but I don't think he intended fraud. He thought he was right. He wasn't.

There have been worse. Real, absolute frauds have made it to the top of their fields before being caught. I'm thinking about a recent one involving stem cells...


I don't think that there was any fraud involved in Benveniste's case, but I do believe that he had already settled on his conclusion and was unwittingly "massaging" the data in that direction.

Plonka said...

I don't think he was intentionally fraudulent either. Just a nutter with a degree...:)

Dikkii said...

Right. That's the third comment in three days that Blogger hasn't told me about.

Benveniste was nuts, certainly. But this is the bit that worries me - he wanted to believe his conclusion, and was looking for facts that supported it.

This is a recipe for disaster - good science has facts first, hypothesis afterwards. And this is why skeptics get so mad when people like Benveniste do stuff the other way round.

Plonka said...

That's the third comment in three days that Blogger hasn't told me about.

You too eh?

This is a recipe for disaster

Sounds more like a recipe for religion to me...;)

Dikkii said...

Sounds more like a recipe for religion to me...;)

Almost exactly.

Anonymous said...

To dicky,
Are u stupid, jealous or both that you can only work out how to open a can of beans when others are struggling to contribute to the body of science and well being of people. One thing is for sure, Benveniste's work is carried on today and is being found to be correct to a certain degree.
What have you found aside an increasing awareness that you havent found anything?

Anonymous said...

Another point is that you clearly have the wrong interests in your life as no decent feedback pulls facts from the wikipedia... a source which in which i can change the information myself...check your source first idiot!

Dikkii said...

Are u [sic] stupid, jealous or both that [sic] you can only work out how to open a can of beans when others are struggling to contribute to the body of science and well being [sic] of people.

Let's look at these one by one:

1. Stupid. I'll come to this a little later on.

2. Jealous. Jealous of a bloke who was exposed as a phony in embarrassing circumstances? Mate, if you believe this, then go down to where I discuss "stupid" shortly.

3. Both.

I tell you what, Anonymous. (If that is your real name) How about you have a guess at this? Then, I'll tell you how you scored.

One thing is for sure, [sic] Benveniste's work is [sic] carried on today and is being found to be correct to a [sic] certain degree.

Really? Show me a reference where this is the case. We are talking about his "memory of water" work aren't we?

What have you found aside [sic] an increasing awareness that you havent found anything?

I have no idea what you mean by this question, so I'll just call you a goose. It adds nothing to the debate, is completely irrelevant and is somewhat childish and immature to boot.

But it feels so damned good, and if you're reading this, you'll be getting so mad just reading this.

Another point is that you clearly have the wrong interests [sic] in your life as no decent feedback pulls facts from the wikipedia [sic]... a source which in which [sic] i [sic] can change the information myself...check your source first [sic] idiot!.

Careful with those dots, Anon. Any more and you'll need surgery. It's a good thing that Wikipedia makes its editors cite their sources - it's one thing that others who make claims without providing references could learn from. How is that one showing Benveniste's work "being found to be correct to a certain degree" coming along?

Goose.

Anonymous said...

Yes im so mad!
You took the time to write this page, why wud you if you were not jealous ey? Why show interest if hes just a phoney ey?

anyway, you were jealous of his status before this, the man was asked by the president for minister of health, now then ask yourself how is it a man working for people's health with such a reputation be challenged by someone like you? Dikkie? who's that aside someone bouncing off his work to get attention? otherwise why are you even here?

Ok heres a reference of some scientific work to argue your quest to be destructive:
Rey, L. (2003), Thermo-luminescence of Ultra-high Dilutions of Lithium Chloride and Sodium Chloride. Physica A, Vol. 323:67-74

Or if you use internet as a pose to books with your silly wikipedia, then heres a link to a review of it:
http://www.newscientist.com/article/dn3817-icy-claim-that-water-has-memory.html

If you understand this then you might be enlightnened, if not then keep peddling your bike. Ill send you some more info when i think you can understand it's meaning

Dikkii said...

Anon, I'm pleasantly surprised that you came back to check for a response to your comment, and I thank you for dropping by again.

I don't thank you for the nightmarish mix of slang and typos. It makes your text very hard to decypher.

So you're punting for "jealous", eh? I'm going to score you as zero for this. Jealous I ain't.

Nor would it be relevant if I was: click here to find out why.

Why show interest if hes just a phoney ey?

Maybe because the whole episode was, wait for it, funny?

Maybe because certain credulous boys and girls still hopelessly believe in this cock and bull concept blindly without even looking at the evidence, of which there is none?

Maybe it's my way of trying to prevent honest folk flushing their hard-earned down the shitter and possibly dying because some idiot told them that homeopathy would help whatever condition they were suffering from. Did you ever think of that?

Incidentally, your second paragraph was next to incomprehensible. I don't normally respond to someone who chooses not to spell or punctuate at an acceptable level. Consider yourself warned.

I loved your Louis Rey reference, by the way. Amongst other things, Benveniste himself criticised Rey's work for not even being blinded (let alone double blinded).

Did you even read Rey's study? A copy of it is here, for your benefit. In it, firstly, there is no mention from the author about "memory of water" or any other such related homeopathic concept. Secondly, Rey didn't compare his "energized" samples of H2O or D2O with non-energized control amounts of H2O or D2O. A bit of an omission, don't you think?

Lastly, any comments relating to connections with Benveniste's work appear to have been made by Rey and associates of Rey after the work was slated for publication. This appears to contain the following presuppositions:

1. That there is evidence to suggest that highly dilute solutions of anything actually have a measurable effect on anything;
2. That the term "memory of water" even has an accepted definition; and
3. That the sole means of achieving a "memory" of anything in water is through manipulation of hydrogen bonding.

Given that the first two are certainly not established by a long , long shot, and that the third is merely ad-hoc hypothesising, this study proves nothing.

Actually, to put this in perspective, the third point is such a stab in the dark, I'll wager that it is equivalent to dropping a ball bearing from a 747 flying at 40,000 ft over a sparsely populated part of the Sahara desert and expecting to hit an egg and lettuce sandwich which might not even be there.

Would you take me up on such a wager?