Doctors and homeopathy...

Re: Doctors and homeopathy...

Mark Hedberg — Wed, 04 Sep 2013 14:04:22 GMT

[quote user="Arlo Guthrie"]

[quote user="Mark Hedberg"]I thought you meant page 34 was going to be an interview with some noted homeopath - lucky it was instead an interview with Arlo [/quote]

Well, I'm not sure about 'lucky'. You could well ask which would be worse: me blithering on about nothing, or a homeopath extolling the virtues of water.

Still, I was very surprised and extremely flattered to be asked to do the interview. Mind you, I've have had to explain to family that 'Companion' is not the monthly magazine of a singles club for middle-aged gentlemen (or that 'The Companion Interview', as it is styled, isn't the BSAVA equivalent of the Playmate of The Month).

[/quote]

Oh come now - it would have been hilarious to let them keep thinking that! :) Add in the detail that there are pictures of dogs in the magazine too, and let their imagination fly like the wind!

Re: Doctors and homeopathy...

Arlo Guthrie — Wed, 04 Sep 2013 13:46:38 GMT

[quote user="Mark Hedberg"]I thought you meant page 34 was going to be an interview with some noted homeopath - lucky it was instead an interview with Arlo [/quote]

Well, I'm not sure about 'lucky'. You could well ask which would be worse: me blithering on about nothing, or a homeopath extolling the virtues of water.

Re: Doctors and homeopathy...

Mark Hedberg — Tue, 03 Sep 2013 20:36:52 GMT

[quote user="Arlo Guthrie"]

Update: Big feature on homeopathy in September edition of Companion, the magazine for BSAVA members, which includes quite a bit about our survey.

But if you really want to find yourself reaching for the sick bag, turn to page 34

[/quote]

I thought you meant page 34 was going to be an interview with some noted homeopath - lucky it was instead an interview with Arlo

Re: Doctors and homeopathy...

John Flynn — Sat, 31 Aug 2013 10:49:49 GMT

"Moreover, for many current scientific fields, claimed research findings may often be simply accurate measures of the prevailing bias."

Couldn't have put it better

[quote user="Thomas Johnson"]But can you believe published research findings that say that most published research findings are false...[/quote]

...and all Irishmen always lie.

Re: Doctors and homeopathy...

Thomas Johnson — Sat, 31 Aug 2013 08:31:01 GMT

[quote user="Lyn Thomson"]

Looks like their is no hope for any of us .............an essay titled "Why Most Published Research Findings are False"

http://www.plosmedicine.org/article/info:doi/10.1371/journal.pmed.0020124

[/quote]

But can you believe published research findings that say that most published research findings are false...

Re: Doctors and homeopathy...

Lyn Thomson — Sat, 31 Aug 2013 05:40:14 GMT

Looks like their is no hope for any of us .............an essay titled "Why Most Published Research Findings are False"

http://www.plosmedicine.org/article/info:doi/10.1371/journal.pmed.0020124

Re: Doctors and homeopathy...

Francisco Gomez — Fri, 30 Aug 2013 11:43:05 GMT

Re: Doctors and homeopathy...

John Flynn — Fri, 30 Aug 2013 11:38:36 GMT

Fair enough.

I suspect we're actually in pretty good agreement that "failure to demonstrate a significant difference" is miles apart from "there is no significant difference" (or to avoid the double negative: "there is equivalence").

Re: Doctors and homeopathy...

Francisco Gomez — Fri, 30 Aug 2013 11:20:56 GMT

I agree this paper doesn't apply 'equivalence testing' very well. I.e. the groups are not sufficiently 'equivalent'.

The aim of the study (last paragraph of Introduction) says ''was to determine whether treatment regimen affected outcome in a retrospective analysis of cases diagnosed with IMHA at a veterinary referral hospital' ... and later on in the Discussion 'The apparent difference in mortality between the groups treated with prednisolone alone and prednisolone with cyclosporine, although not significant...' so to me that actually makes me think that the authors were open to to expect 'no significant difference' and that is what they got (even tho that wasn't necessarily what they were pursuing).

Anyway, no I don't remember any paper that complies with all your 4 bullet points. I'll give you that

Re: Doctors and homeopathy...

John Flynn — Fri, 30 Aug 2013 11:04:36 GMT

I'm not seeing any properly applied equivalence testing in support of a hypothesis of "no difference" in this paper.

I only skim-read the paper, but I missed the bit where the authors claimed that their results indicate that there is no difference between treatment methods - if they did then this would be an example of what I was talking about.

What I'm looking for is a vet paper published which claims there is no difference between e.g. 2 treatment methods and makes this claim based on a study designed to test that claim.

This would need:

1) A clear hypothesis of no difference (e.g. "the hypothesis is that there is no difference between treatment A and treatment B"; thus the null hypothesis is that there IS a difference between these treatments assuming frequentist statistics are about to be applied).

2) A sensibly decided on pre-hoc definition of the equivalence limits (i.e. what range of outcomes are to be accepted as equivalent for the purpose of the statistical testing that is to follow).

3) The application of appropriate statistical testing at least briefly outlined in the methods prior to looking at results.

4) A positive result to the study (i.e. conclusion of equivalence of the two treatments).

What I tend to see instead, is studies that fail to demonstrate any difference, often having failed to state a clear hypothesis in the first place, and then all too frequently conclude that their study has demonstrated there isn't a difference.

This is pretty basically wrong in my books when using classical stats. I'd rather have the results reported without a pile of meaningless p-values and invalid conclusions.

It is many years since I did stats/science as a school-boy, and I may either be unecessarily pedantic or have learned a grossly simplified state of affairs, but I can't get my head round what I'm missing then.

Re: Doctors and homeopathy...

Francisco Gomez — Fri, 30 Aug 2013 09:48:28 GMT

[quote user="Francisco Gomez"]

[quote user="John Flynn"]

When was the last time that you saw a veterinary paper properly applying equivalence testing in support of a hypothesis of "no difference" rather than failing to demonstrate a difference (p>0.05 - arrggghhh!) and then claiming that this is evidence that there is no difference?

[/quote] Last week?

[/quote]

[quote user="John Flynn"]

Roll on up and name that paper

[/quote]

Swann, J. W. & Skelly, B. J. (2011) 'Evaluation of immunosuppressive regimens for immune-mediated haemolytic anaemia: a retrospective study of 42 dogs'Journal of Small Animal Practice • Vol 52 • July 2011, 353–358

Now that you've been commenting on a IMHA case, this is a paper that supports that there is almost 'no difference' (well, at least, it claims that more studies would be necessary if you wanted to demonstrate a difference) in the treatment of IMHA.

Before you go ahead and criticise it, I'll save you the post: I agree that the paper is not that great, it compares animals receiving blood transfusions with others that didn't and then it tries to make the case whether prednisolone alone maybe similar to treatment with a combination of other drugs, but the animals receiving prenisolone alone where 'less sick' anyway..., it's not double blinded and not prospective either...

Anyway, I hope this shows my opinion about papers, and people 'half reading' our posts don't get confused and understand that this is NOT a support of homeopathy, which I can't endorse in anyway... In my view, mainly because there aren't studies that can properly support it, and to have studies supporting it, you would need (I agree on this) to have a properly applied scientific ground where those studies could be extracted from. (As you well said, if you put garbage in, you'll get garbage out )

Re: Doctors and homeopathy...

Arlo Guthrie — Thu, 29 Aug 2013 17:44:00 GMT

Update: Big feature on homeopathy in September edition of Companion, the magazine for BSAVA members, which includes quite a bit about our survey.

But if you really want to find yourself reaching for the sick bag, turn to page 34

Re: Doctors and homeopathy...

Arlo Guthrie — Fri, 09 Aug 2013 13:58:13 GMT

[quote user="alex gough"]

I saw that. As a PR minded person, Arlo, can you tell me what would make a publication take sides in such a way?

Alex[/quote]

I don't think this was deliberate. I think they have just not put enough thought into the headline. Simple as that. Also, they've just taken our press release and allowed the BAHVS to respond, seemingly without questioning the response (I mean, it's laughable to suggest that 2.4% is not representative of opinion in a profession). Nor did they investigate whether there was any truth to the BAVHS claim that the survey was biased, they just reported the claim.

Re: Doctors and homeopathy...

Mark Hedberg — Fri, 09 Aug 2013 13:22:03 GMT

[quote user="alex gough"]

I saw that. As a PR minded person, Arlo, can you tell me what would make a publication take sides in such a way?

Alex

[/quote]

Easy - people read controversial stuff. If it increases pageviews or clicks, they'll do it. Calm, level headed, even, unbiased reporting is not what Fox News is famous for, after all!

(This is not intended as a ding against Arlo or anything like that - just saying that's the way newspapers these days work.)

Re: Doctors and homeopathy...

Alex Gough — Fri, 09 Aug 2013 12:06:52 GMT

I saw that. As a PR minded person, Arlo, can you tell me what would make a publication take sides in such a way?

Alex

Re: Doctors and homeopathy...

Arlo Guthrie — Fri, 09 Aug 2013 11:40:42 GMT

[quote user="alex gough"]I was paraphrasing, but the gist was not vets condemn homeopathy, but homeopathis hit out at survey. Hence BAHVS being quoted first, and you just getting a line at the bottom. [/quote]

See what you mean.

Vet Times has taken a not dissimilar tack. Page two today.

But John Bonner has written a really interesting feature, coming up soon, which discusses the issues raised, rather than focusing on the BAVHS response.

Re: Doctors and homeopathy...

Alex Gough — Fri, 09 Aug 2013 10:51:47 GMT

I was paraphrasing, but the gist was not vets condemn homeopathy, but homeopathis hit out at survey. Hence BAHVS being quoted first, and you just getting a line at the bottom.

Whats in the pipeline?

Re: Doctors and homeopathy...

Arlo Guthrie — Fri, 09 Aug 2013 09:46:05 GMT

[quote user="alex gough"]

Pretty biassed reporting! Is it Dogs World or Dogs Today or both that have regular alt med columns? I thought it was interesting that the story was not "vets condemn homeopathy", but "homeopathic vets condemn surgery."

Alex[/quote]

Alex, I didn't see that headline. Are you sure you were looking at the same article. The headline on the one I linked to was; "Survey raises questions about homeopathy", which seemed fair enough.

[quote user="alex gough"]And, almost incredulously, "VetSurgeon.org editor Arlo Guthrie still questioned the validity of homoeopathic remedies." (my italics)[/quote]

Yes, that made me laugh. As if it should have read "Following the BAVHS statement, VetSurgeon.org editor Arlo Guthrie immediately realised the grave error of his ways and just before falling prostrate on his sword, sold all his assets and donated them to further homeopathic research."

[quote user="alex gough"]Im afraid your partner in crime still didn't get a mention, Arlo.[/quote]

Think it's my silly name. But there's something in the pipeline ....

Re: Doctors and homeopathy...

John Flynn — Fri, 09 Aug 2013 07:36:29 GMT

[quote user="Francisco Gomez"]I am genuinely scared of doing so... :-)[/quote]

It was the thought of an Irishman in a balaclava, wasn't it - no balaclava smilies on here I note

Re: Doctors and homeopathy...

John Flynn — Fri, 09 Aug 2013 07:34:44 GMT

perhaps best... I see we seem to have left the thread title as a distant memory

Re: Doctors and homeopathy...

Francisco Gomez — Fri, 09 Aug 2013 07:30:08 GMT

I am genuinely scared of doing so... :-)

Re: Doctors and homeopathy...

John Flynn — Fri, 09 Aug 2013 07:26:24 GMT

[quote user="Francisco Gomez"]

Last week?[/quote]

SOunds like you may be decidedly better read than me , but just to check we're not talking cross-purposes and separated by a sea of jargon:

[Sounding a little like a quiz show host] Roll on up and name that paper

Re: Doctors and homeopathy...

Francisco Gomez — Fri, 09 Aug 2013 07:00:07 GMT

[quote user="John Flynn"]

[quote user="Francisco Gomez"]At least one of the decks' makes it biased and therefore unacceptable. You are running a battery of single studies and choosing the one with your desired results. You'll find that most sudies nowadays avoid this gross handicap.[/quote]

Do they?

[/quote] Yes [quote]

[/quote] Last week?

Re: Doctors and homeopathy...

John Flynn — Thu, 08 Aug 2013 23:48:10 GMT

[quote user="Francisco Gomez"]John, not ignoring you...[/quote]

I frequently ignore myself, so feel free to - I strongly suspect (p<0.05) I am best ignored

[quote user="Francisco Gomez"] I bet it would be scary for you and I to turn up in a house in Mornington Crescent wearin balaclavas... I once went 'all in' with 7 2 post-flop being last three remaining in the game with a raised blind... Nobody called.. And won the night at the end...[/quote]

I have a safety catch to stop players raising my blinds now, but still wear my balaclava indoors just in case they do.

Re: Doctors and homeopathy...

John Flynn — Thu, 08 Aug 2013 23:43:24 GMT

Do they?

http://www.alltrials.net/blog/the-alltrials-campaign/

I might not even be a naughty pharmaceutical company, I might be a genuine convert to a form of medicine (say homoepathy) trying to prove a point. I might not be anything but a studious and honest investigator who rightly conducts a good study trying to disprove any beneficial effect from the medicine I have selected and only if I fail to do so after a rigorous study with a level of significance pre-set at p<0.05 accept my original hypothesis that the drug works.

The problem is that I might have 15 equally studious colleagues who already tried this drug in a trial and failed to demonstrate any benefit. Their results never made the light of day for a number of reasons. Most never got round to writing them up for publication (an onerous task at the best of times) as the results were not exactly earth-shattering or interesting. Some did bother to take the time, but failed to find a journal to publish a speculative study into a random drug that failed to show any difference and was probably not expected to anyway (their readers would cancel their subscription if they filled their pages with such "irrelevant drivel"). Some of course could have their results deliberately suppressed, a form of scientific fraud up there with inventing results.

So if Roger has access to 800, high-quality, peer-reviewed studies investigating the phenomenon of "homopathy" (I'm guessing he doesn't, but let's pretend), then to send you a single double-blinded placebo control study with a significance of p<0.05 is so easy, in the absence of "homopathy" actually doing anything, that it would be 99.9999999999999998% likely that he would have been able to do so - I'm not a betting man, but I make it (again correct me if I get these numbers wrong someone!) that I'd be massively more likely to win the lottery this weekend than Roger would fail to produce such a study from his 800.

So if Roger can't produce such a study, then that's simply because the studies haven't been done - not because there is no merit in the concept of "homeopathy". If you lead him to believe that the production of such a study will open your eyes to whatever it is that he is saying, then he will only accuse you of never being satisfied when he produces you such a study, done quite legitimately. As I said, him and his friends need only conduct a mere 16 decent studies between them to your specifications before one of them can have a reasonable expectation (>50%) to provide the results you seek. If they conduct 800 more studies having taken on board your feedback of their poor study design, then they would expect, on average, 40 of these studies to "prove" homepathy to your standards.

[quote user="Francisco Gomez"]However, if homeotherapy people came up with them and other studies failed to disregard their results (yes, studies with negative outcomes get also published and they aren't always funded by drug companies), they will have a start but my feeling is that they wont be able to do so (tho I'm not closed minded and will read/hear)...[/quote]

Roger produces a well-performed study beyond reproach with a p-value of <0.05 into his concept.

I could do 20 smaller studies all with "negative outcomes" (i.e. p>0.05) just by using smaller numbers of cases each time, even if there was an effect of the treatment he proposed.

What there are very few of are studies aimed at demonstrating that something doesn't work.

Unless I'm mistaken again, you'd likely need much higher sample sizes to demonstrate the same level of statistical significance with equivalence testing than you required to "prove" the difference in the first place.

It's a matter of garbage-in-garbage-out.

If his hypothesis makes no sense to start with, then his results will invariably be uninteresting. The addition of double-blinding, placebos and p<0.05 would not change this.