P.Mean: Are certain CAM therapies undeserving of further study (created 2010-12-01).

I have become something of a celebrity on the Science Based Medicine site, as I have noted in an earlier webpage. In addition to the blog post I noted earlier, there is a new post: Of SBM and EBM Redux. Part I: Does EBM Undervalue Basic Science and Overvalue RCTs? These posts are reminding me how important it is to write precisely, which is good. I largely agree with many of the comments written in these particular entries and in others at the Science Based Medicine site, but there are still areas of fundamental disagreement. One of the major areas where we disagree is over the value of running randomized control trials for certain CAM (Complementary and Alternative Medicine) therapies that are biologically implausible.

The blog post by Dr. Atwood points out a critical distinction between "biologically implausible" and "no known mechanism of action" and I must concede this point. There are certain therapies in CAM that take the claim of biological plausibility to an extreme. It's not as if those therapies are just implausible. It is that those therapies must posit a mechanism that "would necessarily violate scientific principles that rest on far more solid ground than any number of equivocal, bias-and-error-prone clinical trials could hope to overturn." Examples of such therapies are homeopathy, energy medicine, chiropractic subluxations, craniosacral rhythms, and coffee enemas.

The Science Based Medicine site would argue that randomized trials for these therapies are never justified. And it bothers Dr. Atwood when a systematic review from the Cochrane Collaboration states that no conclusions can be drawn about homeopathy as a treatment for asthma because of a lack of evidence from well conducted clinical trials. There's plenty of evidence from basic physics and chemistry that can allow you to draw strong conclusions about whether homeopathy is an effective treatment for asthma. So the Cochrane Collaboration is ignoring this evidence, and worse still, is implicitly (and sometimes explicitly) calling for more research in this area.

There are a host of issues worth discussing here, but let me limit myself for now to one very basic issue. Is any research justified for a therapy like homeopathy when basic physics and chemistry will provide more than enough evidence by itself to suggest that such research is futile. Worse still, the randomized trial is subject to numerous biases that can lead to erroneous conclusions.  

I disagree for a variety of reasons.

Reason #1. It's good for business. I don't want to sound shallow, but there's money to be made by statisticians when research is done, and if I make a few bucks and in the process help to make the research more rigorous, that's a win-win situation. I've not done much work with CAM, but I have helped on several projects at Cleveland Chiropractic College. Some Chiropractors bristle at the thought that Chiropractic treatment is a part of CAM, but it is close enough in some people's minds to CAM that it is worth mentioning here.

I'm also partially supported by a grant looking at economic expenditures associated with patients with back pain who seek out CAM therapies. It is also a borderline case because the goal of the research is not to establish efficacy, but instead to look at insurance claims among people who see a CAM therapist versus those who do not. Still, it is better to note these because they do constitute a financial conflict of interest and they have the potential of coloring my responses.

Reason #2. Everyone deserves their day in court. I believe that if someone is sincere in testing whether a therapy is effective or not, then they deserve my help. I won't help someone who is trying to manipulate statistics to build a case for their therapy, of course. But the researchers at Cleveland Chiropractic College, for example, are truly interested in finding out what works and what doesn't work. If they want to spend their time and money to answer these questions, I want to help them.

It's a bit more complicated than that, of course, because they also want to spend some of the tax money that you and I send to Uncle Sam and which finds its way into places like the National Center for Complementary and Alternative Medicine. This leads to a difficult question of competing resources--money spent on CAM research is money that is not being spent on research in more promising areas. Which leads to ...

Reason #3. CAM therapies represent an enormous expenditure of limited health care dollars, and if research can help limit the fraction of CAM expenditures that are inappropriate then that represents a good use of scarce research dollars. There are some people who would not believe the results of carefully run trials, of course. There's a famous quote "You see, that is why we never do double-blind testing anymore. It never works!" that is representative of some perspectives among CAM practitioners, but there are enough others who would be swayed to make this worthwhile. The research can also serve as a shield for insurance companies who want to deny claims for some of the more outrageous CAM therapies.

Now I wouldn't be too upset if NCCAM funding dried up completely. There's an argument to be made that NCCAM research to date has been pretty much a bust. It might make more sense for CAM projects to compete directly with other therapies. There's no national center for exercise physiology or no national center for surgical interventions, so why is CAM the only therapy that has its own center. Take the money in NCCAM, and redistribute it to the other centers and then encourage CAM researchers to apply to the appropriate center. A CAM therapy that reduces the risk of heart attacks would be sent to the National Heart, Lung, and Blood Institute. A CAM therapy for asthma would go to the National Institute of Allergy and Infectious Diseases.

It will never happen, of course, because there are some pretty strong CAM defenders in high places in government. As long as money is flowing to NCCAM, I want to help insure that it is being spent as effectively as possible.

Now part of me says things like, no funding of research into therapeutic touch until someone can replicate the Emily Rosa experiment and show different results than Ms. Rosa did. So I'm kind of split on this issue.

Reason #4. We have to trust that the system can work. Randomized trials are indeed subject to many biases, and it is worth noting them. But are the biases so serious that they will lead to incorrect conclusions about CAM? It's possible to manipulate the randomized trial, as many drug companies have shown us, so it is possible for CAM practitioners to use some of the same tricks. But I don't think that CAM practitioners will be able to get away with running shoddy trials and then claiming efficacy based on these trials. The randomized trial is remarkably easy to document and as long as there is transparency in the process things should be okay. Drug companies have tried very hard at times to game the system, but we're learning from this and are in a better position today to insure that randomized trials provide fair and unbiased answers.

Reason #5. Scientific testing is the norm for other claims that lack scientific plausibility. I am a regular regular of Skeptical Inquirer and Skeptic Magazine, and when someone makes a claim about ghosts, telekinesis, or reincarnation, they'll point out all the existing knowledge that makes such claims unbelievable. But then they'll still go to the haunted house or set up a spoon bending experiment or reinterview people who remember past lives. These claims have even less credibility than much of CAM research, but they are still being tested. So why not test CAM the same way?

There are some other issues worth discussing when I have time. Some CAM therapies are not so implausible (herbal medicine, perhaps), but how much negative data has to accumulate on these therapies before we call it a day. Also how can we invoke scientific plausibility in a world where intelligent people differ strongly on what is plausible and what is not? Finally, is there a legitimate Bayesian way to incorporate information about scientific plausibility into a Cochrane Collaboration systematic overview.

It's worth noting another conflict of interest. Traffic to my website has jumped since my webpages have been cited on the Science-Based Medicine blog. There's an order or two of magnitude difference between our sites. I'm lucky if I get two or three people to send comments to me about my newsletter. The Science Based Medicine blog entry cited above has 139 responses so far. So to Dr. Atwood and anyone else at the Science-Based Medicine blog. Go ahead and criticize this web entry. Just be sure to include a link when you do, as I appreciate getting even a fraction of the attention that your blog gets. It's a testament to the quality of your writing and the importance of your topic that you are getting so much traffic.

Creative Commons License This work is licensed under a Creative Commons Attribution 3.0 United States License. This page was written by Steve Simon and was last modified on 2010-12-03. Need more information? I have a page with general help resources. You can also browse for pages similar to this one at Category: Critical appraisal.