The Monthly Mean newsletter, June 2013. Released 2013-07-18.
It's been a busy summer, but I do want to try my best to release these newsletters every month. I'm only 18 days behind schedule.
--> How the AIDS crisis changed the nature of placebo controlled trials
--> Looking at two by two tables with a larger contingency table
--> Article: Assessing potential sources of clustering in individually randomised trials
--> Joke: x-squared says to x-cubed...
--> Nick News: Nick at Camp Bartle
--> Trivia: What television character...
--> Join me on Facebook, LinkedIn and Twitter
--> Permission to re-use any of the material in this newsletter
--> Introduction. Welcome to the Monthly Mean newsletter, a newsletter with articles about Statistics with occasional forays into research ethics and evidence based medicine. If you are having trouble reading this newsletter in your email system, please go to the web version (www.pmean.com/news/201306.html). If you are not yet subscribed to this newsletter, you can sign on at the newsletter page (www.pmean.com/news). If you no longer wish to receive this newsletter, there is a link to unsubscribe at the bottom of this email.
--> How the AIDS crisis changed the nature of placebo controlled trials. The placebo offers you many advantages in a research study. It prevents your expectations or the expectations of your patients from coloring the results of the trial. It allows you to control for effects like regression to the mean. It is useful in preventing fraud. But the placebo has always been surrounded by controversy. The attitudes of the research community towards placebos has changed markedly, and some of the early trials of AZT and other related drugs for AIDS patients played a major role in this change.
The first few AIDS trials were placebo controlled and this was not (in my humble opinion) a good idea. You are facing a rapidly fatal disease (at least that's what AIDS was before the era of AZT), and you want to enroll in a trial where there is a 50% chance of getting a placebo? Many people voiced strong objections to the use of placebos in these trials. Others came to the defense of the placebo. But more interesting than the debate among scientists was how the patients themselves reacted to the placebo.
This is described in the book, Impure Science: AIDS, Activism, and the Politics of Knowledge by Steven Epstein.
It turns out that the drug being tested (AZT) had a bitter taste and you
could tell whether you were on the placebo by just opening up the
capsule and putting a small pinch on your tongue. After the drug company changed
the composition of the placebo to match the bitter taste, AIDS patients
would take one of the pills to a local chemist for analysis.
I'm speculating, but if you found out you were in the placebo group, you
would probably drop out and then re-enroll under a different name.
Still other patients pooled their pills with others figuring out that a
100% chance of getting a dose at 50% potency was better than a 50%
chance of getting a dose at 0% potency.
Now Epstein does admit that much of this was "rumor" and that the actual
level of "abuse" (if you can call it that) was very low. But let's think about this for a bit. The AIDS patients were subverting the clinical trial by unmasking the placebo, so you could argue that they were harming society. But they were doing what they felt was in their best individual interest. If there is a strong discrepancy between the needs of society and the needs of the individual, I believe that you need to rethink the rationale for the original research study.
In a disease that (at the time) was 100% fatal, there was no real need for a control group. Any drug that was even partially effective will stand out clearly in a clinical trial without a placebo arm. Once a drug like AZT is proven to work, then you can compare it to other drugs in a blinded fashion.
As a statistician, I am not thrilled with the prospect of sacrificing your placebo arm, but I am even less thrilled with the prospect of asking an unreasonably big sacrifice from the 50% of those patients who are randomized to the placebo arm of the trial. It doesn't matter if the drug being tested has an unknown impact. It's unfair to ask half of your patients to forgo a promising new drug when the stakes are so high.
--> Looking at two by two tables with a larger contingency table. Dear Professor Mean, I am comparing a categorical outcome variable with four levels across three different treatment groups. I have arranged this in a four row by three column contingency table and have run the Chi-squared test. It is statistically significant. I now want to look at where the differences lie. I could analyze a bunch of two by two tables within this larger table, but how do I adjust for multiple comparisons? I could use the Bonferroni correction, but I hear that it is very conservative.
There is no consensus in the research community about how to handle multiple comparisons for a large contingency table. I would mention, however, that Bonferroni is not too conservative. Bonferroni is ultraconservative when you have a very large number of tests or the tests are highly correlated. Neither is true here.
I would encourage you, however, to consider ordinal logistic regression if your outcomes are ordinal, as the chi-square test for the big four by three table ignores order completely. That makes it markedly inefficient.
--> Article: Kahan BC, Morris TP. Assessing potential sources of clustering in individually randomised trials. BMC Medical Research Methodology. 2013;13(1):58. doi:10.1186/1471-2288-13-58. Abstract: "Recent reviews have shown that while clustering is extremely common in individually randomised trials (for example, clustering within centre, therapist, or surgeon), it is rarely accounted for in the trial analysis. Our aim is to develop a general framework for assessing whether potential sources of clustering must be accounted for in the trial analysis to obtain valid type I error rates (non-ignorable clustering), with a particular focus on individually randomised trials." Available at: http://www.biomedcentral.com/1471-2288/13/58
--> Joke: I had to use MathType for this joke.
You can find this joke at the Online Math Learning website.
--> Nick News: Nick at Camp Bartle. Nick graduated from Cub Scouts to Boy Scouts this Spring. One of the big differences is that the campouts for Boy Scouts are more intense. The best example of this is Camp Bartle. It's a summer camp that goes for nine consecutive days. During that time, your home is a tent. Your neighbors are ticks and they try their best to get up close and personal. The only concession to civilization is some showers attached to the swimming pool. Nick wanted to go, and since this was his first year, I volunteered to go as well. The adults at the campsite help out with a variety of tasks. My major task was insuring that the first year scouts got to the proper locations during the day.
Here's a picture of Nick waiting for the bus to take him to camp.
Notice the footlocker. That held enough clothes for nine days. It ended up being used for something else midway through camp.
One of the best parts of camp is the chance that kids have to do things that they cannot or should not do at home. Nick signed up for an optional activity called "Mountain Man" which included firing black powder rifles at a shooting range.
Nick loves to catch insects, but at camp he also found several lizards. Here's a skink. Nick did not know this, but when you hold up a skink by the tail, the tail breaks off, and it runs away to a safe place where it can grow another tail. Nick didn't make that mistake twice. The second lizard was given a living space in a foot locker. A foot locker whose clothes were dumped all around inside his tent. A footlocker with dirt, water, rocks, and twigs inside it to make a more livable lizard habitat.
Another highlight of camp was the last day of mammal studies, where teams of kids were asked to build shelters that a small animal might take refuge in. Nick's team created a shelter was a bit larger. It was a shelter for a moose, not a mouse.
The last night at camp, one of the leaders at the campfire asked if the kids were looking forward to going home again. Some cheered, but others (including Nicholas) chanted "Nine more days! Nine more days!"
--> Trivia: What television character had the following line: "You're worth two 43s, 86."?
Last month's trivia question was a trick: What is the number who sing the song "Heigh Ho" in the classic Disney cartoon, "Snow White"? The film title was actually "Snow White and the Seven Dwarves" so you might think that the correct answer is "seven." But Jane Yank was not fooled. Here's her correct response: "6 of the 7 dwarves sing Heigh Ho as they head to the mines. Dopey goes along to work but can't speak (or sing, I suppose), but we love him anyway! So six is the answer." Congratulations, Jane! That's two in a row for you.
--> Website: Wickham H, Maeland B. It's crantastic! Description: One of the best features of R but also one of its most worrisome, is the large selection of user contributed packages. It gives you a lot of options, but sometimes it seems like too many options. Crantastic is a web site that tracks, tags, and reviews these packages.
--> Tell me what you think. How did you like this newsletter? Give me some feedback by responding to this email. Unlike most newsletters where your reply goes to the bottomless bit bucket, a reply to this newsletter goes back to my main email account. Comment on anything you like, but I am especially interested in answers to the following three
questions.
--> What was the most important thing that you learned in this newsletter?
--> What was the one thing that you found confusing or difficult to follow?
--> What other topics would you like to see covered in a future newsletter?
If you send a comment, I'll mention your name and summarize what you said in the next newsletter. It's a small thank you and acknowledgement to those who take the time to help me improve my newsletter. If you send feedback and you want to remain anonymous, please let me know.
I received feedback from five people.
Several people asked for the full reference of the article by Sir Austin Bradford Hill. It is Hill AB. The Environment and Disease: Association or Causation? Proc R Soc Med. 1965;58(5):295-300. You can find it at PubMed Central.
I got a compliment from Nayak Pollisar that my writing has a warm, enjoyable, friendly style. Thanks. Here's one hint if you want to emulate that style in your writing. Pronouns, in general, make writing warmer, but you should especially try to use the pronoun "you" rather than "I" whenever possible. Also, if you can get away with it, replace "he/she/they" with "you" as well. Even think about using "you" for generic people. So instead of "when patients are assigned to the placebo group" say "when you are assigned to the placebo group." I learned this from the famous Internet usability expert, Jakob Nielsen.
An unsigned email from a brand new subscriber also complimented my newsletter and mentioned that he/she was going to look through the earlier newsletters. If you didn't know it, there is an archive page that links to each individual newsletter.
Tzippy Shocat had an interesting comment on the article Solving Equation of a Hit Film Script, With Data, The New York Times, May 5, 2013. Let me quote directly from her email: "Using statistics to decide on movie productions may be a bane to creativity. This kind of statistics can only analyze past preferences but may have little to say on a totally new idea. Do we really only want more of the same?"
The article did talk about this and there are many who share Tzippy's concerns. The opposing viewpoint, though, is that this method provides feedback to the writer before the script is completed. This empowers the writer in a way that other forms of testing (e.g., a preview focus group screening after most of the filming and editing is done) cannot. Also, they point out that all of the feedback is optional. The script writer is free to use or ignore any of the statistical advice given. It should go without saying, but when I mention an article like this, it is because I found the article interesting not because I agreed with or endorsed any particular finding or conclusion in that article. On this particular article, I'm on the fence. But when Hollywood calls on me to replicate this research, I will become a fervent advocate. Is there an Oscar for best data miner?
Ed Gracely offered two very helpful comments. First he noted that a borderline p-value like 0.078, if associated with a strong effect, is evidence that the sample size is way too small. He also commented on two important advantages of very large sample sizes. First, the very narrow confidence intervals that result allow you to assess clinical or practical significance much more clearly. Second, a very large sample size gives you the luxury of looking interactions among variables without losing much power.
Bengt Brorsson liked the reminder of the importance of the 1965 article by Sir Austin Bradford Hill.
An anonymous reader liked my article on everything being significant with a very large sample size and characterized it nicely as "think first and analyze after."
--> Join me on Facebook, LinkedIn, and Twitter. I'm just getting started with social media. My Facebook page is www.facebook.com/pmean, my page on LinkedIn is www.linkedin.com/in/pmean, and my Twitter feed name is @profmean. If you'd like to be a Facebook friend, LinkedIn connection (my email is mail (at) pmean (dot) com), or tweet follower, I'd love to add you. If you have suggestions on how I could use these social media better, please let me know.
--> Permission to re-use any of the material in this newsletter. This newsletter is published under the Creative Commons Attribution 3.0 United States License. You are free to re-use any of this material, as long as you acknowledge the original source. A link to or a mention of my main website, www.pmean.com, is sufficient attribution. If your re-use of my material is at a publicly accessible webpage, it would be nice to hear about that link, but this is optional.
What now?
Sign up for the Monthly Mean newsletter
Review the archive of Monthly Mean newsletters
Take a peek at an early draft of the next newsletter
This work is licensed under a
Creative
Commons Attribution 3.0 United States License. This page was written by
Steve Simon and was last modified on
2010-12-31.
