P.Mean >> The Monthly Mean >> Filler material for future newsletters (created 2010-02-04)
The following is filler material which I plan to use in future newsletters.
Feedback for November 2008 newsletter. I got some very positive feedback from three people last month. Two people cited the article on ANOVA versus regression as being the most helpful. The one suggestion for improvement was to drop the section on Wikipedia. I agree and in the next newsletter I might replace it with a "current events" entry that highlights a recent newspaper article touching on issues in Statistics. some suggested topics for future newsletters are principles for good graphical displays (such as the guidance from Edward Tufte) and important events in the history of Statistics.
Feedback for December 2008 newsletter. I only received feedback from two people (that's okay, since feedback is totally optional). One person had an interesting take on my newsletter. I don't think I read your newsletter to learn specific things as much as to be reminded of a way of thinking. I see so much use of statistics that I suspect is really lousy that it is useful to just immerse myself in the words of someone who knows what he's talking about, loves the field, and is able to teach/communicate well. I think I read your newsletter for inspiration ... for the fact that it returns me to a place of clarity and dare I say hope?? I'm not sure I always know what I'm talking about, but I do try to get people to think about how statistics are used in the real world. You don't have to be a rocket scientist or a brain surgeon to be able to critically evaluate the use of statistics in the real world. If one person finds this inspiring, then I'm happy. Another respondent liked my discussion on overfitting, but found my discussion on combining measures on different scales of measurement confusing. I'll see if I can clarify the latter point in a future webpage or newsletter entry. Both respondents had interesting suggestions for future discussion. One suggestion was about the use of propensity score, and whether it should used as a matching factor, a weighting factor, or a categorizing factor. This is something I am actively working on for a client, and I hope to put some of this up soon. Another suggestion was an explanation of adjusted odds ratios in logistic regression. I have some material on this at my old website and I'll try to elaborate further on this. A third suggestion was for "basic statistics" although there was some concern that this might be boring for other readers. I generally have found that people who are interested in help will invariably use the term "statistics for beginners." Actually, they will use a more perjorative term like "idiots" or "complete dummies" but this is not true. The people who ask for help are almost always very well educated and well read. They just lack experience in a particular area that I happen to know a few things about. In all the classes I've taught recently and all the webpages I've written, I've only received one request to make the discussion more technical. Even in people who I recognize understand statistics well, their desire is still for more basic information. I'm all in favor of getting a more solid understanding of the fundamentals. A final request came in more of the form of a complaint. In some studies, a single patient may have more than one procedure. This can lead to some confusing situations and the researchers themselves often fail to distinguish between patients and procedures. I think it is critical to always know how much data you have and of what type. The classic example that I encountered was a breastfeeding study with a data set of 84 infants born to 72 mothers because 12 of the mothers had given birth to twins. I'll try to write up something about this and about a closely related topic, pseudo-replication, for a future newsletter.
Feedback for January 2009 newsletter. January's newsletter produced three comments on the web survey and two emails. Someone was nice enough to point out a broken link in the January newsletter. When I link to my own website, iContact gets confused, so I need to manually insert the proper URLs for these links. I forgot one, which was a plug for my book. The correct link is http://www.pmean.com/Evidence.html. The same person also pointed out that when adults fall, it's not only the greater mass, but also the greater velocity due to the fact that we are up higher than kids. Greater mass and greater velocity is a double whammy. Cathy's pretty much recovered from the skating experience, by the way. I got a compliment for my section on crude versus adjusted comparisons. I was worried that it was a bit too technical. That person also wanted to see a discussion of generalized estimating equations in a future newsletter. I'm not sure I understand GEE models well enough to explain them clearly, but I will try to place them in the general context of repeated measures designs as a starting point. It may take a while for me to get something coherent written about this. This person also thought that "Nick News" was cute. Thanks! Two people emphasized a desire for basic statistics, how-to tutorials, and general concepts. That's my general goal. I'm not writing for people who already know Statistics inside and out. I also got a nice email from someone who liked my description of CART and the reference on overlapping confidence intervals. This person suggested some material on Bayesian models (more specifically, the Bayes factor). I had just come back from a conference on Bayesian Biostatistics, so the material is fresh in my mind. The hard part will be trying to make it accessible.
Feedback for Feburary 2009 newsletter. I got only one response to my feedback webpage. The respondent was relieved by my comments about small sample sizes, a reaction I had not expected. "The small samples problem isn't a big as i thought it was -- I worry about paediatric trials with 20-30 patients, but shouldn't so much." The entry was "Is there a sample size too small to allow efficient randomization?" and it is only one of several considerations. I elaborate on this in more detail in this month's newsletter. Still, if you're able to conduct a research study with 20-30 patients and get good precision on your confidence intervals, don't let anyone convince you that your sample size is too small. That person also asked for more material on generalized estimating equations (GEE), and other closely related topics. I've added a definition of GEE in this newsletter and hope to show some examples of various models in this area in future newsletters. I did get some nice compliments by email. Thanks! I'd brag about all the nice things you've said, but my email folders are in limbo thanks to a malfunctioning laptop.
Feedback for March/April 2009 newsletter. I got several nice comments about the March/April newsletter. There was appreciation for the explanation of GEE models, the rule of 15 for logistic regression, and the loss of power caused by dichotomization. Someone pointed out that the Java application on dichotomization is also available at www.bolderstats.com/jmsl/doc/medianSplit.html. I probably need to explain GEE models in more detail, though, and further clarify the Bayesian approach. There was a suggestion to explain what heterogeneity is. I presume this is heterogeneity in meta-analysis.
Feedback for May/June 2009 newsletter. Only two people sent feedback (that's okay, I know that you all love my newsletter). I got compliments on my description of mosaic plots. I also got compliments on the material on covariate adjustment from one person, but another person found that material confusing. Suggestions of future topics included meta-analysis statistics, Bayesian statistics, various methods for smoothing and checking for interactions with applications to survival analysis, and the problem with ROC curves. That's quite an ambitious list.
Feedback for July/August 2009 newsletter. Two people provided feedback to the last newsletter. I got positive comments about my article on weighted means and the JAMA article about changes to the primary outcome measure. Both were confused by the article on Bayesian data analysis. One had difficulty with interpreting conditional probability, especially with the order (A given B versus B given A). In spite of this, I was encouraged to write more about Bayesian statistics.
Feedback for September/October 2009 newsletter. Three people provided feedback to the last newsletter. I got praise for the article on rounding and re-ordering and the article on early stopping of a study and general praise for the newsletter. The only concern was about the basic formatting. I don't know what this means and I'd appreciate additional information from the person who left this comment or anyone else. Topics suggested for future newsletters include longitudinal analysis and robust analysis. I have been intending to write up a series of webpages about longitudinal analyses, but it's a complicated area and takes a lot of time to develop. I do hope to have something soon about this.
Feedback for November 2009 newsletter. Only two people provided feedback to the last newsletter. The Stephen Jay Gould article "The median is not the message" and the description of Kaplan-Meier curves drew praise. One person liked the simple explanation about the case mix index but the other person thought it was confusing. I got the excellent suggestion to talk about problems associated with missing data. It will take some effort, but I'll see what I can do.
Feedback for December 2009 newsletter. Two people provided feedback to the last newsletter. Both liked the description of the ROC curve. For one person, it was more a review and a clarification. I'm glad to do this. Sometimes the first time you learn something, it doesn't sink in. It's when you hear a second explanation, from a slightly different perspective, that it takes for an abstract idea to take hold. One person found the description about power and the minimum clinically relevant difference to be tough to follow. There were no suggestions this month for new topics. I do want to consider earlier suggestions about mixed models, generalized estimating equations, and multlilevel models in future newsletters, but these topics take a lot of time to prepare well. I'm also going to try to talk about missing data. It's a very important topic, but again one that takes some work to prepare well.
Feedback for January 2010 newsletter. Two people provided feedback to the last newsletter. One liked the description of the stem and leaf diagram, especially the explanation of how you might use two digits in the stems. The other liked the recent entry in the seven deadly sins of researchers (wrath) and the comments about heterogeneity in clinical trials. There was nothing unclear to either respondent. One suggestion for future topics was some of the issues associated with random effects meta-analysis, such as the use of confidence intervals versus prediction intervals. The other suggestion was how to do power calculations when you have the wrong standard deviation.
Feedback for February/March 2010 newsletter. Three people provided feedback to the last newsletter. Two liked my description of sensitivity and specificity and the other liked my explanation about weights. One person wanted more explanation of how to use control charts. I'm thinking of putting together a more elaborate set of webpages about quality improvement in general and that would include several different variations of the control chart. Suggestions for future newsletters include fixed versus random effects models and survival analysis. For the latter topic, I would note that my definition in the November 2009 newsletter was for the Kaplan-Meier plot. There is more to survival analysis than just Kaplan-Meier, of course, so I'll see what I can do. Let me apologize in advance though for this and earlier suggestions that I have not found the time to write just yet. These webpages take quite a bit of effort, especially the ones that describe some of the more advanced perspective.
Feedback for April 2010 newsletter. Only one person provided feedback to the last newsletter. That person liked the articles I had highlighted (Can we rely on the best trial? A comparison of individual trials and systematic reviews, and Convincing the Public to Accept New Medical Guidelines). That person also suggested that I review some more basic review topics like Type I and Type II errors. I do have definitions of these two terms at my old website but I still like the suggestions and will try to elaborate more on these definitions in a future newsletter.
Feedback for May/June 2010 newsletter. Three people provided feedback to the last newsletter. I got compliments on the article about why randomization doesn't always work, about Simpson's paradox (I think this person was thinking about my interaction article), and the link to the journal article about overdiagnosis of cancer. Several people were confused, though, about the description of interactions among two continuous variables in a linear regression model. I'll see if I can simplify it. There weren't a lot of suggestions about future topics and one person liked being surprised. A comment about the advantages of meta-analysis over simply counting the number of positive/negative studies was offered, though.
Seven deadly sins summary:
Ideas for me to write about
L'Abbe plot
Power for comparison of two Poisson means
Quotes
"In God we trust. All others must bring data."
David Cox: "There are no routine statistical questions, only questionable statistical routines"
"... no scientific worker has a fixed level of significance at which from year to year, and in all circumstances, he rejects hypotheses; he rather gives his mind to each particular case in the light of his evidence and his ideas." - Sir Ronald A. Fisher (1956)
"All scientific work is incomplete - whether it be observational or experimental. All scientific work is liable to be upset or modified by advancing knowledge. That does not confer upon us a freedom to ignore the knowledge we already have or postpone the action that it appears to demand at a given time." Sir Austin Bradford Hill, as quoted at Toxipedia http://toxipedia.org/display/toxipedia/Sir+Austin+Bradford+Hill
Jokes
There are 10 types of programmers in the world: those who understand binary numbers and those who do not understand binary numbers.
There's a story about two doctors who are floating above the countryside in a hot air balloon. They are drifting with the wind and enjoying the scenery, but after a couple of hours, they realize that they are totally lost. They see someone down on the ground, and shout down "Hello! Can you tell us where we are?" The person on the ground replies, "you're fifty feet up in the air, in a hot air balloon." One doctor turns to the other and says, "That person on the ground must be a statistician." "How did you know?" came astonished reply from the ground. "Only a statistician would provide an answer that was totally accurate and totally useless at the same time." In my stories, of course, the statistician always has the last word. "Very good. But I can also tell that you two are doctors." It was the doctors' turn to be astonished. The statistician explained. "Only a doctor would have such a good view of the area and still not have any idea they were."
Books
The Statistical Evaluation of Medical Tests for Classification and Prediction (Oxford Statistical Science Series) by Margaret Sullivan Pepe
Mistakes were made, but not by me, by Carol Tavris and Elliot Aronson.
How We Know What Isn't So: The Fallibility of Human Reason in Everyday Life, by Thomas Gilovich
Risk Adjustment for Measuring Healthcare Outcomes, Third Edition Lisa I. Iezzoni (Editor)
Articles
Results Unproven, Robotic Surgery Wins Converts. http://www.nytimes.com/2010/02/14/health/14robot.html
In the City’s Vital Statistics, So Many Ways to Die by Accident http://www.nytimes.com/2010/02/16/nyregion/16accidents.html
Khamis, H. and Kepler, M.: Sample size in multiple regressions: 20 + 5k, Journal of Applied Statistical Science, to appear in Vol. 17. Also see http://ablejec.nib.si/as2007/appliedstatistics2007booklet.pdf (page 64).
It's not too recent, but an outstanding overview is: Biases in the interpretation and use of research results. RJ MacCoun. Annu Rev Psychol
1998: 49; 259-87.
* http://socrates.berkeley.edu/~maccoun/ar_bias.html
* http://ist-socrates.berkeley.edu/~maccoun/MacCoun_AnnualReview98.pdfAmong the fascinating findings summarized in this paper:
"partisans on both sides of a dispute tend to see the exact same
media coverage as favoring their opponents� position""24 students favoring capital punishment and 24 opposing it were recruited; each group believed the existing evidence favored their views. They were then given descriptions of two fictitious studies, one supporting the deterrence hypothesis, the other failing to support it. For half the respondents, the prodeterrence paper used a cross-sectional methodology (cross-state homicide rates) and the antideterrent paper used a longitudinal methodology (within-state rates before and after capital punishment was adopted); for the remaining respondents, the methodologies were reversed. Each description contained a defense of the particular methodology and a critique of the opposing approach. Students received and provided initial reactions to each study's results before being given methodological details to evaluate. Analyses of student ratings of the quality and persuasiveness of these studies revealed a biased assimilation effect: students more favorably evaluated whichever study supported their initial views on the deterrent effect, irrespective of research methodology. Students' open-ended comments reveal how either methodology, cross-sectional or longitudinal, could be seen as superior or inferior, depending on how well its results accorded with one's initial views."
"people will feel less safe after a noncatastrophic technological breakdown if they already oppose the particular technology, but will feel more safe after such a breakdown if they support the technology."
I think that MacCoun also mentions somewhere about a study that showed people two contradictory articles on the same political topic. After reading both articles people became STRONGER in their beliefs. After all, they found a study that supported what they knew all along, and that other study must have been flawed somehow.
This paper should be required reading for anyone in Evidence-Based Medicine.
John P. A. Ioannidis. Why Most Published Research Findings Are False. PLoS Med. 2005;2(8):e124. Abstract: "There is increasing concern that most current published research findings are false. The probability that a research claim is true may depend on study power and bias, the number of other studies on the same question, and, importantly, the ratio of true to no relationships among the relationships probed in each scientific field. In this framework, a research finding is less likely to be true when the studies conducted in a field are smaller; when effect sizes are smaller; when there is a greater number and lesser preselection of tested relationships; where there is greater flexibility in designs, definitions, outcomes, and analytical modes; when there is greater financial and other interest and prejudice; and when more teams are involved in a scientific field in chase of statistical significance. Simulations show that for most study designs and settings, it is more likely for a research claim to be false than true. Moreover, for many current scientific fields, claimed research findings may often be simply accurate measures of the prevailing bias. In this essay, I discuss the implications of these problems for the conduct and interpretation of research." [Accessed January 6, 2009]. Available at: http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=1182327.
Unsung heroes
John F. Hall (http://surveyresearch.weebly.com)
The R Foundation.
Websites
Gordon Guyatt. McMaster University > Evidence-Based Clinical Practice Workshop. Description: This website provides details on how to teach evidence-based clinical practice, through workshops offered at McMaster University, as well as through videos, such as "making Sense of Likelihood Ratios" and "Understanding Odds Ratios." The site also offers some of the teaching handouts as PDF files. [Accessed January 24, 2010]. Available at: http://ebm.mcmaster.ca/.
What now?
Sign up for the Monthly Mean newsletter
Take a peek at an early draft of the next newsletter
Review the archive of Monthly Mean newsletters
This work is licensed under a
Creative
Commons Attribution 3.0 United States License. This page was written by
Steve Simon and was last modified on
2010-07-07. Need more
information? I have a page with general help
resources. You can also browse for pages similar to this one at
Category: Website details.