Stats #18: Quality Control: A Hands-On Workshop

Content: This training class will show you how to use statistical tools to assess the quality of an on-going laboratory or medical process.

Objectives: In this class, you will learn how to:

Teaching strategies: Didactic lectures and small group exercises.

Notes: There are no pre-requisites for this class. Please bring a pocket calculator for some simple arithmetic calculations.

Web pages included in this handout:

The workshop will conclude with a summary and moderated discussion. Ample time for discussion of all topics has been allocated.

A condensed version of this handout is available at

Information about my book, Statistical Evidence in Medical Trials

I recently published a book, Statistical Evidence in Medical Trials, What do the Data Really Tell Us? through Oxford University Press. A good summary of what this book is about appears on the back cover:

"Statistical Evidence in Medical Trials is a lucid, well-written and entertaining text that addresses common pitfalls in evaluating medical research. Including extensive use of publications from the medical literature and a non-technical account of how to appraise the quality of evidence presented in these publications, this book is ideal for health care professionals, students in medical or nursing schools, researchers and students in statistics, and anyone needing to assess the evidence published in medical journals."

A review by Rebecca Rooney in the International Journal of Epidemiology states:

"This book is a clear, concise, and interesting read and should prove to be a useful guide. The examples and case studies make it easy to understand difficult concepts and the jokes and stories make it fun. There are some salient points and hopefully the reader will be enthused about looking at the published research and be more confident about distinguishing between the good and the bad."

More information about the book (supporting materials, answers to the exercises, and other updates) can be found on the web at http://www.childrensmercy.org/stats/evidence.asp.

Where can you find this handout?

This handout and the handouts that I use for all of my seminars and training classes are a compilation of individual web pages at www.childrensmercy.org/stats. I use the "Include Page" feature of Microsoft FrontPage to combine these into a single page. You can always find the most recent version of this compilation by going to the web address listed at the bottom of this page. Links for the handouts for other seminars and classes appear at www.childrensmercy.org/stats/training.asp.

Why don't I use PowerPoint?

I stopped using PowerPoint for my presentations in the mid 1990's. This was based on Edward Tufte's advice that presenting information in a paper handout is more effective than presenting the information on a projected screen. I found this to be excellent guidance. I enjoy talking when I don't have to wrestle with a laptop computer. I look at my audience more and interact with them better. I elaborate on this in greater detail at www.childrensmercy.org/stats/weblog2004/powerpoint.asp.

Stats #18: Practice Exercise

Form a team of four to six people. The size may be slightly bigger or smaller if the instructor agrees, but teams of size one are not teams. You will receive a packet that includes a toy and a measuring tape. Here are some examples, but your toy may be different.

Flying saucerToy car with "pull-back". Foam disc gun.

Please note that some of the toys are choking hazards, so take appropriate precautions if one of the members of your group is less than three years old.

Create a target on the floor or on a table at a reasonable distance so that there is some challenge in hitting the target with your toy. Each team member should get two practice attempts with your toy. Every member of the team is required to use the toy unless they are too young or too old. Do not measure anything at this point, but do try your best to get as close to the target as possible.

If you cannot get your toy to work, call your child on your cell phone and ask for advice. If this fails, ask the instructor for a different toy.

After every team member has had a practice turn, discuss what strategy you will use to insure consistent and high quality performance from each team member. You may wish to adjust the location of your target if you believe that this would help.

Repeat the practice runs, but now record how close each team member is to the target on two consecutive attempts. Use metric measurements if possible.

If any result is so bad that the results are not measurable (e.g., further away than the maximum length of the tape measure), you are allowed to "take a mulligan" that is, to repeat the run. If any team member requires more than two mulligans, write an outstanding performance review for this person and see if you can get him/her hired by a different team.

Write down the results for each team member and compute a mean and a standard deviation. Hint: when there are only two observations, the standard deviation is equal to the range divided by the square root of two (1.414).

Draw an Analysis of Means (ANOM) chart for your data. For four team members, the critical value for h is 3.889. For five, it is 3.724, and for six team members, it is 3.622.

Here is a sample example using real data. As a classroom exercise similar to the one described above, a group of three volunteers (labeled A, R, and V to protect their anonymity) were asked to take turns hitting a target with a hand launched foam rocket.

The equipmentShooting at the targetMeasuring the result

Since the group was small, I asked each team member to shoot twice with their dominant hand and twice with their non-dominant hand. For each shot, the distance from the target was measured in centimeters. Here is the data (D=dominant hand, N=non-dominant hand).

A-D 14 39
A-N 60 20
R-D 26  9
R-N  9 12
V-D 36 21
V-N 53 18

The means, variances, and standard deviations are:

    Mean Var   Stdev
A-D 26.5 312.5 17.68
A-N 40.0 800.0 28.28
R-D 17.5 144.5 12.02
R-N 10.5   4.5  2.12
V-D 28.5 112.5 10.61
V-N 35.5 612.5 24.75

The overall mean is 26.42 and the pooled standard deviation is 18.20. The ANOM limits are

26.4 - 3.724 * 18.2 * sqrt(5/12) = -17.3 (round this to zero)
26.4 + 3.724 * 18.2 * sqrt(5/12) =  70.1

Here is a graph of the results:

Now modify your procedure, offer extra practice trials and repeat the experiment.

-->

08/training/www.childrensmercy.org/definitions/ControlChart.htm

08/training/www.childrensmercy.org/definitions/SpecialCause.htm

08/training/www.childrensmercy.org/definitions/CommonCause.htm

The Busy Tailor

When it was his turn to explain his recent work, Student Leaf stood up and portrayed an elegant experiment that used a central composite design with four factors. Master Stem asked, "Is this process ready for such an experiment?"

Student Leaf replied, "I do not understand."

Master Stem looked at him with an air of amusement. "If this process is not ready for an experiment, then you will make yourself very busy for no good reason."

"How can I tell, Master Stem, if a process is ready?"

"Have you computed a control chart for this process? Do you know if the process is in control?"

"I have not computed a control chart, but I do know that the process is too variable. I want to run an experiment to reduce that variation."

"I have a tailor I would like you to meet. He makes all the clothes for my family. I brought my oldest child in for a fitting and the tailor made measurements and started sewing. When I visited the next time, I had my youngest child with me. I apologized, but the tailor still insisted on doing the fitting. This required ripping out all the old seams, remeasuring and resewing. 'I am almost done with the clothes for your youngest child,' he told me, 'please come back tomorrow.' So I returned the next day, but this time I was accompanied by my middle child. 'No matter,' replied the tailor, 'I will rip out all the seams again and make the clothes fit your middle child.'"

"That is a very foolish tailor, Master Stem."

"And you, too, are foolish if you run an experiment without looking at the control chart first. If your process is out of control, that tells you that your process is not a single process, but is many instead. And you do not know which process is visiting at any time. Your experiment, carefully optimized for one process, will fit poorly for the other processes."

08/weblog2007/PocketCalculator.asp

08/weblog2007/XbarSControlChart.asp

08/weblog2007/PControlChart.asp

08/weblog2007/AnomCalculations1.asp

08/weblog2007/AnomTable05Part1.asp

08/weblog2006/FishboneDiagram.asp

08/weblog2007/XbarSControlChartAnswers.asp

08/weblog2007/PControlChartAnswers.asp

08/weblog2007/AnomAnswers.asp