P.Mean: Reliable diagnosis of cataracts (created 2008-07-28).

This page is moving to a new website.

Can you help with this question? Cataracts of the eye may be difficult to diagnose, especially in the early stages. In a study of the reliability of their diagnoses, two physicians each examined the same 1,000 eyes, without knowing the other's diagnoses. Each physician found 100 eyes with cataracts. Does this mean that the diagnoses are reliable?

I get lots of questions like this and they are obviously homework questions. The two tip-offs are the perfectly round numbers (100 and 1000) and the very generic medical condition (cataracts) with few supporting details. Now I could be wrong of course. But I would be willing to wager a considerable sum of money on this conjecture.

Now, I don't have a problem with helping people with their homework, but I don't encourage it. First, I don't want to step on anyone's toes. Second, I don't have understand the classroom context in which this problem was presented. So I usually provide broad hints rather than an actual solution. Here's what I wrote back.

Is this a homework question? If so, you're far better off asking your teacher than asking me. If it is not a homework question, or even if it is, you might start by reviewing the exact definition of reliability. If it is similar to the Wikipedia entries on reliability and inter-rater reliability, then you need more information to answer the question. The key point is that you need to measure consistency, using a statistic like Cohen's Kappa. I hope this helps. If not, feel free to send back another email detailing your problem in further detail.

This work is licensed under a Creative Commons Attribution 3.0 United States License. This page was written by Steve Simon and was last modified on 2010-04-01. Need more information? I have a page with general help resources. You can also browse for pages similar to this one at Category: Measuring agreement.