Updated: The post-modern assault on evidence-based medicine (created 2007-12-18, updated 2010-10-26).

I never got around to submitting an article to Skeptic Magazine, but here is an update of something I wrote back in 2007 about post-modern philosophy and evidence based medicine.

As mentioned in a December 7, 2007 weblog entry, I sent an email to Michael Shermer, editor of Skeptic Magazine. In it, I included a brief pitch for an article, "The post-modern assault on evidence-based practice" and he gave me some encouragement to work on this. Guidelines for contributions are at http://www.skeptic.com/the_magazine/contribute.html and they accept articles of varying lengths (500 to 5000 words) though they prefer shorter articles. The current draft is 5800 words (as of February 11). So I'll have to find a way to condense many of these thoughts.

Here are some of the talking points in the article. I'm including heading in this draft, but they are just to help me organize my thoughts. This draft still needs a lot of work, but I wanted to have something to show to people.

Introduction. There's an old joke about philosophy: The famous French philosopher, Rene Descartes walks into a bar. The bartender asks "Do you want something to drink?" to which Descartes replies "I think not." Then poof! he disappears.

This is the sort of joke that appeals to those of us who see philosophy as a bunch of silly old fools talking about things that have no practical impact on the world. I certainly am one of the people who Bertrand Russell refers to as

"many men, under the influence of science or of practical affairs, are inclined to doubt whether philosophy is anything better than innocent but useless trifling, hair-splitting distinctions, and controversies on matters concerning which knowledge is impossible." (Bertrand Russell's The Problems of Philosophy, as quoted at http://skepdic.com/russell.html).

But I am also inclined to agree when he later asserts that

"The man who has no tincture of philosophy goes through life imprisoned in the prejudices derived from common sense, from the habitual beliefs of his age or his nation, and from convictions which have grown up in his mind without the co-operation or consent of his deliberate reason."

So it is both with a sense of trepidation, but also a sense of excitement that I want to tackle a recent philosophical critique of evidence based practice.

Post-modern philosophers have been attacking many institutions that provide (from the post-modern perspective) a false sense of objectivity in their work. It was only a matter of time before they would turn their attention on evidence based practice (EBP), an approach that tries to incorporate greater use of objective research into the practice of healthcare. Their rhetoric is surprisingly harsh, but unnecessarily so. All the criticisms that post-modern writers lay at the feet of EBP are criticisms that EBP itself has been able to successfully identify. EBP is perfectly capable on its own to understand the areas where objectivity is an illusion and to take corrective action.

What is evidence based practice? Evidence Based Practice has radically changed our health care system. This is an umbrella term for Evidence Based Medicine, Evidence Based Nursing, Evidence Based Dentistry, Evidence Based Mental Health, and so forth. The term Evidence Based Practice is preferred by many because it incorporates the interests of all health care professionals.

There are many competing definitions for EBP. A classic definition from one of the most popular books on the topic is

"the integration of best research evidence with clinical expertise and patient values." (Sackett 2000)

The three pillars of EBP (best evidence, clinical expertise, and patient values) are all important, but some definitions will fail to emphasize all three elements equally. Interestingly, the first edition of this popular reference uses

"integrating individual clinical expertise with the best available clinical evidence from systematic research," (Sackett 1997)

a definition that fails to mention patient values at all.

The five steps in applying EBP in a clinical situation are:

  1. CONVERT the need for information into answerable questions.
  2. TRACK DOWN the best evidence with which to answer the questions.
  3. CRITICALLY APPRAISE the evidence for its validity, impact, and applicability.
  4. INTEGRATE the critical appraisal with our clinical expertise and with our patient's unique biology, values, and circumstances.
  5. EVALUATE our effectiveness and efficiency in executing steps 1-4 and seek ways to improve them both for next time. (Source: www.oucom.ohiou.edu/ebm/def.htm).

Most proponents and critics of  EBP focuses only on the third step, critical appraisal. I fall into this trap myself often. It's easy to do because this is the most visible and concrete aspect of EBP. This article may have the unfortunate tendency to perpetuate the reductionist notion that EBP is all about critical appraisal.

What was life like before EBP? It is very important to place EBP in context by noting what it has replaced. Before EBP became prominent, changes in medicine occurred when a small group of respected experts opined that changes were needed. This is practice is labeled eminence-based medicine.

"Eminence based medicine---The more senior the colleague, the less importance he or she placed on the need for anything as mundane as evidence. Experience, it seems, is worth any amount of evidence. These colleagues have a touching faith in clinical experience, which has been defined as 'making the same mistakes with increasing confidence over an impressive number of years.' The eminent physician's white hair and balding pate are called the "halo" effect." (Isaacs 1999)

The Cochrane Collaboration. At the forefront of the EBP movement is the Cochrane Collaboration. The Cochrane Collaboration is organization of a large number of medical professionals who donate their services to produce evidence based systematic reviews on a variety of health topics. Some of the readers may be more familiar with the term meta-analysis. Meta-analysis constitutes the statistical tools used to quantitatively combine the results of multiple research studies. A systematic overview is a careful and reproducible method for gathering all available research on a particular topic, which may or may not include a quantitative pooling (meta-analysis) as part of the process. Thus, systematic overview is a more general term.

The Cochrane Collaboration uses a fairly rigid set of guidelines, again developed by medical professionals, to insure uniformity and a high level of quality for these systematic reviews.

The Cochrane Collaboration was named in honor of a famous physician, Archie Cochrane, who argued that

"because resources would always be limited, they should be used to provide equitably those forms of health care which had been shown in properly designed evaluations to be effective. In particular, he stressed the importance of using evidence from randomised controlled trials (RCT's) because these were likely to provide much more reliable information than other sources of evidence." (Source: www.cochrane.org/docs/archieco.htm)

Post modern critique of EBP. Any movement that makes major changes to the way that health care is provided is going to have its share of supporters and critics. But recently, EBP has come under a vigorous criticism assault from writers who apply post-modern techniques in their criticism. I'm a big fan of both EBP and post-modern philosophy, though both groups, particularly the latter, will sometimes take their arguments to ridiculous extremes. I also see EBP as a logical tool for investigating and validating some of the social, political, and historical influences on the process of combing best available evidence with clinician knowledge and patient values.

Some critics attempt to deconstruct EBP with a sense of dispassion

"more useful than either arguing for or against it [EBP], is to understand the policy background and sociological reasons for its emergence and spread" (Traynor 2002)

"postmodernism fundamentally challenges the apparent 'objectivity' of evidence-based practice but it does not challenge the fundamental rules for acquiring and testing evidence. Rather it is the selection of questions to be asked and answered by evidence-based practice/practitioners that is the true limitation. This is the ground upon which fruitful argument can be had about the significance of evidence without undermining the requirement that there be evidence and standards to judge such evidence." (Griffiths 2005)

Others are harshly critical. The most vicious of these assaults (Holmes 2006) describes EBP as "microfascism," an "ossifying discourse," and a "hegemony" that calls for "vigilant resistance." This criticism misses the mark and totally mischaracterizes the EBP movement. This criticism does, however, provide an opportunity to understand how EBP fits into a post-modern view of the world.

The heart of the critique is that EBP is "dangerously exclusionary" because it relies on rules developed by the Cochrane Collaboration. And the Cochrane Collaboration (according to Holmes 2006), will accept only randomized clinical trials and therefore rejects 98% of the available evidence.

The charge of fascism. The analogy between EBP and fascism deserves to be put aside quickly. The problem with analogies to fascism; they rapidly lose any sense of perspective. The prospect of an insurance company using an EBP argument to deny women under the age of 50 a free mammogram is indeed bad, but can it compare to life under a brutal dictator like Saddam Hussein? You can tell that the authors are nervous about using such a strong term. They immediately qualify the term by pointing out that the "fascism of the masses" (as practiced by Hitler and Mussolini) has been replaced by microfascisms,

"polymorphous intolerances that are revealed in more subtle ways."

But micro doesn't mean less serious. In fact the authors point out that microfascisms are

"less brutal, [but] they are nevertheless more pernicious."

You and I would probably look on charges of fascism with an air of bemusement, but at least one journalist noted the historical work of Archie Cochrane, who fought true fascists rather than inventing imaginary fascists.

"But Archie Cochrane, on the other hand, pioneering epidemiologist, inspiration for the Cochrane Library, a prisoner of war for four years in Nazi Germany, who has, from his abstracted position, probably saved more lives than any doctor you know, might see it differently, since in 1936, he went to Spain to join the International Brigade, and fight the fascists of General Franco. Now, what did you do with your summer holidays?" (Goldacre 2006)

It's unfortunate that the charge of fascism has gained so much prominence, because it immediately polarizes the debate about post-modern philosophy and EBP. There are serious charges hiding behind the nasty language, and it is these charges rather than the harsh rhetoric that deserve a careful review.

What is post-modern philosophy? Postmodern philosophy is difficult to define in a brief form. One common theme among most post-modern writers is a distrust of the grand narrative, an all encompassing system used to identify universal truths (Lyotard 1988). Religious and superstitious beliefs characterize the grand narrative of the pre-modern world. Science is held up as the prime example of the grand narrative of the modern world, though Marxism and Freudian theory can also be cited as examples of grand narratives. In the post-modern world, grand narratives are eschewed. They represent an attempt of those in power to define ways of knowing in an exclusionary way that preserves their power by marginalizing dissenting voices. Postmodern philosophy has been derided as the belief that there are no universal truths, but a fairer characterization is that they believe that there is no single path to establishing truth. Instead, truth is established in a way that is dependent on social context.

Post-modern philosophy is harshly critical towards logical positivism, the belief that

"that all meaningful statements must consist solely of empirically verifiable facts" (ddthesis.wordpress.com/2007/12/13/definition-logical-positivism/)

and empiricism, the belief that

"all hypotheses and theories must be tested against observations of the natural world, rather than resting solely on a priori reasoning, intuition, or revelation" (en.wikipedia.org/wiki/Empiricism).

Post-modern analysis commonly relies on a process called deconstruction for critical analysis. This is a process that

"works to demonstrate how concepts or ideas are contingent upon historical, linguistic, social and political discourses, to name but a few." (Holmes 2006)

It is tempting to use this word negatively as a synonym for destruction , but that oversimplifies the notion of deconstruction (http://jamesfaulconer.byu.edu/deconstr.htm).

Post-modern philosophers have noted that any text or narrative has absences or omissions, not necessarily because the writer is sloppy, but because writing about anything except the utterly trivial will require more space and time to produce a complete account than any author is capable of. Deconstruction examines these absences and omissions. Much like Debussy defined music as the spaces between the notes, post-modern philosophers derive the meaning of a text by the the unstated assumptions that these texts are built upon.

What do post-modern critics want to replace EBP with? Holmes (2004) uses the analogy of the rhizome. A ginger bulb is an example of a rhizhome. The rhizhome is a good analogy for post-modern nursing because

"the rhizome is open at both ends. It has no central or governing structure; it has neither beginning nor end. As a rhizome has no centre, it spreads continuously without beginning or ending and basically exists in a constant state of play. It does not conform to a unidirectional or linear reasoning. The rhizome challenges the sense of a unique direction because it emerges and grows in simultaneous, multiple ways." (Holmes 2004)

"The rhizomatic thought permits to bear ambivalence, allegory, chaos and diversity because the thinker is not attached to an official structure, a rigid pattern, an imposed and straightforward stream of thought. It is postmodern, in its very essence. This type of ‘counter-thought’ offers new possibilities because it does not follow a logic characterized by dichotomy or binary positions."

"the rhizomatic thought would acknowledge, accept and promote multiple discourses within nursing, even if they compete with one another."

"We believe that the existing paralysing ‘nursing order’, as it pertains to knowledge development, must be replaced by ‘nursing chaos’. From chaos will emerge a brand new and fragmented order, one that will dare to tolerate multiplicities of thoughts."

Another post-modern critic uses the term "methodological pluralism."

"Codes for the production of nursing knowledge have been skewed towards knowledge that is statistically verifiable, rupturing the methodological pluralism that the nursing community had previously accepted as suitable for the production of nursing knowledge." (Winch 2002b)

No conspiracy theories, please. First it's worth noting that the Cochrane Collaboration is not a shadowy organization that secretly pulls the strings of the entire research community. Cochrane Collaboration reviews are not immune from criticism (Alderson 2003, Eysenbach 2005). To the extent that the research community has adopted models developed by the Cochrane Collaboration, it is largely out of respect for the quality of work that they produce (Oleson 1998).

Reliance solely on RCTs? Reliance solely on RCTs is indeed exclusionary, but perhaps what is being exercised here is not exclusion but discretion. The belief of many in the EBP movement is that a single well-conducted randomized trial will trump any number of observational studies (research comparisons that do not use randomization).

Is this belief true? The most common argument for the superiority of RCTs is (irony of ironies) anecdotal evidence. There are specific medical therapies (two commonly cited examples are mammary artery ligation and post-menopausal hormone replacement therapy) which that achieved wide acceptance on the basis of observational research but were later discredited by carefully conducted RCTs.

If there is any strong evidence about RCTs, it is evidence that would indicate that RCTs really aren't that much better than observational studies (Concato 2000). While there is still debate in the research community about the meaning of this study, I take it as evidence that the folks who conduct observational studies have learned how to do them with sufficient care that their credibility approaches that of randomized studies. Much of the anecdotal evidence against observational studies comes from decades ago. The mammary artery ligation studies, for example were performed in 1959. Many observational studies back then relied on historical controls and did not incorporate complex adjustments to control for bias.

An article with the hilarious title "Parachute use to prevent death and major trauma related to gravitational challenge: systematic review of randomised controlled trials" mocks the rejection of therapies not studied by randomized trials.

"As with many interventions intended to prevent ill health, the effectiveness of parachutes has not been subjected to rigorous evaluation by using randomised controlled trials. Advocates of evidence based medicine have criticised the adoption of interventions evaluated by using only observational data. We think that everyone might benefit if the most radical protagonists of evidence based medicine organised and participated in a double blind, randomised, placebo controlled, crossover trial of the parachute." (Smith 2003)

All of this would be a scathing indictment of EBP, except for the fact that EBP does not rely solely on RCTs. Holmes (2006) badly misreads one of their sources here. The 98% comes from a quote by David Sackett, in his famous book about EBP (Sackett 2000). Sackett claimed claimed that only 2% of the published research is sound with valid conclusions. If you read the whole book, however, you will see that nowhere does Sackett equate randomized with sound/valid nor observational with unsound/invalid. In fact, a careful reading of this book will show you that there are criteria both for randomized and observational studies that you can use to gauge the validity of either type of study. Sackett does suggest that randomized studies are more appropriate for evaluating efficacy and observational studies for evaluating harm, but even here, it is obviously more a convention to simplify the presentation. Sackett recognizes that randomized trials are sometimes more appropriate for some studies of harm and observational studies are sometimes more appropriate for some studies of efficacy.

"Evidence based medicine is not restricted to randomised trials and meta-analyses. It involves tracking down the best external evidence with which to answer our clinical questions. To find out about the accuracy of a diagnostic test, we need to find proper cross sectional studies of patients clinically suspected of harbouring the relevant disorder, not a randomised trial. For a question about prognosis, we need proper follow up studies of patients assembled at a uniform, early point in the clinical course of their disease. And sometimes the evidence we need will come from the basic sciences such as genetics or immunology. It is when asking questions about therapy that we should try to avoid the non-experimental approaches, since these routinely lead to false positive conclusions about efficacy. Because the randomised trial, and especially the systematic review of several randomised trials, is so much more likely to inform us and so much less likely to mislead us, it has become the "gold standard" for judging whether a treatment does more good than harm. However, some questions about therapy do not require randomised trials (successful interventions for otherwise fatal conditions) or cannot wait for the trials to be conducted. And if no randomised trial has been carried out for our patient's predicament, we must follow the trail to the next best external evidence and work from there." (Sackett 1996)

What evidence does the Cochrane Collaboration accept? Does the Cochrane Collaboration only accept randomized trials? Well yes and no. The Cochrane review relies on the best available evidence. Although there are exceptions to this rule (see below), the Cochrane Collaboration will generally exclude non-randomized studies from the systematic overview if good quality randomized trials are available. If randomized trials are not available, or if they are uniformly flawed, then they use non-randomized evidence. This is not unlike the coffee drinker who always chooses cream over milk for their coffee if both are available, but will tolerate milk as a last resort.

An interesting example where the Cochrane Collaboration did mix randomized and observational data occurs in a systematic overview that examines the benefits of breastfeeding.

"We selected all internally-controlled clinical trials and observational studies comparing child or maternal health outcomes with exclusive breastfeeding for six or more months versus exclusive breastfeeding for at least three to four months with continued mixed breastfeeding until at least six months. Studies were stratified according to study design (controlled trials versus observational studies), provenance (developing versus developed countries), and timing of compared feeding groups (three to seven months versus later)." (Source: www.cochrane.org/reviews/en/ab003517.html

If any group would reject evidence based on observational studies, it would be those very same  post-modern philosophers. A study like the breastfeeding study mentioned above would suffer, from a post-modern perspective, from logical positivism since randomized and observational studies both represent an attempt to apply scientific methods to uncover an objective reality.

Does Cochrane exclude qualitative data? Perhaps the dangerously exclusionary EBP is excluding qualitative research. There are methods to combine qualitative and quantitative research (Jick 1979), but it is unclear how to do this in the context of EBP. Although there are a few systematic overviews that summarize qualitative research studies (such as Munro 2007), this has been a self-acknowledged failing of EBP. In response, the Cochrane Collaboration has developed the Cochrane Qualitative Research Methods Group.

"The Cochrane Qualitative Research Methods Group develops and supports methodological work on the inclusion in systematic reviews of findings from studies using qualitative methods and disseminates this work within and beyond the Collaboration's Review Groups. The Cochrane Qualitative Methods Group focuses on methodological matters arising from the inclusion of findings from qualitative studies into systematic reviews." (Source:  www.joannabriggs.edu.au/cqrmg/)

The role of individual patient preference. Perhaps EBP is dangerously exclusive because is does not allow a role for individual patient preferences. This is belied by the classic definition of EBP which  incorporates individual patient preference, as well as exhortations among EBP proponents not to ignore the individual patient (Sackett 2000). There is an effort to encourage value based medicine, which involves greater consideration of patient values (Brown 2005), but this is not a rejection of EBP but rather incorporates it under a broader umbrella.

"Values-based medicine can incorporate all the other paradigms of medicine, including scientific and evidence-based medicine, within it, because it can include anything that contributes to human security and flourishing." (Source: www.mja.com.au/public/issues/177_06_160902/lit10253_fm.html)

The use of intuition. Unmentioned by Holmes (2006) in this list of other ways of knowing is the word "intuition." But intuition is mentioned by other post-modern critics.

"Methods that are likely to capture the intuitive base of nursing, such as the case study or conversational analysis, are not credible within the current evidence-based practice framework." (Winch 2002).

"Ultimately, the postmodern ironist reader of the research report must make a judgement without criteria, based on her own practical wisdom or 'prudence'."  (Rolfe 2006)

What role should intuition play in the practice of health care? My hunch is that intuition is overrated as a way of knowing. I could cite a whole bunch of empirical studies that criticize our process of intuition, but that research would not be all that persuasive to someone who views intuition as an important way of knowing.

One irreconcilable problem is deciding how to deal with two people offering conflicting intuitions? You can choose the one who agrees with my current intuition (which makes you ask, why did you bother searching for those other intuitive insights) or you could choose the intuitive insight from the person you respect and trust more. This brings us back to the days of eminence based medicine.

I know of several ways to reconcile two conflicting empirical studies. You can split the difference, you can search for a methodological  explanation of the disparity, you can use a third replication to break the tie, or you can select the results from the empirical study that is higher on the research hierarchy. Unlike intuition, empirical data lends itself to resolution of conflicting viewpoints.

The best argument I can make against reliance on intuition comes from a very pragmatic argument. Think back to the last time that you were able to convince a skeptical audience that your viewpoint was correct. Were they persuaded by your powers of intuition? Or did it take something else? There's a famous saying among statisticians:

"In God we trust, all others bring data." (Source: klabs.org/richcontent/Misc_Content/Quotes.htm)

That's an quote that falls decidedly in the empiricist camp, but it rings true in many situations. Most of us do not have the persuasive power of Oprah Winfrey, who send authors to skyrocketing fame and fortune with just a brief mention of their books. The rest of us need to present persuasive arguments and most persuasive arguments rely on data rather than personal intuition. Perhaps the greatest empiricist of the 19th century was Florence Nightingale.

"Florence Nightingale had exhibited a gift for mathematics from an early age and excelled in the subject under the tutorship of her father. She had a special interest in statistics, a field in which her father, a pioneer in the nascent field of epidemiology, was an expert. She made extensive use of statistical analysis in the compilation, analysis and presentation of statistics on medical care and public health. Nightingale was a pioneer in the visual presentation of information. Among other things she used the pie chart, which had first been developed by William Playfair in 1801. After the Crimean War, Nightingale used the polar area chart, equivalent to a modern circular histogram or rose diagram, to illustrate seasonal sources of patient mortality in the military field hospital she managed. Nightingale called a compilation of such diagrams a "coxcomb", but later that term has frequently been used for the individual diagrams. She made extensive use of coxcombs to present reports on the nature and magnitude of the conditions of medical care in the Crimean War to Members of Parliament and civil servants who would have been unlikely to read or understand traditional statistical reports. In her later life Nightingale made a comprehensive statistical study of sanitation in Indian rural life and was the leading figure in the introduction of improved medical care and public health service in India. In 1859 Nightingale was elected the first female member of the Royal Statistical Society and she later became an honorary member of the American Statistical Association." (Source: en.wikipedia.org/wiki/Florence_Nightingale)

Florence Nightingale made major changes in health care in a time when women's voices were routinely ignored and when the profession of nursing was in its infancy. She couldn't make any serious headway by making intuitive arguments, but had to present hard facts. Today's world is different, of course, but the persuasive nature of data still remains strong.

But the point to remember is the EBP does not force you to abandon your intuition. Your intuition is part of your clinical judgment, and clinical judgment is explicitly acknowledged in the definition of EBP. I wouldn't want it any other way. If all of the research said that a certain therapy was good for you, but your doctor or nurse had a nagging suspicion that you were an exception, I'd want to hear about it, wouldn't you?

Good doctors use both individual clinical expertise and the best available external evidence, and neither alone is enough. Without clinical expertise, practice risks becoming tyrannised by evidence, for even excellent external evidence may be inapplicable to or inappropriate for an individual patient. Without current best evidence, practice risks becoming rapidly out of date, to the detriment of patients. (Sackett 1996)

Does EBP allow outsiders to dictate to the nursing profession? Some postmodern critics see EBP as a way for outsiders to meddle in the practice of nursing.

"If nursing was to marginalize itself within initiatives promoting evidence-based practice ..., the possibility exists that research on nursing practice would come to be led by others." (Bonnell 1999).

In contrast, other post-modern philosophers criticize such a concern as linear (the ultimate post-modern insult) and prescriptive and correctly point out that a true post-modern perspective would encourage

"transdisciplinarity, diversity and plurality." (Holmes 2004)

After all, you can't complain about dangerous exclusions out of one side of your mouth while excluding other voices from consideration because they are not nursing voices.

This fear of outsider influence is not new. When the practice of meta-analysis became popular in the mid 1990's there was a surge of resentment among doctors because all their hard work designing and running clinical trials became a mere piece of data in the meta-analysis that often included (horror of horrors!) a statistician as the lead author.

"Meta-analysis has provoked acrimony in every discipline-from psychology to physics- where it has been applied. 'To some people,' say Richard Kronmal, a biostatistician at the University of Washington, 'it seems like little more than an attempt by statisticians to put themselves on the top of the totem pole. Individual researchers with their individual experiments see themselves reduced to becoming a cog in the great statistical wheel. And they're saying, well, no, that's not how science works.'" (Mann 1990)

This resentment subsided when the statisticians recognized that they could not produce credible meta-analysis without substantial medical input, and the process of producing meta-analytic studies became a cross-disciplinary team effort.

Is EBP a surrogate for cost containment at the expense of quality of care?

There is certainly a lot of concern about how EBP is a covert way of introducing cost containment into the health care system. One postmodern critic of EBP argues that it

"is driven by the economic imperatives of the healthcare system." (Winch 2002b)

It is certainly debatable whether EBP encourages cost reduction at the expense of quality health care, but if this is happening, this clearly is an abuse of the EBP methodology.

Some fear that evidence based medicine will be hijacked by purchasers and managers to cut the costs of health care. This would not only be a misuse of evidence based medicine but suggests a fundamental misunderstanding of its financial consequences. Doctors practising evidence based medicine will identify and apply the most efficacious interventions to maximise the quality and quantity of life for individual patients; this may raise rather than lower the cost of their care. (Sackett 1996)

Deconstructing EBP using EBP. The post-modern criticisms that EBP excludes observational data, denigrates qualitative data, ignores individual patient preference, or substitutes cost containment for quality health care are all weak charges. Proponents of EBP have already recognized these issues and have taken steps to address them. It is worth noting that EBP does a better job of deconstructing itself than the postmodern critics have done.

For example, what influence does money have on EBP? Quite a bit. Researchers with financial ties to pharmaceutical companies are more likely to produce publications favorable to that company's products (Stelfox 1998) and have a greater tendency to write conclusions that are not supported by the data. [[Reference needed]]. Published conflicts of interest also influence reader's perceptions of the articles (Schroter 2004).

What influence does the dominance of the English language have on EBP? Articles published in English, compared to articles published in other languages are more likely to produce positive results (Egger 1997). Ignoring non-English language publications is a dangerously exclusionary practice that was discovered through EBP.

What impact does the availability of information on the Internet have on EBP? Articles published with full free text on the web (FUTON) are more often cited than their counterparts without full text on the net. This is known as FUTON bias, and can potentially lead to seriously flawed conclusions (Murali 2004; Wentz 2002).

I noted earlier the evidence based explorations of the relative value of randomized versus observational studies.

Is it fair to use a flawed tool, EBP, to evaluate the flaws of EBP?

There are limitations to any methodology, including EBP. EBP relies on the best available evidence, but the production of evidence is a social process that is influenced by economic considerations, social norms and expectations, political meddling, and so forth. If no one uses women as research subjects, then EBP cannot make intelligent recommendations about the treatment of women. But post-modern critics err when they claim that EBP proponents

"seldom question the authority of their own discourses, but deploy them unknowingly."

or that EBP

entails an inability to appreciate (even tolerate) contrasting ideas and/or 'see a bigger picture'. (De Simone 2006)

These statement is demonstrably false. EBP is a system with built-in self-correction. If EBP produces bad conclusions, one can study this and propose remedies using EBP itself.

In fact, EBP is, to a large extent, self-critical and self-correcting. It's ironic that Holmes (2006) uses words like "hegemonic" to describe EBP. EBP actually disbanded the prior hegenomy of eminence based medicine, a hegemony, it is worth noting, that excluded for the most part the voices of women, the voices of people from developing countries, and the voices of non-physicians.

Today, thanks to EBP, the process of impacting clinical practice is much more democratic.  If you don't like the status quo, conduct your own research and publish it. Unless you conduct this research sloppily, it will get into the next systematic overview. If you don't trust the people conducting systematic overviews, conduct your own systematic overview.

I won't pretend that it's easy to conduct your own systematic overview, but for the record, a systematic overview is scientific in that its methods are transparent and open to replication. The recipe is out there for anyone who has the stamina to follow it.

If you believe that a systematic overview is dangerously exclusionary, replicate it but with a broader set of inclusion criteria. And finally, if you don't like the rules used by the Cochrane Collaboration, study the empirical impact of using a different set of rules. If EBP is as dangerous as claimed in Holmes, then there ought to be plenty of empirical studies out there that could demonstrate this. I hope that the post-modern critics don't think that only non-quantitative tools can be used to critique EBP, because if they did, they would be dangerously exclusionary.

EBP is not perfect, but it is the wholesale rejection of this methodology by postmodern critics that is dangerously exclusionary. The threat to quality health care comes not from EBP, but from the

"association of quantitative/experimental methods with lots of unsavoury philosophical and methodological baggage [that] prevents them from giving these methods a full consideration." (Bonell 1999).

What can EBP learn from post-modern philosophy? While I disagree strongly with many of the published post-modern criticisms of EBP, I still think that there is much that can be gained by viewing EBP from a post-modern perspective.

It is a mistake, for example, to presume that a systematic overview provides objective results.  The Cochrane Collaboration, for example, produced a systematic overview of the use of mammography screening (Issue 4, 2001, www.cochranelibrary.net). The original review examined a small number studies and rejected others for poor methodological quality. The summary of these studies showed that mammograms did not produce an overall benefit to these women. Dissenting voices claimed that the exclusion of additional studies was wrong and when this data was combined with the studies used in the Cochrane analysis, mammography was shown to be justified (Olson 2001). So which result is correct? You can argue back and forth, but I would point out that both approaches have merit and that if two reasonable competing approaches produce conflicting results, the only safe conclusion is that the research base is ambiguous. One commentary on this controversy notes that

"even when scientists tried very hard to be rigorous and methodologically sound they brought some subjectivity into their work. "Despite all the efforts we make even when we undertake rigorous systematic reviews, interpretations may differ. Different people faced with the same raw data will not necessarily come to the same conclusion," Professor Liberati said." (Mayor 2001)

and a Lancet editorial on the controversy points out that

"even in the best organisations raw evidence alone is sometimes insufficient to influence opinion." (Horton 2001)

If one systematic overview produces such difficulty, it is safe to conclude that others may as well. I do believe that most systematic overviews provide a sufficient approximation to reality that they can be treated as objective. But one should never neglect the potential for ambiguity, even in an open, transparent, and repeatable process like a systematic overview.

A commentary on a different controversy reminds us that the process of producing a systematic overview is not so simple that you just turn the crank and out pops the result.

In the end there is no escape from a return to "the expert," who tells us which trial to believe, not only on the basis of methodology but also on the basis of insights in pathophysiology, pharmacology, and perhaps type of publication (supplements, special interest or "throw away" journals, etc). All that we can ask from the expert is a careful explanation of what arguments he or she used in accepting or dismissing the evidence from certain trials. Jan P Vandenbroucke, Experts’ views are still needed, BMJ 1998;316:469 (7 February)

There is also a mistaken belief among some that if you can reduce something to a quantitative value, you have produced an objective evaluation. There is a famous quote by Lord Kelvin,

"In physical science the first essential step in the direction of learning any subject is to find principles of numerical reckoning and practicable methods for measuring some quality connected with it. I often say that when you can measure what you are speaking about, and express it in numbers, you know something about it; but when you cannot measure it, when you cannot express it in numbers, your knowledge is of a meagre and unsatisfactory kind; it may be the beginning of knowledge, but you have scarcely in your thoughts advanced to the state of Science, whatever the matter may be." (Source: zapatopi.net/kelvin/quotes/).

and another by Leonardo da Vinci

"No human investigation can be called real science if it cannot be demonstrated mathematically."

but as much as I would love to believe these quotes, I know in my heart that I can't. I always have to remind myself of the competing quote,

"The government is very keen on amassing statistics. They collect them, add them, raise them to the nth power, take the cube root and prepare wonderful diagrams. But you must never forget that every one of these figures comes in the first instance from the village watchman, who just puts down what he damn well pleases." Sir Josiah Stamp (Source: www.trends.net/~roversct/Quotes/humour.html)


Not everything that can be counted counts, and not everything that counts can be counted. - Albert Einstein

The tendency to view a statistic as an objective measure is best tempered by reading the excellent book "Damned Lies and Statistics. Untangling Numbers from the Media, Politicians, and Activists." (Best 2001).

"Every statistic must be created, and the process of creation always involves choices that affect the resulting number and therefore affect what we understand after the figures summarize and simplify the problem. People who create statistics must choose definitions-they must define what it is they want to count-and they must choose their methods-the ways they will go about their counting. Those choices shape every good statistic, and every bad one. Bad statistics simplify reality in ways that distort our understanding, while good statistics minimize that distortion. No statistic is perfect, but some are less imperfect than others. Good or bad, every statistic reflects its creators' choices." (Best 2001, page 161)

Finally, while EBP is a good methodology, it is always subject to being hijacked by groups with hidden agendas. There is a term

Sackettisation ... the artificial linkage of a publication to the evidence-based medicine movement in order to improve sales. bmj.bmjjournals.com/cgi/content/full/320/7244/1283

that can equally apply for people with other motivations.

Conclusion. The post-modern criticism of Evidence Based Practice is grossly overstated. EBP has flaws and an appreciation of post-modern philosophy can help you recognize these flaws. But EBP is a vast improvement over the previous reliance on small group of experts. EBP is broadly democratic. It is open, transparent, and repeatable. It is able to recognize its own flaws, and can take actions to correct itself.


Phil Alderson, Iain Chalmers. Survey of claims of no effect in abstracts of Cochrane reviews. BMJ. 2003;326(7387):475. Excerpt: "It is never correct to claim that treatments have no effect or that there is no difference in the effects of treatments. It is impossible to prove a negative or that two treatments have the same effect. There will always be some uncertainty surrounding estimates of treatment effects, and a small difference can never be excluded." [Accessed October 26, 2010]. Available at: http://www.bmj.com/content/326/7387/475.short.

Joel Best. Damned Lies and Statistics: Untangling Numbers from the Media, Politicians, and Activists. 1st ed. University of California Press; 2001. Description from the publisher's website: "Does the number of children gunned down double each year? Does anorexia kill 150,000 young women annually? Do white males account for only a sixth of new workers? Startling statistics shape our thinking about social issues. But all too often, these numbers are wrong. This book is a lively guide to spotting bad statistics and learning to think critically about these influential numbers. Damned Lies and Statistics is essential reading for everyone who reads or listens to the news, for students, and for anyone who relies on statistical information to understand social problems." Available at: http://www.ucpress.edu/book.php?isbn=9780520219786.

C Bonell. Evidence-based nursing: a stereotyped view of quantitative and experimental research could work against professional autonomy and authority. J Adv Nurs. 1999;30(1):18-23. Abstract: "In recent years, there have been calls within the United Kingdom's National Health Service (NHS) for evidence-based health care. These resonate with long-standing calls for nursing to become a research-based profession. Evidence-based practice could enable nurses to demonstrate their unique contribution to health care outcomes, and support their seeking greater professionalization, in terms of enhanced authority and autonomy. Nursing's professionalization project, and, within this, various practices comprising the 'new nursing', whilst sometimes not delivering all that was hoped of them, have been important in developing certain conditions conducive to developing evidence-based practice, notably a critical perspective on practice and a reluctance merely to follow physicians' orders. However, nursing has often been hesitant in its adoption of quantitative and experimental research. This hesitancy, it is argued, has been influenced by the propounding by some authors within the new nursing of a stereotyped view of quantitative/experimental methods which equates them with a number of methodological and philosophical points which are deemed, by at least some of these authors, as inimical to, or problematic within, nursing research. It is argued that, not only is the logic on which the various stereotyped views are based flawed, but further, that the wider influence of these viewpoints on nurses could lead to a greater marginalization of nurses in research and evidence-based practice initiatives, thus perhaps leading to evidence-based nursing being led by other groups. In the longer term, this might result in a form of evidence-based nursing emphasizing routinization, thus--ironically--working against strategies of professional authority and autonomy embedded in the new nursing. Nursing research should instead follow the example of nurse researchers who already embrace multiple methods. While the paper describes United Kingdom experiences and debates, points raised about the importance of questioning stereotyped views of research should have international relevance." [Accessed October 26, 2010]. Available at: http://www.ncbi.nlm.nih.gov/pubmed/10403976.

Melissa M. Brown, Gary C. Brown, Sanjay Sharma. Evidence-Based To Value-based Medicine. 1st ed. American Medical Association Press; 2005. Excerpt: "Medical research and practice is currently evolving from evidence-based medicine to an even higher quality of patient care: value based medicine. Value-based medicine, which measures the patient-perceived value and integrates relevant costs provided by healthcare interventions, allows a more accurate measure of the overall worth of interventions to a patient and other stakeholders. Evidence-Based to Value-Based Medicine explains this evolution and explains the uses and practice of value-based medicine in today's healthcare environment. Through detailed explanations about how to integrate value-based medicine and macroeconomic issues, this resource will teach practitioners how to deliver a higher quality of clinical care and to measure and demonstrate the value of their interventions to patients, policymakers, and payers. Also included are innovative input variable methodologies, a critical appraisal of health-related quality-of-life instruments, and an overview of healthcare economics. " Available at: https://catalog.ama-assn.org/Catalog/product/product_detail.jsp?productId=prod240179.

John Concato, Nirav Shah, Ralph I. Horwitz. Randomized, Controlled Trials, Observational Studies, and the Hierarchy of Research Designs. N Engl J Med. 2000;342(25):1887-1892. Background In the hierarchy of research designs, the results of randomized, controlled trials are considered to be evidence of the highest grade, whereas observational studies are viewed as having less validity because they reportedly overestimate treatment effects. We used published meta-analyses to identify randomized clinical trials and observational studies that examined the same clinical topics. We then compared the results of the original reports according to the type of research design. Methods A search of the Medline data base for articles published in five major medical journals from 1991 to 1995 identified meta-analyses of randomized, controlled trials and meta-analyses of either cohort or case-control studies that assessed the same intervention. For each of five topics, summary estimates and 95 percent confidence intervals were calculated on the basis of data from the individual randomized, controlled trials and the individual observational studies. Results For the five clinical topics and 99 reports evaluated, the average results of the observational studies were remarkably similar to those of the randomized, controlled trials. For example, analysis of 13 randomized, controlled trials of the effectiveness of bacille Calmette-Guerin vaccine in preventing active tuberculosis yielded a relative risk of 0.49 (95 percent confidence interval, 0.34 to 0.70) among vaccinated patients, as compared with an odds ratio of 0.50 (95 percent confidence interval, 0.39 to 0.65) from 10 case-control studies. In addition, the range of the point estimates for the effect of vaccination was wider for the randomized, controlled trials (0.20 to 1.56) than for the observational studies (0.17 to 0.84). Conclusions The results of well-designed observational studies (with either a cohort or a case-control design) do not systematically overestimate the magnitude of the effects of treatment as compared with those in randomized, controlled trials on the same topic. [Accessed March 7, 2009]. Available at: http://content.nejm.org/cgi/content/abstract/342/25/1887.

John De Simone. Reductionist inference-based medicine, i.e. EBM. J Eval Clin Pract. 2006;12(4):445-449. RATIONALE, AIMS AND OBJECTIVES: Unbeknown to many, reductionist and postmodern worldviews competitively coexist in science and society. The debate on evidence-based medicine (EBM) is at the tip of this 'iceberg'. Via systems thinking and complexity science EBM reveals crucial flaws and its reductionism entails an inability to appreciate (even tolerate) contrasting ideas and/or 'see a bigger picture'. An interdisciplinary approach provides insight into novel explanations. Thereafter, the conceptual barrier shifts to communication, a challenge which mandates attempts to steer the discourse by reframing the debate. METHOD: Interdisciplinary perspectives serve to illustrate a 'bigger picture'. Also, 'wicked' questions stimulate reflection, discern leverage points and dismantle resilient defences. Lastly, a proposal: exploring the value of 'glasses half full'. CONCLUSION: Some may realize that postmodern concepts behind compelling criticisms to EBM have already taken root, being shared by policymakers, practitioners and patients as well. [Accessed October 26, 2010]. Available at: http://www.ncbi.nlm.nih.gov/pubmed/16907688.

M Egger, T Zellweger-Zähner, M Schneider, et al. Language bias in randomised controlled trials published in English and German. Lancet. 1997;350(9074):326-329. Abstract: "BACKGROUND: Some randomised controlled trials (RCTs) done in German-speaking Europe are published in international English-language journals and others in national German-language journals. We assessed whether authors are more likely to report trials with statistically significant results in English than in German. METHODS: We studied pairs of RCT reports, matched for first author and time of publication, with one report published in German and the other in English. Pairs were identified from reports round in a manual search of five leading German-language journals and from reports published by the same authors in English found on Medline. Quality of methods and reporting were assessed with two different scales by two investigators who were unaware of authors' identities, affiliations, and other characteristics of trial reports. Main study endpoints were selected by two investigators who were unaware of trial results. Our main outcome was the number of pairs of studies in which the levels of significance (shown by p values) were discordant. FINDINGS: 62 eligible pairs of reports were identified but 19 (31%) were excluded because they were duplicate publications. A further three pairs (5%) were excluded because no p values were given. The remaining 40 pairs were analysed. Design characteristics and quality features were similar for reports in both languages. Only 35% of German-language articles, compared with 62% of English-language articles, reported significant (p < 0.05) differences in the main endpoint between study and control groups (p = 0.002 by McNemar's test). Logistic regression showed that the only characteristic that predicted publication in an English-language journal was a significant result. The odds ratio for publication of trials with significant results in English was 3.75 (95% CI 1.25-11.3). INTERPRETATION: Authors were more likely to publish RCTs in an English-language journal if the results were statistically significant. English language bias may, therefore, be introduced in reviews and meta-analyses if they include only trials reported in English. The effort of the Cochrane Collaboration to identify as many controlled trials as possible, through the manual search of many medical journals published in different languages will help to reduce such bias." [Accessed May 19, 2010]. Available at: http://www.ncbi.nlm.nih.gov/pubmed/9251637.

Gunther Eysenbach, Per Egil Kummervold. "Is Cybermedicine Killing You?" - The Story of a Cochrane Disaster. J Med Internet Res. 2005;7(2):e21. Abstract: "This editorial briefly reviews the series of unfortunate events that led to the publication, dissemination, and eventual retraction of a flawed Cochrane systematic review on interactive health communication applications (IHCAs), which was widely reported in the media with headlines such as "Internet Makes Us Sick," "Knowledge May Be Hazardous to Web Consumers' Health," "Too Much Advice Can Be Bad for Your Health," "Click to Get Sick?" and even "Is Cybermedicine Killing You?" While the media attention helped to speed up the identification of errors, leading to a retraction of the review after only 13 days, a paper published in this issue of JMIR by Rada shows that the retraction, in contrast to the original review, remained largely unnoticed by the public. We discuss the three flaws of the review, which include (1) data extraction and coding errors, (2) the pooling of heterogeneous studies, and (3) a problematic and ambiguous scope and, possibly, some overlooked studies. We then discuss "retraction ethics" for researchers, editors/publishers, and journalists. Researchers and editors should, in the case of retractions, match the aggressiveness of the original dissemination campaign if errors are detected. It is argued that researchers and their organizations may have an ethical obligation to track down journalists who reported stories on the basis of a flawed study and to specifically ask them to publish an article indicating the error. Journalists should respond to errors or retractions with reports that have the same prominence as the original story. Finally, we look at some of the lessons for the Cochrane Collaboration, which include (1) improving the peer-review system by routinely sending out pre-prints to authors of the original studies, (2) avoiding downplay of the magnitude of errors if they occur, (3) addressing the usability issues of RevMan, and (4) making critical articles such as retraction notices open access." [Accessed October 26, 2010]. Available at: http://www.jmir.org/2005/2/e21/.

Ben Goldacre. Objectionable 'objectives'. The Guardian. 2006. Excerpt: "Deconstructing the evidence-based discourse in health sciences: truth, power and fascism. Even from looking at the title, you just know this academic paper from the September edition of the International Journal of Evidence-based Healthcare is going to be an absolute corker. And it uses the word "fascist" (or elaborate derivatives) 28 times in six pages, which even Rik Mayall in The Young Ones might have described as "overdoing it"." [Accessed October 26, 2010]. Available at: http://www.guardian.co.uk/science/2006/aug/19/badscience.uknews.

Dave Holmes, Amélie Perron, Gabrielle Michaud. Nursing in corrections: lessons from France. J Forensic Nurs. 2007;3(3-4):126-131. Abstract: "This article presents the results of a qualitative study (grounded theory) comparing nursing practice in corrections in both France and Canada. In Canada, nurses work as both agents of care and agents of social control. In contrast, French legislation has separated the responsibilities of health care and corrections. The effects of this split are illustrated, and the French model examine for important perspectives for restructuring nursing services in Canada." [Accessed October 26, 2010]. Available at: http://www.ncbi.nlm.nih.gov/pubmed/18027532.

Dave Holmes, Dan Warner. The anatomy of a forbidden desire: men, penetration and semen exchange. Nurs Inq. 2005;12(1):10-20. Abstract: "The rising popularity of unprotected anal sex (bareback sex) among men who have sex with men (MSM) is perplexing healthcare providers working in sexual health clinics. Epidemiological research on the topic overlooks several socio-cultural and psychological dimensions. Our research attempts to construct an appropriate theoretical edifice by which we can understand this sexual practice. In order to achieve this objective, a qualitative design was selected and 18 semiconductive in-depth interviews were carried out with barebackers from five European and North American cities. We then analyzed the data using two theoretical approaches that were sensitive to the issues of desire, transgression and pleasure. These theories are those of the late French psychoanalyst, Jacques Lacan, and those of poststructural thinkers, Gilles Deleuze and Felix Guattari. These theoretical frameworks helped shed light on the significance of bareback sex, and can potentially influence healthcare providers in gaining a better understanding not only of their clients, but also of their own role in the circuitry of desire at work within bareback. We found that while the exchange of semen constitutes a dangerous and irrational practice to healthcare professionals, it is nevertheless a significant variable in the sexual lives of barebackers that needs to be taken into consideration in the provision of healthcare services." [Accessed October 26, 2010]. Available at: http://www.ncbi.nlm.nih.gov/pubmed/15743438.

David Isaacs, Dominic Fitzgerald. Seven alternatives to evidence based medicine. BMJ. 1999;319(7225):1618. Excerpt: "We, two humble clinicians ever ready for advice and guidance, asked our colleagues what they would do if faced with a clinical problem for which there are no randomised controlled trials and no good evidence We found ourselves faced with several personality based opinions, as would be expected in a teaching hospital. The personalities transcend the disciplines, with the exception of surgery, in which discipline transcends personality. We categorised their replies, on the basis of no evidence whatsoever, as follows." [Accessed October 26, 2010]. Available at: http://www.bmj.com/content/319/7225/1618.short.

Todd D. Jick. Mixing Qualitative and Quantitative Methods: Triangulation in Action.. Administrative Science Quarterly. 1979;24(4):602-11. Excerpt: "There is a distinct tradition in the literature on social science research methods that advocates the use of multiple methods. This form of research strategy is usually described as one of convergent methodology, multimethod/multitrait (Campbell and Fiske, 1959), convergent validation or, what has been called "triangulation" (Webb et al., 1 966). These various notions share the conception that qualitative and quantitative methods should be viewed as complementary rather than as rival camps. In fact, most textbooks under- score the desirability of mixing methods given the strengths and weaknesses found in single method designs." Available at: http://www.jstor.org/stable/pdfplus/2392366.pdf.

Salla A Munro, Simon A Lewin, Helen J Smith, et al. Patient Adherence to Tuberculosis Treatment: A Systematic Review of Qualitative Research. PLoS Med. 2007;4(7):e238. Excerpt: "From a systematic review of qualitative research, Munro and coauthors found that a range of interacting factors can lead to patients deciding not to complete their course of tuberculosis treatment." [Accessed October 26, 2010]. Available at: http://dx.doi.org/10.1371/journal.pmed.0040238.

Narayana S Murali, Hema R Murali, Paranee Auethavekiat, et al. Impact of FUTON and NAA bias on visibility of research. Mayo Clin. Proc. 2004;79(8):1001-1006. Abstract: "OBJECTIVE: To determine whether availability of journals on MEDLINE as FUTON (full text on the Net) affects their impact factor. MATERIAL AND METHODS: A comprehensive search identified 324 cardiology, nephrology, and rheumatology/immunology journals on-line until May 2003. The status of these journals was ascertained in MEDLINE as having FUTON, abstracts only, and NAA (no abstract available). Impact factors for all available journals from the Institute for Scientific Information (ISI) were abstracted. RESULTS: Of the 324 Journals, 124 (38.3%) were FUTON, 138 (42.6%) had abstracts only, and 62 (19.1%) had NAA. The mean (+/-SEM) impact factor was 3.24 (+/-0.32), 1.64 (+/-0.30), and 0.14 (+/-0.45), respectively. Of the 324 current journals, 159 existed in both the pre- and the post-Internet era. An analysis of the change (ie, delta) in impact factor from the pre- to post-Internet era revealed a trend between journals with FUTON and abstracts only (P=.17, Wilcoxon rank sum test). Similar analyses of the delta of cardiology journals revealed a statistically significant difference between Journals with FUTON and abstracts only (P=.04, Wilcoxon rank sum test). CONCLUSION: FUTON bias is the tendency to peruse what is more readily available. This is the first study to show that on-line availability of medical literature may increase the impact factor and that such increase tends to be greater in FUTON journals. Failure to consider this bias may affect a journal's impact factor. Also, it could limit consideration of medical literature by ignoring relevant NAA articles and thereby influence medical education akin to publication or language bias." [Accessed May 19, 2010]. Available at: http://www.mayoclinicproceedings.com/content/79/8/1001.abstract.

Ole Olsen, Philippa Middleton, Jeanette Ezzo, et al. Quality of Cochrane reviews: assessment of sample from 1998. BMJ. 2001;323(7317):829 -832. Abstract: "Objective: To assess the quality of Cochrane reviews. Design: Ten methodologists affiliated with the Cochrane Collaboration independently examined, in a semistructured way, the quality of reviews first published in 1998. Each review was assessed by two people; if one of them noted any major problems, they agreed on a common assessment. Predominant types of problem were categorised. Setting: Cyberspace collaboration coordinated from the Nordic Cochrane Centre. Studies: All 53 reviews first published in issue 4 of the Cochrane Library in 1998. Main outcome measure: Proportion of reviews with various types of major problem. Results: No problems or only minor ones were found in most reviews. Major problems were identified in 15 reviews (29%). The evidence did not fully support the conclusion in nine reviews (17%), the conduct or reporting was unsatisfactory in 12 reviews (23%), and stylistic problems were identified in 12 reviews (23%). The problematic conclusions all gave too favourable a picture of the experimental intervention. Conclusions: Cochrane reviews have previously been shown to be of higher quality and less biased on average than other systematic reviews, but improvement is always possible. The Cochrane Collaboration has taken steps to improve editorial processes and the quality of its reviews. Meanwhile, the Cochrane Library remains a key source of evidence about the effects of healthcare interventions. Its users should interpret reviews cautiously, particularly those with conclusions favouring experimental interventions and those with many typographical errors." [Accessed October 26, 2010]. Available at: http://www.bmj.com/content/323/7317/829.abstract.

Amélie Perron, Carol Fluet, Dave Holmes. Agents of care and agents of the state: bio-power and nursing practice. J Adv Nurs. 2005;50(5):536-544. Abstract: "AIM: This paper presents a conceptual analysis of the concept of bio-power in the context of nursing, including a critique of the widespread rhetoric that nursing is deprived of power and consequently is an apolitical agency. BACKGROUND: Traditionally, power tends to be defined in terms of repression, interdiction and punishment. On the contrary, work by Michel Foucault with regard to bio-power brings into evidence the productive and positive nature of power at the heart of society. Despite being often used by various academic and professional disciplines, the concept of bio-power is rarely cited in nursing. FINDINGS: Nursing as a profession is at the heart of bio-power in that nurses lie at the crossroads between the anatomo-political and bio-political ranges of power over life. They therefore contribute to social regulation through a vast array of diverse political technologies. Nurses are at the flexing point of the state's requirements and of individual and collective aspirations. They occupy a strategic position that allows them to act as instruments of governmentality. Consequently, nurses constitute a fully-fledged political entity making use of disciplinary technologies and responding to state ideologies. CONCLUSION: The concept of bio-power offers a rich theoretical perspective for nursing, as it questions the definition of nursing care as neutral and mainly provided according to patients' best interests." [Accessed October 26, 2010]. Available at: http://onlinelibrary.wiley.com/doi/10.1111/j.1365-2648.2005.03432.x/full.

Amélie Perron, Carol Fluet, Dave Holmes. Agents of care and agents of the state: bio-power and nursing practice. J Adv Nurs. 2005;50(5):536-544. Abstract: "AIM: This paper presents a conceptual analysis of the concept of bio-power in the context of nursing, including a critique of the widespread rhetoric that nursing is deprived of power and consequently is an apolitical agency. BACKGROUND: Traditionally, power tends to be defined in terms of repression, interdiction and punishment. On the contrary, work by Michel Foucault with regard to bio-power brings into evidence the productive and positive nature of power at the heart of society. Despite being often used by various academic and professional disciplines, the concept of bio-power is rarely cited in nursing. FINDINGS: Nursing as a profession is at the heart of bio-power in that nurses lie at the crossroads between the anatomo-political and bio-political ranges of power over life. They therefore contribute to social regulation through a vast array of diverse political technologies. Nurses are at the flexing point of the state's requirements and of individual and collective aspirations. They occupy a strategic position that allows them to act as instruments of governmentality. Consequently, nurses constitute a fully-fledged political entity making use of disciplinary technologies and responding to state ideologies. CONCLUSION: The concept of bio-power offers a rich theoretical perspective for nursing, as it questions the definition of nursing care as neutral and mainly provided according to patients' best interests." [Accessed October 26, 2010]. Available at: http://onlinelibrary.wiley.com/doi/10.1111/j.1365-2648.2005.03432.x/full.

Sackett DL. Pronouncements about the need for "generalizability" of randomized controlled trial results are humbug. Control. Clinical Trials 2000: 21; 82S.

Gordon C S Smith, Jill P Pell. Parachute use to prevent death and major trauma related to gravitational challenge: systematic review of randomised controlled trials. BMJ. 2003;327(7429):1459 -1461. Objectives To determine whether parachutes are effective in preventing major trauma related to gravitational challenge. Design: Systematic review of randomised controlled trials. Data sources: Medline, Web of Science, Embase, and the Cochrane Library databases; appropriate internet sites and citation lists. Study selection: Studies showing the effects of using a parachute during free fall. Main outcome measure: Death or major trauma, defined as an injury severity score > 15. Results: We were unable to identify any randomised controlled trials of parachute intervention. Conclusions: As with many interventions intended to prevent ill health, the effectiveness of parachutes has not been subjected to rigorous evaluation by using randomised controlled trials. Advocates of evidence based medicine have criticised the adoption of interventions evaluated by using only observational data. We think that everyone might benefit if the most radical protagonists of evidence based medicine organised and participated in a double blind, randomised, placebo controlled, crossover trial of the parachute." [Accessed October 26, 2010]. Available at: http://www.bmj.com/content/327/7429/1459.abstract.

H T Stelfox, G Chua, K O'Rourke, A S Detsky. Conflict of interest in the debate over calcium-channel antagonists. N. Engl. J. Med. 1998;338(2):101-106. Abstract: "BACKGROUND: Physicians' financial relationships with the pharmaceutical industry are controversial because such relationships may pose a conflict of interest. It is unknown to what extent industry support of medical education and research influences the opinions and behavior of clinicians and researchers. The recent debate over the safety of calcium-channel antagonists provided an opportunity to examine the effect of financial conflicts of interest. METHODS: We searched the English-language medical literature published from March 1995 through September 1996 for articles examining the controversy about the safety of calcium-channel antagonists. Articles were reviewed and classified as being supportive, neutral, or critical with respect to the use of calcium-channel antagonists. The authors of the articles were asked about their financial relationships with both manufacturers of calcium-channel antagonists and manufacturers of competing products (i.e., beta-blockers, angiotensin-converting-enzyme inhibitors, diuretics, and nitrates). We examined the authors' published positions on the safety of calcium-channel antagonists according to their financial relationships with pharmaceutical companies. RESULTS: Authors who supported the use of calcium-channel antagonists were significantly more likely than neutral or critical authors to have financial relationships with manufacturers of calcium-channel antagonists (96 percent, vs. 60 percent and 37 percent, respectively; P<0.001). Supportive authors were also more likely than neutral or critical authors to have financial relationships with any pharmaceutical manufacturer, irrespective of the product (100 percent, vs. 67 percent and 43 percent, respectively; P< 0.001). CONCLUSIONS: Our results demonstrate a strong association between authors' published positions on the safety of calcium-channel antagonists and their financial relationships with pharmaceutical manufacturers. The medical profession needs to develop a more effective policy on conflict of interest. We support complete disclosure of relationships with pharmaceutical manufacturers for clinicians and researchers who write articles examining pharmaceutical products." [Accessed October 26, 2010]. Available at: http://www.ncbi.nlm.nih.gov/pubmed/9420342.

Sharon E. Straus, Finlay A. McAlister. Evidence-based medicine: a commentary on common criticisms. CMAJ. 2000;163(7):837-841. Abstract: "Discussions about evidence-based medicine engender both negative and positive reactions from clinicians and academics. Ways to achieve evidence-based practice are reviewed here and the most common criticisms described. The latter can be classified as "limitations universal to the practice of medicine," "limitations unique to evidence-based medicine" and "misperceptions of evidence-based medicine." Potential solutions to the true limitations of evidence-based medicine are discussed and areas for future work highlighted." [Accessed October 26, 2010]. Available at: http://www.cmaj.ca/cgi/content/abstract/163/7/837.

Sarah Winch, Debra Creedy, And Wendy Chaboyer. Governing nursing conduct: the rise of evidence-based practice. Nurs Inq. 2002;9(3):156-161. Abstract: "Drawing on the Foucauldian concept of 'governmentality' to analyse the evidence-based movement in nursing, we argue that it is possible to identify the governance of nursing practice and hence nurses across two distinct axes; that of the political (governance through political and economic means) and the personal (governance of the self through the cultivation of the practices required by nurses to put evidence into practice). The evaluation of nursing work through evidence-based reviews provides detailed information that may enable governments to target and instruct nurses regarding their work in the interest of preserving the health of the population as a whole. Political governance of the nursing population becomes possible through centralised discursive mechanisms, such as evidence-based reviews that present nursing practice as an intelligible field whose elements are connected in a more or less systematic manner. The identity of the evidence-based nurse requires the modern nurse to develop new skills and attitudes. Evidence-based nursing is an emerging technology of government that judges nursing research and knowledge and has the capacity to direct nursing practice at both the political and personal level." [Accessed October 26, 2010]. Available at: http://onlinelibrary.wiley.com/doi/10.1046/j.1440-1800.2002.00148.x/full.

This page was written by Steve Simon while working at Children's Mercy Hospital. Although I do not hold the copyright for this material, I am reproducing it here as a service, as it is no longer available on the Children's Mercy Hospital website. Need more information? I have a page with general help resources. You can also browse for pages similar to this one at Category: Critical appraisal.