How to Read a Paper: The Basics of Evidence-Based Medicine (4 page)

A recent large survey by an Australian team looked at 1000 patients treated for the 22 most commonly seen conditions in a primary care setting. The researchers found that whilst 90% of patients received evidence-based care for coronary heart disease, only 13% did so for alcohol dependence [6]. Furthermore, the extent to which any individual practitioner provided evidence-based care varied in the sample from 32% of the time to 86% of the time. These findings suggest room for improvement all round.

Let's take a look at the various approaches that health professionals use to reach their decisions in reality—all of which are examples of what EBM
isn't
.

Decision-making by anecdote

When I was a medical student, I occasionally joined the retinue of a distinguished professor as he made his daily ward rounds. On seeing a new patient, he would enquire about the patient's symptoms, turn to the massed ranks of juniors around the bed, and relate the story of a similar patient encountered a few years previously. ‘Ah, yes. I remember we gave her such-and-such, and she was fine after that’. He was cynical, often rightly, about new drugs and technologies and his clinical acumen was second to none. Nevertheless, it had taken him 40 years to accumulate his expertise, and the largest medical textbook of all—the collection of cases that were outside his personal experience—was forever closed to him.

Anecdote (storytelling) has an important place in clinical practice [7]. Psychologists have shown that students acquire the skills of medicine, nursing and so on by memorising what was wrong with particular patients, and what happened to them, in the form of stories or ‘illness scripts’. Stories about patients are the unit of analysis (i.e. the thing we study) in grand rounds and teaching sessions. Clinicians glean crucial information from patients' illness narratives—most crucially, perhaps, what being ill
means
to the patient. And experienced doctors and nurses rightly take account of the accumulated ‘illness scripts’ of all their previous patients when managing subsequent patients. But that doesn't mean simply doing the same for patient B as you did for patient A if your treatment worked, and doing precisely the opposite if it didn't!

The dangers of decision-making by anecdote are well illustrated by considering the risk–benefit ratio of drugs and medicines. In my first pregnancy, I developed severe vomiting and was given the anti-sickness drug prochlorperazine (Stemetil). Within minutes, I went into an uncontrollable and very distressing neurological spasm. Two days later, I had recovered fully from this idiosyncratic reaction, but I have never prescribed the drug since, even though the estimated prevalence of neurological reactions to prochlorperazine is only one in several thousand cases. Conversely, it is tempting to dismiss the possibility of rare but potentially serious adverse effects from familiar drugs—such as thrombosis on the contraceptive pill—when one has never encountered such problems in oneself or one's patients.

We clinicians would not be human if we ignored our personal clinical experiences, but we would be better to base our decisions on the collective experience of thousands of clinicians treating millions of patients, rather than on what we as individuals have seen and felt. Chapter 5 (Statistics for the non-statistician) describes some more objective methods, such as the number needed to treat (NNT), for deciding whether a particular drug (or other intervention) is likely to do a patient significant good or harm.

When the EBM movement was still in its infancy, Sackett emphasised that evidence-based practice was no threat to old-fashioned clinical experience or judgement [1]. The question of
how
clinicians can manage to be both ‘evidence-based’ (i.e. systematically informing their decisions by research evidence) and ‘narrative-based’ (i.e. embodying all the richness of their accumulated clinical anecdotes and treating each patient's problem as a unique illness story rather than as a ‘case of X’) is a difficult one to address philosophically, and beyond the scope of this book. The interested reader might like to look up two articles I've written on this topic [8] [9].

Decision-making by press cutting

For the first 10 years after I qualified, I kept an expanding file of papers that I had ripped out of my medical weeklies before binning the less interesting parts. If an article or editorial seemed to have something new to say, I consciously altered my clinical practice in line with its conclusions. All children with suspected urinary tract infections should be sent for scans of the kidneys to exclude congenital abnormalities, said one article, so I began referring anyone under the age of 16 with urinary symptoms for specialist investigations. The advice was in print, and it was recent, so it must surely replace what had been standard practice—in this case, referring only the small minority of such children who display ‘atypical’ features [10].

This approach to clinical decision-making is still very common. How many doctors do you know who justify their approach to a particular clinical problem by citing the results section of a single published study, even though they could not tell you anything at all about the methods used to obtain those results? Was the trial randomised and controlled (see section ‘Cross-sectional surveys’)? How many patients, of what age, sex and disease severity, were involved (see section ‘Whom is the study about?’)? How many withdrew from (‘dropped out of’) the study, and why (see section ‘Were preliminary statistical questions addressed?’)? By what criteria were patients judged cured (see section ‘Surrogate endpoints’)? If the findings of the study appeared to contradict those of other researchers, what attempt was made to validate (confirm) and replicate (repeat) them (see section ‘Ten questions to ask about a paper that claims to validate a diagnostic or screening test’)? Were the statistical tests that allegedly proved the authors' point appropriately chosen and correctly performed (see Chapter 5)? Has the patient's perspective been systematically sought and incorporated via a shared decision-making tool (see Chapter 16)? Doctors (and nurses, midwifes, medical managers, psychologists, medical students and consumer activists) who like to cite the results of medical research studies have a responsibility to ensure that they first go through a checklist of questions like these (more of which are listed in Appendix 1).

Decision-making by GOBSAT (good old boys sat around a table)

When I wrote the first edition of this book in the mid-1990s, the commonest sort of guideline was what was known as a
consensus statement
—the fruits of a weekend's hard work by a dozen or so eminent experts who had been shut in a luxury hotel, usually at the expense of a drug company. Such ‘GOBSAT (good old boys sat around a table) guidelines’ often fell out of the medical freebies (free medical journals and other ‘information sheets’ sponsored directly or indirectly by the pharmaceutical industry) as pocket-sized booklets replete with potted recommendations and at-a-glance management guides. But who says the advice given in a set of guidelines, a punchy editorial or an amply referenced overview is correct?

Professor Mulrow [11], one of the founders of the science of systematic review (see Chapter 9) showed a few years ago that experts in a particular clinical field are
less
likely to provide an objective review of all the available evidence than a non-expert who approaches the literature with unbiased eyes. In extreme cases, an ‘expert opinion’ may consist simply of the lifelong bad habits and personal press cuttings of an ageing clinician, and a gaggle of such experts would simply multiply the misguided views of any one of them.
Table 1.1
gives examples of practices that were at one time widely accepted as good clinical practice (and which would have made it into the GOBSAT guideline of the day), but which have subsequently been discredited by high-quality clinical trials.

Table 1.1
Examples of harmful practices once strongly supported by ‘expert opinion’

Chapter 9 takes you through a checklist for assessing whether a ‘systematic review of the evidence’ produced to support recommendations for practice or policymaking really merits the description, and Chapter 10 discusses the harm that can be done by applying guidelines that are not evidence-based. It is a major achievement of the EBM movement that almost no guideline these days is produced by GOBSAT!

Decision-making by cost-minimisation

The popular press tends to be horrified when they learn that a treatment has been withheld from a patient for reasons of cost. Managers, politicians and, increasingly, doctors can count on being pilloried when a child with a rare cancer is not sent to a specialist unit in America or a frail old lady is denied a drug to stop her visual loss from macular degeneration. Yet in the real world, all healthcare is provided from a limited budget and it is increasingly recognised that clinical decisions must take into account the economic costs of a given intervention. As Chapter 11 argues, clinical decision-making
purely
on the grounds of cost (‘cost-minimisation’—purchasing the cheapest option with no regard to how effective it is) is generally ethically unjustified, and we are right to object vocally when this occurs.

Expensive interventions should not, however, be justified simply because they are new, or because they ought to work in theory, or because the only alternative is to do nothing—but because they are very likely to save life or significantly improve its quality. How, though, can the benefits of a hip replacement in a 75-year-old be meaningfully compared with that of cholesterol-lowering drugs in a middle-aged man or infertility investigations for a couple in their twenties? Somewhat counter-intuitively, there is no self-evident set of ethical principles or analytical tools that we can use to match limited resources to unlimited demand. As you will see in Chapter 11, the much-derided quality-adjusted life year (QALY), and similar utility-based units are simply attempts to lend some objectivity to the illogical but unavoidable comparison of apples with oranges in the field of human suffering. In the United Kingdom, the National Institute for Health and Care Excellence (see
www.nice.org.uk
) seeks to develop both evidence-based guidelines and fair allocation of NHS resources.

There is one more reason why some people find the term
evidence-based medicine
unpalatable. This chapter has argued that EBM is about coping with change, not about knowing all the answers before you start. In other words, it is not so much about what you have read in the past but about how you go about identifying and meeting your ongoing learning needs and applying your knowledge appropriately and consistently in new clinical situations. Doctors who were brought up in the old school style of never admitting ignorance may find it hard to accept that a major element of scientific uncertainty exists in practically every clinical encounter, although in most cases, the clinician fails to identify the uncertainty or to articulate it in terms of an answerable question (see next section). If you are interested in the research evidence on doctors' [lack of] questioning behaviour, see an excellent review by Swinglehurst [13].

The fact that none of us—not even the cleverest or most experienced—can answer all the questions that arise in the average clinical encounter means that the ‘expert’ is more fallible than he or she was traditionally cracked up to be. An evidence-based approach to ward rounds may turn the traditional medical hierarchy on its head when the staff nurse or junior doctor produces new evidence that challenges what the consultant taught everyone last week. For some senior clinicians, learning the skills of critical appraisal is the least of their problems in adjusting to an evidence-based teaching style!

Having defended EBM against all the standard arguments put forward by clinicians, I should confess to being sympathetic to many of the more sophisticated arguments put forward by philosophers and social scientists. Such arguments, summarised in Chapter 17 (new for this edition), address the nature of knowledge and the question of how much medicine really rests on decisions at all. But please don't turn to that chapter (which is, philosophically speaking, a ‘hard read’) until you have fully grasped the basic arguments in the first few chapters of this book—or you risk becoming confused!

Before you start: formulate the problem

When I ask my medical students to write me an essay about high blood pressure, they often produce long, scholarly and essentially correct statements on what high blood pressure is, what causes it and what the different treatment options are. On the day they hand their essays in, most of them know far more about high blood pressure than I do. They are certainly aware that high blood pressure is the single most common cause of stroke, and that detecting and treating everyone's high blood pressure would cut the incidence of stroke by almost half. Most of them are aware that stroke, although devastating when it happens, is a fairly rare event, and that blood pressure tablets have side effects such as tiredness, dizziness, impotence and getting ‘caught short’ when a long way from the lavatory.

But when I ask my students a practical question such as ‘Mrs Jones has developed light-headedness on these blood pressure tablets and she wants to stop all medication; what would you advise her to do?’, they are often foxed. They sympathise with Mrs Jones' predicament, but they cannot distil from their pages of close-written text the one thing that Mrs Jones needs to know. As Smith (paraphrasing TS Eliot) asked a few years ago in a BMJ editorial: ‘Where is the wisdom we have lost in knowledge, and the knowledge we have lost in information?’[14].