How to Read a Paper: The Basics of Evidence-Based Medicine (10 page)

Cohort studies

In a cohort study, two (or more) groups of people are selected on the basis of differences in their exposure to a particular agent (such as a vaccine, a surgical procedure or an environmental toxin), and followed up to see how many in each group develop a particular disease, complication or other outcome. The follow-up period in cohort studies is generally measured in years (and sometimes in decades) because that is how long many diseases, especially cancer, take to develop. Note that RCTs are usually begun on people who already have a disease, whereas most cohort studies are begun on people who may or may not develop disease.

A special type of cohort study may also be used to determine the prognosis of a disease (i.e. what is likely to happen to someone who has it). A group of people who have all been diagnosed as having an early stage of the disease or a positive screening test (see Chapter 7) is assembled (the inception cohort) and followed up on repeated occasions to see the incidence (new cases per year) and time course of different outcomes. (Here is a definition that you should commit to memory if you can:
incidence
is the number of new cases of a disease per year, whereas
prevalence
is the overall proportion of the population who suffer from the disease.)

The world's most famous cohort study, whose authors all won knighthoods, was undertaken by Sir Austen Bradford Hill, Sir Richard Doll and, latterly, Sir Richard Peto. They followed up 40 000 male British doctors divided into four cohorts (non-smokers, and light, moderate and heavy smokers) using both all-cause (any death) and cause-specific (death from a particular disease) mortality as outcome measures. Publication of their 10-year interim results in 1964 [15], which showed a substantial excess in both lung cancer mortality and all-cause mortality in smokers, with a ‘dose–response’ relationship (i.e. the more you smoke, the worse your chances of getting lung cancer), went a long way to demonstrating that the link between smoking and ill health was causal rather than coincidental. The 20-year [16], 40-year [17] and 50-year [18] results of this momentous study (which achieved an impressive 94% follow-up of those recruited in 1951 and not known to have died) illustrate both the perils of smoking and the strength of evidence that can be obtained from a properly conducted cohort study.

Given here are clinical questions that should be addressed by a cohort study.

• Does smoking cause lung cancer?
  
• Does the contraceptive pill ‘cause’ breast cancer? (Note, once again, that the word ‘cause’ is a loaded and potentially misleading term. As Guillebaud has argued in his excellent book ‘The Pill…’[19], if a thousand women went on the oral contraceptive pill tomorrow, some of them would get breast cancer. But some of those would have got it anyway. The question that epidemiologists try to answer through cohort studies is, ‘what is the
  additional
  risk of developing breast cancer which this woman would run by taking the pill, over and above the baseline risk attributable to her own hormonal balance, family history, diet, alcohol intake, and so on?’.)
  
• Does high blood pressure get better over time?
  
• What happens to infants who have been born very prematurely, in terms of subsequent physical development and educational achievement?
  

Case–control studies

In a case–control study, patients with a particular disease or condition are identified and ‘matched’ with controls (patients with some other disease, the general population, neighbours or relatives). Data are then collected (e.g. by searching back through these people's medical records, or by asking them to recall their own history) on past exposure to a possible causal agent for the disease. Like cohort studies, case–control studies are generally concerned with the aetiology of a disease (i.e. what causes it), rather than its treatment. They lie lower down the conventional hierarchy of evidence (see subsequent text), but this design is usually the only option when studying rare conditions. An important source of difficulty (and potential bias) in a case–control study is the precise definition of who counts as a ‘case’, because one misallocated individual may substantially influence the results (see section ‘Was systematic bias avoided or minimised?’). In addition, such a design cannot demonstrate causality—in other words, the
association
of A with B in a case–control study does not prove that A has
caused
B.

Clinical questions that should be addressed by a case–control study are listed here.

• Does the prone sleeping position increase the risk of cot death (sudden infant death syndrome)?
  
• Does whooping cough vaccine cause brain damage? (see section ‘Was systematic bias avoided or minimised?’).
  
• Do overhead power cables cause leukaemia?
  

Cross-sectional surveys

We have probably all been asked to take part in a survey, even if it was only a woman in the street asking us which brand of toothpaste we prefer. Surveys conducted by epidemiologists are run along essentially the same lines: a representative sample of participants is recruited and then interviewed, examined or otherwise studied to gain answers to a specific clinical (or other) question. In cross-sectional surveys, data are collected at a single time point but may refer retrospectively to health experiences in the past—for example, the study of patients' medical records to see how often their blood pressure has been recorded in the past 5 years.

A cross-sectional survey should address the following clinical questions .

• What is the ‘normal’ height of a 3-year-old child? This, like other questions about the range of normality, can be answered simply by measuring the height of enough healthy 3-year-olds. But such an exercise does not answer the related clinical question ‘when should an unusually short child be investigated for disease?’ because, as in almost all biological measurements, the physiological (normal) overlaps with the pathological (abnormal). This problem is discussed further in section ‘Likelihood ratios’.
  
• What do psychiatric nurses believe about the value of antidepressant drugs and talking therapies in the treatment of severe depression?
  
• Is it true that ‘half of all cases of diabetes are undiagnosed’? This an example of the more general question, ‘What is the prevalence (proportion of people with the condition) of this disease in this community?’ The only way of finding the answer is to do the definitive diagnostic test on a representative sample of the population.
  

Case reports

A case report describes the medical history of a single patient in the form of a story (‘Mrs B is a 54 year old secretary who developed chest pain in June 2010…’). Case reports are often run together to form a
case series
, in which the medical histories of more than one patient with a particular condition are described to illustrate an aspect of the condition, the treatment or, most commonly these days, adverse reaction to treatment.

Although this type of research is traditionally considered to be relatively weak as scientific evidence (see section ‘The traditional hierarchy of evidence’), a great deal of information that would be lost in a clinical trial or survey can be conveyed in a case report (see Chapter 12). In addition, case reports are immediately understandable by non-academic clinicians and by the lay public. They can, if necessary, be written up and published within days, which gives them a definite edge over clinical trials (whose gestation period can run into years) or meta-analyses (even longer). There are certainly good theoretical grounds for the reinstatement of the humble case report as a useful and valid contribution to medical science, not least because the story is one of the best vehicles for
making sense
of a complex clinical situation. Richard Smith, who edited the British Medical Journal for 20 years, recently set up a new journal called
Cases
dedicated entirely to ‘anecdotal’ accounts of single clinical cases (see
http://casesjournal.com/casesjournal
).

The following are clinical situations in which a case report or case series is an appropriate type of study.

• A doctor notices that two babies born in his hospital have absent limbs (phocomelia). Both mothers had taken a new drug (thalidomide) in early pregnancy. The doctor wishes to alert his colleagues worldwide to the possibility of drug-related damage as quickly as possible [20]. (Anyone who thinks ‘quick and dirty’ case reports are never scientifically justified should remember this example.)
  
• A previously healthy patient develops spontaneous bacterial peritonitis—an unusual problem that the average doctor might see once in 10 years. The clinical team looking after her search the literature for research evidence and develop what they believe is an evidence-based management plan. The patient recovers well. The team decide to write this story up as a lesson for other clinicians—a so-called evidence-based case report [21].
  

The traditional hierarchy of evidence

Standard notation for the relative weight carried by the different types of primary study when making decisions about clinical interventions (the ‘hierarchy of evidence’) puts them in the given order.

1.
Systematic reviews and meta-analyses (see Chapter 9);

2.
RCTs with definitive results (i.e. confidence intervals that do not overlap the threshold clinically significant effect (see section ‘Probability and confidence’));

3.
RCTs with non-definitive results [i.e. a point estimate that suggests a clinically significant effect but with confidence intervals overlapping the threshold for this effect (see section ‘Probability and confidence’)];

4.
Cohort studies;

5.
Case–control studies;

6.
Cross-sectional surveys;

7.
Case reports.

The pinnacle of the hierarchy is, quite properly, reserved for secondary research papers, in which all the primary studies on a particular subject have been hunted out and critically appraised according to rigorous criteria (see Chapter 9). Note, however, that not even the most hard-line protagonist of EBM would place a sloppy meta-analysis or an RCT that was seriously methodologically flawed above a large, well-designed cohort study. And as Chapter 12 shows, many important and valid studies in the field of qualitative research do not feature in this particular hierarchy of evidence at all.

In other words, evaluating the potential contribution of a particular study to medical science requires considerably more effort than is needed to check off its basic design against the above-mentioned 7-point scale. A more recent publication on hierarchies of evidence suggests we should grade studies on four dimensions: risk of bias, consistency, directness and precision—an approach that would complicate any simple pyramid of evidence [22]. The take-home message is, don't apply the hierarchy of evidence mechanically—it's only a rule of thumb.

A more complex representation of the hierarchy of evidence geared to the domain of the question (therapy/prevention, diagnosis, harm, prognosis) was drawn up by a group of us in 2011 [23] and is available for download on the Centre for Evidence-Based Medicine website (
http://www.cebm.net/index.aspx?o=5653
). But before you look that one up, make sure you are clear on the traditional (basic) hierarchy described in this section.

A note on ethical considerations

When I was a junior doctor, I got a job in a world-renowned teaching hospital. One of my humble tasks was seeing the geriatric (elderly) patients in casualty. I was soon invited out to lunch by two charming mid-career doctors, who (I later realised) were seeking my help with their research. In return for getting my name on the paper, I was to take a rectal biopsy (i.e. cut out a small piece of tissue from the rectum) on any patient over the age of 90 who had constipation. I asked for a copy of the consent form that patients would be asked to sign. When they assured me that the average 90-year-old would hardly notice the procedure, I smelt a rat and refused to cooperate with their project.

At the time, I was naïvely unaware of the seriousness of the offence being planned by these doctors. Doing
any
research, particularly that which involves invasive procedures, on vulnerable and sick patients without full consideration of ethical issues is both a criminal offence and potential grounds for a doctor to be ‘struck off’ the medical register. Getting formal ethical approval for one's research study (for UK readers, see corec.org.uk), and ensuring that the research is properly run and adequately monitored (a set of tasks and responsibilities known as
research governance
[24–26]) can be an enormous bureaucratic hurdle. Ethical issues were, sadly, sometimes ignored in the past in research in babies, the elderly, those with learning difficulties and those unable to protest (e.g. prisoners and the military), leading to some infamous research scandals [24].

These days, most editors routinely refuse to publish research that has not been approved by a research ethics committee. Note, however, that heavy-handed approaches to research governance by official bodies may be ethically questionable. Neurologist and researcher Professor Charles Warlow [27] argued some years ago that the overemphasis on ‘informed consent’ by well-intentioned research ethics committees has been the kiss of death to research into head injuries, strokes and other acute brain problems (in which, clearly, the person is in no position to consider the personal pros and cons of taking part in a research study). More recently, exasperated researchers published a salutary tale entitled ‘Bureaucracy stifles medical research in Britain’ [28]. The bottom line message for this book is: make sure that the study you are reading about has had ethical approval, while also sympathising with researchers who have had to ‘jump through hoops’ to get it.