I Think You'll Find It's a Bit More Complicated Than That (40 page)

But Dr Julie Sharp, senior science information manager at Cancer Research UK, said: ‘This study is not about breast cancer, it’s a study showing how different diet patterns affect weight loss and it’s misleading to draw any conclusions about breast cancer from this research.’

The late caveat, torpedoing the central premise of a news piece, is a common strategy in many newspapers. But what use is this information, at the end of a long article, in paragraph number 19?

The way people read newspapers has been studied widely, using eye-tracking technology. It’s through this that we discover, for example, that when presented with a full-length photograph of a man, men are more likely to
look at the penis
area than women.

Most of this research is more interested in adverts than news, because research in all fields is driven by money (top left of the page is best, apparently): but there is plenty of other useful stuff, much of it by the Poynter Institute.

They did an
early study in 1990
, with predictable findings: photos attract attention; eyes travel from the dominant photo to the biggest headline, then teasers, and finally text; text is read the least, headlines the most; and so on.

But their
most recent project
was far bigger: they took a representative sample of 582 people from four cities in the US, and invited them to read a newspaper and a website as they normally would, wearing the eye-tracking equipment, over five days in 2006, for fifteen minutes each. This yielded a dataset of more than 102,000 eye stops.

Here’s what they found: once a story gets longer than eleven paragraphs, on average, your readers will read only half. A tiny minority will make it to paragraph number 19, where, on this occasion, a fraction of the readers of the
Daily Mail
would have discovered that the central premise of the news story – that a new trial had found a 40 per cent reduction in cancer through intermittent dieting – was false.

Caveats in paragraph 19 are standard practice for stories with outlandish health claims. Like nipple tassels in 1950s burlesque, they’re a way to keep it legal, but titillating; and in many cases, when the late rebuttal comes from an authority figure – calling for calm in patrician tones – it can feel as if it’s only even there to accentuate the excitement. But if your interest is informing a reader, they are plainly misleading.

Why Don’t Journalists
Link to Primary Sources?

Guardian
, 19 March 2011

Why don’t journalists link to primary sources? Whether it’s a press release, an academic journal article, a formal report, or even the full transcript of an interview, the primary source contains more information for interested readers: it shows your working, and it allows people to check whether what you wrote is true. Here are three short stories.

This week the
Telegraph
ran the headline ‘
Wind farms blamed
for stranding of whales’. ‘Offshore wind farms are one of the main reasons why whales strand themselves on beaches, according to scientists studying the problem,’ it continued. Baroness Warsi even cited the story on BBC
Question Time
this week, arguing against wind farms.

But anyone who read the
open-access academic paper
in
PLoS One
, titled ‘Beaked Whales Respond to Simulated and Actual Navy Sonar’, would see that the study looked at sonar, and didn’t mention wind farms at all. At best, the
Telegraph
story was a massive exaggeration of one brief, contextual aside about general levels of manmade sound in the ocean, made by one author at the
end of the press release
(titled ‘Whales Scared by Sonars’). This release didn’t mention wind farms, and it didn’t say they were ‘one of the main reasons why whales strand themselves on beaches’. Anyone reading the press release could see that the study was about naval sonar.

This
Telegraph
article (
now deleted
, with a
miserly correction
) was a distortion, perhaps driven by the paper’s odd editorial line about the environment. But there is a bigger fish here: if we had a culture of linking to primary sources – if they were always just a click away – then shame alone would probably have stopped it going online. Outright misrepresentations are only worth risking in an environment where the reader is routinely deprived of information.

Sometimes the examples are sillier. Professor Anna Ahn published a paper recently, showing that people with shorter heels have larger calves. For the
Telegraph
this became ‘Why stilettos are the
secret to shapely legs
’, for the
Mail
‘Stilettos give women
shapelier legs than flats
’, for the
Express
‘
Stilettos tone up
your legs’.

But anybody who
read even the press release
would immediately see that this study had nothing to do with shoes. It wasn’t about shoe-heel height: it looked at anatomical heel length, the distance from the back of your ankle joint to the insertion of the Achilles tendon. The participants were all barefoot, and the paper was just a nerdy insight into the engineering of a human body: if you have a shorter lever at the back of your foot, you need a bigger muscle in your calf. Once again, this story was a concoction by journalists. But more than that, no sane journalist could possibly have risked writing the story about stilettos, if there was a culture and tradition of linking to the academic paper, or even the press release: they’d have looked like idiots, and fantasists, to anyone who bothered to click.

Lastly, on Wednesday the
Daily Mail
ran with the scare headline ‘Swimming too Often in Chlorinated Water “Could Increase Risk of Developing Bladder Cancer”, Say Scientists’. There’s hardly any point documenting the errors in
Daily Mail
health stories any more, but if you
read the original paper
,
or even the press release
, again, anyone can see that bladder cancer wasn’t measured, and the
Mail
’s story was a
simple distortion
. It’s worth mentioning that these press releases were fairly readable pieces of popular science in themselves.

Of course, this is a problem that occurs well beyond science. Over and again, you read comment pieces that purport to be responding to an earlier piece, but distort the earlier arguments, or miss out the most important ones: they count on it being inconvenient for you to check. It’s also an interesting difference between different forms of media: most bloggers have no institutional credibility, so they must build it, by linking transparently, and allowing you to easily double-check their work.

But more than anything, because linking sources is such an easy thing to do, and the motivations for avoiding links are so dubious, I’ve detected myself using a new rule of thumb: if people don’t link to primary sources, I don’t trust them, and I don’t read them.

A Fishy Friend
, and His Friends

Guardian
, 5 June 2010

‘
Fish oil helps
schoolchildren to concentrate’ was the headline in the
Observer
. The
omega-3 fish-oil pill
issue has dragged on for almost a decade now: the entire British news media repeatedly claim that trials show it improves school performance and behaviour in mainstream children, but no such trial has ever been published. There is something very attractive about the idea that solutions to complex problems in education can be found in a pill.

So, have things changed? The
Observer
’s health correspondent,
Denis Campbell
, is on the case, and it sounds as if they have. ‘Boys aged eight to 11 who were given doses once or twice a day of docosahexaenoic acid, an essential fatty acid known as DHA, showed big improvements in their performance during tasks involving attention.’ Great. ‘The researchers gave 33 US schoolboys 400mg or 1,200mg doses of DHA or a placebo every day for eight weeks. Those who had received the high doses did much better in mental tasks involving mathematical challenges.’ Brilliant news.

Is it true? After some effort, I have tracked down the academic paper. This was not a trial of whether fish-oil pills improve children’s performance: it was a brain-imaging study. They took thirty-three kids, divided them into three groups (of ten, ten and thirteen) and then gave them either no omega-3, a small dose, or a big dose. Then the children performed some attention tasks in a brain scanner, to see if bits of their brains lit up differently.

Why am I saying ‘omega-3’? Because it wasn’t a study of fish oil, as the
Observer
says: it was a study of omega-3 fatty acids derived from algae. But that’s small print.

If this had been a trial to detect whether omega-3 improves performance, it would be laughably small: about ten people in each treatment group. While small studies aren’t entirely useless, as amateurs often claim, you do have a very small number of observations to work from, so your study is much more prone to error from the simple play of chance. A study with thirty-three children, like this one, could conceivably detect an effect; but only if the fish oil caused a gigantic and unambiguous improvement in all the children who got it, and none of the children on placebo improved.

This paper showed no difference in performance at all. Since it was a brain-imaging study, not a trial, the results of the children’s actual performance on the attention task are reported only in passing, in a single paragraph, but the researchers are clear: ‘There were no significant group differences in percentage correct, commission errors, discriminability, or reaction time.’

So this is all looking pretty wrong. Are we even talking about the same academic paper? I’ve been trying to get mainstream media to link to original academic papers when they write about them, at least online, with some limited success on the BBC website. I asked Denis Campbell which academic paper he was referring to, but he declined to answer, and passed me on to Stephen Pritchard, the Readers’ Editor for the
Observer
, who answered a couple of days later to say that he didn’t understand why he was being involved. Eventually Denis confirmed, but through Stephen Pritchard, that it
was indeed the paper
I had found, from the April edition of the
American Journal of Clinical Nutrition
.

If we are very generous, is it informative, in any sense, that a brain area lights up differently in a scanner after some pills? Intellectually, it may be. But doctors get very accustomed to drug company sales reps and enthusiastic researchers who approach them with an exciting
theoretical
reason why one treatment should be better than another: maybe their intervention works selectively on only one kind of receptor molecule, for example, so it should therefore have fewer side effects. Similarly, drug reps and researchers will often announce that their intervention has an effect on some kind of elaborate laboratory measure: maybe a molecule in the blood goes up in concentration, or down, in a way that suggests the intervention might be effective.

This is all very well. But it’s not the same as showing that something really does work, back here in the real world, and medicine is overflowing with unfulfilled promises from early theoretical research. This stuff is interesting: but it’s not even in the same ballpark as showing that something works.

Oddly enough, though, someone really has finally conducted a proper trial of fish-oil pills in mainstream children to see if they work. It’s the trial that journalists have long been waiting for: a well-conducted, randomised, double-blind, placebo-controlled trial, in 450 children aged eight to ten from a mainstream school population. It was
published in full this year
, and it found no improvement. Show me the news headlines about that paper.

Meanwhile, Euromonitor
estimates global sales
for fish-oil pills at $2 billion, having doubled in five years, with sales projected to reach $2.5 billion by 2012, and they are now the single best-selling product in the UK food-supplement market. This has only been possible with the kind assistance of the British media, and their eagerness to write stories about the magic intelligence pill.

The week after this piece appeared, the
Independent
’s health correspondent wrote an angry column, explaining that health correspondents can’t be expected to check facts. In the interests of balance, his piece is
reproduced below in full
.

Jeremy Laurance: Dr Goldacre Doesn’t Make Everything Better

Is Ben Goldacre, the celebrated author of
Bad Science
and scourge of health journalists everywhere, losing it? So accustomed has he become to swinging his fists at the media when they get a science story wrong, I fear he may one day go nuclear and take out three rows of medical correspondents with a single lungful of biting sarcasm.

He was at it again in Saturday’s
Guardian
, pistol-whipping his
Guardian
and
Observer
colleague, health correspondent Denis Campbell, over a report he wrote about fish oil and its supposed role in improving children’s intelligence.

Campbell had reported claims made at a press conference that fish oil improved mental performance in children taking supplements. His crime, however, was to fail to check the claims against the academic paper on which they were based. That showed that the fish oil ‘enhanced the function of those brain regions that are involved in paying attention’, as revealed by a brain scanner.

Not quite the same as ‘improving their performance’, as Goldacre rightly pointed out. Indeed the paper revealed that there had been no improvement in the children’s performance. Time, then, for Goldacre to deliver his customary knee-capping. He did so because Campbell declined to help him with his inquiries. Small wonder, given it is the second occasion the hapless Campbell has found himself in Goldacre’s sights.