I Think You'll Find It's a Bit More Complicated Than That (21 page)

The next interesting figure misleads about a trend (
we’ve seen this a lot
from health ministers recently) and attempts to take the credit for a long-standing change. The leaflet says: ‘Since May 2010 the NHS has gained 2,550 more doctors and has 3,000 fewer managers.’ This is correct: full-time equivalent figures (my favourite) from
NHS workforce data
show 97,720 doctors in May 2010, and 100,197 in December. But
NHS Information Centre figures
show that between 1999 and 2009 the
total number of doctors
increased from 88,693 to 132,683, GPs from 28,354 to 36,085, and consultants from 21,410 to 34,654. Doctors take a while to grow, and they’ve been growing in number for a good long while.

Then we have choice: ‘95 per cent want more choice over their healthcare’. The source given is the twenty-fifth
British Social Attitudes Survey
. Interestingly, the government has just announced that it’s going to stop funding the health questions in the British Social Attitudes Survey, so this valuable resource won’t be around for long. The data was collected in 2007, it’s free to download, and if you do so you’ll see it didn’t ask about ‘more choice’.
Question 583
asks how much choice you think NHS patients
currently
have (‘a lot’, ‘a little’, and so on), and Question 584 asks how much choice you think they
should
have.

How those responses were aggregated to get 95 per cent of people in favour of ‘more choice’ – a key justification for reform – is a mystery: many people will have said they have ‘a little’ choice, and that they should have ‘a little’ choice (we can’t see how many from the aggregated data in tables of course; we need respondent-level data, because that’s the only way we can link together an individual’s responses to a sequence of questions). I asked the government how it produced its figure, since BSA25 doesn’t have data on the question ‘Would you like
more
choice?’ I was told the source was table 3.1 in Chapter 3 of a book
that costs £52
, called
Do People Want More Choice and Diversity of Provision in Public Services
.

I got that book: it’s the same old BSA25 data. It doesn’t contain anything on ‘more choice’, and they got the title wrong: it’s not
Do People Want More Choice
…
It’s
Do People Want Choice …
which shows how misleading they’re being, and wasted me a lot of time in the library. None of this should be difficult. The facts in this plainly political pamphlet should be clean, correct, transparent and justified. As the government
defies all reason
by claiming that NHS staff support its reforms, we can only fear the results of its new ‘listening exercise’.

Andrew Lansley
and His Imaginary Evidence

Guardian
, 5 February 2011

I have never heard one politician use the word ‘evidence’ so persistently, and so misleadingly, as Health Secretary Andrew Lansley defending his NHS reforms. Since he repeatedly claims that the evidence supports his plan, let’s skim through what we can find on whether GP consortiums work, the benefits of competition, and the failures of the NHS.

Are GP consortiums better than Primary Care Trusts for commissioning? There have been fifteen major reorganisations of the
NHS in thirty years
. We’ve had GP fundholders, GP multifunds, primary care groups, primary care trusts, family practitioner committees, purchasing consortiums, and more. After all this change, lots of data should have been gathered on the impact of specific strategies.

In reality,
few of them were properly studied
. Here are four papers on GP fundholding, which is broadly similar to Lansley’s GP consortiums.
Kay in 2002
found it was introduced and then abolished without any evidence of its effects. In 2006
Greener and Mannion
found a mix of good and bad but no evidence that it improved patient care.
In 1995 Coulter
found nothing but gaps in the evidence and no evidence of any improvement in efficiency, responsiveness, or quality.
Petchley found
there was insufficient data to make any judgement. Lansley says he is following the evidence. I see no evidence to follow here.

Next, competition. Andrew Lansley has repeatedly denied that he is introducing competition on price. This is disturbing behaviour: his Bill explicitly introduces price-based competition – it’s in paragraph 5:43 of his
‘NHS Operating Framework’
.

Does variable-price competition work in healthcare markets?
Working from first principles
, markets for healthcare in which people compete on price as well as quality might be expected to produce lower-quality healthcare, because prices are easy to measure, while quality, in healthcare, is surprisingly difficult to measure: so quality suffers.

It’s hard to research this kind of thing, but even the
evidence on
fixed-price
competition
– where you compete on quality – is mixed. There are various ways to assess it. Often people choose an outcome – like the number of people who survive a heart attack – and compare this outcome in areas of more intense or less intense competition. Sometimes competition makes things worse, sometimes better.

For
variable-price
competition, which is what we’re facing, things don’t look good. Its introduction in New Jersey in the 1990s was associated with a worsening
death rate from heart attacks
,
while in the UK
, stopping variable-price competition was associated with improvement. These aren’t clean, easy interventions to assess, but despite his using the word repeatedly, again the ‘evidence’ does not support Lansley here.

Lastly, there is the justification for reform. Both Lansley and David Cameron are – rather shamefully – overstating our mortality figures, in order to claim that the NHS is failing. Everyone wants more improvement, but money or a structural change does not produce an immediate and visible reduction in mortality from one thing, so it’s hard to use these figures to pin blame or credit on anyone; interventions take time to have an impact, especially on things that kill you slowly; and NHS treatment isn’t the only factor affecting how many people die of something. But since you’re interested,
to take just two things
: mortality from cancer has fallen every year since 1995, and heart attack deaths have halved since 1997.

The government claims that our rate of death from heart attacks is double that in France, even though we spend the same on health. Health economist
John Appleby
instantly debunked this claim in the
BMJ
, and his piece will become a citation classic. From static 2006 figures in isolation, the government is right. But the trajectory of improvement in the UK is so phenomenal that if the straight line continues – as it has done for thirty years – we will be better than France within a year.

I’m not in favour of – or against – anything here: genuinely, all health-service administrative models baffle and bore me equally. But when Andrew Lansley says all the evidence supports his interventions, as he has done repeatedly, he is simply wrong. His wrongness is not a matter of opinion, it is a fact, and his pretence at data-driven faux neutrality is not just irritating, it’s also hard to admire. There’s no need to hide behind a cloak of scientific authority, murmuring the word ‘evidence’ into microphones. If your reforms are a matter of ideology, legacy, whim and faith, then like many of your predecessors you could simply say so, and leave ‘evidence’ to people who mean it.

Why Is Evidence
So Hard for Politicians?

Guardian
, 12 February 2011

One thing you hope for, with politicians, is that they won’t make the same mistakes over and over again.

Last week we saw
that the government has overstated the problems in the NHS by using dodgy figures (to be precise, it used
misleading static figures
instead of time trends). We also saw that Andrew Lansley’s frequently repeated claim – that his reforms are justified by evidence – is untrue: the evidence doesn’t show that price-based competition improves outcomes (if anything, it makes things worse); and the evidence also doesn’t show that GP consortiums improve outcomes (unless you cherry-pick only the positive findings). It’s OK if your reforms aren’t supported by existing evidence: you just shouldn’t claim that they are.

Now Lansley’s Junior Minister
Paul Burstow MP has kindly responded
, repeating the exact same mistakes again, only more clumsily. From a Minister, this is frightening, so we should see how he does it.

Burstow’s letter:

In Ben Goldacre’s pursuit of the evidence for NHS modernisation (‘Evidence? What evidence?’, 5 February), he appears to have overlooked the impact assessment we published alongside the health and social care bill, where we present a thorough analysis of the evidence for and against our plans. As he will see, studies show that GP fundholding and practice-based commissioning delivered shorter waits and fewer referrals to hospitals for patients. The evidence on competition demonstrates that when it is well-designed and conducted on the basis of quality (as we are proposing), rather than price, it can drive up quality and efficiency.

We have not sought to understate the achievements of the NHS – but a 2008 study by Martin McKee and Ellen Nolte, citing OECD data, concluded that the UK had one of the worst rates of mortality amenable to healthcare among rich nations. If the NHS was to perform as well as the best-performing countries, thousands of lives could be saved each year. We make no apology for that.

Finally, Goldacre appears to have misunderstood the aims of our plans. We are not advocating reform for the sake of ideology. The changes we are proposing are designed to put patients first, improve health outcomes, empower clinicians so they can design services that meet the needs of their local communities, and put the NHS on a more sustainable footing so it is better able to meet the challenges of the 21st century.

Paul Burstow MP

Minister for Care Services

The government initially claimed that UK heart attack death rates were twice as bad as those in France. This was an overstatement: they are, but following recent interventions the
gap is closing so rapidly
that on current trends it will have disappeared entirely by 2012. In response, Burstow cites a 2008 paper by
McKee and Nolte
which he says ‘concluded that the UK had one of the worst rates of mortality amenable to healthcare among rich nations’.

Burstow either misunderstands or misrepresents this very simple and brief paper. It is a study explicitly looking at time trends, not static figures, and it once again finds that comparing 2003 with 1998, the UK still had fairly high rates of avoidable mortality, but these were falling faster than in all but one of the other eighteen industrialised countries they examined. (Meanwhile, in the US, avoidable mortality improved at a disastrously slow pace, although they spent more money).

This is a paper showing the recent
success
of the NHS, and the fact that we are discussing such a massive improvement in avoidable mortality from Labour’s first term in government is not my being partisan: this is the paper that was cited by the Tory Minister as evidence, bizarrely, of the NHS’s recent failures.

Next, Burstow says I
‘overlooked the impact assessment
we published alongside the health and social care bill, where we present a thorough analysis of the evidence for and against our plans … studies show that GP fundholding and practice-based commissioning delivered shorter waits and fewer referrals to hospitals for patients’.

In its section on GP fundholding, this ‘thorough analysis’ ignores
the four peer-reviewed academic papers
I described last week, which sadly found no evidence of an overall benefit from GP fundholding. It makes a series of five assertions about outcomes, though not one of these is referenced to any single paper or study, anywhere at all.

I contacted the Department of Health, which ferreted out the sources especially for me. It turned out there was just one, a document from the King’s Fund. It’s not a peer-reviewed academic journal article, but the King’s Fund is pretty good, in my view. If you read this document, once again, as with other reviews of the literature, it finds that the results of GP fundholding were mixed: some things got better, some things got worse.

So, the Minister has cherry-picked only the good findings, from only one report, while ignoring the peer-reviewed literature. Most crucially, he cherry-picks findings he likes while explicitly claiming that he is fairly citing the totality of the evidence from a thorough analysis. By using this approach, I can produce good evidence that I have a magical two-headed coin, if I just disregard all the throws where it comes out tails.

Here is what politicians apparently cannot understand, repeatedly, over and again: it’s fine to make policy based on ideology, whim, faith, principles, and all the other things we’re used to. It’s also fine for evidence to be mixed. And it’s absolutely fine if your reforms aren’t supported by existing evidence: just don’t pretend that they are.

Politicians Can Divine
Which Policy Works Best by Using Their Special Magic Politician Beam

Guardian
, 22 May 2010

We have accidentally elected a coalition. This week all good citizens are poring over the
‘Programme For Government’
, and there is much to be pleased with. Labour wasn’t all about unbridled credit and fun public sector spending sprees: they kept all our emails, kept records of the websites we visited, used ‘anti-terrorism’ legislation on people who plainly weren’t terrorists, and so on.