The following is an edited transcript of a symposium held at a festival of ideas in Bath. The chair, Dr Hannah Rees, was joined by Priya Desai, who analyses performance data for a group of hospitals; Martin Kerr, a retired head teacher; and Tomasz Nowak, a philosopher who writes on the ethics of measurement.
Hannah Rees: Our title tonight is deliberately provocative. 'Tyranny' is a strong word for a spreadsheet. Priya, are numbers really our masters now? Priya Desai: I'd say they're more often our whistle-blowers. Before hospitals routinely published infection rates or waiting times, patients had almost no way of knowing whether one ward was safer than another. Measurement has exposed failings that had been hidden for years behind reassuring words. I'm wary of any argument that begins by sneering at data, because the people who benefit from an absence of data are rarely the weakest. Hannah Rees: Martin, you spent thirty years in schools. Does that ring true?
Martin Kerr: Partly. I've no wish to return to the days when a school could be failing its pupils and nobody would know. But I watched something happen over my career that worried me. A measure would be introduced, sensibly, to give us a picture of how pupils were doing. Then it would be attached to league tables, then to inspections, then to people's jobs. And at each stage the picture became a little less accurate, because everyone, quite rationally, started working on the number rather than on the thing the number was meant to show. Teaching to the test isn't a moral failing of individual teachers. It's what any intelligent person does when their livelihood depends on a single figure. Tomasz Nowak: Which is, more or less, what economists call Goodhart's law. The popular version goes: when a measure becomes a target, it ceases to be a good measure. A number is like a window; the moment you start polishing it for the inspectors, you stop looking through it.
Hannah Rees: Can you give us an example from outside schools? Tomasz Nowak: The classic ones are almost comic. Suppose a call centre rewards staff for keeping calls short. Calls get shorter, certainly, and customers ring back three times to finish the conversation. Or an office is judged on how quickly it answers letters, so it sends a prompt reply saying, in effect, 'We have received your letter.' The figure improves; the service doesn't. The deeper problem, though, isn't cheating. It's displacement: what gets measured gets attention, and what doesn't gets quietly neglected. Kindness, curiosity, a nurse's sense that a patient is frightened: these are real, and they don't show up on a dashboard. Priya Desai: I'd push back a little on that. 'Some things can't be measured' is true, but it's also a very convenient thing to say if you'd rather not be measured at all. We can measure far more than people assume, including, imperfectly, how patients feel about their care. The answer to a bad measure is usually a better one, or several, not none.
Martin Kerr: Several, yes. A single figure is a tyrant; a handful of them, read with judgement, is more like a council of advisers. What made the difference in the best schools I knew wasn't fewer numbers but a different attitude to them. The data started a conversation instead of ending one. Hannah Rees: Tomasz, is 'judgement' just a polite word for 'trust me'? Tomasz Nowak: It can be, and that's Priya's fair worry. But the opposite illusion is just as dangerous: that if we only had enough data, we could do without judgement entirely. There's an old joke about a man searching for his keys under a streetlight. A passer-by asks whether he dropped them there. 'No,' he says, 'I dropped them in the park, but the light's better here.' Organisations do that all the time. They search where the numbers are, not where the answers are.
Hannah Rees: We're nearly out of time. One practical suggestion each. Priya Desai: Use data to learn before you use it to punish. If staff believe a figure will be used against them, you'll get the figure you deserve, not the one you need. Martin Kerr: Ask the people being measured to help design the measures. They know where the loopholes are, usually because they've been tempted by them. Tomasz Nowak: And keep asking what a number is for. A measure is a servant with a short memory; it forgets its purpose unless someone keeps reminding it. Hannah Rees: On that note of cautious agreement, thank you all.