Eenglish-tests.com

C2 Text 63 · Feature · 804 words

Algorithmic Fairness

A feature on a council's repair-scoring software shows why making algorithms fair is a moral question as well as a technical one.

In the spring of last year, the housing department of Brackenford Council, in the north of England, introduced a piece of software to decide which tenants' repair requests should be dealt with first. The old system had been a mixture of phone calls, spreadsheets and the memory of a few long-serving staff. The new one promised to be faster, more consistent and, above all, fairer. Within six months, it had become the subject of a heated public meeting, a petition with four thousand signatures and a quietly commissioned review. What went wrong is instructive, not because the software was unusually bad, but because it was rather ordinary.

The program was trained on ten years of the council's own records. It learned which requests had, in the past, turned out to be urgent: damp that spread, boilers that failed in winter, leaks that brought down ceilings. It then gave each new request a score. On paper, it never considered a tenant's age, ethnicity or income. In practice, it did something subtler. Households that had complained often in the past had, historically, been visited sooner, so their problems were more likely to have been recorded as serious. Households that rarely complained, often older residents or people less confident in English, had problems that were logged late, if at all. The algorithm, faithfully learning from the past, concluded that their requests were less likely to matter.

This is what researchers call a proxy problem. An algorithm may be forbidden from looking at a sensitive characteristic, yet still infer it from something that travels alongside it: a postcode, a pattern of phone calls, the time of day a form is submitted. 'People imagine that bias is something you put into a system,' says Dr Helena Marsh, a statistician who advised the review. 'More often, it's something you fail to take out. The data are a record of what an organisation did, not of what was needed. If you train a model on your history, you inherit your history.'

The difficulty runs deeper than bad data, however. Even with perfect records, there is no single, agreed definition of fairness to aim at. Should the system be equally accurate for every group? Should it flag the same proportion of urgent cases in every neighbourhood? Should a given score mean the same level of risk whoever it is attached to? Each sounds reasonable. In 2016 and 2017, several groups of researchers showed, independently, that when groups differ in their underlying rates of need, some of these aims cannot all be met at once. Choosing between them is not a technical decision but a moral one, and it is usually made, often without anyone noticing, by whoever writes the code.

British readers may remember a more public example. In the summer of 2020, when exams in England were cancelled because of the pandemic, a statistical model was used to adjust the A-level grades that schools had estimated for their students. Many students received lower grades than expected, and the model was widely criticised for appearing to penalise able pupils at schools with weaker past results. After protests, the government reversed course and allowed students to receive their schools' estimated grades wherever these were higher. The episode left a lasting mark on public trust: for many young people, 'the algorithm' became shorthand for a faceless judgement that could not be argued with.

Back in Brackenford, the review's recommendations were modest but telling. The scoring system was not abandoned; instead, it is now audited every quarter, its results broken down by area and age group. Tenants are told when software has been used and given a simple way to challenge a decision. Most importantly, a named officer is responsible for the outcomes. 'The worst thing about the old software wasn't the maths,' says Tom Adeyemi, who chaired the tenants' association during the dispute. 'It was that nobody would own it. Every time we asked why, we were told the computer had decided. Now there's a person whose job it is to answer.'

Supporters of automated decisions make a fair point in return: human judgement was never neutral either. Caseworkers have moods, favourites and blind spots; at least an algorithm's prejudices can be measured. Dr Marsh agrees, up to a point. 'The promise of these tools is real. You can interrogate a model in a way you can't interrogate a tired person at the end of a long shift. But that's an opportunity, not a guarantee. Fairness isn't a feature you install. It's a practice you keep up.'

Perhaps that is the most useful lesson of the Brackenford affair. The question was never whether a machine could be fair in the abstract. It was whether an institution would accept that fairness is a decision it keeps on making: in public, and with its working shown.

Words in colour are explained in the glossary below.

Glossary

WordMeaning
heated/ˈhiːtɪd/full of strong, angry feelings
commissioned/kəˈmɪʃənd/officially asked for and paid for
instructive/ɪnˈstrʌktɪv/teaching a useful lesson
proxy/ˈprɒksi/something that stands in for something else
infer/ɪnˈfɜː/work something out from the evidence
inherit/ɪnˈherɪt/receive something from the past or from people before you
flag/flæɡ/mark something so that people notice it
penalise/ˈpiːnəlaɪz/punish someone or put them at an unfair disadvantage
reversed course/rɪˌvɜːst ˈkɔːs/changed a decision to the opposite one
shorthand/ˈʃɔːthænd/a short way of referring to a bigger idea
audited/ˈɔːdɪtɪd/officially checked in detail
blind spots/ˈblaɪnd ˌspɒts/areas where a person fails to notice or understand things
prejudices/ˈpredʒədɪsɪz/unfair opinions formed without good reason
interrogate/ɪnˈterəɡeɪt/question or examine very closely
in the abstract/ɪn ði ˈæbstrækt/in theory, not in a real situation
with its working shown/wɪð ɪts ˌwɜːkɪŋ ˈʃəʊn/showing how it reached its answer, as pupils do in maths

Algorithmic Fairness: tests

Found this text useful? Share it with a friend or your class.