Eenglish-tests.com

C1 Text 2 · Article · 628 words

The Replication Crisis

A feature article on why many published scientific findings could not be repeated, what caused the problem and how researchers are trying to fix it.

One of the founding principles of science is that a result should not depend on who obtains it. If a study is carried out carefully, another team following the same method ought to reach the same conclusion. This process, known as replication, is what turns a single interesting finding into reliable knowledge. It was therefore deeply unsettling when, in the 2010s, a series of large projects attempted to repeat well-known studies in psychology and found that a substantial proportion of them could not be reproduced.

The findings that failed were not obscure. Some had appeared in leading journals, been cited thousands of times and even found their way into popular books and talks. Researchers in medicine, economics and biology soon began asking uncomfortable questions about their own fields, and the phrase 'replication crisis' entered the vocabulary of science.

How could so many apparently solid results turn out to be so fragile? Few experts blame deliberate fraud, which remains rare. The causes, most agree, are subtler and lie largely in the way research is rewarded. Journals have traditionally favoured studies with striking, positive results; a careful experiment that finds no effect is far less likely to be published. Scientists, whose careers depend on publications, are therefore under pressure to produce exciting findings. Many early studies were also based on small samples, which makes results produced purely by chance more likely. On top of this, researchers sometimes tried several ways of analysing their data and reported only the one that worked – a practice often called 'p-hacking', which can create the illusion of a real effect where none exists.

'Most of the people involved weren't cheating,' says Dr Amira Haddad, a psychologist at a university in Bristol who has led several replication studies. 'They were following the habits they had been taught. When you have dozens of small decisions to make about your data, it is remarkably easy to fool yourself without realising it. What the crisis exposed was a weakness in the system, not a handful of bad individuals.'

Encouragingly, the response has been vigorous. A growing number of researchers now pre-register their studies, publicly recording their hypotheses and methods before collecting any data, so that they cannot quietly change their plans afterwards. Some journals agree to publish a study on the strength of its design, whatever the results turn out to be. Sharing data and computer code openly has become far more common, allowing others to check the analysis, and large collaborations involving many laboratories have made it possible to test findings on much bigger samples.

Not everyone is comfortable with the language of crisis, however. Professor Martin Clegg, who studies the history of science, argues that the word is misleading. 'Science has always produced findings that later prove wrong,' he says. 'What is new is that scientists went looking for their own mistakes on a large scale and published what they found. That is not a sign of a discipline in collapse; it is a sign of one that is working.' He also warns that a failed replication does not automatically mean that the original study was wrong. Differences in the people tested, the setting or the exact procedure can all affect the outcome.

Dr Haddad partly agrees. 'I'd call it a crisis of confidence rather than a crisis of science itself,' she says. 'The danger is that people outside research hear the word and conclude that nothing scientists say can be trusted. That would be exactly the wrong lesson.' Indeed, it is precisely because science has ways of checking itself that the problem came to light at all. The challenge now is to make those checks a routine part of research rather than an afterthought, so that the next generation of findings rests on firmer foundations.

Words in colour are explained in the glossary below.

Glossary

WordMeaning
replication/ˌreplɪˈkeɪʃən/repeating a study to see if you get the same result
unsettling/ʌnˈsetlɪŋ/making people feel worried and uncertain
substantial/səbˈstænʃəl/large in amount or importance
obscure/əbˈskjʊə/not well known
cited/ˈsaɪtɪd/mentioned by other writers as a source of information
fragile/ˈfrædʒaɪl/easily broken or shown to be weak
fraud/frɔːd/dishonestly making up or changing results
chance/tʃɑːns/luck; things happening without any real cause
p-hacking/ˈpiː ˌhækɪŋ/trying many ways of analysing data until one seems to show a result
illusion/ɪˈluːʒn/something that seems real but is not
vigorous/ˈvɪɡərəs/strong and full of energy
pre-register/ˌpriː ˈredʒɪstə/to publish your plan for a study before you start it
misleading/ˌmɪsˈliːdɪŋ/giving people the wrong idea
collapse/kəˈlæps/a sudden failure or breakdown
afterthought/ˈɑːftəθɔːt/something that is added later and not planned carefully
foundations/faʊnˈdeɪʃənz/the base that something is built on

The Replication Crisis: tests

Found this text useful? Share it with a friend or your class.