The benefits of mistakes in science

Science can be wrong, that is well known. What is less well known on the other hand, is that failure is an integral part of it. "If your work is cutting-edge, you are bound to make mistakes. It's inevitable," says Manu Kapur, professor of education at ETH Zurich and father of the "fruitful failure" theory, which investigatesthe role of mistakes in learning.

Science offers many opportunities for failure: experiments that don't work, results that are different from what we expected, work that contradicts our theory or, quite simply, a manuscript submitted to a journal that is rejected.

Results on the shelf

The scientific community does not encourage highlighting studies that did not work as expected. This creates a major problem called publication bias. Journals almost only publish positive results and most often reject work with so-called "negative" results. These include experimental or statistical data that are not "statistically significant", i.e. they fail to clearly demonstrate the strength of an observation or to establish clear correlations between a cause and an effect. Most scientists do not make the effort to submit such studies, which then disappear and are never discussed by the community. This is a real problem when it comes to establishing a scientific consensus,  biased by the artificially dominant position of positive results.

Scientists may even shy away from certain research for fear of not being able to publish it. In 2018, Andrea Popp, then a doctoral student at Eawag, published a commentary with colleagues calling for greater acceptance of negative results in her field of hydrology. “A survey had revealed a decline in the amount of hydrological research that is conducted in the field, and that this is partly due to the fact that scientists consider this work to be riskier than modelling studies," explains the researcher. “They fear that these experimental results will be negative and cannot be published. This is a real problem, because this type of work is absolutely essential to advance our discipline."

According to the hydrologist, who now works at the University of Oslo, her colleagues should dare to submit their negative results for publication and talk more about the difficulties encountered in their work: "You always learn from your mistakes, as much from your own as from others.”

Learning from mistakes

“Things are changing, but slowly," says Thomas Jung, physicist at the Paul Scherrer Institute. There is still a lot of pressure from the community to publish spectacular results as quickly as possible. “However, not all scientists have the same inclination to question their work. I always seek external advice when my results are out of the ordinary. I developed this habit following a mistake I made at the beginning of my career, almost thirty years ago.”

As a young doctoral student, Thomas Jung and a colleague submitted a paper on an atomic force microscopy experiment. A visiting physicist took one look at the manuscript and expressed doubts as to whether the results were due to an artificial effect. Thomas Jung and his colleague had to face the facts: their interpretation of the experimental data was wrong. "This experience convinced me not only to be open to criticism, but to actively seek it out," says Thomas Jung.

Changing the system

For Manu Kapur, one tool that can encourage greater transparency is the online pre-registration of studies before they start: "First, the experimental protocol is assessed by experts in the field, which ensures that it is of good quality. Secondly, this step pushes the research team to publish even negative results, as the community knows that the study is ongoing and therefore expects results. This reduces the chances that they will remain unseen." As an editor, he says he accepts manuscripts with negative results, as long as they are not due to faulty design or misinterpretation.

“It is of course normal to want to talk about one's successes, but one should also discuss one's failures," he says. "I am not reassured to hear only about successes when I visit an institute. I get the impression that the teams are either not doing enough work, or that they are hiding their failures.”

Manu Kapur himself is conducting research into the role failure plays in learning. “I see a clear link with scientific research," he explains. “In both cases you are at the limit of your knowledge. In general, teaching focuses on imparting new knowledge, including solutions to problems given to students. Our work shows that people learn much more grounded when they have the opportunity to look for the solution themselves. However,  for this they must be allowed to make mistakes. It's the same in science: failure is an inherent part of the process.

Thomas Jung  shares this opinion: "It is too often forgotten, but science is made by people, and to err is human. Our imperfection is something we have to deal with every day. We are constantly making mistakes, and that is normal.”