New recommendations for designing animal studies could improve research reproducibility
New recommendations for the design of animal studies could improve reproducibility of scientific results, reducing the number of studies required to confirm results and thus the overall number of animals used in research, according to an international team of researchers.
“Reproducibility, or the ability to produce similar results by independent replicate studies, is a key part of scientific advancement,” said Naomi Altman, professor emeritus of statistics at Penn State. “Results are of scientific interest only if they can be reproduced.”
Animal research, such as preclinical studies to validate new drugs for human health, typically relies on standardization of animal characteristics and the laboratory environment, for example by using only one strain, age, or sex of animal under strict housing conditions. While standardization is meant to improve reproducibility of scientific results, in reality reproducibility among studies is surprisingly low.
“This so-called ‘reproducibility crisis’ produces economic costs and scientific uncertainty, potentially attenuating medical progress,” said Hanno Würbel, professor of animal welfare at the University of Bern.
Last year, Altman, Würbel, and other experts in animal biology, experimental design, and biostatistics convened in Blonay, Switzerland to discuss strategies to address this challenge. They concluded that a paradigm shift in experimental design is urgently needed and present their conclusions in a perspectives article from that workshop appearing June 2 in the journal Nature Reviews Neuroscience.
Strict standardization of both the animals’ characteristics and their environment is the norm for studying the effects of a treatment, such as a drug. The intent of this standardization is to eliminate all sources of variation other than the experimental treatment, which is meant to increase the precision of the results and thus allow fewer animals to be used in each experiment.
However, this rigorous standardization reduces the range of animals and conditions to which their findings can reliably be generalized. For example, the results of a strictly standardized study in young male mice of a particular strain may not extend to older or female or other strains of mice, reducing the likelihood of reproducing these results in future studies.
The research team therefore advocates the deliberate inclusion of biological variation into the design of animal experiments to improve the ability to extend results beyond the specific study conditions and thus improve reproducibility. This “heterogenization” of animals and their environment can be produced using a technique for allocating animals to different treatments called a randomized block design.
“With this design we can balance the need to compare treatments under similar conditions, with the ability to introduce heterogeneity, which allows us to determine whether effects are robust over a range of conditions,” said Altman.
Researchers can introduce biological variation to study populations in many ways, for example, by including different strains of animals, sexes, age groups, or animals housed under different environmental conditions. Alternatively, they may split experiments into several independent batches of animals or even conduct multi-laboratory studies.
“There is no single best solution for every experiment,” said lead author and workshop host Bernhard Voelkl, senior scientist at the Animal Welfare Division of the University of Bern. “Therefore, we recommend heterogenization of animals and environmental conditions should be recommended in general terms. Researchers should justify their choices with respect to the range of animals and conditions to which their findings should generalize.”
Future research should explore these strategies to provide researchers better guidance in their choices.
The researchers suggest that by introducing biological variation into study design, fewer studies, and thus fewer animals, will be necessary to reproduce results. Therefore, although in some cases the number of animals used in a single study may increase, the overall number of animals used in research will be reduced.
“We propose a paradigm shift to increase the benefit of the research and reduce the overall number of animals used in research,” said Würbel, who co-hosted the workshop. “Instead of minimizing the number of animals per experiment, we need to maximize the amount of knowledge we gain per animal or study.”
To promote this paradigm shift, the team recommends that funders and editors ask researchers to use heterogenization as the default option and to justify experimental designs in terms of the intended range of conditions to which their findings should apply.
The workshop was funded by the Swiss National Science Foundation (SNSF) and the Animal Welfare Division of the University of Bern.