In Vitro Reproducibility and Experimental Conditions:

If another lab tried to reproduce your results, would they fail?

It’s a scary thought.

Amongst the recent publications on the Crisis in Reproducibility, there has been extensive discussion of the importance of factors from raw materials through published reports. These include; improper statistics [1], experimental design, controls, reagent validation, and reference standards [2], cell line misidentification [3], animal cage environment [4], raw data availability [5], and data reporting [6].

Beyond equipment validation, only one author has addressed variability in the in vitro cell microenvironment, arguing that it is one factor that makes the push for replication unwise. [7]

Were the cells out of optimum conditions for the same length of time this experiment as the last one?

As cell culturists, we were all taught to minimize the length of time that cells are out in the Biological Safety Cabinet (BSC). We all know that even if it is clean, room air is suboptimal for cell cultures.

Consider the Cytocentric approach, figuratively taking your cells’ point of view. Compared to a standard 37 degree, CO2-controlled incubator environment, room air in a standard BSC is hyperoxic, hypocapnic and uncomfortably cold. When incubating cells at physiologically relevant oxygen levels, room air is even further out of optimum.

Moving cell cultures out of the incubator for routine cell handling results in variations in the cellular microenvironment that are difficult to replicate over time. Different cell culturists in different laboratories handle cells for different lengths of time. Even in a GLP or GMP cleanroom, where every part of a cell production process has been planned in excruciating detail, cells are routinely handled in variably suboptimal conditions.

Altered critical cell parameters induce mechanical and phenotypic changes in cultured cells.

When exposing cells to suboptimal conditions on a regular basis, researchers are relying upon the cells’ own abilities to compensate. There is a limited capacity for cells to bounce back from cell handling in room air. Cell culture temperature changes produce instantaneous effects on cellular mechanics [8] as well as longer term variations in cell signaling and phenotype [9]. Even once the cells are returned to the incubator, dissolved CO2 changes alter culture pH, which affects cellular function [10]. It takes time for critical cell parameters to return to optimum once cultures are returned to the incubator after handling in HEPA-filtered room air, and during that time all these changes affect cells.

Length of time in suboptimal conditions is not often considered a variable in cell-based experiments. The effects will remain unseen unless researchers make the relevant comparisons.

Incubator conditions vary, beyond temperature and CO2 recovery, after room air incursion.

Due to high CO2 and humidity, the partial pressure of oxygen in an incubator is necessarily lower than room air, and it is driven down further every time the door closes and CO2 levels charge back up to 5%. The smaller the incubator and the more frequently the door is opened, the more variable the oxygen levels inside. The presence of other experiments in the same incubator is another factor that contributes to variable frequency of incubator breaches. See a recent blog post here for relevant data.

Was the incubator door opened the same number of times today as yesterday? Unless researchers actively monitor incubator oxygen levels, the effect of varying laboratory operations upon the cells in culture are impossible to know.

Pericellular oxygen levels vary.

How long has that medium been sitting in the fridge? It’s looking a little purple. The pH is off. It needs some CO2. Is the dissolved O2 different, too?

The time to equilibrate medium oxygen levels to the incubator is dependent upon factors such as starting oxygen partial pressure, medium depth, vessel type and air circulation in the incubator. Overall, oxygen is quite slow to equilibrate in static culture. Cultured cells could be out of optimum conditions for hours to days depending upon the protocol and that length of time could vary from experiment to experiment, batch to batch, and lab to lab.

These are factors that make the length of time that cells are out of optimum an important issue in the Reproducibility discussion.

The practical aspects of tackling variability in the cellular microenvironment are not unconquerable.

It is possible to eliminate variation in critical cell parameters with full-time optimization of cellular handling and incubation conditions.

Is this an issue that is simply overlooked when we discuss issues of reproducibility or are key opinion leaders avoiding this topic? Is there an elephant in the room air?

Contact us here at the Cytocentric blog with your opinion.

References:

1. 1. Ioannidis, J.P.,Why most published research findings are false. Chance, 2005.18(4): p. 40-47.
2. 2. Freedman, L.P. and M.C. Gibson,The impact of preclinical irreproducibility on drug development. Clin Pharmacol Ther, 2015.97(1): p. 16-8.
3. 3. Freedman, L.P.,Know Thy Cells: Improving Biomedical Research Reproducibility. Sci Transl Med, 2015.7(294): p. 294ed7.
4. 4. Toth, L.A.,The influence of the cage environment on rodent physiology and behavior: Implications for reproducibility of pre-clinical rodent research. Exp Neurol, 2015.270: p. 72-7.
5. 5. Ioannidis, J.P.,Anticipating consequences of sharing raw data and code and of awarding badges for sharing. J Clin Epidemiol, 2015.
6. 6. Kenall, A., et al.,Better reporting for better research: a checklist for reproducibility. Gigascience, 2015.4: p. 32.
7. 7. Bissell, M.,The risks of the replication drive. Nature, 2013.503(7476): p. 333-334.
8. 8. Kießling, T.R., et al.,Thermorheology of living cells—impact of temperature variations on cell mechanics. New Journal of Physics, 2013.15(4): p. 045026.
9. 9. Digel, I., P. Kayser, and G. Artmann,Molecular processes in biological thermosensation. Journal of Biophysics, 2008.2008.
10. 10. Lo, C.-M., C.R. Keese, and I. Giaever,pH changes in pulsed CO 2 incubators cause periodic changes in cell morphology. Experimental cell research, 1994.213(2): p. 391-397.