Impacts of Inter- and Intralaboratory Variations on the Reproducibility of Microbial Community Analyses

Abstract

With the advent of molecular biological techniques, especially next-generation sequencing and metagenomics, the number of microbial biogeography studies is rapidly increasing. However, these studies involve the synthesis of data generated by different laboratories using different protocols, chemicals, etc., all with inherent biases. The aim of this study was to assess inter- as well as intralaboratory variations in microbial community composition when standardized protocols are applied to a single soil sample. Aliquots from a homogenized soil sample from a rice field in Italy were sent to five participating laboratories. DNA was extracted by two investigators per laboratory using an identical protocol. All DNA samples were sent to one laboratory to perform DNA quantification, quantitative PCR (QPCR), and microarray and denaturing gradient gel electrophoresis (DGGE) analyses of methanotrophic communities. Yields, as well as purity of DNA, were significantly different between laboratories but in some cases also between investigators within the same laboratory. The differences in yield and quality of the extracted DNA were reflected in QPCR, microarray, and DGGE analysis results. Diversity indices (Shannon-Wiener, evenness, and richness) differed significantly between laboratories. The observed differences have implications for every project in which microbial communities are compared in different habitats, even if assessed within the same laboratory. To be able to make sensible comparisons leading to valid conclusions, intralaboratory variation should be assessed. Standardization of DNA extraction protocols and possible use of internal standards in interlaboratory comparisons may help in rendering a “quantifiable” bias.

Microorganisms comprise a major part of total biomass and biodiversity (21, 41-43, 49). They play a critical role in biogeochemical processes and ecosystem functioning (16). However, knowledge of ecology and functioning of environmental microbial communities is still far from complete, mainly due to our inability to grow the majority of environmental microbes under laboratory conditions. The introduction of culture-independent DNA- and RNA-based techniques has led to a revolution in environmental microbiology, yielding a wealth of information on community compositions in an ever-growing range of habitats. Phylogenetic as well as functional microarrays (51) and metagenomic techniques (41, 47) enable in-depth analyses and comparison of whole microbial communities in a high-throughput manner.

The collective goal of all environmental microbial ecology studies is 2-fold: (i) to obtain an overall understanding of microbial community composition, dynamics, and functioning and (ii) to identify regulating mechanisms. Reaching these goals will necessitate the integrated analyses of data generated in different laboratories and from different habitats. The first step in most if not all environmental microbial community studies is the extraction of total DNA from environmental samples in a way that reflects the in situ community composition as closely as possible. Numerous methods, protocols, and commercial kits have been developed to improve and optimize quantity and quality of extracted community DNA from a wide range of natural environments (4, 8, 28, 37, 39). However, up-to-date bias-free extraction methods are not available, especially not for complex and highly variable matrices, like soil. Beside the challenge of lysing all cells, the incomplete removal of compounds interfering with downstream processing render the development of a bias-free protocol a “mission impossible.” Assessments of the bias introduced by DNA extraction with different methods and kits on microbial community profiling revealed that a perfect protocol fitting all types of environments is not feasible (10, 17, 20, 45). However, in light of the global biodiversity debate, assessment of local and global patterns of microbial diversity and their controlling factors (19, 26) necessitates the comparison of data collected in multiple habitats and processed in different laboratories.

In contrast to other scientific disciplines, intercalibration of protocols is not common practice in environmental microbiology. Interlaboratory comparisons (ring analyses) have been applied commonly in food control, veterinary, forensic, and soil studies to evaluate, for example, Salmonella diagnostic accuracy (25), virus isolation (18), enzyme-linked immmunosorbent assay methods (2), mitochondrial DNA sequencing (30), soil microbial biomass C (3), and quantitative PCR (QPCR) (11). Ring analyses assessing the reproducibility of DNA extraction and subsequent community analyses between different laboratories have not been carried out so far in environmental microbial ecology.

A microbial functional guild that has been investigated intensively using molecular techniques is represented by the methanotrophs (aerobic methane-oxidizing bacteria [MOB]), which can be found in a wide variety of environments (27). The unique contribution of these bacteria to the global methane cycle has rendered the diversity and ecology of MOB hot topics for decades (9, 14, 34, 46, 48). By using methane as single source of carbon and energy, these microbes represent the only biological sink of the greenhouse gas methane under aerobic conditions (13). Aerobic MOB belong to the Gamma- and Alphaproteobacteria and the Verrucomicrobia (13, 34) and have the following features that enable linking function and identity. Assimilating methane facilitates the application of stable isotope probing of diagnostic lipids and of RNA/DNA (6, 29, 33). Besides this, the key gene in methane oxidation (for methanemonooxygenase subunit A, pmoA) reflects the phylogeny of these bacteria, facilitating a direct link between methane consumption and taxonomy. These features have made this group of microbes a model group for studies in environmental microbial ecology. Combined with the broad distribution and high environmental relevance, this group is highly suited to perform a ring analysis on reproducibility of DNA extraction and subsequent community profiling.

In the present study, five independent laboratories from Norway, Finland, Netherlands, Germany, and Austria extracted DNA from the same rice field soil sample, using identical protocols and performed by two different investigators per laboratory. Subsequently, the extracted DNA was sent to one laboratory, where DNA quantification, QPCR, microarray, and denaturing gradient gel electrophoresis (DGGE) analyses were performed by one and the same person. The impacts of inter- as well as intralaboratory variations of DNA extraction are discussed, and recommendations for comparative studies are presented.

MATERIALS AND METHODS

Soil.

Soil samples were collected in rice fields in Vercelli, Italy, on 30 March 2009 before flooding. The characteristics of this rice field soil have been described previously (24). Soil was frozen and freeze-dried in the laboratory of the Max-Planck Institute for Terrestrial Microbiology (Lab D; Marburg, Germany). After homogenizing by mortar and pestle and sieving through a 2-mm mesh, equal 10-g portions were sent to four additional laboratories (Lab A [Finland], Lab B [Netherlands], Lab C [Austria], and Lab E [Norway]) for DNA extraction.

DNA extraction.

DNA was extracted using a modification of the method described by Yeates and Gillings (50), based on the FastDNA spin kit for soil (MP Biomedicals, LLC, Solon, OH). In every laboratory two investigators (investigators 1 and 2) processed four replicate soil samples in parallel on the same day, using identical chemicals and machinery.

Soil (0.3 g) and 780 μl lysis buffer (200 mM NaPO₄ [pH 7.0], 1% cetyltrimethylammonium bromide, 1.5 M NaCl, 2% polyvinylpyrrolidone K30, and 5 mg/ml lysozyme [added right before use]) was added into a multimix FastPrep tube and incubated at 37°C for 30 min. MT buffer (122 μl) was added, and tubes were shaken in the FastPrep instrument (MP Biomedicals, LLC, Solon, OH) for 30 s at 5.5 m s⁻¹. Subsequently, samples were centrifuged for 15 min at 10,000 rpm, and 700 μl of supernatant was collected. The pellet was reextracted by adding lysis buffer (500 μl) and 50 μl MT buffer to the FastPrep tubes, which were shaken in the FastPrep instrument for 30 s at 5.5 m s⁻¹ again followed by the transfer of the second 700 μl of supernatant into separate Eppendorf tubes. At this step, two 700-μl aliquots of supernatant were obtained from each sample. Five microliters of a 10-mg ml⁻¹ freshly made proteinase K solution was added to each tube. Tubes were incubated at 65°C for 30 min. Samples were extracted with phenol-chloroform-isoamyl alcohol (25:24:1), followed by a chloroform-isoamyl alcohol (24:1) extraction. A 125-μl volume of 7.5 M potassium acetate was added, and samples were incubated on ice for 5 min and then centrifuged at 10,000 rpm for 10 min. Supernatants (two 700-μl aliquots per soil sample) were transferred to new tubes, 700 μl of binding matrix was added, and tubes were mixed for 5 min on a rotator. Binding matrix, with bound DNA, was pelleted by 1 min of centrifugation at 10,000 rpm. The supernatant was discarded, and the pellet was resuspended in 500 μl wash buffer. The resulting suspension was added into a Spinfilter and centrifuged for 1 min at 10,000 rpm. The eluate was discarded, and the pellet was washed again in 500 μl wash buffer. After discarding the second eluate, the Spinfilter was centrifuged for another 10 s to dry the pellet. The filter was placed in a new tube, and 50 μl of Tris-EDTA (TE; pH 8.0) was added. The filter was incubated at room temperature for 1 min and centrifuged for 1 min. The filter was reeluted in the same way with 50 μl of TE (pH 8.0). The eluate collected in the catch tube contained the purified DNA. The DNA was subsequently lyophilized at 40°C maximally and shipped within 24 h to the laboratory of the Netherlands Institute of Ecology (Nieuwersluis, Netherlands) for further processing. Upon arrival, all lyophilized DNA samples were dissolved in equal amounts (100 μl) of water, after which the analyses described below were executed.

DNA quantification.

The DNA concentration was assessed using two independent methods, NanoDrop and PicoGreen. The NanoDrop spectrophotometer (ND-1000; Nanodrop Technology, Wilmington, DE) gives the DNA concentration as well as purity indices (A₂₆₀/A₂₈₀, A₂₆₀/A₂₃₀, and A₃₂₀). The PicoGreen (Molecular Probes, Eugene, OR) assay kit uses an ultrasensitive fluorescent nucleic acid stain for quantitating double-stranded DNA in solution. A standard curve was constructed by a dilution series of 0, 0.002, 0.001, 0.01, 0.1, and 2 ng/μl of standard lambda phage DNA (100 ng/μl). For each sample, duplicate 50-μl aliquots of the DNA extracts supplemented with 50 μl of 1× PicoGreen solution were transferred to a 96-well microtiter plate. The plate was shaken for 5 min; fluorescence of DNA extracts was measured at 520 nm after excitation at 480 nm by using a microplate reader (BioTek). The DNA concentration was calculated from the standard curve. During the whole procedure, all samples were protected from light.

Diagnostic pmoA microarray analyses.

Microarray analysis was carried out using a modification of the method described in reference 44, as described below.

Target preparation.

The PCR amplification was based on a two-step seminested protocol. The first-step PCR mixture comprised 25 μl of 2× Premix F (Epicentre Biotechnologies, Madison, WI), 25 pmol of primers A-189f and A-682r (21a) each, 1 unit of Taq polymerase (Invitrogen), and 50 ng of genomic DNA as template, in a total volume of 50 μl. PCR was performed using a touchdown protocol with an initial incubation of 5 min at 94°C, then 25 cycles of 1 min at 94°C, 1 min at the annealing temperature, and 1 min at 72°C, followed by a final incubation of 10 min at 72°C. The annealing temperature was lowered from 62°C to 52°C over the first 11 cycles, after which it was maintained for a further 14 cycles at 52°C. Five microliters of 1/100-diluted PCR product from the first step was used as template in a subsequent nested amplification with primers A-189f and T7-661r. The second-step PCR was performed with a total 25 cycles, with an initial incubation of 5 min at 94°C, then 1 min at 94°C, 1 min at the annealing temperature, and 1 min at 72°C, followed by a final incubation of 10 min at 72°C. The annealing temperature was lowered from 62°C to 52°C over the first 11 cycles, after which it was maintained for a further 14 cycles at 52°C.

IVT.

In vitro transcription (IVT) was carried out under RNase-free conditions using the following procedure: 7 μl purified PCR product (50 ng μl⁻¹; Qiagen kit), 4 μl of 5× T₇ RNA polymerase buffer, 2 μl dithiothreitol (100 mM), 0.5 μl RNasin (40 U μl⁻¹; Promega, Madison, WI), 1 μl each of ATP, CTP, and GTP (10 mM), 0.5 μl of UTP (10 mM), 1 μl T₇ RNA polymerase (40 U ml⁻¹; Invitrogen), and 1 μl Cy3-UTP (5 mM; GE Healthcare, Uppsala, Sweden) were added into a 1.5-ml tube and incubated at 37°C for 4 h. RNA was purified immediately by using the RNeasy minikit (Qiagen, Valencia, CA). Eight microliters of diethyl pyrocarbonate (DEPC)-treated water, 350 μl RLT, and 250 μl ethanol were added to the IVT mixture, followed by thorough mixing. Samples were transferred to an RNeasy minitube. Subsequently, 500 μl of RPE was added, followed by centrifugation at 10,000 rpm for 15 s. Another 500 μl of RPE was added, followed by 2 min of centrifugation at 10,000 rpm. The purified RNA was eluted into 50 μl DEPC-treated water and subsequently fragmented by incubation with 9.5 mM ZnCl₂ and 24 mM Tris-Cl (pH 7.4) at 60°C for 30 min. Fragmentation was stopped by the addition of 12 mM EDTA (pH 8.0) to the reaction mixture and putting it on ice. RNasin (1 μl of a 40-U/μl solution) was added to the fragmented target.

Hybridization.

Microarray slides from the same production batch, spotted with the same batch of probes, were used throughout the analyses in this study to minimize effects that could be introduced by spotting. Hybridization was carried out in a Belly Dancer hybridization water bath (Stovall Life Sciences, Greensboro, NC) that was preheated to 55°C for at least 1 h. For each hybridization mixture, 62 μl DEPC-treated water, 1 μl 10% SDS, 30 μl 20× SSC (1× SSC is 0.15 M NaCl plus 0.015 M sodium citrate), 2 μl 50× Denhardt's reagent, and 5 μl target RNA were added into a 1.5-ml tube and incubated at 65°C for 1 min. The preheated hybridization mixture was applied onto assembled slides with a HybriWell (Grace BioLabs, Bend, OR). In the hybridization chamber, the slides were incubated overnight at 55°C at maximum bending and the lowest rpm setting of the Belly Dancer shaking plateau. In the washing procedure following the hybridization, the slides were washed by shaking at room temperature for 5 min in 2× SSC, 0.1% (wt/vol) SDS, twice for 5 min in 0.2× SSC, and finally for 5 min in 0.1× SSC. Slides were dried individually using pressurized air.

Scanning and data analysis.

Hybridized slides were scanned at 10-μm resolution with a GenePix 4000 laser scanner (Axon, Foster City, CA) at a wavelength of 532 nm. Fluorescent images were analyzed with the GenePix software (Axon). Microsoft Excel was used for statistical analyses and presentation of results.

QPCR.

Three methanotrophic subgroups (Table 1) were quantified by pmoA-based quantitative PCR based on assays described elsewhere (22). The type Ia and II assays were carried out as described previously (7). For the type Ib assay, DNA standards were prepared by dilution of a known amount of PCR product amplified from a reference clone by using the 189-Mc468 primer set (22). A 25-μl reaction mixture contained 12.5 μl 2× SYBR green mix (AB Gene, Epsom, United Kingdom), 2.5 μl of diluted DNA template, and 0.8 mM (each) primers. The samples were diluted accurately to 1 ng/μl. The thermal cycle started with an initial denaturation at 95°C for 15 min, followed by 45 cycles of denaturation at 95°C for 20 s, annealing at 64°C for 20 s, and extension at 72°C for 45 s. Fluorescence was recorded at 84°C, and DNA melting curve analysis was performed at temperatures ranging from 70°C to 99°C. All three assays were performed with a RotorGene 6000 thermal cycling system (Corbett Research, Eight Mile Plains, Concorde, QLD, Australia), and samples were added to aliquots of the master mixture by using a CAS-1200 liquid handling system (Corbett Robotics, Eight Mile Plains, Concorde, QLD, Australia). Every sample was analyzed in duplicate. Quantification analysis was performed by using the RotorGene software. Cell numbers were calculated assuming PCR product lengths of 412 bp, 279 bp, and 423 bp for type Ia, type Ib, and type II methanotrophs, respectively.

TABLE 1.

Explanation of MOB group indicators used in the microarray and QPCR analyses^a

Group indicator	Genera detected	Phylogenetic affiliation
Type Ia	Methylobacter, Methylomonas, Methylomicrobium, Methylosarcina	Gammaproteobacteria
Type Ib	Methylococcus, Methylocaldum	Gammaproteobacteria
Type II	Methylocystis, Methylosinus	Alphaproteobacteria

^aShown are the genera covered by the group indicators (types) and the respective phylogenetic affiliations of these genera.

16S rDNA gene PCR-DGGE.

DGGE analysis was performed as described previously (5), with the following exception. The PCR mixture contained 25 μl of 2× Premix F (Epicentre Biotechnologies), 25 pmol of each primer, 1 unit of Taq polymerase (Invitrogen), and 50 ng of genomic DNA as template. One microliter of the first-step PCR product was added to the second step. The gels were loaded with 2 μl of loading buffer and 23 μl of PCR product.

Statistical analyses.

The effects of laboratory and investigator on DNA extraction method, DNA concentration, purity, and QPCR results were analyzed using a nested analysis of variance design, where the investigator was nested within laboratory. Post hoc comparisons between laboratories were analyzed using Student's t test with a Bonferroni correction for the number of comparisons. All data were checked for normality and homogeneity of variances and log transformed, if necessary. The analysis was carried out using the Statistica software package, version 8.0 (Statsoft Inc., Tulsa, OK).

Analyses of microarray data.

The normalized signal intensities derived from the scanned microarrays formed the matrix that was used for nonmetric multidimensional scaling (MDS) analyses. The inputs of MDS analyses were Bray-Curtis similarity matrices generated using the log(x + 1)-transformed signal intensity values, to even out the contributions of very rare and very dominant probe signals. The MDS analyses result in a two-dimensional plot, where the distance between samples indicates the similarity of the samples relative to other samples in the plot. The accuracy of the two-dimensional representation is indicated by the “stress” value (Kruskall's stress formula). Stress values of <0.1 indicate a good ordination with no prospect of misleading interpretation. Stress values of <0.2 still give a good two-dimensional representation where not too much reliance should be put on the detail. In this case other methods of representation should be used in parallel, such as clustering analyses. The ANOSIM procedure was used to test whether microarray community profiles obtained in different laboratories differed. All MDS analyses were performed using the Primer-E software (Plymouth Marine Laboratory, Plymouth, United Kingdom). Theoretical aspects of the cluster and MDS analyses used have been described elsewhere (12). Due to repetition of single-sample DNA extraction in different countries, the effect of country on positive microarray probe was tested with a linear mixed effect model that was fit for observations taken on related individuals. Analysis used the R program add-on package nlme (38).

RESULTS

DNA concentration and quality.

DNA concentration was analyzed using two different methods, the NanoDrop and PicoGreen assays (Fig. 1). It is obvious that the DNA concentrations measured with the NanoDrop assay were on average double the amounts measured with PicoGreen. Besides this, for both assays there were significant effects of laboratory as well as for investigator (Fig. 1A; see also Table S1 in the supplemental material). Laboratories A and B extracted more DNA than the other labs, while in three laboratories there were significant differences between investigators (Fig. 1B; see also Table S1 in the supplemental material). Differences between laboratories were also observed in the purity of the DNA (i.e., the A₂₆₀/A₂₈₀) (Fig. 2; see also Table S1). However, yield and purity only correlated for the PicoGreen method and not for DNA concentrations assessed by the NanoDrop method (see Table S4 in the supplemental material). An investigator effect on DNA purity was only observed in laboratory C (Fig. 2).

FIG. 1.

DNA concentrations (means ± 1 standard deviation) as analyzed with NanoDrop or PicoGreen, showing the comparisons between laboratories (A) and between investigators in the various laboratories (B). Different letters in panel A indicate significant differences between countries (P < 0.05, unequal honestly significant differences test). In panel B, the asterisk indicates a significant difference between investigators within one laboratory (as assessed using Students's t test; P < 0.01).

FIG. 2.

Purity of extracted DNA, measured as the A₂₆₀/A₂₈₀ ratio (means ± 1 standard deviastion). Different letters indicate significant differences between laboratories (unequal n honestly significant differences test, P < 0.05). The arrow indicates a significant difference between investigators within that laboratory.

Quantitative PCR.

To test the effect of different DNA concentration measurements on QPCR results, both methods were compared for the samples processed in laboratory C (Fig. 3). QPCR assays targeted three groups of MOB, type Ia, type Ib, and type II. In all assays, numbers of MOB were significantly higher when initial target input amounts were based on the NanoDrop than the PicoGreen method (Fig. 3; see also Table S2 in the supplemental material). Except for type II pmoA copy numbers, there were also significant differences between the investigators (see Table S4). In Fig. 4, the QPCR data for all laboratories are presented for three different groups of MOB. Again, there were significant differences between laboratories, which were largest for type Ia, for which the numbers from laboratories E and C were lower than the rest (Fig. 4; see also Table S3 in the supplemental material). For all QPCR assays, there was no investigator effect (see Table S3). Copy numbers in all assays correlated strongly with the DNA purity (i.e., the A₂₆₀/A₂₈₀ ratio) and DNA concentration based on the PicoGreen assay (see Table S4).

FIG. 3.

Effect of DNA quantitation method on pmoA gene copy number (means ± 1 standard deviation) in rice soil assessed using three different assays targeting subgroups of methane-oxidizing bacteria, executed in one laboratory only. The numbers 1 and 2 in parentheses indicate the results of the two different investigators from the Austrian lab.

FIG. 4.

pmoA gene copy numbers (means ± 1 standard deviation) in rice soil of three different subgroups of methane-oxidizing bacteria as assessed in five different laboratories. Different letters per subgroup of MOB indicate significant differences between laboratories (unequal n honestly significant differences test, P < 0.05; n = 8, except for Lab E [n = 6]).

pmoA microarray analysis.

The diversity of the MOB was assessed using a pmoA diagnostic microarray. In analogy to the QPCR analysis, microarray analysis was carried out with samples from laboratory C and an initial target concentration based on the NanoDrop as well as PicoGreen DNA concentration assays. The nonmetric multidimensional scaling shown in Fig. 5 displays the dissimilarities between methanotrophic community patterns derived from DNA batches extracted in one laboratory. What is graphically obvious was also confirmed by ANOSIM: community profiles based on different DNA concentrations differed significantly from each other (Fig. 5). Microarray community profiles generated with DNA from different laboratories showed obvious differences (Fig. 6). A global ANOSIM for the laboratory effect was highly significant (P < 0.001), while there was no effect of investigator. Linear mixed effect model analyses indicated that only the profiles from laboratories A and B did not differ from each other (see Table S5 in the supplemental material). Overall, 56 probes showed positive signals. DNA from laboratory D gave higher signal intensities for eight probes than results from laboratories A and B. On the other hand, DNA from laboratories C and E gave 15 and 14 significantly reduced probe intensities, respectively. For all laboratories and investigators, 59% of potentially positive probes were not detected or showed a reduced intensity at least once. Most of these changes happened within species-specific type II probes (67% out of 21 probes were affected). However, group-specific probes for type II methanotrophs were consistently positive, while type Ia- and type Ib-specific probes differed between laboratories (see Table S5).

FIG. 5.

Nonmetric multidimensional scaling plot using log-transformed Bray-Curtis dissimilarity matrices based on signal intensity values of pmoA microarray analyses, performed on the basis of the NanoDrop or PicoGreen DNA quantitation method. Distances between symbols represent relative dissimilarity between MOB communities. Analyses of similarity (ANOSIM) resulted in a significant difference between MOB community structure when based on different DNA concentration measurements (n = 8).

FIG. 6.

Nonmetric multidimensional scaling plot using log-transformed Bray-Curtis dissimilarity matrices based on signal intensity values of the pmoA microarray analyses on DNA extracted in five different laboratories. Distances between symbols represent relative dissimilarity between MOB communities. Analyses of similarity (ANOSIM) resulted in a significant difference between MOB community structure analyzed in the different laboratories. Only samples from laboratory A and B did not differ from each other (n = 8, except for laboratory E [n = 6]).

DGGE analysis.

The DGGE patterns were rather similar in terms of the number of bands (i.e., operational taxonomic units) detected (see Fig. S1 in the supplemental material). However, the intensities of the bands differed. Since equal amounts of extracted DNA were added to the PCR mixture and equal volumes of PCR product were loaded on the gels, the intensity differences could be related to the amounts of targets in the extracts. There are differences between laboratories as well as between investigators. DNA from laboratory A, also representing the largest amount of extracted DNA, gave the highest band intensities for type I as well as for type II. Laboratories with the highest investigator differences (B and C) observed in the pmoA-based analyses also displayed the highest variability by 16S-based PCR-DGGE.

DISCUSSION

Source of variation.

The data presented in this paper clearly demonstrate that even the application of identical protocols to a single soil sample leads to significantly different community composition. It is well known that different DNA extraction protocols differ in their efficiency of liberating and purifying DNA from various environmental matrices with consequences for microbial community composition (32). This study adds another component of variation, the laboratory or the researcher carrying out the analyses. Applying the same protocol to the same sample, different laboratories with different investigators ended up with different MOB community profiles. Different methods of assessing the community composition as well as different markers (the 16S rRNA gene and pmoA) confirmed the conclusion that the origin of the differences can only reside in the DNA extraction procedure. The dependency of the outcome of microbial diversity analyses on the laboratory and even on an individual researcher has serious implications for any project or meta-analysis where results from different laboratories have to be compared.

Since all analyses following the DNA extraction were carried out in the same laboratory, using the same equipment and chemicals and executed by the same person, the source of variation between laboratories came from the extraction procedure. One explanation may be that not all laboratories used the same bead-beating machine. This may have caused the rather low DNA yield and purity in laboratory E, which used a different bead beater than the other laboratories involved. The bead beating has been found to increase the yield of soil DNA (23); however, so far there are no reports presenting data on DNA extraction efficiency as a function of the type of bead beater used.

However, laboratories using the same bead-beating machine (i.e., Labs B, C, and D) also arrived at different results. Here, the source of variation may reside in the chemicals (buffers, enzymes, solvents, etc.) used in the extraction procedures. However, the intralaboratory comparisons between two investigators using exactly the same equipment as well as chemicals also yielded significant differences in some cases. The latter can only be caused by variation in parts of the protocol that are subject to handling variation by the investigator, which can only be the pipetting routine. In the protocol, there are phase separation steps where the DNA is in the upper phase of the extract. The upper layer has to be removed without taking anything from the lower layers containing the contaminants. The efficiency of the removal of the upper phase will determine the DNA yield in a 1:1 ratio. This step requires pipetting practice and may very well differ between investigators. Nevertheless, the effect between laboratories is still greater than that between investigators. Hence, the observed variation is a mix of local differences in equipment or chemicals and also the technical skills of the person carrying out the analyses.

In addition to the phase separation step, the quality of phenol used in this part of the protocol may affect the results. Phenol (pK_a 9.95) is a sensitive and reactive organic compound, having a hydroxyl group in a benzole backbone. It has a crucial role in the phase separation step by removing proteins, polyphenols, and polysaccharides from the aquatic phase, in which the DNA resides. If the phenol is not in optimal condition, the quality of the extracted DNA will be lower. In our study all laboratories used the same mixing ratios of phenol, chloroform, and isoamyl alcohol in the phase separation step except for laboratory D, which used water-saturated phenol of pH 4.0 that contained 8-hydroxyquinoline as a stabilizer. The other laboratories used Tris-HCl-buffered phenol of pH 7.0 to 8.0 and without stabilizer. The difference in pH itself is not crucial, but the storage at low pH in the presence of the stabilizer preserves the phenol better, possibly leading to a higher purity of DNA and higher values for microarray probes in the case of laboratory D.

A critical factor investigated in this study was the method of DNA concentration measurement. The popular NanoDrop method is very fast and needs only small aliquots. However, it measures DNA as well as RNA, and it is affected by the impurities in the extract. This method gave double the amount of DNA as the PicoGreen method, which only detects double-stranded DNA. The suggested overestimation by the NanoDrop method may add to the variability of final results: when starting to dilute the DNA in order to add a fixed amount to the PCR, the extract is diluted more than it would be when using the concentration measured using PicoGreen. The former method may lower the influence of residual PCR-inhibiting compounds. This may contribute to interlaboratory differences when the DNA concentration is measured with different assays. It is difficult to say which assays perform better. There are reports that above concentrations of 10 ng μl⁻¹ both assays perform similarly, while below this concentration PicoGreen performs better (1, 15). The reproducibility is in the same range for both assays, as indicated by the coefficient of variation (see Fig. S2 in the supplemental material) and the positive correlation (see Table S4).

Consequences for MOB community structure assessment.

To the best of our knowledge, there are no combined inter- and intralaboratory studies on methanotrophic community structure against which to compare our results. However, it is obvious that quantitative comparisons between MOB communities have to be treated with caution, even if done in one laboratory but by different investigators. DNA extraction yields and purity may be significantly different, as are, e.g., the subsequent QPCR results. Diversity measures considering quantitative data may also be compromised. The Shannon diversity index based on the richness and evenness (microarray analyses) shows that there are significant differences between laboratories (Fig. 7). These differences already occur at the very beginning of every molecular investigation, at the DNA extraction procedure. The purity of DNA (the A₂₆₀/A₂₈₀ ratio) correlated positively with the Shannon and eveness indices as well as MOB abundance as assessed by QPCR, indicating that yield as well as purity influence the downstream results. In addition, higher single-probe intensities in the case of the eight probes of laboratory D could be connected to the higher purity of the DNA (see Table S5 in the supplemental material). Comparing diversity and/or abundance between habitats can be almost impossible without some information on the bias introduced at the extraction stage. A strategy overcoming these problems may be to use internal controls during DNA extraction, as previously described (31, 35, 40). However, the relative abundance as measured by QPCR stayed rather constant between laboratories. Hence, the ratios between numbers of types Ia, Ib, and II were not significantly affected by laboratory or investigator and were also not correlated to DNA quantity or quality (see Table S4). Therefore, statements such as “in this habitat there is more type II than in another habitat” or “this habitat is more diverse than another” cannot be made when analyses have been carried out in different laboratories or by different investigators, without a proper assessment of laboratory or investigator bias. What is valid is to test the effects of environmental change (assessed in different laboratories) on normalized relative abundances of MOB, which were not affected by DNA purity or yield. In a recent study it was shown that denitrification gene abundance was correlated with activity and environmental parameters only when relative abundances were used in the analyses (36). This may be caused by the fact that relative abundance levels as measured with independent assays are not influenced by biases in DNA extraction. Also, changes in community structure as assessed in one laboratory are valid results, because the same bias will be introduced at every time point, assuming that DNA yield, purity, and the investigator are identical.

FIG. 7.

Shannon index, species richness (i.e., positive probes), and evenness (means ± 1 standard deviation) of MOB communities assessed by pmoA microarray analyses, on DNA extracted in five different laboratories. Different letters indicate statistically significant differences (unequal honestly significant differences test, P < 0.05; n = 8, except for laboratory E [n = 6]).

Conclusions.

The quality and quantity of extracted DNA from a single soil sample by using identical protocols can differ between laboratories and investigators, resulting in significant bias in downstream molecular analyses. Comparison of results from different experiments necessitates assessment of this bias. To minimize the bias, we recommend performing all DNA extractions in one laboratory using identical chemicals and machinery. In a large-scale project with multiple research groups involved, all downstream processing steps should be executed using PCR ingredients purchased from the same company, and preferably from the same production batch. However, in many projects these criteria are difficult to meet. An alternative strategy would be to develop an internal control system which gives information on inter- and intralaboratory biases. Internal standards should not only help validate the quality of downstream processing but also, preferably, the DNA extraction step. In this way a correction factor might be developed, which would allow a reproducible interpretation of molecular data. For post hoc meta-analyses not meeting these requirements, however, only robust algorithms should be used, in order to avoid overinterpretation of differences.