Skip to main content
PMC home page
PLOS ONE logoLink to PLOS ONE
. 2012 Feb 17;7(2):e30572. doi: 10.1371/journal.pone.0030572

Human Tumor Cell Proliferation Evaluated Using Manganese-Enhanced MRI

Rod D Braun 1,2,*, David Bissig 1, Robert North 1, Kerry S Vistisen 1, Bruce A Berkowitz 1,3
Editor: Juri G Gelovani4
PMCID: PMC3281834  PMID: 22363447

Abstract

Background

Tumor cell proliferation can depend on calcium entry across the cell membrane. As a first step toward the development of a non-invasive test of the extent of tumor cell proliferation in vivo, we tested the hypothesis that tumor cell uptake of a calcium surrogate, Mn2+ [measured with manganese-enhanced MRI (MEMRI)], is linked to proliferation rate in vitro.

Methodology/Principal Findings

Proliferation rates were determined in vitro in three different human tumor cell lines: C918 and OCM-1 human uveal melanomas and PC-3 prostate carcinoma. Cells growing at different average proliferation rates were exposed to 1 mM MnCl2 for one hour and then thoroughly washed. MEMRI R1 values (longitudinal relaxation rates), which have a positive linear relationship with Mn2+ concentration, were then determined from cell pellets. Cell cycle distributions were determined using propidium iodide staining and flow cytometry. All three lines showed Mn2+-induced increases in R1 compared to cells not exposed to Mn2+. C918 and PC-3 cells each showed a significant, positive correlation between MEMRI R1 values and proliferation rate (p≤0.005), while OCM-1 cells showed no significant correlation. Preliminary, general modeling of these positive relationships suggested that pellet R1 for the PC-3 cells, but not for the C918 cells, could be adequately described by simply accounting for changes in the distribution of the cell cycle-dependent subpopulations in the pellet.

Conclusions/Significance

These data clearly demonstrate the tumor-cell dependent nature of the relationship between proliferation and calcium influx, and underscore the usefulness of MEMRI as a non-invasive method for investigating this link. MEMRI is applicable to study tumors in vivo, and the present results raise the possibility of evaluating proliferation parameters of some tumor types in vivo using MEMRI.

Introduction

Uncontrolled cellular proliferation is the hallmark of cancer, and proliferation rate, i.e., the rate of tumor cell division, is linked to prognosis for several types of cancer [1], [2], [3], [4]. Currently the only method to spatially monitor local tumor cell proliferation in vivo is positron emission tomography (PET), which uses the accumulation of 18F-labeled 39-deoxy-39-fluorothymidine (18F-FLT), fluorodeoxyglucose (18F-FDG), or 2–11C thymidine (11CTdR) as a proliferation marker [5]. While application of PET as a method of detecting proliferation in vivo remains promising, its spatial resolution is limited compared to other imaging modalities, such as MRI. Bading and Shields acknowledge that “an effective and clinically practical means for the imaging of cell proliferation is still an unrealized objective.”[5].

Cell proliferation is usually associated with an increase in cytoplasmic calcium ion, either from the extracellular space or from intracellular calcium stores [6], [7], [8]. Much of the extracellular Ca2+ enters the cell via calcium-permeable channels [6], [7], [8]. Indeed, tumor cell proliferation has been specifically linked to calcium ion channel activity in some, but not all, tumors [6], [7], [9], [10], [11], suggesting that calcium ion channel activity could be a useful surrogate marker of tumor cell proliferation. A powerful method for investigating calcium ion channel activity in vivo is monitoring the extent of tissue uptake of manganese ion, Mn2+, a Ca2+ analog [12], [13]. Manganese can enter cells via calcium ion channels, particularly through voltage-gated channels [12], [13], although other routes, including transferrin receptor-mediated or DMT1-dependent routes, may also contribute [14], [15]. Importantly, Mn2+ accumulates intracellularly due to a slow rate of efflux and acts as an MRI contrast agent by increasing the tissue longitudinal relaxation rate (R1 = 1/T1) in proportion to manganese concentration [16], [17]. Manganese-enhanced MRI (MEMRI) has been successfully used to functionally image brain [16], [18], [19], [20], [21] and retinal [22], [23], [24] activity, as well as the activity of other tissues [25]. These considerations suggest that MEMRI might be usefully applied to monitor tumor cell proliferation.

Free Mn2+ ion is known to accumulate in tumors in vivo [25], [26], [27], [28], [29]. Previously, using MEMRI in a nude rat model, we demonstrated significant Mn2+ uptake by C918 uveal melanoma xenografts relative to surrounding tissues and speculated that the tumor-specific Mn2+ uptake may have been a result of high proliferation rates within the tumor [30].

In this study, we test the hypothesis that tumor cell uptake of a calcium surrogate, Mn2+ (measured with MEMRI), is linked to tumor cell proliferation rate and will be a biomarker of proliferation in at least some tumors. Specifically, the growth of three human tumor cell lines was characterized in vitro and their proliferation rates were correlated to MEMRI R1 (1/T1) values.

Materials and Methods

Human Tumor Cell Lines

Three different human tumor cell lines were used in this study. The human uveal melanoma cell lines C918 and OCM-1 were used, because we had previously shown that C918 cells took up Mn2+ in vitro and in vivo [30], and we wished to further investigate this class of tumor. The OCM-1 cells were originally cultured from a human choroidal melanoma specimen in the 1980's [31], while the C918 cells were derived from a patient tumor in 1996 at the University of Iowa [32]. To extend the analysis of Mn2+ uptake to a type of cancer with a higher clinical incidence, we also investigated the human prostate carcinoma line PC-3, which was originally generated from a bone metastasis of a grade IV prostatic adenocarcinoma [33]. All cells were maintained in RPMI media +10% fetal bovine serum (FBS) + antibiotic under standard incubating conditions. Cells were seeded into 6-well plates or standard tissue culture flasks at a density of 20.8 cells/mm2.

Measurement of Proliferation Rate of Human Tumor Cell Lines

To describe the growth of the tumor cells in vitro, cells were trypsinized, harvested, and counted on different days after seeding. The cell density was determined by dividing the cell number by the area of the well or flask. The number of cell divisions over that period of days, D, was calculated as:

graphic file with name pone.0030572.e001.jpg (1)

where D = the number of cell divisions, C = tumor cell density (cells/mm2), and C0 = initial cell density, i.e., the seeding concentration (cells/mm2). These data were fit to a Weibull growth model [34] using nonlinear least-squares regression, while setting C0 = 20.8 cells/mm2 (GraphPad Prism, GraphPad Software, Inc., La Jolla, CA). The Weibull model was chosen after fit comparisons with other common growth models, including the Gompertz and sigmoid logistic models [35]. Model comparisons using Akaike's Information Criterion (AIC) method [36] showed that the Weibull model was the better model (AIC probabilities>99.5%) compared to either the Gompertz model or the sigmoid logistic model.

Since the Weibull model best described the growth of the cells, we used the corresponding equation to describe the number of cell divisions as a function of time:

graphic file with name pone.0030572.e002.jpg (2)

where Cmax = maximum cell density (cells/mm2), κ = the inverse of the time constant (1/daysν), and ν = dimensionless constant. The proliferation rate, dD/dt (divisions/day), is given by:

graphic file with name pone.0030572.e003.jpg (3)

MEMRI of Human Tumor Cell Pellets

Based on the tumor growth curves, cells were used for these experiments on different days after seeding. First 19.8 µl of stock MnCl2 solution (10 mg MnCl2•4H2O/ml of 0.9% saline; 50.5 mM MnCl2) was added to each ml of RPMI medium in the flask to reach a concentration of 1 mM MnCl2. The solution was left on the cells for one hour in the incubator. The cells were then rinsed with Hanks' balanced salt solution and trypsinized. After cell detachment was evident, the trypsin was quenched with RPMI media +10% FBS. The cells were centrifuged, and the resultant pellet was washed twice more in RPMI +10% FBS. The cells were counted and resuspended to a concentration of 1.5×106 cells/100 µl. Two hundred µl of the suspension were placed in a 0.65 ml microcentrifuge tube, yielding a total of 3×106 cells in the tube. The cells were pelleted by gravity for at least 20 minutes. At early time points, e.g., Days 2 or 3, it was sometimes necessary to use more than one flask to obtain 3×106 cells. The microcentrifuge tubes were placed in a 7 T magnet (Bruker ClinScan , Billerica, MA), and the average longitudinal relaxation time (T1) for each pellet was determined using a partial saturation T1 approach as follows. Several spin-echo images (two 2.0 mm thick slices – one through pellets and one through supernatants – spaced 4 mm apart; matrix size 464×576, field of view 46×57 mm, TE 13 ms) were acquired at different repetition times in the following order (with number of acquisitions per TR in parentheses): TR 0.15 s (6), 3.50 s (1), 1.00 s (2), 1.90 s (1), 0.35 s (4), 2.70 s (1), 0.25 s (5), and 0.50 s (3). Images acquired with the same TR were averaged offline using ImageJ (http://rsbweb.nih.gov/ij/). Average signal intensity (SI, based on a circular region-of-interest encompassing most of the cells in each tube) from each sample varies as a function of TR according to a monoexponential function:

graphic file with name pone.0030572.e004.jpg

where a, b, and T1 are the fitted parameters. This function was fit to data using the Levenberg-Marquardt nonlinear least-squares algorithm in the minpack.lm library (v.1.1.1, by Timur V. Elzhov and Katharine M. Mullen) for R [37]. The R1 (1/T1) values (longitudinal relaxation rates) directly reflect manganese levels [16], [17].

Each pellet likely contained a heterogeneous cell population, e.g., cells in different phases of the cell cycle, and, potentially, correspondingly distinct values for R1, and this could confound our estimates of the pellet R1. As a first approximation, we assumed a fast-exchange limit wherein the exchange of water between heterogeneous compartments (including cell populations) is fast relative to the acquisition of the T1 data set. In this case, if the individual populations had different T1 values, that information would be blurred and would show up as a single exponential. To check this assumption, in a preliminary study using images from a typical run of nine pellets, we compared fits of monoexponential and biexponential functions to the intensity vs. TR data. When the data were fit to the monoexponential model, the fitted parameters had tight confidence intervals. In contrast, the biexponential fits resulted in parameter values with very large confidence intervals, indicative of non-unique parameter values (data not shown). Since these models are nested, i.e., one model is an extension of the other, an extra sum-of-squares F test was used to statistically compare the fits [36]. The F-test showed that the monoexponential fit was adequate to describe the data in every case and that there was no reason to invoke the more complex biexponential model (p>0.13, n = 9). These results are consistent with results from phantom experiments, which showed that T1 values have to differ by a factor of 2 to 3 to be reliably distinguished by a biexponential fit [38]. Therefore, all of the signal intensity data in this study were fit to the monoexponential equation noted above.

Cell Cycle Analysis

For C918 cells, separate flasks were seeded at a concentration of 20.8 cells/mm2, and cells were harvested on different days after seeding. The washed cells were fixed in 70% ethanol by adding absolute ethanol dropwise to the cell suspension. The cells were stored at −20°C until flow cytometry could be performed on multiple samples. Three hours before flow cytometric analysis, the cells were washed three times in Hanks' balanced salt solution and were stained with a 50 µg/ml propidium iodide solution in the presence of 0.5 µg/ml RNase A. DNA content of each sample was determined by flow cytometry on a BD LSR II flow cytometer (BD Biosciences, San Jose, CA). The percentages of cells in G0/G1 phase, S phase, and G2/M phase were determined using ModFit LT software (Verity Software House, Topsham, ME). Flow cytometry was performed by the Microscopy, Imaging, and Cytometry Resources Core at The Karmanos Cancer Institute, Wayne State University.

For the PC-3 cells, a portion of the cells from the same flask that was used to perform the MEMRI experiments was processed for flow cytometry and analyzed in the same manner as described above.

To functionally describe the relationship between the percentage of cells in S phase and the proliferation rate, dD/dt, three different models were compared using Akaike's Information Criterion (AIC) method [36]: the sigmoid logistic model, the Weibull model, and a simple monoexponential model. The 3-parameter logistic model resulted in fits with lower sum-of-squares errors than the 3-parameter Weibull model for both cell lines. The AIC test revealed that the symmetric sigmoid logistic model was superior to the monoexponential model (AIC probability>70%). Therefore, the relationship is best described by:

graphic file with name pone.0030572.e005.jpg (4)

where S = percentage of cells in S phase (%), dD/dt = proliferation rate (divisions/day), Smax = maximum percentage of cells in S phase (%), dS = constant (days/division), and τS = value of dD/dt at which S is half its maximum value (divisions/day).

Forms of the same three functions were similarly compared to test their ability to adequately describe the relationship between the percentage of cells in G0/G1 phase and dD/dt. The 4-parameter logistic model resulted in fits with lower sum-of-squares errors than the 4-parameter Weibull model for both cell lines. Since the AIC test revealed that the symmetric sigmoid logistic model was the better model (probability>60%) compared to the monoexponential model, the sigmoid logistic model was used:

graphic file with name pone.0030572.e006.jpg (5)

where G01 = fraction of cells in G0/G1 phase (%), GA = constant (%), GB = constant (%), dG = constant (days/division), and τG = constant (divisions/day). Note that GA or GB taken individually has no biological significance and could be >100, but GA-GB is the minimum percentage of cells in G0/G1 phase (%).

The sum of all three cell fractions must be 100:

graphic file with name pone.0030572.e007.jpg

where G2M = fraction of cells in G2/M phase (%). Solving this equation for G2M yields:

graphic file with name pone.0030572.e008.jpg (6)

Substituting Equations 4 and 5 into Equation 6 yields the following expression for G2M:

graphic file with name pone.0030572.e009.jpg (7)

Statistical Analysis

The fits of the cellular growth data to the logistic, Gompertz, and Weibull models were compared using Akaike's Information Criterion (AIC) method [36]. This test was used rather than an extra sum-of-squares F test, because the models are not interrelated, i.e., they are non-nested [36]. The AIC method was also used to compare the fits of the percentage of cells in S phase or G0/G1 phase versus proliferation rate data to the sigmoid logistic, Weibull, and exponential models.

Two-way ANOVA analysis was used to compare intracellular R1 values among the three cell lines with or without Mn2+ exposure. Tukey's HSD post-hoc comparison tests were used to check for differences between any two groups. Two-way ANOVA and Tukey's HSD were performed using R [37]. A p-value<0.05 was considered statistically significant.

Correlations between R1 values and proliferation rate were determined using linear regression analysis (GraphPad Software, Inc., La Jolla, CA). A regression p-value<0.05 was considered statistically significant.

The lack of fit of the weighted-average model to the experimental data was determined using the lack-of-fit ANOVA-based F-test [39], [40]. Using the sum of squares of the measurement and modeling errors, it tests the hypothesis that the lack of fit of the model curve is much greater than the measurement error [40]. A p-value<0.05 was considered statistically significant, indicating the model provided an incomplete description of the data.

Results

Tumor Cell Growth and Proliferation Rate

As shown in Figure 1, the growth data for all three cell lines were well described by the Weibull model (Equation 2), with all coefficients of determination equal to 0.97. Proliferation rates were calculated from the fitted parameters using Equation 3 (Figure 1B). C918 cells had the highest maximum proliferation rate of 2.1 divisions/day, which occurred 1.7 days after seeding. The OCM-1 cell line showed a maximum proliferation rate of 1.5 divisions/day at 2.4 days after seeding. The PC-3 cells grew the slowest with a maximum proliferation rate of 1.3 divisions/day at 2.5 days after seeding.

Figure 1. Tumor Cell Line Growth and Proliferation Rate.

Figure 1

Open in a new tab

A) Growth of three human tumor cell lines, expressed as number of divisions after seeding at a concentration of 20.8 cells/mm2 on Day 0. PC-3 (▾, n = 84), C918 (•, n = 102), and OCM-1 (▪, n = 72) data were fit to the Weibull model (Equation 2). For clarity the mean ± SEM on each day are shown. Fitted parameters: PC-3: Cmax = 1988, κ = −0.00115, ν = 3.30, r2 = 0.972; C918: Cmax = 9636, κ = −0.000844, ν = 3.52, r2 = 0.969; OCM-1: Cmax = 3748, κ = −0.000553, ν = 3.62, r2 = 0.969. B) Proliferation rates of the cell lines, as calculated from Equation 3: PC-3 (——), C918 (— —), and OCM-1 (----).

Tumor Cell Mn2+ Uptake

A 2×3 two-way ANOVA analysis revealed a significant impact of Mn2+ exposure (p<0.0001) and of cell type (p<0.0001) on the R1 value of the cell pellets, regardless of their proliferation rate (Figure 2). Post-hoc analysis demonstrated that, as expected, for each cell line, the R1 of cells exposed to Mn2+ for one hour and then rinsed was significantly greater than the R1 of corresponding cells not exposed to Mn2+ (p<0.0001, Tukey's HSD test, Figure 2). R1, and thus Mn2+ uptake, in the PC-3 and OCM-1 cell lines was greater than R1 in the C918 cells (p<0.0001). In addition, the R1 value of the PC-3 cells was significantly less than that of the OCM-1 cells (p = 0.001). When no Mn2+ was added, there were no differences among any of the pellet R1 values (Tukey's HSD test, p = 1.000). Supernatant R1 values, regardless of whether the cells were exposed to Mn2+ or not, ranged between 0.63 and 0.87 sec−1.

Figure 2. Comparison of Tumor Cell Pellet and Supernatant R1 Values.

Figure 2

Open in a new tab

Average R1 values for tumor cell pellets (solid symbols) and supernatants (open symbols) after exposure to 1 mM MnCl2 or media without added Mn2+. Two-way ANOVA analysis of cell pellet R1 values revealed a significant impact of Mn2+ exposure (p<0.0001) and cell type (p<0.0001). Tukey's HSD test, Mn2+-exposed vs. no Mn2+: *p<0.0001. Tukey's HSD test, cell lines: a: p<0.0001 vs. C918 pellet, b: p = 0.001 vs. OCM-1 pellet. When no Mn2+ was added, there were no differences among the pellet R1 values (Tukey's HSD test, p = 1.000).

Relationship between Tumor Cell Mn2+ Uptake and Proliferation Rate

The R1 values for pellets of cells following MnCl2 exposure are shown as a function of proliferation rate in Figure 3. In PC-3 prostate cancer cells, the Mn2+-enhanced R1 values for the cell pellets were positively correlated with proliferation rate (Figure 3A, r2 = 0.284, p = 0.005, n = 26), while the R1 values of cell pellets in the absence of Mn2+ were not correlated with proliferation rate (r2 = 0.146, p = 0.618, n = 4). C918 human uveal melanoma cells also took up Mn2+ (Figure 3B), and there was a significantly positive correlation between MEMRI R1 and proliferation rate (r2 = 0.502, p<0.0001, n = 40). Again, R1 values of cell pellets in the absence of Mn2+ were not correlated with proliferation rate (r2 = 0.111, p = 0.317, n = 11). The OCM-1 cells showed no correlation between Mn2+-dependent R1 and proliferation rate (Figure 3C, r2 = 4.94×10−5, p = 0.975, n = 22) or between R1 and proliferation rate in the absence of Mn2+ (r2 = 0.040, p = 0.799, n = 4).

Figure 3. Relationships between Tumor Cell Pellet R1 and Proliferation Rate.

Figure 3

Open in a new tab

Correlation between MEMRI R1 values in tumor cell pellets exposed to Mn2+ (solid symbols) and cellular proliferation rate as calculated from Figure 1B for A) PC-3 prostate carcinoma (▾), B) C918 uveal melanoma (•), and C) OCM-1 uveal melanoma (▪). Linear regressions: PC-3: R1 = 0.310(proliferation rate)+1.239; r2 = 0.284; n = 26, p = 0.0005. C918: R1 = 0.239(proliferation rate)+0.744; r2 = 0.502; n = 40, p<0.0001. OCM-1: R1 = -0.00294(proliferation rate)+1.63; r2 = 4.94×10−5; n = 22, p = 0.975. Open symbols represent R1 values of cell pellets in the absence of Mn2+. Note that the abscissas and ordinates have different scales in each panel.

Phases of the Cell Cycle and Proliferation Rate

Since there was a positive correlation between MEMRI R1 values and proliferation rate in the PC-3 and C918 cell lines, cell cycle changes in those cell lines were also investigated. As expected, the fraction of cells in each of the phases of the cell cycle changed with proliferation rate (Figure 4). In both cell lines, the fraction of cells in S phase increased with proliferation rate, while the fraction of cells in G0/G1 decreased. The percentage of cells in S phase as a function of proliferation rate, dD/dt, was well described by Equation 4 for both the PC-3 cells (r2 = 0.921, n = 29, Figure 4A) and the C918 cells (r2 = 0.920, n = 20, Figure 4B). Similarly, Equation 5 adequately described the percentage of cells in the G0/G1 phase as a function of dD/dt for both the PC-3 cells (r2 = 0.967, n = 29, Figure 4A) and the C918 cells (r2 = 0.848, n = 20, Figure 4B). Equation 10 reasonably described the changes in the G2/M fraction of cells as well (Figure 4).

Figure 4. Relationships between Cell Cycle Fractions and Proliferation Rate.

Figure 4

Open in a new tab

Percentage of cells in different phases of the cell cycle as a function of proliferation rate for PC-3 (A) and C918 (B) cells. S-phase data were fit to Equation 4. PC-3 (A): Smax = 43.6, ds = 5.0, τs = 0.54 (r2 = 0.921, n = 29); C918 (B): Smax = 40.1, ds = 4.7, τs = 0.63 (r2 = 0.920, n = 20). G0/G1 data points were fit to Equation 5. PC-3 (A): GA = 104.2, GB = 62.37, ds = 4.05, τG = 0.392 (r2 = 0.967, n = 29); C918 (B): GA = 75.9, GB = 30.48, ds = 11.13, τG = 0.791 (r2 = 0.848, n = 20). The curves describing the G2/M were calculated from Equation 7, using the above parameter values. For clarity the mean ± SEM on each day are shown (n = 3–10 values per point).

Modeling the Relationship between Tumor Cell Mn2+ Uptake and Proliferation Rate

It is possible to derive a simple, general model to describe the positive correlation between Mn2+-induced changes in R1 and proliferation rate in the PC-3 and C918 cell lines (Figure 3). The cell pellets at each proliferation rate have at least three distinct subpopulations of cells in different phases of the cell cycle: G0/G1, S, and G2/M (Figure 4). If each subpopulation takes up a different amount of Mn2+, each subpopulation would have its own R1 value, and the average R1 of any mixed cell population would be the weighted average of these individual cell cycle-specific values. Thus, the overall R1 of the pellet is described by the following weighted average:

graphic file with name pone.0030572.e010.jpg (8)

where (R1)pellet = R1 value of cell pellet (sec−1), RS = R1 value of a cell in S phase (sec−1), R01 = R1 value of a cell in G0/G1 phase (sec−1), and RM = R1 value of a cell in G2/M phase (sec−1). From Figure 4, it is known that the size of each subpopulation in the pellet changes with proliferation rate, and the parameters S, G01, and G2M can be expressed as functions of proliferation rate. Substituting Equation 6 into Equation 8 and simplifying yields:

graphic file with name pone.0030572.e011.jpg (9)

Substituting Equations 4 and 5 into Equation 9 yields:

graphic file with name pone.0030572.e012.jpg (10)

The only unknowns in this equation are RS, R01, and RM. It is not known whether the amount of Mn2+ taken up by each subpopulation remains constant or changes with proliferation rate, i.e., RS, R01, and RM are either constants or functions of dD/dt. Based on this reasoning, there are only two possible explanations for the positive correlation between the average cell pellet R1 and proliferation rate. Either it is attributable to proliferation rate-dependent changes in the fraction of cells in each subpopulation alone (RS, R01, and RM are constant) or it is caused by proliferation rate-dependent changes in both the subpopulation distribution and in Mn2+ uptake of at least one of the subpopulations.

With the available data, it is possible to directly test the first possibility, i.e., that the three subpopulations demonstrate unique amounts of ion flux (R01, RS, and RM), but these amounts are not affected by proliferation rate. In other words, the correlation between R1 and proliferation is solely due to the relative size of each subpopulation. Application of this simple model will help guide future studies by indicating whether the second more complicated model needs to be further investigated.

The PC-3 and C918 cellular R1 values following MnCl2 exposure as a function of proliferation rate shown in Figure 3 were fit to Equation 10 (with RS, R01, and RM as constants) using nonlinear least-squares regression (GraphPad Prism,GraphPad Software, Inc., La Jolla, CA). Cell line-specific constant parameters were obtained from the fits shown in Figure 4.

For the PC-3 cells, the model fit of all 26 data points (Figure 5A, solid line, r2 = 0.304, n = 26) passes through the mean values (open symbols). A lack-of-fit ANOVA-based F-test [39], [40] revealed that the model adequately described the data (p = 0.978). In other words, given the variance in the PC-3 data and the quality of the present model fit, it is not possible for an alternative model to produce a significantly better fit. The best-fit values of R01, RS, and RM with the 95% confidence intervals (95% CI) were 1.16 (95% CI: 0.82 to 1.51) sec−1, 1.86 (95% CI: 0.66 to 3.07) sec−1, and 2.02 (95% CI: −1.87 to 5.91) sec−1, respectively. Despite the wide confidence intervals (a consequence of uncertainty in RM), if the value for R01 is fixed within its 95% CI (between 0.82 and 1.51 sec−1), all solutions yield fits in which R01<RS. As an additional check that R01<RS, the four mean values were fit to the model (Figure 5A, dashed line). In this case, the fit was indistinguishable from the curve obtained when all of the individual points were fit to the model (solid line). The resulting fit yielded an r2 of 1.000 (n = 4) and the following parameter values: R01 = 1.16 (95% CI: 1.07–1.25), RS = 1.86 (95% CI: 1.54–2.18), and RM = 2.02 (95% CI: 0.98–3.05). From the confidence intervals, it is evident that R01 is always less than RS, but that RM cannot be as reliably predicted. Based on these results, the correlation between PC-3 pellet R1 and proliferation rate is consistent with a change in the sizes of the cellular subpopulations during proliferation, without a change in their individual R1 values.

Figure 5. Weighted-Average Model Describing Cell Pellet MEMRI R1 Values as Function of Proliferation Rate.

Figure 5

Open in a new tab

Solid symbols are the individual R1 values for PC-3 (A: ▾) or C918 cells (B: •) harvested on the same day of growth at the corresponding proliferation rate. Open symbols are the mean ± SEM of the R1 values for PC-3 (A: ▽) or C918 cells (B: ○). Solid lines show the fits of individual R1 values to Equation 10, using constants given in Figure 4. Fitted parameters: A) PC-3: R01 = 1.16 sec−1, RS = 1.86 sec−1, and RM = 2.02 sec−1; r2 = 0.304; n = 26. B) C918: R01 = 0.67 sec−1, RS = 1.57 sec−1, and RM = 1.07 sec−1; r2 = 0.436; n = 40. Dashed lines show the fits of the mean R1 values to Equation 10, using constants given in Figure 4. Fitted parameters: A) PC-3: R01 = 1.16 sec−1, RS = 1.86 sec−1, and RM = 2.02 sec−1; r2 = 1.000; n = 4. B) C918: R01 = 0.64 sec−1, RS = 1.62 sec−1, and RM = 1.26 sec−1; r2 = 0.722; n = 7. Note that the dashed line underlies the solid line in panel A.

For the C918 cells, a fit of the 40 cell pellet R1 values following MnCl2 exposure to Equation 10 yielded the curve shown in Figure 5B (solid line, r2 = 0.436, n = 40). The best-fit values of R01, RS, and RM with the 95% confidence intervals (95% CI) were 0.67 (95% CI: 0.48 to 0.86) sec−1, 1.57 (95% CI: 1.17 to 1.97) sec−1, and 1.07 (95% CI: −0.20 to 2.33) sec−1, respectively. Although, on visual inspection, the model does follow the general trend of the data, there are notable deviations. At low proliferation rates, the model predicts a more rapid change in pellet R1 than seen in the data, resulting in a large overestimate of R1 at a proliferation rate of 0.9 divisions/day (Figure 5B). Similarly, at high proliferation rates, the model predicts a plateau in the pellet R1 value, which is not evident in the experimental data. Consistent with these observations, a lack-of-fit ANOVA-based F-test [39], [40] revealed that the model does not adequately describe the data (p = 0.017). The assumption of constant R1 values for C918 cells in the different phases of the cell cycle appears questionable.

Discussion

In this study, in two of three cancer cell lines, we found that MEMRI R1 values, which reflect cellular Mn2+ uptake, changed with tumor cell proliferation rate. These data underscore the usefulness and sensitivity of MEMRI to the known heterogeneous nature of different tumor cell proliferation rates. Although we and others have shown that tumor cells accumulate Mn2+ when exposed to MnCl2 and that Mn2+ changes the cellular MEMRI R1 values [25], [27], [28], [29], [30], this is the first time that the Mn2+-induced R1 changes have been correlated with changes in proliferation rate. In other studies, increased tumor Mn2+ uptake has been correlated with increased tumor neuroendocrine activity [27] or tumor cell Mn-superoxide dismutase (Mn-SOD) levels [29].

Tumor Cell Mn2+ Uptake

Tumor cell Mn2+ uptake could occur by several different mechanisms, including expression and/or activity of Ca2+ channels, transferrin receptors, and divalent metal-ion transporter-1 (DMT1) channels [14], [15]. Both voltage-gated Ca2+ channels (VGCC) [13], [41], [42] and some TRP channels [43], [44], [45] are permeable to Mn2+, and the expression of these channels has been linked to tumor cell proliferation [6], [7], [10], [11], [46], [47], [48]. Although we cannot rule out a role for transferrin receptor-mediated or DMT1-dependent Mn2+ uptake, preliminary studies in our laboratory suggest that VGCCs, most likely T-type, are important regulators of Mn2+ uptake in the PC-3 and C918 cell lines. Therefore, we speculate that the difference in MEMRI R1 values, i.e., Mn2+ uptake, among the three cell lines, is related primarily to differences in the expression or activity of VGCCs.

The differences in Mn2+ uptake and cellular R1 values among the three cell lines (Figure 2) are most likely the result of different expression or activity of various calcium ion channels or other transport mechanisms responsible for Mn2+ entry into the cells. Interestingly, the C918 cells took up the least Mn2+, i.e., had the lowest pellet R1 values, even though they had the highest proliferation rates (Figure 1B). The OCM-1 cells, in which Mn2+ uptake was not correlated with proliferation rate, had the highest R1 values and an intermediate proliferation rate. These results demonstrate that the absolute level of Mn2+ uptake (R1 value) is not correlated with proliferation rate across cell types. In other words, a high R1 value does not mean that a particular cell line proliferates more rapidly than another cell line with a lower average R1. Rather, it is the change in the pellet R1 value for a specific cell line that is indicative of a change in cellular proliferation rate.

As expected, substantial leakage of Mn2+ out of the cells into the overlying supernatant did not occur, suggesting stable intracellular accumulation of manganese and adequate wash procedures. The lack of any significant Mn2+ leakage out of the cells after removal of the MnCl2 and washing of the cells implies a relatively slow rate of Mn2+ efflux out of the cells, similar to what has been reported in neuronal tissues [23], [49].

Relationship between Tumor Cell Pellet MEMRI R1 and Proliferation Rate

Our initial hypothesis was based on reports that intracellular Ca2+ levels change during cellular proliferation and that these changes may, at least in part, be caused by an increase in the uptake of extracellular Ca2+ [6], [7], [8]. Since Mn2+ is a calcium surrogate, this uptake of Ca2+ might be detectable as a change in intracellular Mn2+ measured using MEMRI. Thus, we hypothesized a positive correlation between average MEMRI tumor cell pellet R1 value and cellular proliferation rate.

Two of the three tumor cell lines tested, PC-3 and C918, revealed such a positive correlation between average cell pellet MEMRI R1 and proliferation rate. No correlation was found for the OCM-1 uveal melanoma cell line. There are several possible, but not mutually exclusive, explanations for this result. First, it is possible that the proposed link between increased Ca2+ influx and proliferation is not universal to all tumor cell lines. For example, in the OCM-1 cells, the cytosolic Ca2+ change that helps drive proliferation may be dominated by release of intracellular Ca2+ stores, rather than uptake of extracellular Ca2+ [6], [7], [8]. Alternatively, extracellular Ca2+ may enter the OCM-1 cells through Mn2+-impermeable Ca2+ channels, so that the Ca2+ changes are not detected by MEMRI. Third, while a proliferation-related increase in Mn2+ uptake may occur through certain routes, the change in MEMRI R1 could be too small to be detected over the background of high baseline Mn2+ permeability through alternative proliferation-independent routes. This last possibility is consistent with the fact that the OCM-1 cells had the highest Mn2+ uptake (R1) of all three cell lines, regardless of the proliferation status (Figure 2). At this point, additional work is needed to identify the underlying reasons for a lack of a significant correlation in the OCM-1 cells. In any event, these data highlight the sensitivity of MEMRI to differential cell calcium handling in various tumor cell types.

As presented in the Results section, the simple weighted-average model (Equation 10) assumes that the changes in pellet R1 can be fully explained by changes in the relative distribution of cells in different phases of the cell cycle (each with a cell-cycle specific constant R1) as proliferation rate changes. For the PC-3 cells, the relationship between the average pellet R1 and proliferation rate was adequately described by this model (Figure 5A). The results of the modeling suggest that PC-3 cells in the G0/G1 phase of the cell cycle took up less Mn2+, i.e., had a lower R1 value, than cells in S phase. The dramatic increase in the fraction of cells in S phase during proliferation and concomitant decrease in the fraction of G0/G1 cells (Figure 4A) seems most likely responsible for the positive correlation between PC-3 pellet R1 and proliferation rate (Figure 2A).

For the C918 cells, the poorer fit of the model to the data (Figure 5B) suggests that the simple weighted-average model with constant cell cycle-specific R1 values is insufficient to completely describe the relationship between average cell pellet R1 and proliferation rate. This result suggests that the Mn2+ uptake and the R1 value of one or more of the cell subpopulations changed with proliferation rate, but more work is needed to investigate this possibility.

In summary, MEMRI is a useful non-invasive method for accurately measuring the link between tumor calcium channel activity and tumor proliferation in vitro. Future studies will investigate whether these proliferation-related changes in MEMRI R1 can be confirmed in vivo.

Acknowledgments

The authors wish to thank Drs. Mary Hendrix and Karla Daniels for supplying the C918 cells, Dr. June Kan-Mitchell for providing the OCM-1 cells, and Dr. Lisa Anne Polin for supplying the PC-3 cells. The Microscopy, Imaging and Cytometry Resources Core is supported, in part, by NIH Center grant P30CA22453 to The Karmanos Cancer Institute, Wayne State University and the Perinatology Research Branch of the National Institutes of Child Health and Development, Wayne State University.

Footnotes

Competing Interests: The authors have declared that no competing interests exist.

Funding: This work was supported by a Pilot Project Grant from Karmanos Cancer Institute (RDB, BAB), National Institutes of Health (NIH) EY018109 (BAB), NIH National Eye Institute Departmental Core Grant (P30EY04068), NIH Center Grant to the Karmanos Cancer Institute (P30CA22453), Juvenile Diabetes Research Foundation (BAB), NIH AG034752 (DB), Wayne State University School of Medicine MD/PhD program (DB), and an unrestricted grant from Research to Prevent Blindness (Kresge Eye Institute). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

References

  • 1.Buhmeida A, Pyrhonen S, Laato M, Collan Y. Prognostic factors in prostate cancer. Diagn Pathol. 2006;1:4. doi: 10.1186/1746-1596-1-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 2.Minner S, Jessen B, Stiedenroth L, Burandt E, Köllermann J, et al. Low level Her2 overexpression is associated with rapid tumor cell proliferation and poor prognosis in prostate cancer. Clin Cancer Res. 2010;16:1553–1560. doi: 10.1158/1078-0432.CCR-09-2546. [DOI] [PubMed] [Google Scholar]
  • 3.Urruticoechea A, Smith IE, Dowsett M. Proliferation marker Ki-67 in early breast cancer. J Clin Oncol. 2005;23:7212–7220. doi: 10.1200/JCO.2005.07.501. [DOI] [PubMed] [Google Scholar]
  • 4.Valera V, Yokoyama N, Walter B, Okamoto H, Suda T, et al. Clinical significance of Ki-67 proliferation index in disease progression and prognosis of patients with resected colorectal carcinoma. Br J Surg. 2005;92:1002–1007. doi: 10.1002/bjs.4858. [DOI] [PubMed] [Google Scholar]
  • 5.Bading JR, Shields AF. Imaging of cell proliferation: status and prospects. J Nucl Med. 2008;49:64S–80. doi: 10.2967/jnumed.107.046391. [DOI] [PubMed] [Google Scholar]
  • 6.Lipskaia L, Lompre AM. Alteration in temporal kinetics of Ca2+ signaling and control of growth and proliferation. Biol Cell. 2004;96:55–68. doi: 10.1016/j.biolcel.2003.11.001. [DOI] [PubMed] [Google Scholar]
  • 7.Munaron L, Antoniotti S, Fiorio Pla A, Lovisolo D. Blocking Ca2+ entry: a way to control cell proliferation. Curr Med Chem. 2004;11:1533–1543. doi: 10.2174/0929867043365008. [DOI] [PubMed] [Google Scholar]
  • 8.Berridge MJ, Bootman MD, Lipp P. Calcium - a life and death signal. Nature. 1998;395:645–648. doi: 10.1038/27094. [DOI] [PubMed] [Google Scholar]
  • 9.Fiske J, Fomin V, Brown M, Duncan R, Sikes R. Voltage-sensitive ion channels and cancer. Cancer Metast Rev. 2006;25:493–500. doi: 10.1007/s10555-006-9017-z. [DOI] [PubMed] [Google Scholar]
  • 10.Panner A, Wurster RD. T-type calcium channels and tumor proliferation. Cell Calcium. 2006;40:253–259. doi: 10.1016/j.ceca.2006.04.029. [DOI] [PubMed] [Google Scholar]
  • 11.Munaron L, Antoniotti S, Lovisolo D. Intracellular calcium signals and control of cell proliferation: how many mechanisms? J Cell Mol Med. 2004;8:161–168. doi: 10.1111/j.1582-4934.2004.tb00271.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12.Anderson M. Mn ions pass through calcium channels. A possible explanation. J Gen Physiol. 1983;81:805–827. doi: 10.1085/jgp.81.6.805. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13.Narita K, Kawasaki F, Kita H. Mn and Mg influxes through Ca channels of motor nerve terminals are prevented by verapamil in frogs. Brain Res. 1990;510:289–295. doi: 10.1016/0006-8993(90)91379-u. [DOI] [PubMed] [Google Scholar]
  • 14.Erikson KM, Thompson K, Aschner J, Aschner M. Manganese neurotoxicity: a focus on the neonate. Pharmacol Ther. 2007;113:369–377. doi: 10.1016/j.pharmthera.2006.09.002. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 15.Roth JA. Homeostatic and toxic mechanisms regulating manganese uptake, retention, and elimination. Biol Res. 2006;39:45–57. doi: 10.4067/s0716-97602006000100006. [DOI] [PubMed] [Google Scholar]
  • 16.Lin YJ, Koretsky AP. Manganese ion enhances T1-weighted MRI during brain activation: an approach to direct imaging of brain function. Magn Reson Med. 1997;38:378–388. doi: 10.1002/mrm.1910380305. [DOI] [PubMed] [Google Scholar]
  • 17.Silva AC, Lee JH, Aoki I, Koretsky AP. Manganese-enhanced magnetic resonance imaging (MEMRI): methodological and practical considerations. NMR Biomed. 2004;17:532–543. doi: 10.1002/nbm.945. [DOI] [PubMed] [Google Scholar]
  • 18.Koretsky AP, Silva AC. Manganese-enhanced magnetic resonance imaging (MEMRI). NMR Biomed. 2004;17:527–531. doi: 10.1002/nbm.940. [DOI] [PubMed] [Google Scholar]
  • 19.Wadghiri YZ, Blind JA, Duan X, Moreno C, Yu X, et al. Manganese-enhanced magnetic resonance imaging (MEMRI) of mouse brain development. NMR Biomed. 2004;17:613–619. doi: 10.1002/nbm.932. [DOI] [PubMed] [Google Scholar]
  • 20.Yu X, Wadghiri YZ, Sanes DH, Turnbull DH. In vivo auditory brain mapping in mice with Mn-enhanced MRI. Nat Neurosci. 2005;8:961–968. doi: 10.1038/nn1477. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21.Bissig D, Berkowitz BA. Manganese-enhanced MRI of layer-specific activity in the visual cortex from awake and free-moving rats. NeuroImage. 2009;44:627–635. doi: 10.1016/j.neuroimage.2008.10.013. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 22.Berkowitz BA, Gradianu M, Bissig D, Kern TS, Roberts R. Retinal ion regulation in a mouse model of diabetic retinopathy: natural history and the effect of Cu/Zn superoxide dismutase overexpression. Invest Ophthalmol Vis Sci. 2009;50:2351–2358. doi: 10.1167/iovs.08-2918. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 23.Berkowitz BA, Roberts R, Goebel DJ, Luan H. Noninvasive and simultaneous imaging of layer-specific retinal functional adaptation by manganese-enhanced MRI. Invest Ophthalmol Vis Sci. 2006;47:2668–2674. doi: 10.1167/iovs.05-1588. [DOI] [PubMed] [Google Scholar]
  • 24.Berkowitz BA, Roberts R, Oleske DA, Chang M, Schafer S, et al. Quantitative mapping of ion channel regulation by visual cycle activity in rodent photoreceptors in vivo. Invest Ophthalmol Vis Sci. 2009;50:1880–1885. doi: 10.1167/iovs.08-2958. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 25.Seshadri M, Hoy A. Manganese-enhanced MRI of salivary glands and head and neck tumors in living subjects. Magn Reson Med. 2010;64:902–906. doi: 10.1002/mrm.22452. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 26.Tamano H, Enomoto S, Oku N, Takeda A. Preferential uptake of zinc, manganese, and rubidium in rat brain tumor. Nucl Med Biol. 2002;29:505–508. doi: 10.1016/s0969-8051(02)00289-5. [DOI] [PubMed] [Google Scholar]
  • 27.Cross DJ, Flexman JA, Anzai Y, Sasaki T, Treuting PM, et al. In vivo manganese MR imaging of calcium influx in spontaneous rat pituitary adenoma. AJNR. 2007;28:1865–1871. doi: 10.3174/ajnr.A0693. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 28.Banerjee D, Hegedus B, Gutmann DH, Garbow JR. Detection and measurement of neurofibromatosis-1 mouse optic glioma in vivo. NeuroImage. 2007;35:1434–1437. doi: 10.1016/j.neuroimage.2007.02.019. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 29.Hasegawa S, Koshikawa-Yano M, Saito S, Morokoshi Y, Furukawa T, et al. Molecular imaging of mesothelioma by detection of manganese-superoxide dismutase activity using manganese-enhanced magnetic resonance imaging. Int J Cancer. 2010 doi: 10.1002/ijc.25547. [DOI] [PubMed] [Google Scholar]
  • 30.Braun RD, Gradianu M, Vistisen KS, Roberts RL, Berkowitz BA. Manganese-enhanced MRI of human choroidal melanoma xenografts. Invest Ophthalmol Vis Sci. 2007;48:963–967. doi: 10.1167/iovs.06-1156. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 31.Kan-Mitchell J, Mitchell MS, Rao N, Liggett PE. Characterization of uveal melanoma cell lines that grow as xenografts in rabbit eyes. Invest Ophthalmol Vis Sci. 1989;30:829–834. [PubMed] [Google Scholar]
  • 32.Daniels KJ, Boldt HC, Martin JA, Gardner LM, Meyer M, et al. Expression of type VI collagen in uveal melanoma: its role in pattern formation and tumor progression. Lab Invest. 1996;75:55–66. [PubMed] [Google Scholar]
  • 33.Kaighn ME, Narayan KS, Ohnuki Y, Lechner JF, Jones LW. Establishment and characterization of a human prostatic carcinoma cell line (PC-3). Invest Urol. 1979;17:16–23. [PubMed] [Google Scholar]
  • 34.Weibull W. A statistical distribution function of wide applicability. J Appl Mech. 1951;18:293–297. [Google Scholar]
  • 35.López S, Prieto M, Dijkstra J, Dhanoa MS, France J. Statistical evaluation of mathematical models for microbial growth. Int J Food Microbiol. 2004;96:289–300. doi: 10.1016/j.ijfoodmicro.2004.03.026. [DOI] [PubMed] [Google Scholar]
  • 36.Motulsky H, Christopoulos A. Fitting models to biological data using linear and nonlinear regression: a practical guide to curve fitting. New York: Oxford University Press; 2004. [Google Scholar]
  • 37.R Development Core Team. R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing. ISBN: 3-900051-07-0. 2009. Available: http://www.R-project.org. Accessed 2012 Jan 9.
  • 38.Kjaer L, Thomsen C, Larsson HB, Henriksen O, Ring P. Evaluation of biexponential relaxation processes by magnetic resonance imaging. A phantom study. Acta Radiol. 1988;29:473–479. [PubMed] [Google Scholar]
  • 39.Ritz C, Martinussen T. Lack-of-fit tests for assessing mean structures for continuous dose-response data. Environ Ecol Stat. 2011;18:349–366. [Google Scholar]
  • 40.Schokker EP, van Boekel AJS. Kinetic Modeling of Enzyme Inactivation: Kinetics of Heat Inactivation at 90–110°C of Extracellular Proteinase from Pseudomonas fluorescens 22F. J Agr Food Chem. 1997;45:4740–4747. [Google Scholar]
  • 41.Drapeau P, Nachshen DA. Manganese fluxes and manganese-dependent neurotransmitter release in presynaptic nerve endings isolated from rat brain. J Physiol. 1984;348:493–510. doi: 10.1113/jphysiol.1984.sp015121. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 42.Shibuya I, Douglas WW. Indications from Mn-quenching of Fura-2 fluorescence in melanotrophs that dopamine and baclofen close Ca channels that are spontaneously open but not those opened by high [K+]O; and that Cd preferentially blocks the latter. Cell Calcium. 1993;14:33–44. doi: 10.1016/0143-4160(93)90016-y. [DOI] [PubMed] [Google Scholar]
  • 43.Grimm C, Kraft R, Sauerbruch S, Schultz G, Harteneck C. Molecular and functional characterization of the melastatin-related cation channel TRPM3. J Biol Chem. 2003;278:21493–21501. doi: 10.1074/jbc.M300945200. [DOI] [PubMed] [Google Scholar]
  • 44.Monteilh-Zoller MK, Hermosura MC, Nadler MJS, Scharenberg AM, Penner R, et al. TRPM7 provides an ion channel mechanism for cellular entry of trace metal ions. J Gen Physiol. 2003;121:49–60. doi: 10.1085/jgp.20028740. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 45.Zhu X, Jiang M, Birnbaumer L. Receptor-activated Ca2+ influx via human Trp3 stably expressed in human embryonic kidney (HEK)293 cells. J Biol Chem. 1998;273:133–142. doi: 10.1074/jbc.273.1.133. [DOI] [PubMed] [Google Scholar]
  • 46.Taylor JT, Zeng XB, Pottle JE, Lee K, Wang AR, et al. Calcium signaling and T-type calcium channels in cancer cell cycling. World J Gastroenterol. 2008;14:4984–4991. doi: 10.3748/wjg.14.4984. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 47.Thebault S, Flourakis M, Vanoverberghe K, Vandermoere F, Roudbaraki M, et al. Differential role of transient receptor potential channels in Ca2+ entry and proliferation of prostate cancer epithelial cells. Cancer Res. 2006;66:2038–2047. doi: 10.1158/0008-5472.CAN-05-0376. [DOI] [PubMed] [Google Scholar]
  • 48.Zeng X, Sikka SC, Huang L, Sun C, Xu C, et al. Novel role for the transient receptor potential channel TRPM2 in prostate cancer cell proliferation. Prostate Cancer P D. 2009 doi: 10.1038/pcan.2009.55. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 49.Valois AA, Webster WS. Retention and distribution of manganese in the mouse brain following acute exposure on postnatal day 0, 7, 14 or 42: an autoradiographic and gamma counting study. Toxicology. 1989;57:315–328. doi: 10.1016/0300-483x(89)90119-4. [DOI] [PubMed] [Google Scholar]

Articles from PLoS ONE are provided here courtesy of PLOS

RESOURCES