Validating laboratory results in a national observational cohort study without field centers: The Reasons for Geographic and Racial Differences in Stroke cohort

Abstract

Objectives

The REasons for Geographic and Racial Differences in Stroke (REGARDS) study is a prospective cohort of 30,239 Americans in the contiguous United States; the first of this scale to use home visits to obtain, process, and ship biologic samples to a core laboratory. Pre-analytical factors resulting from this study design may affect the results of some laboratory assays. We investigated the impact of REGARDS processing on a variety of analytes.

Design and methods

In REGARDS, blood samples were processed in the field by technicians who were trained on standardized methods for phlebotomy and sample processing. Field processing included centrifugation using varying non-uniform equipment and shipping overnight on ice to the University of Vermont, where samples were re-centrifuged for 30,000 ×g-minutes and stored at −80 °C. We assessed the effects of REGARDS sample handling by processing split samples from 20 volunteers using either ideal procedures or simulated REGARDS procedures. Assays for 19 analytes for potential study in REGARDS were then run on both samples and results compared.

Results

Spearman correlation coefficients for analytes measured in ideal versus REGARDS processed samples ranged from 0.11 to 1.0. Thirteen of 19 analytes were highly correlated (>0.75), but platelet proteins were more variable.

Conclusions

Simulation of non-optimal field processing and shipment to a central laboratory showed high variability in analytes released by platelets. The majority of other analytes produced valid results, but platelet contamination in REGARDS samples makes measurement of platelet proteins unadvisable in these samples. Future analytes considered by REGARDS or similar studies should undergo similar pilot testing.

Introduction

Unlike epidemiologic studies that use study centers for clinical exams and blood sample collection, the REasons for Geographic and Racial Differences in Stroke (REGARDS) study conducted in-home visits with participants. This allowed recruitment of a nationally representative sample. While standardized training and methods were used, since field processing and sample shipping may affect the quality of samples [1–5], the validity of assays run on these samples required investigation. Here, we provide results on analyte validity using split samples from 20 volunteers, and results on sample yield and quality from >30,000 REGARDS participants from a baseline in-home visit.

Materials and methods

REGARDS enrolled 30,239 adults ≥45 years old, 42% black and 58% white, 45% men and 55% women, 56% from the stroke belt (Alabama, Arkansas, Georgia, Louisiana, Mississippi, North Carolina, South Carolina, and Tennessee) and 44% from the 40 other contiguous United States [6]. Institutional review boards approved the study. After written informed consent, Examination Management Services, Inc. (EMSI) (Scottsdale, AZ) technicians collected biologic samples following telephone-based screening and enrollment. Trained technicians followed detailed instructions for phlebotomy and sample processing using centrally assembled kits. Fasting morning blood was drawn using butterfly needle systems in the following order: 1 9-mL serum separator tube (SST; BD, Franklin Lakes, NJ), 1 10-mL EDTA plasma tube (BD), 1 5-mL sample collection/anticoagulant tube (SCAT-1) plasma tube (Hematologic Technologies, Inc., Essex Junction, VT), and 1 4-mL SST. SCAT-1 tubes are designed to prevent in vitro clotting activation and contain, in whole blood, 4.5 mmol/L EDTA, 0.15 kIU/L aprotinin, and 20 mol/L D-Phe-Pro-Arg-chloromethylketone [7]. All tubes were placed in a biohazard bag with absorbent pad and placed in a styrofoam mailer on frozen gel ice packs (0 °C) for the remainder of the EMSI visit. Processing was required within 120 minutes of phlebotomy; centrifugation was for 10 minutes, then serum or plasma, and packed cells from the EDTA and SCAT-1 tubes were transferred into mailer tubes. Specimens were shipped overnight with two frozen gel ice packs to the University of Vermont Laboratory for Clinical Biochemistry Research. Upon receipt, samples were catalogued and serum and plasma re-centrifuged at 4 °C for 30,000 ×g-minutes, then stored at −80 °C.

Real-time feedback on sample condition was provided to EMSI. Sample redraws were required for errors including mislabeled tubes, missing or extra tubes, whole blood sent instead of serum or plasma, <2 ice packs, spilled/leaked specimens, or samples not shipped the day of draw. Redraws were not requested due to participant refusal of phlebotomy, technical problems with phlebotomy, shipping delay attributed to the shipping carrier, or central laboratory processing errors.

To determine whether analytes proposed for measurement in REGARDS could be accurately measured, split samples from 20 volunteers were processed using both ideal (immediate) procedures and a simulated REGARDS processing protocol. Ideal processing included allowing the SST tube to clot at room temperature for 30 minutes before centrifuging for 30,000 ×g-minutes at 4 °C; EDTA and SCAT tubes were placed on ice for <5 minutes following phlebotomy then centrifuged for 30,000 ×g-minutes at 4 °C. Serum and plasma were then immediately stored at −80 °C. To simulate REGARDS processing, draw tubes were placed in an insulated cooler with 2 gel ice packs for 60–120 minutes, and then centrifuged for 5000 ×g-minutes at room temperature. Plasma and serum were separated into mailer tubes and sealed in an insulated shipping container with 2 gel ice packs overnight. The following day, mailer tubes were re-centrifuged for 30,000 ×g-minutes at 4 °C, and plasma and serum stored at −80 °C. Ideal and REGARDS processed samples from each individual were then assayed for analytes listed in Table 1.

Table 1.

Correlations between plasma samples processed in REGARDS vs. ideal fashion.

Assay	Inter-assay CV	Reference range	Sample type	Spearman rank correlation (ρ)	Coefficient of determination (R2)	% Bias	Linear regression equation
ADAMTS-13*	8%**	0.6–1.6 μg/mL	EDTA	0.61	0.27	20%	y = 0.74x − 0.01
Adiponectin	4–13%	3 to 14 μg/ml	EDTA	0.92	0.87	−11%	y = 0.87x + 1.29
			SCAT	0.98	0.97	−1%	y = 0.96x + 0.76
D-Dimer*	5–17%	0.22–4.0 μg/mL	EDTA	0.77	0.57	33%	y = 0.77x − 0.26
			SCAT	0.75	0.72	0%	y = 0.99x − 0.18
Factor VII antigen	6–9%	70–130%	SCAT	0.93	0.93	−8%	y = 0.94x − 2.22
Factor VIII antigen (ERL)	4–7%	40–145%	EDTA	0.89	0.83	3%	y = 0.86x + 17.1
			SCAT	0.92	0.80	7%	y = 0.94x + 12.1
Factor IX antigen (ERL)	9–15%	80.3–152%	EDTA	0.90	0.93	1%	y = 0.90x + 7.87
			SCAT	0.88	0.90	0%	y = 0.87x + 10.4
Factor XI antigen (ERL)	6–9%	40–111.4%	EDTA	0.91	0.85	0%	y = 0.86x + 16.9
			SCAT	0.75	0.72	0%	y = 0.80x + 27.2
Fibrinogen antigen	3–8%	180–350 mg/dl	EDTA	0.87	0.71	6%	y = 1.13x − 22.6
			SCAT	0.96	0.94	−2%	y = 0.94x + 14.0
HGF	3–10%	80–380 pg/mL	EDTA	0.88	0.67	−20%	y = 0.77x + 5.02
			SCAT	0.94	0.77	17%	y = 1.07x + 10.3
IL-8*	3–7%	1.2–16.7 pg/mL	EDTA	0.69	0.45	30%	y = 0.93x + 0.17
			SCAT	0.90	0.86	10%	y = 0.92x + 0.10
IL-10*	4–11%	0.16–12.70 pg/mL	EDTA	0.88	0.73	1%	y = 0.08x + 1.16
			SCAT	0.99	0.84	0%	y = 0.10x + 1.11
Leptin*	1–4%	2000–11,100 pg/mL	EDTA	1.0	1.0	−4%	y = 1.02x − 0.20
			SCAT	1.0	1.0	3%	y = 1.00x + 0.00
PAI-1	4–16%	5–66 ng/mL	SCAT	0.65	0.33	676%	y = 1.10x + 59.0
PDGF	6.5–8.5%	155.7–643.0 pg/mL	EDTA	0.23	0.02	253%	y = 0.32x + 1855.2
Protein C* antigen (ERL)	9%	70–140%	EDTA	0.53	0.29	4%	y = 0.37x + 2.61
			SCAT	0.62	0.44	−8%	y = 0.50x + 0.13
Protein C antigen (Stago)	5–8%	70–140%	EDTA	0.94	0.87	1%	y = 1.04x − 3.85
			SCAT	0.61	0.48	1%	y = 0.50x + 73.8
Resistin	2–6%	6.39–26.4 ng/ml	EDTA	0.59	0.29	−10%	y = 0.38x + 6.52
			SCAT	0.73	0.53	−5%	y = 0.39x + 6.22
s-EPCR*	6–7%	65.3–197.3 ng/mL	EDTA	0.93	0.96	1%	y = 0.93x + 0.37
TFG-β1	6.4–9.3%	100.0–1234.0 pg/mL	EDTA	0.11	0.03	126%	y = 4.12x + 16,110
			SCAT	0.31	0.06	64%	y = 0.85x + 16,031
vWF antigen	3–13%	50–160%	EDTA	0.60	0.48	−18%	y = 0.94x − 13.4

CV = coefficient of variation, SCAT = sample collection/anticoagulant tube, EDTA = ethylenediaminetetraacetic acid, ADAMTS-13 = A disintegrin and metalloproteinase with a thrombospondin type 1 motif, member 13, ERL = Enzyme Research Laboratories, HGF = hepatocyte growth factor, IL = interleukin, PAI-1 = plasminogen activator inhibitor 1, PDGF = platelet-derived growth factor, s-EPCR = soluble endothelial protein C receptor, TFG-β1 = transforming growth factor beta-1, vWF = von Willebrand factor.

^*Log transformed for linear regression.

^**Based on 2 plates.

Results and discussion

Between February 2003 and December 2007, 31,170 sample kits (including re-draws) were received at the core laboratory (Supplemental Fig. 1), representing 96% of the cohort. Errors occurred for 8.2% of the kits; 3.2% from shipper delay, and 5% (n = 1,575) from technician errors that triggered redraw requests. Technician errors were misprocessing (48%), shipping delay (47%), or both (5%). Redraws were received for 54.6% of these and 90% were error-free.

For the split samples experiment, Spearman correlation coefficients, the coefficient of determination, linear regression equations, and percent bias are shown in Table 1. Due to the convention of using quantiles of biomarker distributions to categorize risk exposure, relative rank of biomarkers is more important than complete correlation; thus Spearman correlation coefficients are the best measure of validity. Correlations between ideal and REGARDS-processed samples ranged from 0.11 (transforming growth factor beta 1; TGFβ1) to 1.0 (leptin). Thirteen of 19 had correlation coefficients of >0.75 in at least 1 tube type (Table 1). Fig. 1 shows scatter plots of a representative protein highly affected by processing (von Willebrand factor; vWF) and one unaffected (leptin). Scatter plots of all assays are shown in Supplemental Figs. 2–20.

Fig. 1.

Examples of analytes affected (von Willebrand factor) and unaffected (leptin) by REGARDS processing. Scatter plots show untransformed raw data, and best-fit linear regression lines and equations, with results from ideally processed samples on the x-axis and results from samples processed using simulated REGARDS processing on the y-axis. Grey boxes highlight the reference ranges of the analytes: 50–160% for von Willebrand factor, and 2000–11,100 pg/mL for leptin.

Explanations for poor correlation between ideal and REGARDS-processed samples include incomplete removal of platelets from REGARDS-processed specimens in the remote centrifugation, longer time to centrifugation in REGARDS samples [4], temperature variations in REGARDS-processed samples [5], low variability of the biomarker between subjects, or the CV of the assay.

Exponentially higher levels of platelet alpha granule proteins (transforming growth factor beta 1 (TGFβ1) and platelet-derived growth factor (PDGF)) in REGARDS- compared to ideal-processed samples suggest platelet contamination in REGARDS plasma. vWF [8], ADAMTS-13 [9], and plasminogen activator inhibitor 1 (PAI-1) [10] are also secreted by platelets. PAI-1 was consistently higher in REGARDS-processed samples (mean, 84.9 ng/mL) compared to ideal samples (mean, 23.5 ng/mL), p = 0.0001. Mean ADAMTS-13 was not higher in REGARDS samples (p = 0.19), but 3 of 20 REGARDS-processed samples had results above the detectable maximum of the assay, perhaps due to platelet release of ADAMTS-13 during suboptimal processing. REGARDS-processed samples had lower mean vWF antigen (97%) than ideal samples (118%), p = 0.04, possibly due to protein removal after centrifugation in a clot formed in vitro.

Resistin showed higher variability than other adipokines. Although it is stable within individuals [11] and has wide between-person variability [12], no literature is available on the analytical impact of processing temperature or time to processing. Resistin is not known to be stored or produced in platelets but the effects of platelet contamination on resistin levels is unknown.

Considering protein C antigen measured with the Stago kit (Supplemental Fig. 16), two batches of reagents were used in SCAT and EDTA samples, and the batch used for SCAT had a higher CV. Thus, differences between these two sample types should be interpreted with caution. The Enzyme Research Laboratory protein C kit produced variable results in both EDTA and SCAT samples.

The use of quantiles for division of exposure risk in epidemiologic research rather than clinical cutpoints makes normal reference ranges less important. However, in samples that were highly correlated, REGARDS processing did not result in more values outside of the normal range (Supplemental Fig. 3). For analytes with variability due to platelet contamination in REGARDS-processing often shifted results to above the normal reference range (Supplemental Fig. 14).

Limitations of this study include that volunteers were generally healthy individuals with lower biomarker variability than might be expected in the REGARDS cohort. For example, D-dimer increases with age and is higher in individuals with prothrombotic states. The maximum level and variability of D-dimer within the older REGARDS cohort are likely greater than the volunteers in this study. However, increased between-person variability in this scenario will make the assay CV less important and the distinction between “high” and “normal” values easier. Conversely, low levels of adiponectin are associated with higher risk of cardiovascular [13] and metabolic disease [14]. Adiponectin levels in the REGARDS cohort may be lower than in our volunteers; indeed, many of our participants had values well above the normal range suggesting better than average obesity-related risk profiles. Lower variability and floor effects could therefore affect the measurement of this analyte in REGARDS samples. Another limitation is that we did not actually ship the REGARDS-simulated samples.

Conclusions

REGARDS succeeded in creating a biorepository from over 30,000 individuals enrolled throughout the continental United States. While many biomarkers were relatively unaffected by field processing, as simulated here, analytes secreted by platelets (PDGF, TGFβ1, vWF, ADAMTS-13, and PAI-1) showed wide variability, presumably due to platelet contamination in processing. Resistin and the Stago protein C assay showed unanticipated variability, and further research should characterize the mechanisms. Future biomarker work in REGARDS or similar studies should investigate measurement validity using similar methods.

Abbreviations

ADAMTS-13

a disintegrin and metalloproteinase with a thrombospondin type 1 motif, member 13

CV

coefficient of variation

EDTA

ethylenediaminetetraacetic acid

ELISA

enzyme-linked immunosorbent assay

EMSI

Examination Management Services, Inc

ERL

Enzyme Research Laboratories

HGF

hepatocyte growth factor

IL

interleukin

PAI-1

plasminogen activator inhibitor 1

PDGF

platelet-derived growth factor

REGARDS

REasons for Geographic and Racial Differences in Stroke

SCAT-1

sample collection/anticoagulant tube

s-EPCR

soluble endothelial protein C receptor

TGFβ1

transforming growth factor beta 1

vWF

von Willebrand factor

Appendix A. Supplementary data

Supplementary data to this article can be found online at http://dx.doi.org/10.1016/j.clinbiochem.2014.08.003.