Personalising laboratory medicine in the ‘real world’: Assessing clinical utility, by clinical indication, of serum total B₁₂ and Active-B₁₂® (holotranscobalamin) in the diagnosis of vitamin B₁₂ deficiency

Abstract

Background

Assessing the pre- and post-test probability of disease in the context of routine health care is challenging. We wished to study how test performance parameters relating to clinical utility vary by clinical indication in a ‘real-world’ setting.

Methods

The diagnostic accuracy of serum total B₁₂ and Active-B₁₂® (holotranscobalamin) was evaluated in a primary care population, using serum methylmalonic acid as the reference standard. We used electronic requesting to establish the clinical indication for each request. Routine requests from primary care for serum total B₁₂ were included if creatinine was also measured and estimated glomerular filtration rate was at least 60 mL/min/1.73 m².

Results

Clinical indications included peripheral neuropathy (n = 168), anaemia (n = 168), cognitive decline (n = 125), suspected dietary deficiency (n = 76), other (n = 362). For peripheral neuropathy, the area under the receiver operator curve ± 95% confidence interval (AUC ± CI) was 0.63 (0.54–0.71) (P = 0.002) for total B₁₂ and 0.68 (0.60–0.77) (P < 0.0001) for Active-B₁₂®. For anaemia, AUC ± CI was 0.56 (0.47–0.66) (P = 0.10) for total B₁₂ and 0.69 (0.59–0.78) (P < 0.0001) for Active-B₁₂®. For cognitive decline, AUC ± CI was 0.54 (0.43–0.65) (P = 0.26) for total B₁₂ and 0.69 (0.58–0.80) (P = 0.0002) for Active-B₁₂®. The pre–post-test change in probability of disease varied by clinical indication.

Conclusion

Combining diagnostic accuracy studies and electronic testing in a ‘real-world’ setting allows clinical utility to be assessed by clinical indication. Wider application of this would permit more personalised laboratory medicine. In this study, diagnostic performance of total B₁₂ and Active-B₁₂® varied across all indications. Active-B₁₂® provided better discrimination, but this may have reflected the cut-offs used.

Introduction

Diagnostic investigations are most useful at the point of maximum diagnostic uncertainty, i.e. when the potential gap between pre- and post-test probability of disease is greatest. In the context of routine health care, it is difficult to study the application of this principle to laboratory investigations. Practical challenges include: first, the existence of multiple requests and request sources, each with a potentially different pretest probability of disease; second, the need for information about the clinical indication for the request; third, the need for a validated reference standard. As a result of these difficulties, ‘real-world’ studies of the clinical utility of tests, and how this varies by clinical indication, are rare.

Separately, accurate assessment of vitamin B₁₂ status is problematic.¹ Defining deficiency in terms of tissue stores (e.g. red cell cobalamin)² is not always feasible, and no single analytical measurement is ideal, leading some to propose the use of combined indicators of vitamin B₁₂ status.³ Several diagnostic reports^4–8 have used methylmalonic acid (MMA) as reference standard, despite its acknowledged limitations.⁹ We adopted this approach in a study of the diagnostic accuracy of serum total B₁₂ and Active-B₁₂® (holotranscobalamin, or holoTC) in the assessment of vitamin B₁₂ status in an otherwise unselected primary care population. We used electronic requesting to establish the clinical indication for each request. Our study is the first to report how test performance parameters relating to clinical utility vary by clinical indication in a ‘real-world’ setting.

Methods

General practitioners in 68 practices in Tayside, Scotland (population approximately 400,000) were advised of the intention to perform a study of the diagnostic accuracy of the existing measure of vitamin B₁₂ status (i.e. total serum B₁₂), and Active-B₁₂® (see below for details of methods), in the setting of routine health care. Their cooperation was sought in the provision of information about the clinical indication for individual requests. Clinical information was delivered through an electronic order communications system, the Integrated Clinical Environment (ICE) (Sunquest Information Systems, Uxbridge, UK). The study was performed between 12 June 2014 and 31 July 2014.

Participants

The study population consisted of consecutive patients from whom samples were received for routine measurement of total serum vitamin B₁₂. In order to minimise the impact of reduced glomerular filtration rate (GFR) on serum MMA,¹⁰ patients were excluded if creatinine was not concurrently requested or if estimated glomerular filtration rate (eGFR) was 60 mL/min/1.73 m² or less. Only the results of total serum vitamin B₁₂ were reported.

Clinical indications

Health-care professionals ordering vitamin B₁₂ through the electronic order communications system chose from the following ‘drop-down’ list of clinical indications: unexplained anaemia; macrocytic anaemia; cognitive decline; suspected dietary deficiency; peripheral neuropathy; other.

Analytical methods

Serum total B₁₂ was measured by chemiluminescent microparticle-based competitive immunoassay on an Advia Centaur (Siemens Healthineers, Tarrytown, NY, USA), Active-B₁₂® by chemiluminescent microparticle-based immunoassay on an Abbott Architect i2000 (Abbott Diagnostics, Abbott Park, IL, USA), both at Ninewells Hospital in Dundee. MMA was measured using a MultiPurpose Sampler (Gerstel GmbH, Mülheim an der Ruhr, Germany) coupled to liquid chromatography tandem mass spectrometry (MPS-LC-MS/MS) at the Nutristasis Unit, Viapath, St Thomas Hospital, London.

Cut-offs used to define result categories

The lower limit of the reference interval (200 pg/mL) for serum total B₁₂ was used, in line with the manufacturer’s recommendations. The cut-off for serum holoTC (Active-B₁₂®) was <35 pmol/L, as used by other groups.⁶,¹¹ Two separate cut-offs were used for MMA, reflecting the impact of age-related changes in glomerular function on serum MMA: >0.28 µmol/L for patients ≤65 years and >0.36 µmol/L for patients >65 years. These were defined using age-specific 97.5th percentiles in vitamin B₁₂-replete patients in a previous study.¹² Total B₁₂, Active-B₁₂® and MMA were measured in isolation from each other; respective assessors of each did not have access to results of the other measurements, and only laboratory staff involved in the routine measurement of total B₁₂ had access to the clinical indication for each request.

Statistical methods

Descriptive statistics (mean, 2.5th and 97.5th centiles) were calculated for population characteristics. Test performance parameters and receiver operator curve (ROC) analysis were performed by Analyze-it (Version 2.21; Analyze-it Software Ltd, West Yorkshire, UK). Youden cut-offs were established for the three commonest indications and test performance recalculated for total B₁₂ and Active-B₁₂® by indication.

Results

During the study period, 899 routine requests for serum total B₁₂ were received. The breakdown of clinical indications for requests was as follows: unexplained anaemia (n = 134), macrocytic anaemia (n = 34), cognitive decline (n = 125), suspected dietary deficiency (n = 76), peripheral neuropathy (168), other (n = 362). (The anaemia categories were merged for the purposes of data analysis.) Table 1 summarises group characteristics for the three commonest indications. For the other indications, the characteristics (mean and 2.5–97.5 centiles) were as follows: (1) suspected dietary B₁₂ deficiency: age (years) 57 (15–91), haemoglobin (g/L) 137 (109–165), haematocrit 0.42 (0.35–0.50), mean cell volume (fL) 95 (85–110), white cell count (×10⁹/L) 6.4 (3.5–10.9), platelets (×10⁹/L) 254 (108–406), total vitamin B₁₂ (pg/mL) 370 (164–1004), Active-B₁₂® (pmol/L) 65 (12–256), MMA (µmol/L) 342 (96–1869); (2) other: age (years) 61 (23–92), haemoglobin (g/L) 134 (99–165), haematocrit 0.41 (0.30–0.49), mean cell volume (fL) 94 (81–111), white cell count (×10⁹/L) 6.8 (3.7–11.9), platelets (×10⁹/L) 256 (130–441), total vitamin B₁₂ (pg/mL) 388 (170–974), Active-B₁₂® (pmol/L) 70 (17–256), MMA (µmol/L) 320 (95–1121).

Table 1.

Characteristics of study population.

Group	Parameter		2.5–97.5 percentile	% abnormal results (cut-off value)
Total		n
	Group total	899
	Female	560
	>65 years	479
		Mean
	Age (years)	64	23–92
	Haemoglobin (g/L)	133	96–165	27% (<120 female, <130 male)
	Haematocrit	0.41	0.30–0.50	19% (<0.37)
	Mean Cell Volume (fL)	94	81–110	8% (>105)
	White Cell Count (×109/L)	6.8	3.5–11.9	6% (<4 × 109)
	Platelets (×109/L)	257	126–447	7% (<150 × 109)
	Total B12 (pg/mL)	408	171–1195	5% (<200)
	Active-B12® (pmol/L)	67	15–256	24% (<35)
	MMA (µmol/L)	345	98–1259	32% (>0.28 ≤ 65yo; >0.36 > 65yo)
Cognitive decline		n
	Group total	125
	Female	80
	>65 years	104
		Mean
	Age (y)	77	44–93
	Haemoglobin (g/L)	132	94–166	30% (<120 female, <130 male)
	Haematocrit	0.41	0.29–0.51	21% (<0.37)
	Mean cell volume (fL)	95	83–107	6% (>105)
	White cell count (×109/L)	6.6	3.4–10.3	6% (<4 × 109)
	Platelets (×109/L)	263	135–479	7% (<150 × 109)
	Total B12 (pg/mL)	413	166–1269	7% (<200)yo
	Active-B12® (pmol/L)	62	16–256	34% (<35)yo
	MMA (µmol/L)	346	80–1044	34% (>0.28 ≤ 65yo; >0.36 > 65yo)
Peripheral neuropathy		n
	Group total	168
	Female	109
	>65 years	58
		Mean
	Age (y)	58	23–89
	Haemoglobin (g/L)	142	110–168	7% (<120 female, <130 male)
	Haematocrit	0.43	0.35–0.50	5% (<0.37)
	Mean cell volume (fL)	94	84–106	2% (>105)
	White cell count (×109/L)	7.2	3.8–13.0	4% (<4 × 109)
	Platelets (×109/L)	258	126–434	4% (<150 × 109)
	Total B12 (pg/mL)	410	205–838	1% (<200)
	Active-B12® (pmol/L)	66	12–256	27% (<35)
	MMA (µmol/L)	394	119–2011	33% (>0.28 ≤ 65yo;>0.36 > 65yo)
Unexplained anaemia combined with macrocytic anaemia		n
	Group total	168
	Female	97
	>65 years	105
		Mean
	Age (years)	69	26–91
	Haemoglobin (g/L)	121	87–159	62% (<120 female, <130 male)
	Haematocrit	0.38	0.28–0.50	43% (<0.37)
	Mean cell volume (fL)	95	79–116	12% (>105)
	White cell count (×109/L)	6.8	3.0–11.9	10% (<4 × 109)
	Platelets (×109/L)	258	110–448	11% (<150 × 109)
	Total B12 (pg/mL)	391	144–1541	8% (<200)
	Active-B12® (pmol/L)	67	15–256	21% (<35)
	MMA (µmol/L)	347	84–1149	34% (>0.28 ≤ 65yo; >0.36 > 65yo)

Note: the upper measuring limit of the Active-B₁₂® assay is 256 pmol/L; all samples ≥256 pmol/L were recorded for analysis as 256 pmol/L. This does not affect % abnormal results nor the clinical performance calculations. ‘Unexplained anaemia’ and ‘macrocytic anaemia’ categories were merged. In the category ‘unexplained anaemia’, 8 patients from 134 had an elevated MCV (>105 fL). In the category ‘macrocytic anaemia’, 12 patients from 34 had an elevated MCV.

ROC curves for the three commonest specific indications are shown in Figure 1(a) to (c). For other indications, the areas under the curve ± 95% confidence intervals (AUC ± CI) were as follows for total B₁₂: suspected dietary deficiency 0.50 (0.36–0.64); other 0.47 (0.41–0.54). For Active-B₁₂®, the AUC ± CI were: suspected dietary deficiency 0.61 (0.47–0.75); other 0.66 (0.60–0.72). For the total population, the AUC ± CI for total B₁₂ was 0.53 (0.49–0.57) (P  = 0.075), and for Active-B₁₂®, AUC ± CI was 0.68 (0.64–0.72) (P  < 0.0001). These AUC were significantly different from each other (P  < 0.0001).

Figure 1.

(a) ROC curves for total B₁₂ and Active-B₁₂® in the diagnosis of vitamin B₁₂ deficiency (as defined by raised serum MMA) in anaemia. (b) ROC curves for total B₁₂ and Active-B₁₂® in the diagnosis of vitamin B₁₂ deficiency (as defined by raised serum MMA) in cognitive decline. (c) ROC curves for total B₁₂ and Active-B₁₂® in the diagnosis of vitamin B₁₂ deficiency (as defined by raised serum MMA) in peripheral neuropathy.

Table 2 summarises the pre- and post-test probabilities of vitamin B₁₂ deficiency (as defined by raised serum MMA), before and after measurement of serum total B₁₂, in each of the three commonest specific indications for requesting total B₁₂. Data are presented using the preassigned cut-offs and using Youden cut-offs (designed to optimise differentiating ability).¹³ We wished to calculate the post-test probability of disease in the presence of positive and negative results, respectively; therefore, we replaced the negative predictive values (NPV) by [1–NPV]. Table 3 summarises the corresponding data for Active-B₁₂®.

Table 2.

Clinical utility of Total B₁₂ by clinical indication.

Using pre-assigned cut-off (200 pg/mL)						Using Youden cut-offsa
Indication	Abnormal results (n)	PPV (%)	Δ in probability (%) with low result	1 – NPV (%)	Δ in probability (%) with normal result	Youden cut-off (pg/mL)	Abnormal results (n)	PPV (%)	Δ in probability (%) with low result	1–NPV (%)	Δ in probability (%) with normal result	Prevalence (%)
Anaemian = 168	11	18.2	–18.4	39.0	+2.4	287	46	50.0	+13.4	30.7	–5.9	36.6
Cognitive declinen = 125	9	77.8	+43.4	31.0	–3.4	200	9	77.8	+43.4	31.0	–3.4	34.4
Peripheral neuropathyn = 168	2	50.0	+17.3	32.5	–0.2	481	127	39.4	+6.7	12.2	–20.5	32.7

Δ = change; PPV (positive predictive value) = post-test probability of vitamin B₁₂ deficiency in presence of a low total B₁₂ result; (1–NPV [negative predictive value]) = post-test probability of vitamin B₁₂ deficiency in presence of a normal total B₁₂ result.

Note: Probability of vitamin B₁₂ deficiency (as defined by raised serum MMA) before and after measurement of total B₁₂, for three clinical indications. Pretest probability (prevalence) is shown, along with the post-test probability associated with low and normal results. For ease of reference, the changes in probability are also shown. Data are presented using preassigned cut-offs, and separately using Youden cut-offs.

^aThe cut-offs that optimise differentiating ability when equal weight is given to sensitivity and specificity.

Table 3.

Clinical utility of Active-B₁₂® by clinical indication.

Using pre-assigned cut-off (35 pmol/L)						Using Youden cut-offsa
Indication	Abnormal results (n)	PPV (%)	Δ in probability (%) with low result	1 – NPV (%)	Δ in probability (%) with normal result	Youden cut-off (pg/mL)	Abnormal results (n)	PPV (%)	Δ in probability (%) with low result	1 – NPV (%)	Δ in probability (%) with normal result	Prevalence (%)
Anaemian = 168	29	75.9	+39.3	26.7	–9.9	43	44	68.2	+31.6	22.2	–14.4	36.6
Cognitive declinen = 125	43	55.8	+21.4	23.2	–11.2	44	54	52.6	+18.2	19.1	–15.3	34.4
Peripheral neuropathyn = 168	46	50.0	+17.3	26.2	–6.5	29	30	63.6	+30.9	25.2	–7.5	32.7

Δ = change; PPV (positive predictive value) = post-test probability of vitamin B₁₂ deficiency in presence of a low Active-B₁₂®result; (1–NPV [negative predictive value]) = post-test probability of vitamin B₁₂ deficiency in presence of a normal Active-B₁₂®result.

Note: Probability of vitamin B₁₂ deficiency (as defined by raised serum MMA) before and after measurement of Active-B₁₂®, for three clinical indications. Pretest probability (prevalence) is shown, along with the post-test probability associated with low and normal results. For ease of reference, the changes in probability are also shown. Data are presented using pre-assigned cut-offs, and separately using Youden cut-offs.^aThe cut-offs that optimise differentiating ability when equal weight is given to sensitivity and specificity.

Discussion

Optimal targeting of diagnostic tests in the setting of normal care is challenging. Evaluations of diagnostic accuracy are usually performed in well-defined and pre-selected populations, whereas tests in routine use are usually applied to multiple populations that vary in terms of relevant parameters (e.g. prevalence). In the current study, we have addressed this issue by performing a diagnostic evaluation in the setting of normal care.

We identified several distinct populations, based on the clinical indication for the vitamin B₁₂ request. Patients with cognitive decline were, as anticipated, older than other patient groups, and had higher creatinine, reflecting reduced glomerular function. Interestingly, prevalence of vitamin B₁₂ deficiency, as defined by raised serum MMA, was broadly similar in the patient groups defined by the three commonest specific indications, ranging from 32.7% in patients with peripheral neuropathy to 36.6% in patients with anaemia. This is important, since any differences in PPV and [1–NPV] across clinical indications are therefore not attributable to large differences in prevalence.

For all categories of indication, measurement of Active-B₁₂® reliably differentiated vitamin B₁₂ deficiency from non-deficiency, and for all the AUC for Active-B₁₂® was greater than for total B₁₂, although in the case of peripheral neuropathy, the difference was not statistically significant. However, these findings must be interpreted with caution in the context of the wider literature. The effect on the clinical utility of holoTC of applying different MMA cut-offs,⁸ and more widely, the need to base cut-offs (for all vitamin B₁₂ biomarkers) on adverse outcomes,¹⁴ have previously been highlighted. The age-specific MMA cut-offs applied here, particularly in patients >65 years, may have affected sensitivity and specificity; other limitations include those of using MMA as a reference standard,¹⁵ and the exclusion of patients where creatinine was not requested, or where eGFR was 60 mL/min/1.73 m² or less. On the wider issue, the ability of holoTC to predict, for example, neurological outcomes like cognitive decline in well-designed studies^16–19 has been variable, and its role as a biomarker of clinically meaningful vitamin B₁₂ deficiency remains inconclusive. Recent assessments^15,20 acknowledge the limitations of holoTC and other markers and endorse the adoption of algorithm-based approaches that combine more than one measure of vitamin B₁₂ status.³ This seems reasonable, and our findings do not provide a basis for challenging this position.

With these caveats, low Active-B₁₂® results were more useful than normal ones; the increase in probability of vitamin B₁₂ deficiency seen with a low result was higher, for all three of the commonest specific indications, than the decrease in probability associated with a normal result (see Table 3). This was true both when the preassigned cut-offs and when Youden cut-offs were used. As anticipated, the PPV ranking for these indications corresponded to the prevalence ranking (when the preassigned cut-off was used), with the highest PPV associated with anaemia and the lowest with peripheral neuropathy.

We have shown in the current study that it is possible to apply the principles of evidence-based medicine to routine requesting. Electronic requesting is now commonplace in health care, and, with the co-operation of clinical colleagues, can readily be harnessed to studies of diagnostic accuracy. In the presence of a validated reference standard, it is possible to establish prevalence by clinical indication and requesting source. The cost of diagnostic studies involving validated reference standards should be offset against the potential savings enabled by greater precision in laboratory requesting, for example through the application of local algorithms.

ORCID iD

Michael J Murphy https://orcid.org/0000-0001-6735-125X