Cross-sectional comparison of point-of-care with laboratory HbA_1c in detecting diabetes in real-world remote Aboriginal settings

Abstract

Objectives

To determine if point-of-care (POC) glycated haemoglobin (HbA_1c) is sufficiently accurate in real-world remote settings to predict or exclude the diagnosis of diabetes based on laboratory HbA_1c measurements.

Design

Cross-sectional study comparing POC capillary HbA_1c results with corresponding venous HbA_1c levels measured in a reference laboratory.

Participants

Aboriginal patients ≥15 years old who were due for diabetes screening at the participating clinics were invited to participate. Two hundred and fifty-five Aboriginal participants were enrolled and 241 were included in the analysis.

Setting

6 primary healthcare sites in the remote Kimberley region of Western Australia from September 2011 to November 2013.

Main outcome measures

Concordance and mean differences between POC capillary blood HbA_1c measurement and laboratory measurement of venous blood HbA_1c level; POC capillary blood HbA_1c equivalence value for screening for diabetes or a high risk of developing diabetes; sensitivity, specificity and positive-predictive value for diagnosing and screening for diabetes; barriers to conducting POC testing.

Results

Concordance between POC and laboratory results was good (ρ=0.88, p<0.001). The mean difference was −0.15% (95% limits of agreement, −0.67% to 0.36%). POC HbA_1c measurements ≥6.5%, 48 mmol/mol had a specificity of 98.2% and sensitivity of 73.7% for laboratory measurements ≥6.5%. The POC equivalence value for screening for diabetes or a high risk of developing diabetes was ≥5.7%, 39 mmol/mol (sensitivity, 91%; specificity, 76.7% for laboratory measurements ≥6.0%, 42 mmol/mol). Staff trained by other clinic staff ‘on the job’ performed as well as people with formal accredited training. Staff reported difficulty in maintaining formal accreditation.

Conclusions

POC HbA_1c testing is sufficiently accurate to be a useful component in screening for, and diagnosing, diabetes in remote communities. Limited local training is adequate to produce results comparable to laboratory results and accreditation processes need to reflect this.

Strengths and limitations of this study.

• This was a pragmatic study conducted in ‘real-world’ remote clinics where staff were trained realistically in-house to use the point-of-care glycated haemoglobin (HbA_1c) analyser.

• This study provides insight into how simplified screening for and diagnosing diabetes using HbA_1c tests can be implemented in remote primary healthcare.

• Not all participants with abnormal laboratory HbA_1c measurements were able to be located for follow-up repeat laboratory HbA_1c tests.

• We used a ‘convenience sample’, which may not be representative of the Kimberley adult Aboriginal population.

Introduction

Screening for diabetes in remote Aboriginal Australian settings is challenging, in part due to the difficulty of obtaining fasting blood glucose levels and then completing an oral glucose tolerance test in patients where the glucose level cannot exclude or diagnose diabetes.¹ In 2009 and 2011, the American Diabetes Association (ADA) and WHO, respectively, updated their criteria for diagnosing diabetes to include glycated haemoglobin (HbA_1c) testing.^{2 3} The potential advantages of using HbA_1c include no need for a fasting sample.³ The correlation between HbA_1c levels and presence of retinopathy was used by the ADA and WHO to determine the HbA_1c threshold of ≥6.5% (48 mmol/mol) to diagnose diabetes.^{3 4} ADA also uses HbA_1c values 5.7–6.4% (39–46 mmol/mol) to identify people at future risk of diabetes with 6.0–6.4% (42–46 mmol/mol) classified as high risk.

In Australia, the National Evidence Based Guideline for Case Detection and Diagnosis of Type 2 Diabetes was last updated in 2009 and is based on the 2006 WHO criteria for diagnosing diabetes.^{5 6} It currently does not yet recommend using HbA_1c testing for diagnosis. However, the Australian Medicare Benefits Schedule (MBS) has recently changed to include funding for laboratory HbA_1c testing for diagnosing diabetes from 1 November 2014.⁷

Laboratory HbA_1c tests for patients in the Kimberley region of north Western Australia are analysed in Perth, the state capital, over 1600 km away. This can lead to significant delays in receiving test results for patients located in remote areas of the Kimberley. Once Australian guidelines incorporate the use of HbA_1c testing for diagnosing diabetes, the availability of immediate HbA_1c results is likely to further improve diagnosis in remote areas and timeliness of initiating management, including lifestyle advice and pharmacotherapy.

In a number of studies, point-of-care (POC) testing of HbA_1c has been shown to be closely correlated with laboratory-based results, and the process is well accepted in Aboriginal Medical Services across Australia.^8–11 These studies, in supported research environments with attention to quality control, have not however been repeated in real-world remote settings where maintaining consistency of quality and accreditation may be more difficult. POC HbA_1c testing in Australia is currently used in the management of diabetes, and accreditation with a quality assurance programme, such as the Quality Assurance for Aboriginal and Torres Strait Islander Medical Services (QAAMS) programme, is required for clinics to claim MBS rebates for these tests.¹² Advice submitted to the Medical Services Advisory Committee also recommends that POC HbA_1c testing be funded for diabetes screening if it is performed on an analyser accredited in a recognised external quality assurance programme such as QAAMS;¹³ however, this has not as yet been accepted.

An earlier Kimberley study of POC testing, comparing capillary glucose with venous laboratory glucose, demonstrated that real-world remote primary healthcare delivered accurate POC testing of glucose in the diagnosis of diabetes without formal training.¹⁴ Following this study, the diagnostic pathway in the Kimberley was modified, and the Kimberley Chronic Disease Therapeutic Protocol for type 2 diabetes was updated to reflect these changes.^{15 16} The rationale for POC HbA_1c screening for diabetes is at least as persuasive given the difficulty and delay in accessing laboratory services in remote settings and the benefits of immediate results without the need for retesting in a mobile population.

We aimed to determine if POC capillary blood HbA_1c measurement was sufficiently accurate, in a real-world setting using the existing processes for HbA_1c assessment, to reliably predict or exclude the diagnosis of diabetes based on laboratory venous blood HbA_1c measurements in remote Aboriginal primary healthcare practice.

Methods

Data were collected by local healthcare providers from September 2011 to November 2013 at six sites across the Kimberley. All Aboriginal and Torres Strait Islander people ≥15 years old in the Kimberley are considered to be at high risk of developing diabetes and local protocols recommend that they are screened annually.¹⁵ Aboriginal and/or Torres Strait Islander people who were due for diabetes screening at the participating clinics were invited to participate. The study was carried out on an opportunistic basis when sufficient clinic resources were available to enrol participants.

POC HbA_1c measurement and venepuncture

Blood was expected to be collected for capillary and venous HbA_1c on the same visit; however, this did not always occur. In some remote settings in the Kimberley, samples may not leave the clinic for up to a week and venous samples are often collected just prior to transport. Results up to 7 days apart were therefore included. Data for comparison of POC and laboratory HbA_1c measurements were excluded if the participant's haemoglobin was <90 g/L.

Capillary blood HbA_1c was measured by a primary healthcare provider via a finger-prick blood sample and analysed on a DCA 2000+ Analyzer (Siemens/Bayer, Germany), the only type of POC machine available in Kimberley clinics during the study and used routinely for assessing diabetic control. The study aimed to look at ‘real world’ practice and hence did not attempt to change the way staff learnt how to do the testing or maintain and calibrate the machines. We collected data on clinic processes and calibration after the study stopped recruiting participants by contacting each clinic involved.

Venous whole blood samples were collected in containers with EDTA. Normal procedures at each clinic were used for storage (whole blood was stored at 4°C) and whole blood samples were transported to one of the three laboratories in the Kimberley region before being transported to the main laboratory in Perth (PathWest). The distance to the Kimberley laboratories from the study sites ranged from 2 km by road to 650 km by air, and from the Kimberley laboratories to Perth from 1677 to 2224 km by air. The time for the specimens to reach the Perth laboratory varied from 1 to 8 days.

Venous plasma HbA_1c levels were measured as part of routine work by PathWest (which analyses most Kimberley pathology). PathWest used automated immunoassay with anticoagulated whole blood specimens haemolysed automatically on the Cobas Integra 800 (Roche Diagnostics, Switzerland) using HbA_1c haemolysis reagent in the predilution cuvette. Total haemoglobin was measured colourimetrically while HbA_1c was determined immunoturbidimetrically. The ratio of the concentrations of total haemoglobin and HbA_1c in the specimen yielded the final proportionate HbA_1c measurement.

Readers of the POC and laboratory HbA_1c test results were blinded to the other HbA_1c test results. POC HbA_1c test results were recorded by clinic staff on the participant's data collection form. Laboratory HbA_1c test results were sent by PathWest to the requesting doctor after the POC test was conducted. Participants with laboratory HbA_1c ≥5.7%, 39 mmol/mol were expected to be followed up with another laboratory HbA_1c test to confirm prediabetes or diabetes.

Sample size calculation

On the basis of pilot screening data from the Kimberley, we estimated that prevalence of undiagnosed diabetes is 15% (according to Martin et al⁸ up to 20%) and if HbA_1c has a sensitivity of 60%,^17–20 then 9% would be diagnosed by laboratory HbA_1c. The size of the sample needed depends on the accuracy of sensitivity and specificity required.²¹ Assuming a minimum sensitivity of 90% and specificity of 80% for screening for diabetes (and vice versa for diagnosing diabetes), using calculations by Buderer²² we needed a sample size of 138 and 270 to detect a two-tailed 95% CI with a maximum width of ±5% for sensitivity (ie, 85–95%) and specificity (ie, 75–85%), respectively.

Statistical analysis

Differences in baseline characteristics were compared using χ² tests for categorical data and Mann-Whitney tests for continuous non-parametric data.

Concordance between POC and laboratory blood HbA_1c measurements was assessed by the technique of Lin,²³ which measures the agreement between two methods. Mean difference (bias) and limits of agreement were determined using the techniques of Bland and Altman.²⁴ Linear regression analysis was used to determine whether participant or processing factors affected the relationship between POC and laboratory HbA_1c levels.

The POC HbA_1c diagnostic cut point was based on the laboratory HbA_1c diagnostic cut point recommended by the Australian Diabetes Society expert committee (≥6.5%, 48 mmol/mol).²⁵ The Kimberley POC capillary equivalence value for screening for people with diabetes or at high risk of developing diabetes was determined from receiver operating characteristics (ROC) curves, using a laboratory value of 6% (42 mmol/mol), based on International Expert Committee and ADA recommendations for people at high risk of developing diabetes.^{3 4} Sensitivity, specificity, predictive values and percentage correctly classified for (1) diagnosing diabetes and (2) screening for diabetes and a high risk of developing diabetes were determined based on these cut points.

All analyses were performed using Stata, V.13 (StataCorp, Texas, USA). Point estimates were presented with 95% CIs; p<0.05 was considered statistically significant.

Results

Two hundred and fifty-five participants were enrolled, and 241 included in the analysis. The reasons for exclusion were missing tests in 12 participants, and more than 7 days elapsing between POC measurement and venepuncture in two participants. There was no difference in age (median (range): 36 (17–79) vs 38.5 (17–71); p=0.397) or sex (female (%): 143 (59%) vs 9 (64%); p=0.714) between those included and excluded from the analysis.

Unadjusted POC capillary blood HbA_1c measurements and laboratory-measured venous blood HbA_1c levels showed good concordance (figure 1A). The limits of agreement between POC and laboratory measurements (mean difference: −0.15%; 95% limits of agreement: −0.67% to 0.36%) are presented in figure 1B. The cross tabulation of the classification of participants based on the POC and HbA_1c measurements is shown in table 1.

Figure 1.

Concordance analysis of point-of-care (POC) and laboratory glycated haemoglobin (HbA_1c) results. (A) All unadjusted readings (plasma and whole blood levels; n=241). Dashed line indicates line of perfect concordance. (B) Difference plot (Bland-Altman analysis)²⁴ for all POC versus laboratory HbA_1c. Dotted line indicates 95% limits of agreement.

Table 1.

Cross tabulation of the classification* of participants (n=253) based on POC and laboratory HbA_1c measurements

	Laboratory HbA1c
	Normal	Risk of developing diabetes	High risk of developing diabetes	Diabetes	Missing	Total
POC HbA1c
Normal	79	46	7	0	5	137
Risk of developing diabetes	5	26	16	1	2	50
High risk of developing diabetes	0	6	33	4	0	43
Diabetes	0	1	3	14	0	18
Missing	3	2	0	0	0	5
Total	87	81	59	19	7	253

Bold typeface indicates the participants with diabetes.

*Includes participants with missing results (n=12), but excludes those with POC and laboratory results taken more than 7 days apart (n=2); normal: HbA_1c <5.7%, <39 mmol/mol; risk of developing diabetes: HbA_1c 5.7–5.9%, 39–41 mmol/mol; high risk of developing diabetes: HbA_1c 6.0–6.4%, 42–46 mmol/mol; diabetes: HbA_1c ≥6.5%, ≥48 mmol/mol.

HbA_1c, glycated haemoglobin; POC, point-of-care.

In the regression analysis, the relationship between POC and laboratory HbA_1c levels was not independently influenced by any of the following: the time between POC HbA_1c measurement and venepuncture; the time delay in collecting and processing the laboratory specimen; whether the health worker had performed more than 10 POC tests; health worker type (Aboriginal health worker, nurse, general practitioner); site; or if the POC operator was previously or currently accredited with QAAMS.

Accuracy of POC HbA_1c in predicting diabetes

ROC curves for diagnosing diabetes (laboratory HbA_1c level ≥6.5%, 48 mmol/mol) are shown in figure 2A. Sensitivity, specificity, predictive values and percentage correctly classified for POC and laboratory measurements ≥6.5%, 48 mmol/mol are listed in table 2. As required for diagnostic tests, POC HbA_1c testing had a high specificity (98.2%; 95% CI 95.1% to 99.4%) for predicting diabetes.

Figure 2.

ROC curve for diagnosing and screening for diabetes or a high risk of developing diabetes. ROC curve for (A) diagnosing diabetes (laboratory-measured HbA_1c level, ≥6.5%, 48 mmol/mol) and (B) screening for diabetes or a high risk of developing diabetes (laboratory-measured HbA_1c level ≥6.0%, 42 mmol/mol; HbA_1c, glycated haemoglobin; POC, point-of-care; ROC, receiver operating characteristics).

Table 2.

Sensitivity, specificity, predictive values and classification by point-of-care (POC) glycated haemoglobin (HbA_1c) testing for diagnosing diabetes and screening for participants (n=241) with diabetes or a high risk of developing diabetes

	Diagnosis*	Screening†
Sensitivity (95% CI, %)	73.7 (48.6 to 89.9)	91.0 (81.8 to 96.0)
Specificity (95% CI, %)	98.2 (95.1 to 99.4)	76.7 (69.3 to 82.9)
Positive predictive value (%)	77.8	65.1
Negative predictive value (%)	97.8	94.7
Correctly classified (%)	96.3	81.3

Bold typeface indicates the main criteria for selecting a cut point.

*Diagnosis based on laboratory HbA_1c ≥6.5%, 48 mmol/mol using a POC cut point of 6.5%, 48 mmol/mol.

†Screening based on laboratory HbA_1c ≥6.0%, 42 mmol/mol using a POC cut point of 5.7%, 39 mmol/mol.

For the dichotomous classification of participants as either not having (HbA_1c <6.5%, <48 mmol/mol) or having diabetes (HbA_1c ≥6.5%, ≥48 mmol/mol), based on laboratory measurements, nine of the 241 (3.7%) participants were incorrectly classified with POC HbA_1c. The four participants who were incorrectly classified as having diabetes with POC HbA_1c (6.6–6.8%, 49–51 mmol/mol) had laboratory HbA_1c measurements in the prediabetes range (table 1), with two close to the cut point (6.3–6.4%, 45–46 mmol/mol). The other five participants had laboratory measurements just above the diagnostic cut point (6.5–6.7%, 48–50 mmol/mol) and POC measurements in the prediabetes range (table 1), with four close to the cut point (6.3–6.4%, 45–46 mmol/mol).

Of the 14 participants who had both POC and laboratory measurements ≥6.5%, 48 mmol/mol, 8 were followed up with a repeat laboratory HbA_1c test. The second laboratory result confirmed a diagnosis of diabetes in 7 (88%) participants, with the other participant having a measurement close to the cut point (6.3%, 45 mmol/mol).

Accuracy of POC HbA_1c in screening for diabetes or a high risk of developing diabetes

As a screening test, the most clinically appropriate POC threshold based on ROC curves was 5.7%, 39 mmol/mol (figure 2B). Sensitivity, specificity, predictive values and percentage correctly classified for POC measurements ≥5.7%, 39 mmol/mol and laboratory measurements ≥6.0%, 42 mmol/mol are listed in table 2. As required for screening tests, POC HbA_1c testing had a high sensitivity (91.0%) for detecting people with diabetes or a high risk of developing diabetes.

Using this cut point of 5.7%, 39 mmol/mol, 132 (55%) participants were classified as normal with POC testing. No participant with a diagnostic diabetes laboratory HbA_1c measurement was missed, and only 7 (2.9%) participants would have had a high risk of developing diabetes (HbA_1c ≥6.0%) missed if POC HbA_1c was the only test undertaken.

Accreditation and quality assurance of POC HbA_1c testing

Only two staff members were accredited with QAAMS to perform POC HbA_1c tests for managing diabetes¹² when participating in the study. Three other staff involved in the study had previously received QAAMS accreditation (2004) but were no longer accredited. Many staff were trained to use the analyser by other clinic staff who, in turn, had experience ranging from minimal through to QAAMS accreditation. If no one in the clinic was familiar with the machine, staff taught themselves by reading the instruction poster. Only one site had QAAMS accreditation during the study period and this site calibrated the machine as required to maintain accreditation (two monthly quality checks: internal quality control and external quality assurance testing).¹² An analyser was moved from one remote location to another and was calibrated by staff at the site with QAAMS accreditation prior to being used in the study. Staff from the other four sites, where 140 (59%) samples were tested, reported that they did not calibrate their analyser during the study period.

Barriers to POC testing

Barriers to POC HbA_1c testing included several sites needing new analysers, and in 2011 there was a global shortage of cartridges. Staff reported that they found maintaining accreditation through QAAMS difficult and the quality management framework required for the clinic to maintain QAAMS accreditation¹² was seen as onerous. In the Kimberley, cartridges are usually bought by clinics through the QAAMS programme and staff reported that if their clinic was not accredited, they were unable to order cartridges and claim MBS rebates. These issues meant that at the start of the study only one clinic was doing routine POC HbA_1c testing for diabetes monitoring.

Discussion

Our study demonstrated good concordance between laboratory and real-world remote setting POC HbA_1c results. Measurements using the DCA 2000+ Analyzer were similar to, but slightly lower on average, than laboratory venous samples measured routinely by PathWest. A POC cut point of 6.5%, 48 mmol/mol gave a high specificity (98.2%) for diagnosing diabetes based on the same cut point for laboratory measurements.

We also found that a POC HbA_1c screening cut point of 5.7%, 39 mmol/mol had a high sensitivity for detecting an HbA_1c level ≥6.0%; 42 mmol/mol (91.0%) and detected all participants with diagnostic diabetes laboratory HbA_1c measurements and most participants in the prediabetes range (ie, those at high risk of developing diabetes in 5 years).^26–28 As all Aboriginal and Torres Strait Islander people ≥15 years old in the Kimberley are considered to be at high risk of developing diabetes, misclassification of the seven participants with prediabetes would have limited impact on clinical management. The main interventions for prediabetes are encouraging healthy lifestyles and these should be offered to all patients. In addition, local recommendations for annual screening¹⁵ increase the prospects of progression to diabetes being diagnosed.

If the study population was reasonably typical of Kimberley Aboriginal and Torres Strait Islander residents undergoing screening for diabetes, a POC HbA_1c cut point of <5.7%, 39 mmol/mol to exclude diabetes will reduce the need for venepuncture and a follow-up visit to the clinic to discuss test results by 55%. This is more than twice the reduction found in the Kimberley POC glucose study.¹⁴

Various expert committees suggest that for asymptomatic diagnostic results a confirmatory test should be performed when feasible to confirm diabetes.^{4 25 29} In contrast, the Australian Medical Services Advisory Committee does not consider a confirmatory test to be necessary,³⁰ and there is only one MBS rebate for laboratory HbA_1c for screening in any 12-month period.⁷

If MBS rebates were available for screening using POC HbA_1c analysers in Australia, a POC measurement ≥5.7%, 39 mmol/mol could be followed by a venous laboratory HbA_1c test on the same day. If the laboratory result was in the diagnostic range (≥6.5%, 48 mmol/mol), this would confirm a diagnosis of diabetes in Australia. Given the lower than expected sensitivity (73.7%) for detecting diabetes, it is likely that some patients with POC measurements just below the diagnostic cut point will have a laboratory result in the diagnostic range. Therefore, in high-risk remote populations in countries where a confirmatory test is required if only the venous laboratory result (taken on the same day as the POC measurement) was ≥6.5%, a second laboratory test would need to be performed to confirm the result. Expediting the process and reducing the likelihood of loss to follow-up could result in more timely initiation of management in primary healthcare.

With the dichotomous classification of participants, they were nearly all correctly classified. Of the participants who had initial diagnostic POC and laboratory measurements followed up, a second HbA_1c laboratory measurement confirmed the diagnosis of diabetes in all except one, who had a repeat laboratory measurement close to the cut point. The clinical treatment for these patients is similar (healthy lifestyle advice unless HbA_1c ≥7%, 53 mmol/mol), with the main differences being frequency of follow-up and screening for microvascular disease.

A limitation of this study is that not all participants with laboratory HbA_1c ≥5.7%, 39 mmol/mol were followed up with repeat laboratory HbA_1c tests. A further limitation is that participation was based on inviting patients visiting participating clinics to enrol in the study, that is a ‘convenience sample’. On the basis of population surveys, we predicted that the number of people with undiagnosed diabetes across a range of remote Kimberley towns and communities would be 15%.⁸ We found that only 6% of the study population had previously undiagnosed diabetes. This lower than expected prevalence could be due to the study population not being representative of the Kimberley Aboriginal population or that there is less undiagnosed diabetes in the Kimberley than expected.

Unlike other POC HbA_1c studies,^8–11 our study was conducted in ‘real-world’ remote clinics where staff were mainly trained in-house, and no extra effort was made to standardise health-provider education, or HbA_1c analyser maintenance or calibration. Results demonstrated that despite limited machine calibration and minimal training, POC results were good. The perceived difficulty of maintaining accreditation of staff and clinics to obtain MBS rebates for management of diabetes using POC analysers is a barrier to greater use of HbA_1c for management of diabetes, and this will also be a barrier to POC screening. Although there are a variety of training methods provided by QAAMS, including online training, staff still need to be assessed by someone with current QAAMS accreditation in order to be accredited.¹² This can be difficult in remote areas without easy access to trainers and where workforce turnover is high. The expectations of the quality management framework requirements were also seen to be onerous. While we advocate for, and do encourage, regular machine calibration, it appears that there would be minimal risks and considerable benefits if the quality management requirements and accreditation processes were simplified.

Implications

POC HbA_1c analysers can be used in real-world remote Aboriginal primary healthcare settings for opportunistic screening for diabetes. In remote areas, using POC HbA_1c measurement would expedite the diagnosis of diabetes in high-risk, remote-dwelling populations and provide opportunities for immediate education and initiation of management.