Evaluation of an Over-the-Counter Glycated Hemoglobin (A1C) Test Kit

Abstract

Background

Glycated hemoglobin (A1C) monitoring is an integral component of diabetes management. This study was conducted to evaluate the performance of the A1CNow^® SELFCHECK device when used by lay users and health care professionals (HCPs) to measure A1C.

Methods

Subjects performed two A1CNow SELFCHECK finger-stick self-tests followed by a finger-stick test of the subject’s blood by a HCP. The primary endpoint assessed accuracy of the subject and HCP A1CNow SELFCHECK readings. Secondary endpoints included precision, comprehension of instructional material (written material ± DVD), and product satisfaction. For accuracy comparison, a venous blood sample was drawn from each subject and tested by laboratory (TOSOH) analysis. Subject comprehension of product instructional material was evaluated via first-time failure (FTF) rate as recorded by the HCP, and subject satisfaction was assessed through written survey.

Results

A total of 110 subjects with (n = 93) and without (n = 17) diabetes participated. Of 177 subject A1C values, 165 (93.2%) were within the acceptable range of ±13.5% of the laboratory reference value and considered accurate. Regression analysis showed good correlation of subject values to laboratory and HCP results (R² = 0.93 for both). The average within-subject coefficient of variation was 4.57% (n = 74). The FTF rates with and without instructional DVD were 11.3% (n = 56) and 39.6% (n = 54), respectively. Subjects with diabetes/prediabetes overwhelmingly indicated that they were “very” to “extremely” likely (93.5%) to discuss their home A1C results with their HCP.

Conclusions

Lay users found the A1CNow SELFCHECK easy to use, and both lay users and HCPs were able to measure A1C accurately.

Introduction

As of 2007, an estimated 24 million people in the United States had diabetes, representing nearly 8% of the population.¹ Another 57 million were classified as having prediabetes, which places them at increased risk for developing diabetes, potentially leading to macrovascular and microvascular complications.¹ Glycated hemoglobin [specifically, A1c (A1C)] levels provide an indication of average blood glucose concentration over several months and have a strong predictive value for the occurrence of diabetes complications.^2,3 Monitoring of A1C is an integral component of diabetes management and is an established method of determining glycemic control over time.^2,4

Hemoglobin A1c levels can be measured by a variety of laboratory techniques, and over the past decade, they have expanded to include point-of-care (POC) assays for use in health care provider offices and clinics. These assays provide immediate feedback, allowing for timely treatment decisions and intervention. The availability of A1C results from POC instruments at the time of patient visits has been shown to result in better outcomes, including lower A1C values, owing to increased physician intervention.⁵ Furthermore, providing patients with diabetes immediate feedback on their A1C number has been shown to result in a 1% point reduction in A1C levels in some patients.⁶ Importantly, a 1% point reduction in A1C can lower the risk of serious microvascular and macrovascular complications.⁷ Advances in physician and patient access to A1C monitoring have the potential to benefit the majority of patients with diabetes.⁸

Validation studies have shown the A1CNow+^® POC assay (Bayer HealthCare LLC, Diabetes Care, Tarrytown, NY) to be comparable to laboratory reference measurements.^8,9 Point-of-care assays may be modified to be appropriate for over-the-counter (OTC) use, provided they are relatively simple to perform; the results are clear to the user; and they are approved for at-home use.¹⁰ The A1CNow^® SELFCHECK (Bayer HealthCare LLC, Diabetes Care, Tarrytown, NY) is a fully integrated, hand-held device for the quantification of percentage of A1C in capillary (finger stick) whole blood and is currently approved for OTC use. A test can be performed in 5 min with 5 µl of finger-stick blood. Using combined immunoassay and general chemistry, the device is small and disposable, making it ideal for a home-use environment.

The objectives of the current study were to determine the performance of A1CNow SELFCHECK when used by lay users and health care professionals (HCPs) and to evaluate comprehension of the instructional material by lay users.

Methods

Study Population

The study population consisted of subjects with known diabetes (type 1 or type 2) or prediabetes as well as subjects with no known diagnosis of diabetes (∼15% of the study population). Subjects met inclusion criteria if they were ≥18 years of age (∼80% of subjects were ≤55 years of age) and indicated that they had an interest in performing an at-home test. Subjects were excluded if they had known rheumatoid arthritis or other conditions causing impairment of manual dexterity; had a known hemoglobin variant (e.g., hemoglobin S or C); had a known blood disorder or disorder of a blood-forming organ (such as recovery from blood loss, hemolytic anemia, or iron deficiency anemia); had received a blood transfusion within 4 months prior to enrollment; had a known infection by a blood-borne pathogen; were missing any or part of digits on the hand; had significant visual or hearing impairment, cognitive disorder, or any other condition that could make participation in the study inappropriate as per the investigator’s discretion; were currently taking prescription anticoagulants or had clotting problems (note, use of Plavix^® [Bristol-Myers Squibb/Sanofi Pharmaceuticals Partnership, Bridgewater, NJ] or daily aspirin was allowed); or participated in previous studies on the A1CNow+ product or were working for a competitive medical device company. The protocol, informed consent forms, and supporting documents were approved by an institutional review board, and all subjects completed the informed consent process prior to participating in the study.

Study Design

This clinical trial of the A1CNow SELFCHECK test kit was conducted at two clinical sites in the United States (John Muir Physician Network Clinical Research Center, Concord, CA, and Consumer Product Testing Company, Inc., Fairfield, NJ). Each subject made a single study visit. The primary endpoint was to determine the accuracy of the A1CNow SELFCHECK test kit when used by subjects and trained HCPs. Secondary endpoints included the within-subject precision of the A1CNow SELFCHECK test kit, subject comprehension of the instructional materials (with and without viewing an instructional DVD), and subject satisfaction and ease of use as assessed by subject and HCP surveys.

A1CNow SELFCHECK Testing Procedure

After reviewing the instructional materials, each subject completed two finger-stick self-tests, and a third finger-stick test of the subject’s blood was performed by the HCP. Three different lots of A1CNow SELFCHECK were evenly divided among subjects for use during the study, with each individual subject using only one lot. During the first self-test, each subject was observed by an HCP who recorded the subject’s proficiency during each step. Assistance by the HCP was not permitted during this test. During the second self-test, instruction from the HCP was allowed as needed to ensure that the second test was completed successfully. Accuracy was assessed by comparison of subject- and HCP-obtained results to those obtained by laboratory analysis of A1C levels, whereas precision was evaluated for those subjects who were able to complete two self-tests. After the third test, each subject completed a survey containing questions about product features and clarity of instructional materials. Subjects with diabetes or prediabetes were also asked to complete a survey assessing the potential impact of A1CNow SELFCHECK at-home testing on their diabetes management.

For comparison with finger-stick results, a venous blood sample was collected from each subject and tested at a National Glycohemoglobin Standardization Program (NGSP) level II-certified laboratory (Bayer HealthCare LLC, Diabetes Care, Sunnyvale, CA) by high-performance liquid chromatography (HPLC) on a TOSOH 2.2 laboratory analyzer (TOSOH method; Tosoh Biosciences Inc., South San Francisco, CA). A 1 µl sample was collected from subject finger-stick blood drops in a microhematocrit tube and analyzed for hematocrit concentration within 6 h.

To assess comprehension of the A1CNow SELFCHECK instructional materials, all subjects received written instructions and 50% of subjects viewed a DVD demonstrating the use of the device in addition to receiving the written material. Subject comprehension of the labeling materials was assessed via first-time failure (FTF), defined as follows: (1) the subject could not determine how to use the product based on the instructional material provided without assistance (subject requested professional assistance); (2) the subject attempted to complete the test, realized a mistake was made, but could not continue because one or more of the parts had been rendered unusable because of user error; or (3) the subject managed to complete the test after one or more mistakes and received an error code instead of a result. The FTFs were determined and recorded by the HCP.

Statistical Analysis

The FTF rate was assessed based on the noninferiority hypothesis:

H₀: Pr{FTF} >0.20 versus the alternative H₁: Pr{FTF} ≤0.20.

The null hypothesis was to be rejected if the number of FTFs was less than or equal to the critical value. The critical value is the value that would provide a 90% chance of rejecting the null hypothesis if it is not true. Contingency table analysis was used to compare the FTF rates between subjects in the instructional DVD and non-DVD groups. For regression analysis, outliers determined as described in the Clinical and Laboratory Standards Institute (CLSI) guidelines EP-9A2¹¹ were to be omitted.

An A1C reading was considered accurate if it was within ±13.5% of a corresponding NGSP-certified laboratory reference value (HPLC analysis by the TOSOH method). The noninferiority hypothesis to be tested can be stated as:

H₀: Pr{accurate result} <0.95 versus the alternative

H₁: Pr{FTF} ≥0.95.

The null hypothesis was to be rejected if the number of accurate results met or exceeded the critical value. Sample bias was calculated according to CLSI guidelines EP-9A2¹¹ at 6%, 8%, and 10% A1C.

For precision analysis, the standard deviation (SD) and coefficient of variation (CV) were calculated for each subject who received a A1C value at the end of both self-tests. The average within-subject SD (S_W) was estimated by computing the variance (SD)² for each pair of subject results and computing the square root of the average variances using the following equation:

$$S_w=\sqrt{\frac{\sum _{i=1}^N{S}_i}{N}}$$

where $S_j=\sum _{i=1}^2{(x_{ij}={\overline{x}}_j)}^2$ is the estimate of variance within the ith subject (between duplicate results) and N is thenumber of subjects with duplicate results. The average within-subject CV was computed by using the square of the CV for each pair of subject-generated results and determining the square root of the average for all subjects.

Results

Subject Disposition

Of the 112 subjects who completed the informed consent process, 110 were enrolled in and completed the study; 2 subjects did not meet the inclusion/exclusion criteria. Subject ages ranged from 18 to 77 years, with 20% over 55 years of age, and the majority of subjects were male (64%) and Caucasian (68%; Table 1). All subject hematocrit levels were within the acceptable range for the A1CNow SELFCHECK of 20% to 60%.

Table 1.

Subject Demographic and Diabetes Characteristics

Characteristic	N = 110
Gender
Male, n (%)	70 (64)
Female, n (%)	40 (36)
Median age in years (range)	46 (18–77)
>55 years, n (%)	22 (20)
Racial/ethnic origin, n (%)
Caucasian	75 (68)
Hispanic/Latino	14 (13)
Black or African American	10 (9)
Asian	5 (5)
Native Hawaiian or Pacific Islander	2 (2)
American Indian or Native Alaskan Islander	1 (1)
White and Native Hawaiian or Pacific Islander	1 (1)
Other	2 (2)
Education, n (%)
Less than high school	3 (3)
High school graduate	37 (34)
Some college or associate’s degree	40 (36)
Bachelor’s degree	15 (14)
Graduate or professional degree	15 (14)
Type of diabetes, n (%)
Unknown	1 (1)
Type 1	22 (20)
Type 2	68 (62)
Prediabetes	2 (2)
No diabetes	17 (15)
Length of time with diabetes,an (%)
<1 year	5 (5)
1–2 years	13 (12)
3–5 years	19 (17)
5–10 years	21 (19)
>10 years	35 (32)
Frequency of A1C tests with HCP per yeara
Unknown	7 (6)
>4	7 (6)
4	39 (35)
3	15 (14)
2	16 (15)
1	8 (7)
<1	1 (1)

^aIncludes only subjects with diabetes or prediabetes (n = 93).

A total of 221 subject and 110 HCP A1CNow SELFCHECK tests were completed, with 215 and 109 of the results included in the respective analyses. Six subject tests were excluded from analyses for the following reasons: 2 first test results were excluded because the subjects inadvertently received help from the HCP, 2 results were excluded because the subject used a personal lancing device, 1 result was excluded because it was a third subject self-test, and 1 result was excluded because of a damaged cartridge. The results of these tests included A1C values, error codes, or blank screens at the completion of each test attempt. One HCP result was excluded from analysis because the subject inadvertently picked up the monitor while it was counting down. There were no outliers based on the CLSI EP-9A2 criteria; however, the results from 2 subjects were excluded from the regression analysis because of protocol deviations.

Accuracy

Of the 215 subject self-tests that were included in the analyses, 177 resulted in evaluable readings and were assessed for accuracy; the remaining 38 tests resulted in an error code, no reading at all, or a FTF. Two separate analyses were performed for subject accuracy (Table 2). All samples were pooled together for one analysis; for the other analysis, samples were separated according to treatment group: subjects who were provided only with written instructions (non-DVD group) and subjects who were provided with written instructions and a DVD (DVD group). In each analysis, accuracy assessment was calculated according to the sample size in that group. There were 77 results for subjects in the non-DVD group, 101 results for subjects in the DVD group, and 177 evaluable results for subjects in the pooled data set. As shown in Figures 1A and 1C, a regression analysis showed good correlation between results obtained by subjects using the A1CNow SELFCHECK test kit and the TOSOH method (R² = 0.925) and between subject- and HCP-obtained results (R² = 0.928). As shown in Table 2, in all cases, the number of readings met or exceeded the defined criteria for successes (94 for the DVD group, 71 for the non-DVD group, and 165 for the pooled group). The bias of subject readings from the corresponding laboratory reference result is shown in Figure 2; sample bias at 6%, 8%, and 10% A1C and the corresponding 95% confidence intervals are shown in Table 3.

Table 2.

Accuracy of Subject and Health Care Professional Readings

Accuracy analysis	n	Critical value required	Attained value	Critical value attained?
All subjectsa	177	≥165	165	Yes
DVD group	101	≥93	94	Yes
Non-DVD group	77	≥71	71	Yes
HCP	99	92	93	Yes

^aThe first result for one of the subjects randomized to the non-DVD group was omitted from the pooled analysis because of user error.

Figure 1.

Regression analysis of subject and HCP readings: subject readings compared with TOSOH method (A), HCP readings compared with TOSOH method (B), and subject readings compared with HCP readings (C). Solid lines are lines of regression from the corresponding equation in each panel, and dotted lines are lines of identity (y = x).

Figure 2.

Bland–Altman plot of subject readings compared with the TOSOH method.

Table 3.

Bias Calculations at 6%, 8%, and 10% A1C

A1C value (%)	Bias (standard error)	95% confidence interval
6	0.20 (0.052)	0.09 to 0.30
8	0.22 (0.041)	0.13 to 0.30
10	0.24 (0.067)	0.10 to 0.37

Separately, HCP results were also analyzed. The HCPs performed a total of 109 tests of subject finger-stick blood, 99 of which resulted in evaluable readings and were assessed for accuracy; the remaining 10 resulted in error codes and were excluded from the analysis. Of the 99 evaluable readings, 93 satisfied the accuracy criterion, exceeding the critical value of 92 (Table 2). A regression analysis comparing HCP readings with the TOSOH method showed good correlation (R² = 0.932; Figure 1B).

Precision

Precision analysis was performed using data from all subjects who received a numeric result for both of their self-tests (n = 74). The average SD within-subject CV was 4.57% (0.41).

Comprehension

Comprehension analyses included subjects who were given written instructions and watched the DVD (DVD group, n = 56) and those who were only given written instructions (non-DVD group, n = 54). There was a significant difference between FTF rates for the two groups (Fisher exact test, p = .0056). Subjects in the non-DVD group had a FTF rate of 39.6%, whereas subjects in the DVD group had a FTF rate of 11.3% (Figure 3).

Figure 3.

Subject FTF rates.

The subject errors associated with the most common failure modes during the first self-test are shown in Table 4; the FTF rates for each type of error are also shown. The most common errors were more likely to occur among subjects who did not receive DVD instructions, and FTF rates associated with these errors were higher in the non-DVD group. It should be noted that subjects may have experienced more than one error during their initial self-test; thus it may be difficult to infer which failure was directly related to the FTF for those cases.

Table 4.

Subject Errors

Error	Subjects with this error, na			Failure—no A1C result, nb			FTF rate associated with error (n = 27)c
+DVD	–DVD	Total	+DVD	–DVD	Total	+DVD	–DVD	Total
Did not insert blood collector into shaker properly	9	17	26	4	11	15	15%	41%	56%
Did not fill blood collector properly	2	12	14	1	7	8	4%	26%	30%
Shook shaker for <5 s	12	8	20	2	3	5	7%	11%	19%
Did not shake the shaker at all	1	1	2	0	0	0	0	0	0
Removed shaker base too early	0	1	1	0	1	1	0	4%	4%

^aSubjects may have experienced more than one error during the initial self-test.

^bA total of 27 subjects experienced 29 errors that led to failure.

^cCalculated as (100 × number of failures associated with the error)/total number of subjects that experienced a failure.

Subject Satisfaction

To assess the usability and impact on diabetes manage-ment of the A1CNow SELFCHECK test, subjects were asked to complete written surveys. After the testing was completed, all subjects (n = 110) were asked to assess features of the test kit (including overall experience) as well as the clarity of the accompanying instructions for use. The majority of subjects rated the overall testing experience as “very good” (34.5%) or “excellent” (59.1%; Figure 4). Subjects in the DVD group (n = 56) rated the clarity of the DVD as “excellent” (78.6%) or “very good” (19.6%), and subjects also rated the clarity of the written instructions (all subjects, n = 110) as “excellent” (62.7%) or “very good” (27.3%; Figure 5).

Figure 4.

Feature survey results.

Figure 5.

Clarity of instructional material.

At the conclusion of the testing period, all subjects with diabetes (n = 93) were asked to rate as “not at all,” “somewhat,” “very,” or “extremely” likely or important such factors as their interest in seeking out information about the test when it becomes available (43.0% of subjects chose “extremely likely”), whether or not they would discuss the test with their HCP (55.9% of subjects chose “extremely likely;” Figure 6A), and the importance of the benefits of this product to them (49.5% of subjects chose “extremely important”). In addition, 49.5% of subjects chose “would not change how often I test” when asked whether having this product at home would impact how often they test their blood glucose concentration (Figure 6B). No adverse events occurred during this clinical trial.

Figure 6.

Survey results of subjects with diabetes (n = 93). How likely would subjects be to discuss the results of an at-home A1C test with their doctor or HCP? (A) Assuming subjects had this product, how would it impact how often they test their own blood sugar/glucose levels? (B)

Discussion

The A1C test is an integral component of diabetes management. By providing a measurement of average blood glucose levels over the course of several months, A1C values help to create a more complete picture of blood glucose control when used in conjunction with blood glucose readings.^2,4 The A1C values can also help to assess the accuracy of blood glucose results and the adequacy of the testing schedule.² Though not a replacement for direct blood glucose monitoring, the immediate availability of results attainable through the incorporation of A1C technology at the POC has been shown to lead to more timely treatment decisions and better glycemic control.^6,12–15 To be effective in POC testing, it is important that a A1C test not only provide accurate results comparable to laboratory analysis, but it should also be cost-effective and easy to use; these qualities are even more important in an OTC application for at-home use.¹⁰

The A1CNow+ test kit has been shown to be accurate in POC monitoring of A1C levels.^8,9 The objectives of the current study were to evaluate the performance of the device in the hands of untrained lay users as an OTC application and to evaluate comprehension of the written instructional materials (with and without a training DVD) that are provided with the system. The 13.5% cutoff used to determine accuracy of the A1C readings in this study represents the upper 95% limit on total error and is based on allowing for variation in calibration accuracy and precision of both the test and reference methods. The A1C testing performance landscape at the time that these data were collected included a College of American Pathologists acceptance limit of ±12% and a NGSP criteria of ±0.85% A1C.¹⁶ The A1CNow SELFCHECK has received 2010 NGSP certification under the new tighter criteria of ±0.75% A1C.¹⁷

Subject survey data and failure rate analysis, in conjunction with system accuracy results, indicate that subjects were able to understand the provided instructional materials and perform the A1C test correctly based on these instructions. The instructional DVD component of the system was shown to significantly improve patient under-standing and proper use of the test kit. Because of the significant difference in FTF rates between the DVD and non-DVD subject groups, A1CNow SELFCHECK will include an instructional DVD for users to view prior to beginning the test. This will help ensure that patients are able to accurately measure their A1C levels in a home environment.

For HCPs to make appropriate treatment decisions and better manage a patient’s disease, a critical aspect of at-home A1C testing is that the obtained A1C value be provided to and discussed with the HCP along with results of regular blood glucose monitoring. In this study, the majority of subjects reported that they were “very” or “extremely” likely to discuss their self-obtained A1C results with their HCP. Survey results also suggest that subjects understand the limitations of the A1C test. In particular, patients reported that using the A1CNow SELFCHECK would not change their usual self-monitoring of blood glucose frequency, with most patients anticipating no effect on their blood glucose monitoring habits or indicating that they would test their blood glucose more often.

This study evaluated untrained users in a clinical setting and compared their results with HCP-obtained results as well as laboratory-analyzed A1C values. One limitation of this approach is its failure to capture the true at-home implications of OTC testing, because the study was performed in a simulated environment. However, the results of this study support further evaluation of the A1CNow SELFCHECK system to determine if the impact and advantages of diabetes self-management can be further extended through at-home testing of A1C levels. A potential area of further study is accuracy evaluation of the A1CNow SELFCHECK system in patients with diabetes who have common variants of hemoglobin (e.g., hemoglobin S or C),¹⁸ as a previous version of the system was shown to exhibit positive bias in these patient populations.¹⁹

Conclusions

Overall, lay users found the A1CNow SELFCHECK system easy to use and were able to accurately measure their A1C levels without explanation or assistance from their HCP. As reflected in survey results from this study, subjects recognized the importance of discussing their A1C values with their HCP and understood that A1C values obtained at home were not a substitute for regular blood glucose measurements or regular visits with their HCP. It will be important to determine if these promising results found with the addition of the at-home A1C test to a patient’s diabetes management will translate into improved glycemic control, with the ultimate goal of reducing the risk of associated long-term complications of the disease.

Abbreviations

(A1C)

glycated hemoglobin

(CLSI)

Clinical and Laboratory Standards Institute

(CV)

coefficient of variation

(FTF)

first-time failure

(HCP)

health care professional

(HPLC)

high-performance liquid chromatography

(NGSP)

National Glycohemoglobin Standardization Program

(OTC)

over-the-counter

(POC)

point-of-care

(SD)

standard deviation