Feasibility and Performance of Hemoglobin A1C Self-Testing during COVID-19 among African Americans with Type 2 Diabetes

Abstract

Purpose:

The purpose of the study was to determine the feasibility of implementing A1C self-testing at home using the A1CNow^® Self Check and to compare the accuracy of the A1CNow to a reference standard in African Americans with type 2 diabetes (T2D).

Methods:

African American adults with T2D were recruited from 13 different churches (N=123). Phase 1, conducted during the early phase of the COVID-19 pandemic, examined the feasibility of A1C assessment using the A1CNow performed at home by untrained participants. Phase 2, conducted when in-person research resumed, compared A1C values concurrently measured using the A1CNow and the DCA Vantage^™ Analyzer (reference standard) collected by research staff at church testing sites.

Results:

In Phase 1, 98.8% of participants successfully completed at least one at-home A1C test; the overall failure rate was 24.7%. In Phase 2, the failure rate of staff-performed A1CNow testing was 4.4%. The Bland-Altman plot reveals that A1CNow values were 0.68% lower than DCA values and the mean differences (A1CNow minus DCA) ranged from −2.6 to 1.2% with a limit of agreement between −1.9 to 0.5%.

Conclusions:

A1C self-testing is feasible for use in community settings involving African-American adults with T2D. The A1CNow Self-Check underestimated A1C values when compared with the reference standard. Ongoing improvements in point-of-care devices have the potential to expand research and clinical care, especially in underserved communities.

Type 2 diabetes (T2D) is projected to affect 33% of the United States (U.S.) population by 2050.^1,2 African Americans (AAs) are twice as likely to have T2D as non-Hispanic Whites^3,4 and are less adherent to diabetes medications.⁵ The novel coronavirus disease (COVID-19) pandemic further exposed the detrimental impact of T2D on a wide range of health risks and outcomes and accentuated racial health disparities.^6–9 When diagnosed with COVID-19, AAs with T2D experienced substantially higher rates of hospitalization and mortality compared with AAs without T2D.⁹ As telehealth services and remote intervention programs for T2D become more common, there is a growing demand for A1C self-testing.^10,11 At-home self-testing provides an opportunity to overcome logistical barriers to traditional clinical care. Additionally, expanding the reach of A1C self-testing in community settings would enhance public health programs designed to improve T2D diagnosis, care integration, and self-management, especially for underserved and hard-to-reach populations such as AAs.

The A1CNow^® (PTS Diagnostics, Whitestown, IN) is an A1C point-of-care (POC) instrument that is widely used during clinical visits and in community settings. POC devices have improved the ability of health care professionals to provide care and for patients to manage their glycemic levels with real-time A1C results.^10,12,13 The A1CNow Self Check is a handheld device and is primarily marketed for at-home use with a reported capacity to measure A1C values that range from 4.0 to 13.0%.¹³ Previous studies using the A1CNow have reported that lay users can easily operate the device.^15,16 The device has strong test-retest reliability (r = 0.97)¹⁴ and compares favorably with clinical laboratory values (r² = 0.93 to r² = 0.99)^11,15,16 with a mean bias ranging from −0.55 to 0.50%.^11,15 However, these studies were conducted in controlled environments and included mostly non-Hispanic White middle-aged adults. Currently, no information is available on the ease of use of A1CNow self-testing in community settings among untrained lay users who are older and from racial/ethnic minority groups. Examining the feasibility and performance of using A1C POC devices in underserved community settings has the potential to expand research and clinical care in populations at risk for T2D and its complications.

Modifications to the TX STRIDE (Texas Strength Through Resilience in Diabetes Education) study due to the COVID-19 pandemic allowed investigators to examine the feasibility and performance (i.e., reliability and accuracy) of A1C POC devices in AA adults with T2D. TX STRIDE is an NIH-funded longitudinal randomized controlled clinical trial to determine the efficacy of a resilience-based diabetes self-management education and support program culturally tailored for AA adults with T2D.¹⁷ Soon after the intervention began, all in-person research activities were paused due to the COVID-19 pandemic. When the TX STRIDE study resumed, the intervention was modified from the original in-person format to a remote format using a video conference platform (i.e., Zoom).¹⁷ Prior to the pandemic, A1C was measured in-person using a laboratory-based testing device, the DCA Vantage^™ Analyzer (DCA, Siemens Medical Solutions Diagnostics, Malvern, PA) with a reported capacity to measure A1C values that range from 2.5 to 14.0%. During the pandemic, however, the A1CNow was mailed and used to measure A1C remotely by participants at home with the assistance of research staff. When in-person testing at the church setting was allowed to resume, A1C continued to be measured using the A1CNow for measurement consistency. However, each participant’s A1C was assessed concurrently using both the A1CNow and the DCA. Both devices are National Glycohemoglobin Standardization Program (NGSP) certified A1C POC devices with demonstrated accuracy by successfully meeting the stringent criterion set by the NGSP.¹³ The DCA is considered one of the most accurate POC devices available and is widely used in diabetes care.¹³ According to the most recent NGSP certification, the mean bias of DCA compared with laboratory-analyzed samples was between −0.06% and 0.08%.¹⁸ Therefore, in addition to examining the A1CNow’s feasibility and reliability, the DCA was used as the reference standard to investigate the accuracy of A1CNow.

With this information as background, the aims of this sub-study of the TX STRIDE clinical trial were to: 1) examine the feasibility — successful completion and ease of use — of implementing A1C self-testing at home using the A1CNow; 2) evaluate the test-retest reliability of the A1CNow; and 3) compare A1C values obtained with the A1CNow to those obtained with the DCA as a reference standard.

Methods

Study Design

TX STRIDE is an on-going clinical trial to investigate the effectiveness of a resilience-based diabetes self-management education and support program on physical and mental health outcomes. The present study is part of the parent TX STRIDE clinical trial and occurred in two phases to address the study aims: an A1C self-testing implementation phase at home (Phase 1) and a POC device measurements comparison phase conducted in person at church testing sites (Phase 2). Feasibility for Phase 1 was assessed via successful completion of at-home A1C self-testing using the A1CNow. Ease of use was assessed via failure rate, defined as the number of times participants were unable to obtain an A1C value. Phase 1 also examined the test-retest reliability of the A1CNow measurements. When in-person research resumed, Phase 2 compared A1C values collected by trained research staff using both the A1CNow and the DCA concurrently at church testing sites.

Study Participants

Participants (N = 123) were AA adults with T2D recruited through 13 AA churches in the city of Austin and the surrounding areas. Of the 123 participants in the full sample, 11 performed remote self-testing only, 74 performed remote self-testing and participated in onsite in-person church testing, and 38 were newly recruited and participated in onsite in-person church testing only. The procedures and the pandemic-related modifications were approved by the Institutional Review Board at The University of Texas at Austin. Written informed consent was obtained from all study participants. The sample for this sub-study was predominately female (71%) and was 63±11 years of age. Selected demographic characteristics of study participants are shown in Table 1.

Table 1.

Selected participants characteristics

Characteristics	n	Means ± SD or %
Age (year)	123	62.6 ± 11.1
Sex (male/female)	36/87	29%/71%
Education
High school diploma	35	29%
Some college	51	41%
College degree	21	17%
Graduate degree	16	13%
Body mass index (kg/m2)	123	36.2 ± 8.3
Diabetes diagnosis length (year)	123	10.2 ± 7.6
Diabetes medication use
Oral medications/non-insulin injectable only	77	63%
Insulin only	8	7%
Both	30	24%
No medication	8	7%
Employment status (currently employed)	66	54%
Marital status (currently married)	61	50%

Data Collection

Phase 1.

Participants (n = 85) self-tested their A1C values with the assistance of research staff during data collection sessions. One A1CNow box, which contained supplies for four A1C tests, was mailed to each participant’s home address. A printed step-by-step instruction sheet was also included. Research staff scheduled a remote data collection session to demonstrate how to use the A1CNow kit and guide participants through the testing process. The majority of data collection sessions (91%, 410 tests) were conducted via Zoom while 9% (40 tests) were conducted via phone call. To examine the feasibility and reliability of the A1CNow device, research staff documented the number of tests performed, error codes, and operational mistakes. When possible, at least two A1C values were obtained from each participant to ensure reliability of the A1CNow device. Participants were asked to record their A1C values and a concurrent fasting blood glucose concentration using a glucometer (Trividia Health, Fort Lauderdale, FL) provided by the TX STRIDE study.

Phase 2.

Participants’ (n = 112) A1C values were measured using both the A1CNow and DCA devices by trained research staff at the church locations. Of the 112 participants, 74 were returning participants (participated in Phase 1 data collection), and 38 were newly recruited. A1CNow tests were performed twice to be consistent with the Phase 1 data collection protocol. However, the two tests were performed simultaneously using two A1CNow devices to reduce participant burden (e.g., multiple finger pricks, longer wait time) and to accommodate high-volume community-based testing. The DCA test was performed once as a reference measurement and an A1C value and fasting blood glucose concentration (Cholestech LDX, Alere) were recorded.

Statistical Analyses

For Phase 1, descriptive statistics were used to examine the completion of A1C self-testing, failure rates, and error codes using the A1CNow. The reliability of the A1CNow was assessed via test-retest reliability using Pearson’s correlation coefficient. For Phase 2, linear regression was used to analyze the agreement of A1CNow values compared with the DCA reference standard. In a very small number of cases (n = 6), only a single A1C measurement was available. A paired t-test was performed to determine if there was a significant difference in mean A1C values obtained by the A1CNow compared with the DCA. Agreement between the average values obtained by the two devices was illustrated with a Bland-Altman plot, which compares the mean difference in measurements between two instruments by constructing limits of agreement (i.e., upper and lower limit of 95% confidence interval for the mean difference). The plot is widely used in studies comparing clinical measurements and is a recommended method by the Clinical and Laboratory Standards Institute.¹⁹ Descriptive and inferential statistical analyses were conducted using SPSS software version 25 (IBM Corp., Armonk, NY).

Results

Phase 1

Eighty-five participants performed a total of 450 tests using the A1CNow at home with remote assistance from the research staff; these tests occurred at baseline (n = 224 tests), and at 3-month (n = 108 tests) and 6-month (n = 118 tests) data collection time points (Table 2). The failure rates were 26.3, 21.3, and 24.6% respectively. In total, participants were unable to obtain an A1C value 111 times due to errors, resulting in an overall failure rate of 24.7%. The overall failure rate was 30.0% during phone call meetings and 24.1% during Zoom meetings. In addition, older participants (≥ 75 years; n = 10) had a higher failure rate (37.3%) when compared with participants younger than 75 years (22.8%). However, when the A1CNow self-testing was repeated, almost all participants (n = 84 [98.8%]) were successful in obtaining at least one A1C value. Only one participant (1.2%) failed to obtain an A1C value after using the four test kits contained in one box of A1CNow and needed a second box to complete the self-testing.

Table 2.

Self-performed (n = 85) and research staff-performed (n = 112) A1C tests and error codes

Self-performed A1C Tests and Error Codes	Total Number of Tests (n)	%
Total self-performed A1CNow tests	450
Failed tests	111	24.7%
Error codes for failed tests
OR1	41	36.9%
Operational mistakes	25	22.5%
QC56	17	15.3%
Other codes	16	14.4%
QC7	10	9.0%
Device malfunction	2	1.8%
Staff-performed A1C Tests and Error Codes	Total Number of Tests (n)	%
Total staff-performed A1CNow tests	229
Failed tests	10	4.4%
Error codes for failed tests
QC56	6	60%
Other codes	4	40%
DCA A1C Tests	Total Number of Tests (n)	%
Total DCA Vantage™ Analyzer tests	112
Failed tests	3	2.7%

Note: description of error codes; OR1 (“blood sample may have too little hemoglobin, or too little blood was added”), Operational mistakes (“participants spilled shaker that has blood sample in it by mistake”, etc.) QC56 (“insufficient blood sample was delivered to the test cartridge”), other codes (“the analyzer temperature is above 77^◦ F”, “the quality control checks inside the analyzer did not pass”, etc.) and QC7 (“cartridge remained in the analyzer without blood sample added for 2 minutes after “SMPL” prompt”).

The most frequent error code that was shown during the self-performed testing was OR1 (36.9%), which indicated that the blood sample may have too little hemoglobin or participants added too little blood for the test to work properly. The second most frequent error was participants’ operational mistakes (22.5%), which included spilled shaker and opening the time-sensitive testing material too early.

Seventy-seven pairs of A1C values obtained by participants using A1CNow were compared, and the result indicated a high test-retest reliability (r = 0.97, P < 0.001). The average within-subject coefficient of variance was 3.1%. Participants’ A1C obtained with A1CNow ranged from 4.2 to 13.0% and the mean A1C values of the first and second A1CNow testing values were 7.9 ± 1.7 and 7.8 ± 1.8%, respectively (Table 3).

Table 3.

Self-performed and research staff performed A1C and fasting blood glucose concentrations

Variables	n	Means ± SD
Self-performed tests
1st A1CNow A1C (%)	85	7.9 ± 1.7
2nd A1CNow A1C (%)	77	7.8 ± 1.8
Fasting blood glucose (mg/dL)	82	147 ± 51
Staff-performed tests
1st A1CNow A1C (%)	112	7.2 ± 1.5
2nd A1CNow A1C (%)	106	7.2 ± 1.6
DCA A1C value (%)	109	7.9 ± 1.6
Fasting blood glucose (mg/dL)	112	157 ± 66

Phase 2

Table 2 also shows the total number of A1C tests (A1CNow and DCA) conducted by research staff, failure rates, and error codes. The research staff performed 229 in-person A1CNow tests with 112 participants and were unable to obtain the A1C value 10 times due to errors, a failure rate of 4.4%. The most frequent error code that was shown during the staff-performed testing was QC56, which indicated that an insufficient blood sample was delivered to the test cartridge. The 106 pairs of A1CNow values measured by staff showed a high test-retest reliability (r = 0.93, P < 0.001). The average within-subject coefficient of variance was 4.2%. Participants’ A1C values ranged from 5.0 to 13.0%, and the mean A1C of the first and second A1CNow tests were 7.2 ± 1.5 and 7.2 ± 1.6%. The research staff also performed 112 tests using the DCA and failed to obtain the A1C value 3 times due to low hemoglobin in the blood sample, resulting in a failure rate of 2.7%. Participants’ A1C values from the DCA ranged from 5.5 to 14.0%, with a mean of 7.9 ± 1.6% (Table 3).

The 109 pairs of A1C values measured by A1CNow and DCA were strongly correlated (r = 0.93, P < 0.001), and the linear regression line had a slope of 1 (95% confidence interval [0.93, 1.1]; Figure 1). However, the mean A1C obtained with A1CNow (7.2 ± 1.5%) was lower (t(108) = −11.83, P < 0.001) than the mean obtained with DCA (7.9 ± 1.6%). On average, A1CNow values were 0.68% lower than DCA value. Figure 2A shows the distribution of differences between A1CNow and DCA. The mean differences (A1CNow minus DCA) ranged from −2.6 to 1.2% with a limit of agreement between −1.9% and 0.5%, as shown in the Bland-Altman plot (Figure 2B).

Figure 1.

Association between staff-performed A1CNow and DCA A1C testing results

Figure 2.

Distribution of A1C differences between A1CNow and DCA

Bar graph (A) shows the distribution of differences between A1CNow and DCA, and the dotted line represents 0. The Bland-Altman plot (B) shows the mean A1C values of A1CNow and DCA on the x axis versus the differences (A1CNow minus DCA) on the y axis. The solid line represents the negative mean bias (−0.68) and dotted lines represent limits of agreement from −1.96 SD (−1.87) to + 1.96 SD (0.50).

Discussion

The findings of the present study indicate the potential utility of A1C POC testing devices in a community setting involving AA. More specifically, A1C self-testing at home using the A1CNow was both feasible and reliable among untrained AA adults. Almost all participants were able to obtain two A1C values using the A1CNow with assistance from research staff, and the two measurements demonstrated a high test-retest reliability. However, the A1CNow underestimated A1C values when compared with the DCA reference standard. Prior studies have demonstrated that lay users are able to perform the A1CNow test with minimal^14,15 or no training,¹⁶ and report a positive testing experience.¹⁵ The results of the current study extend this body of work to AA adults with T2D, a population that has been heavily underrepresented in clinical studies.

The overall failure rate of A1CNow self-testing was 24.7%, whereas the overall failure rate was 4.4% when trained research staff performed the A1CNow tests, suggesting that participants experienced challenges in performing the tests. Notably, the failure rate did not decrease when the tests were repeated using the same procedure during the 6-month clinical trial period. The older age of participants, a 3-month gap in data collection time points, and delivery of guidance via phone rather than Zoom may have contributed to the high overall failure rate. Aging is associated with declined cognition and manual dexterity and may have adversely affected the testing process.²⁰ The more frequent failure rate of participants receiving guidance via phone suggests that visual demonstration played a facilitating role in successful completion of the self-testing and that minimal training prior to testing may be beneficial to reduce the failure rate.¹⁵

The A1CNow significantly underestimated A1C values compared with the DCA reference standard, despite the strong correlation of A1C values between the two devices. A systematic review and meta-analysis of the performance of A1C POC devices reported a mean bias ranging from −0.70 to 0.67% for the A1CNow when compared with clinical laboratory samples. However, a majority of studies using A1CNow showed an underestimation of A1C values ranging from −0.70 to −0.04%,¹² which is more consistent with the mean bias from the current study (−0.68%). The DCA has also shown a slight negative mean bias (−0.24%) among adult participants.¹² However, the DCA is reported to have a higher accuracy and sensitivity than other A1C POC devices.¹³ In addition, the maximum reportable range of the A1CNow is 13.0%, compared with the DCA maximum range of 14.0%,¹³ thus reflecting a potential ceiling effect for the A1CNow. Accordingly, healthcare professionals and future intervention programs should be aware of the underestimation of the A1CNow device to avoid inappropriate diagnosis and changes in T2D treatment. Nonetheless, the A1CNow device meets the latest criterion set by the NGSP,²¹ and when compared with laboratory samples, A1CNow results among T2D patients were either accurate (80.2%) or acceptable (17.7%).²² Although underestimation of A1C values may occur, the A1CNow seems appropriate for wider utilization among populations facing barriers to clinical care or for assessing A1C trends over time. Confirming A1C values obtained using POC devices with a clinical reference standard method is desirable whenever possible to avoid inappropriate changes in T2D treatment.

In the present study, implementing an A1C POC device for self-testing in the community setting required significant effort and support from both research staff and participants to ensure the optimal conditions for testing and successful completion of testing. A1CNow testing requires specific testing guidelines (e.g., away from direct sunlight, room temperature, flat surface, and no physical disturbance of the device) for optimal testing. Therefore, research staff and participants were in constant communication with one another, making sure the testing conditions met the manufacturer’s guidelines. A number of participants had difficulty filling the blood collector fully, which resulted in error codes and necessitated retesting. Retesting using the A1CNow required a new blood sample, additional use of a testing kit, and a longer meeting time, taxing both participants and research staff. A few participants declined to perform retesting due to operational challenges. The overall experience from this study suggests that to successfully implement A1C POC devices in the community setting, it is paramount to build rapport with participants and provide sufficient resources, close supervision, and clear communication to reduce participant burden and ensure optimal conditions for testing.

The findings of this study should be considered in light of several limitations. First, detailed feedback from participants on performing at-home A1C self-testing and their subjective ease of using the A1CNow were not systematically gathered. However, most participants expressed positive experiences using the A1CNow during the meeting with research staff. Second, the A1CNow measurements were compared with DCA measurements, not venous blood samples analyzed by a clinical laboratory, which is the gold standard for assessing A1C values. Despite these limitations, this study provides valuable insight into implementing A1C POC devices in the AA community setting and illustrates feasibility, challenges, and performance of A1C POC devices and self-testing of A1C at home. The present study extends previous literature by examining feasibility of at home A1C self-testing using a POC device among AA adults over 60 years. Ongoing improvements in POC devices have the potential to expand research and clinical care, especially for underserved communities most at risk for T2D and in need of easy access to resources to better manage their T2D and reduce diabetes-related complications.

Implications for Practice

The present study of the TX STRIDE clinical trial demonstrates that A1C self-testing using the A1CNow device is feasible in the AA community setting and produces reliable A1C measurements. However, the A1CNow underestimates A1C values when compared with the DCA as a reference standard. The performance of A1C POC devices needs further investigation and improvements. Nonetheless, the widespread use of POC devices for screening purposes and to monitor A1C routinely would be especially beneficial for vulnerable communities at high risk for developing T2D and experiencing significant barriers to effective T2D self-management and clinical care. Having an immediate A1C result available has the potential to translate to improved diabetes management and glycemic control.²³