Ascertainment of Outpatient Visits by Patients with Diabetes: The National Ambulatory Medical Care Survey (NAMCS) and the National Hospital Ambulatory Medical Care Survey (NHAMCS)

Abstract

Aims

To estimate and evaluate the sensitivity and specificity of providers’ diagnosis codes and medication lists to identify outpatient visits by patients with diabetes.

Methods

We used data from the 2006 to 2010 National Ambulatory Medical Care Survey and National Hospital Ambulatory Medical Care Survey. We assessed the sensitivity and specificity of providers’ diagnoses and medication lists to identify patients with diabetes, using the checkbox for diabetes as the gold standard. We then examined differences in sensitivity by patients’ characteristics using multivariate logistic regression models.

Results

The checkbox identified 12,647 outpatient visits by adults with diabetes among the 70,352 visits used for this analysis. The sensitivity and specificity of providers’ diagnoses or listed diabetes medications were 72.3% (95% CI: 70.8% to 73.8%) and 99.2% (99.1% to 99.4%), respectively. Diabetic patients ≥75 years pf age, women, non-Hispanics, and those with private insurance or Medicare were more likely to be missed by providers’ diagnoses and medication lists. Diabetic patients who had more diagnosis codes and medications recorded, had glucose or hemoglobin A1c measured, or made office- rather than hospital-outpatient visits were less likely to be missed.

Conclusions

Providers’ diagnosis codes and medication lists fail to identify approximately one quarter of outpatient visits by patients with diabetes.

INTRODUCTION

Diabetes mellitus is a chronic disease associated with substantial health care utilization and costs. Most studies of diabetes have focused on inpatient utilization and costs. Such studies have consistently shown higher hospitalization rates (1–5) and longer hospital stays (2; 3; 5) for patients with diabetes compared to the general population. A better understanding of the outpatient health care utilization and costs of diabetes would help improve planning for health care resource needs.

The National Ambulatory Medical Care Survey (NAMCS) and the National Hospital Ambulatory Medical Care Survey (NHAMCS) are important resources for understanding the burden of outpatient health care for diabetes. Both surveys collect information on the utilization of ambulatory medical care services (6). In previous studies, diabetes status was usually inferred from diagnosis codes listed for visits (7–12), although two studies (13; 14) also used anti-diabetic medications on the medication list and positive responses to the checkbox to ascertain diabetes mellitus. Unfortunately, the diabetes checkbox were only included on the surveys from 1993 to 1996 at that time (15). Using diagnosis codes and listed anti-diabetic medications might underestimate visits made by patients with diabetes because diagnosis codes for diabetes tend to be under-reported (16; 17) and not listed for visits that are not immediately related to diabetes, and because patients may not be treated with an anti-diabetic medication or their anti-diabetic medications might not be listed.

In 2005, checkboxes for several medical conditions, including diabetes mellitus, became a routine part of the NAMCS and NHAMCS of Outpatient Department (NHAMCS-OPD) Patient Record Forms (15). The checkbox asks if the patient has the condition, regardless of the reasons for the visit. In our recent analysis of the NHAMCS of Emergency Departments (NHAMCS-ED) (18), we found that the sensitivity of identifying diabetes was only 20.5% when using a combination of diagnosis codes and medication lists (excluding biguanides). In other words, the checkbox captured approximately five times more patients with diabetes than the combination of diagnosis codes and medication lists. The sensitivities also depended on the age of the patient, insurance status, the number of diagnosis codes and medications listed, and the nature of the medical care provided in the Emergency Department (ED).

In this study, our objectives were: 1) to estimate the sensitivity and specificity of providers’ diagnosis codes and medication lists to identify patients with diabetes, compared to the checkbox as the gold standard; and 2) to describe and to evaluate the differences in the sensitivity of providers’ diagnosis codes and medication lists based on patients’ characteristics in NAMCS and NHAMCS-OPD. We hypothesized that the sensitivity of diagnosis codes and medication lists to identify patients with diabetes in outpatient settings would be higher than we observed in EDs, as patients with diabetes are more likely to make outpatient visits for diabetes management than they are to make ED visits for diabetes management.

MATERIALS AND METHODS

Study Setting

The NAMCS utilizes a three-stage probability sampling design: 112 primary sampling units (PSU); physicians within PSUs stratified into specialty groups; and patient visits within practices using a 10% to 100% random sample (19). The NAMCS includes patient visits from a sample of visits traditionally made to non-federally employed, office-based physicians who are primarily engaged in direct patient care, as classified by the American Medical Association or the American Osteopathic Association. Physicians who specialize in anesthesiology, pathology, and radiology are excluded. Since 2006, the NAMCS has included a sample of community health centers. This additional sample includes both physicians and mid-level health care providers. Each provider is randomly assigned to a one-week reporting period. Patient Record Forms that gather a systematic, random sample of visits are designed to be completed by the physician or office staff based on medical records, but they are often completed by Census field representatives (20). Patient contacts made outside the physician’s office in hospital and institutional settings, or solely for administrative purposes, are not included in the NAMCS. From 2006 through 2010, the Patient Record Forms collected the following information: patient demographics; injuries/poisoning/adverse effects; diagnostic/screening services; reason for visit; continuity of care; up to three provider’s diagnoses for the visit; vital signs; health education; non-medication treatment; up to eight medications and immunizations that were ordered, supplied, administered, or continued during the visit; provider information; time spent with the provider; and visit disposition (20). From 2006 through 2010, 1,300 to 1,500 physicians participated each year, with an unweighted physician response rate of approximately 60% (19). Each year, 28,000 to 33,000 Patient Record Forms were submitted (19).

The NHAMCS-OPD utilizes a four-stage probability sampling design: 112 PSUs; hospitals in the PSUs; clinics within outpatient departments; and patient visits to the outpatient departments (19). The NHAMCS-OPD includes patient visits from a panel of 600 general and short-stay hospitals in the 50 states and the District of Columbia. Federal, military, and Veterans Administration hospitals are excluded. Each hospital participates in the survey for a four-week reporting period approximately once every 15 months. The patients’ visits to the selected hospitals are randomly sampled, with a target of 200 Patient Record Forms to be completed by hospital staff based on medical records. In this survey, a visit is defined as “a direct, personal exchange between a physician, or a staff member operating under a physician’s direction, for the purpose of seeking care and rendering health services” (19). Trained staff at the hospital complete all survey forms, including the Patient Record Forms. The Patient Record Form in the NHAMCS-OPD requests the same information as does the NAMCS Patient Record Form. From 2006 through 2010, 202 to 236 outpatient departments, including 915 to 1,058 clinics, participated each year, with an overall unweighted, two-stage response rate of 73% (19). Each year, 31,000 to 35,000 Patient Record Forms were submitted (19).

We analyzed the NAMCS and NHAMCS-OPD datasets from 2006 through 2010 (6). Although the checkbox was initiated in 2005, we began our analysis with 2006 because a new medication classification system—the Multum system—was implemented in 2006. We limited our analyses to patients 18 years of age and older who visited primary care physician providers. We chose to limit the analyses to adults because of the small number of outpatient visits by children with diabetes who were included in the NAMCS (N = 49) and NHAMCS-OPD (N = 59) datasets. We limited the analyses to visits to primary care providers because visits to subspecialty providers might not have included diabetes-related diagnosis codes when the reasons for the visits were not related to diabetes. We excluded visits with no valid diagnosis codes or with no valid responses (neither yes nor no) to the question “Were medications prescribed or provided?” In total, 70,352 Patient Record Forms in the combined dataset (40,145 in NAMCS and 30,207 in NHAMCS-OPD) were included in our analysis, representing 1.68 billion encounters (standard error [S.E.]: 0.77 billion), or 1.54 billion from NAMCS and 0.14 billion from NHAMCS-OPD, after appropriate weighting.

Because the NAMCS and NHAMCS datasets are publicly available and de-identified, this study was exempt from regulation by the Institutional Review Boards at the University of Michigan and the University of Tennessee Health Science Center.

Study Definitions

Gold standard definition

Diabetes was considered present when the response box to the question “Does the patient have diabetes?” was checked on the Patient Record Form.

Alternative definition using provider diagnoses and diabetes medications

We classified patients as having diabetes if they were identified as having diabetes by at least one of the provider diagnosis codes described below or at least one of the diabetes medications below.

We classified patients as having diabetes if one of the following International Classification of Disease, Ninth Revision, Clinical Modification (ICD-9-CM) codes were included among the three providers’ diagnosis codes listed on the Patient Record Form: 250 (diabetes mellitus), 357.2 (polyneuropathy in diabetes), 362.0 (eye diseases in diabetes), 366.41 (diabetic cataract), and 648.0 (diabetes mellitus of mother) (21). In our analysis, we did not attempt to distinguish among type 1, type 2, or gestational diabetes. Although either diabetes mellitus or diabetes insipidus may be indicated by a positive response to the checkbox (19), diabetes insipidus is rare. We did not include the diagnosis code for diabetes insipidus on the list of diagnosis codes for the alternative definition.

We also classified patients as having diabetes if they were taking insulin or a glycemic-modifying agent as specified by Health Plan and Employer Data and Information Set (HEDIS) guidelines (21) or if they were prescribed one of these agents at the visit. HEDIS guidelines include the following glycemic-modifying agents: sulfonylureas, meglitinides, alpha-glucosidase inhibitors, thiazolidinediones, anti-diabetic combinations (including combinations of the medications in the classes mentioned and metformin), amylin analogs, and miscellaneous anti-diabetic agents (saxagliptin, sitagliptin, liraglutide, exenatide) (21; 22). HEDIS guidelines exclude biguanides when they are listed as single glycemic-modifying agents because they may be prescribed for indications other than diabetes (21). In a supplemental analysis, we also considered a prescription for a biguanide (metformin) to indicate diabetes. The NAMCS and NHAMCS-OPD use the three drug-level categories defined by the Multum Lexicon Plus system (Cerner Multum, Inc.) (20) to document any medications listed on Patient Record Forms. We specifically used the level 2 category of the Multum Lexicon Plus system to identify insulin and anti-diabetic agents, and we used the level 3 category to exclude buiguanides. The level 2 code 099 (insulin and anti-diabetic agents) includes the following level 3 categories: sulfonylureas, biguanides, insulin, alpha-glucosidase inhibitors, thiazolidinediones, meglitinides, miscellaneous anti-diabetic agents, anti-diabetic combinations, dipeptidyl peptidase 4 inhibitors, amylin analogs, and incretin mimetics. To exclude biguanides, we used the level 3 category of biguanides (code 214). The level 3 category code 214 was previously described as “metabolic agents; antidiabetic agents; non-sulfonylureas,” but all level 3 category codes 214 were coded for biguanides on the Patient Record Forms included in our analyses.

Statistical Analysis

All analyses accounted for the complex survey design using the design variables provided in the NAMCS and NHAMCS public datasets (20). We used SAS (23) and SUDAAN (24) for all analyses. Taylor series methods were applied as the variance estimation method. A type I error of 0.05 was considered statistically significant. We did not include estimates that were based on fewer than 30 observations or had greater than 30% of the relative standard error, because the National Center for Health Statistics (NCHS) considers such estimates to be unreliable (20). We assumed that each Patient Record Form represented a different patient, but it is possible that individual patients are included in the datasets more than once. Because there were no unique identifiers for patients in the NAMCS and NHAMCS data, we were unable to account for the possibility of repeat visits by the same patient.

First, we calculated the sensitivity and specificity with 95% confidence intervals (CI), using the alternative definition relative to the gold standard. We then calculated the sensitivity using the alternative definition by patient characteristics.

We described patient characteristics according to the following factors: the patient’s mid-year age, sex, race/ethnicity, expected source of payment, geographic region of the hospital or the practice, number of previous visits to the clinic or office, number of diagnoses listed on the survey form, number of medications listed on the survey form, diagnostic services and procedures provided in the outpatient visit (e.g., blood test for glucose, hemoglobin A1c, and lipids/cholesterol), and disposition (e.g., referral to other physician, return at specific time, referral to emergency room [ER]/admit to hospital, other, and missing). We re-classified race and ethnicity from the imputed race and ethnicity variables in the NAMCS and NHAMCS-OPD datasets. From 2006 through 2008, race/ethnicity was recoded into seven categories (White Only, Non-Hispanic; Black Only, Non-Hispanic; Hispanic; Asian Only; Native Hawaiian, Other Pacific Islander Only; American Indian/Alaska Native Only; Multiple Races). We grouped the last four categories into one category—Other Race/Multiple Race, Non-Hispanic. From 2009 through 2010, race/ethnicity was recoded into four categories (White Only, Non-Hispanic; Black Only, Non-Hispanic; Hispanic; Other Race/Multiple Race, Non-Hispanic). We re-classified the expected source of payment from the original responses, which allowed multiple responses, into Medicaid, Medicare, private insurance, self-pay, others, (including worker’s compensation, no charge/charity, and other), and unknown. We considered the payment sources to be mutually exclusive and selected the first source appearing on our list to be the payment source. We reclassified disposition into five mutually exclusive groups by hierarchy in the following order: refer to ER/admit to hospital; refer to other physician; return at specific time (including telephone follow-up planned in 2006–2008); other (including no follow-up planned/return if needed in 2006–2008); and missing. We treated the datasets (NAMCS versus NHAMCS-OPD) and years of the surveys as covariates when we combined the datasets.

We developed univariate and multivariate logistic regression models to describe the likelihood of correctly identifying patients with diabetes using the alternative definition when the gold standard was the checkbox for diabetes. Because the likelihood of identifying patients with diabetes (true positives) using the alternative definition was substantial, we estimated prevalence ratios rather than odds ratios (25), using the predicted marginal proportion of the prevalence (26) based on logistic regression models. This ratio of the prevalences is interpreted as the ratio of the sensitivity of the alternative definition for specific patients’ characteristics compared to the reference characteristic. We developed two multivariate models to adjust for potential counfounding effects among the characteristics. Model A included demographic factors (age, sex, race/ethnicity, expected sources of payment, and region), the dataset (NAMCS versus NHAMCS-OPD), and the years of the surveys. Model B included factors representing the medical conditions and care provided at the outpatient visits (number of previous visits to the clinic, number of diagnoses listed on the survey form, number of medications listed on the survey form, a blood test for glucose, hemoglobin A1c, and lipids/cholesterol, and disposition) in addition to the demographic factors included in model A.

Finally, we described the primary diagnoses for outpatient visits grouped into ICD-9-CM chapters (27) in patients with and without diabetes-related diagnoses who had checkboxes marked indicating a diagnosis of diabetes.

RESULTS

In the 2006–2010 NAMCS and NHAMCS-OPD datasets, which included records of patients ≥18 years of age who visited primary care providers and had valid diagnosis codes and medication information, we identified 7,100 office- and 5,547 hospital-outpatient visits—a total of 12,647 outpatient visits (unweighted counts) indicating diabetes based on the checkbox. After appropriate weighting, the estimated number of outpatient visits by patients with diabetes in the United States over the five years period was 291,922,000 (S.E.: 15,470,000), including 264,766,000 (S.E.: 15,479,000) office- and 27,157,000 (S.E.: 2,639,000) hospital-outpatient visits. These represent 17.4% (S.E. 0.4%) of all outpatient visits from 2006 through 2010.

When we used the checkbox as the gold standard, the alternative definition used to identify patients with diabetes showed moderate sensitivity and high specificity. Sensitivity was 72.3% (95% confidence interval [C.I.]: 70.8% to 73.8%) when using a combination of diagnosis codes and medication lists (not including biguanides) in the combined datasets. The specificity was 99.2% (95% C.I.: 99.1% to 99.4%). When we repeated the analysis using the alternative definition but including biguanides as a diabetes medication, sensitivity was 76.9% (95% C.I.: 75.5% to 78.4%) and specificity was 98.8% (95% C.I.: 98.6% to 99.0%).

The sensitivity of the alternative definition, when we assumed that the checkbox was the gold standard, differed according to the characteristics of the patients with diabetes (Table 1 and Table 2). Univariate analysis showed that the alternative definition was less sensitive for patients who were younger than 45 or older than 75 years of age, were female, were Non-Hispanic White, had private insurance or Medicare, had made six or more outpatient visits in the past 12 months, and whose disposition was referral to emergency room or admission to hospital. The alternative definition was more sensitive in identifying diabetes when patients had a greater number of diagnoses/medications or had laboratory orders for glucose, hemoglobin A1c, or lipids/cholesterol.

Table 1.

Characteristics of patients with diabetes identified by the checkbox, either identified or not identified using diagnosis codes and medication lists (excluding biguanides), from the combined datasets of the National Ambulatory Medical Care Survey (NAMCS) and the National Hospital Ambulatory Medical Care Survey (NHAMCS), 2006–2010

	Overall	Diabetes identified using diagnosis codes and medication lists
	No. (S.E.), in 1,000s	Yes No. (S.E.), in 1,000s	No No. (S.E.), in 1,000s	Sensitivity
Age, years
18–44	34,660 (2,169)	22,994 (1,658)	11,666 (1,041)	66.3% (2.2%)
45–64	121,600 (7,204)	89,331 (5,486)	32,269 (2,355)	73.5% (1.1%)
65–74	68,747 (3,558)	51,689 (2,870)	17,059 (1,193)	75.2% (1.3%)
≥75	66,915 (4,657)	47,119 (3,187)	19,795 (1,940)	70.4% (1.6%)
Sex
Female	158,882 (8,567)	111,784 (6,024)	47,098 (3,192)	70.4% (1.0%)
Male	133,040 (7,626)	99,349 (5,948)	33,692 (2,197)	74.7% (0.9%)
Race/ethnicity
Non-Hispanic White	196,060 (12,507)	137,406 (8,999)	58,654 (4,140)	70.1% (0.9%)
Non-Hispanic Black	42,065 (4,210)	32,275 (3,311)	9,790 (1,229)	76.7% (1.8%)
Hispanic	39,055 (4,904)	30,251 (3,784)	8,805 (1,428)	77.5% (2.0%)
Others	14,742 (1,897)	11,202 (1,476)	3,541 (615)	76.0% (2.7%)
Sources of payment
Private	113,091 (7,276)	79,112 (5,379)	33,979 (2,532)	70.0% (1.3%)
Medicaid	41,449 (3,203)	31,071 (2,534)	10,378 (922)	75.0% (1.4%)
Medicare	114,435 (7,287)	83,215 (5,140)	31,220 (2,563)	72.7% (1.1%)
Self-pay	8,086 (919)	6,209 (829)	1,877 (247)	76.8% (2.9%)
Others	7,808 (1,126)	6,117 (911)	1,691 (304)	78.3% (2.5%)
Unknown	7,053 (1,126)	5,408 (916)	1,644 (451)	76.7% (5.0%)
Region
Northeast	50,771 (6,473)	37,048 (4,727)	13,723 (1,946)	73.0% (1.4%)
Midwest	77,123 (8,478)	55,252 (6,349)	21,872 (2,375)	71.6% (1.2%)
South	112,496 (9,039)	81,679 (6,427)	30,818 (3,314)	72.6% (1.5%)
West	51,532 (6,623)	37,154 (5,025)	14,378 (1,791)	72.1% (1.4%)
Number of clinic visits in the past 12 months
0 (new patient)	12,957 (1,078)	9,969 (952)	2,989 (404)	76.9% (2.8%)
1–2	60,092 (4,088)	42,799 (2,989)	17,293 (1,538)	71.2% (1.5%)
3–5	112,249 (6,877)	83,885 (5,414)	28,364 (2,061)	74.7% (1.1%)
≥6	106,625 (6,544)	74,481 (4,870)	32,143 (2,235)	69.9% (1.2%)
Number of diagnosis codes
1	50,547 (3,490)	26,652 (2,238)	23,895 (1,806)	52.7% (2.0%)
2	67,331 (4,145)	47,748 (3,113)	19,583 (1,593)	70.9% (1.5%)
3	174,045 (10,491)	136,733 (8,023)	37,311 (3,127)	78.6% (1.0%)
Number of medications
0–2	81,275 (5,667)	46,628 (3,661)	34,647 (2,660)	57.4% (1.7%)
3–5	84,719 (5,087)	62,337 (3,887)	22,382 (1,792)	73.6% (1.4%)
6–8	125,929 (8,049)	102,168 (6,538)	23,761 (1,958)	81.1% (0.9%)
Blood test for glucose ordered
No	221,248 (11,676)	152,809 (8,212)	68,439 (4,264)	69.1% (0.9%)
Yes	70,675 (5,410)	58,324 (4,589)	12,351 (1,170)	82.5% (1.1%)
Hemoglobin A1c ordered
No	213,479 (10,695)	144,471 (7,234)	69,007 (4,094)	67.7% (0.8%)
Yes	78,444 (6,415)	66,662 (5,528)	11,782 (1,396)	85.0% (1.3%)
Lipids/cholesterol ordered
No	218,534 (11,075)	151,958 (7,837)	66,576 (3,863)	69.5% (0.8%)
Yes	73,389 (5,357)	59,175 (4,294)	14,214 (1,552)	80.6% (1.4%)
Disposition
Return at specific time	213,369 (12,525)	158,671 (9,282)	54,698 (3,831)	74.4% (0.8%)
Refer to other physician	32,492 (2,288)	23,991 (1,700)	8,500 (937)	73.8% (1.9%)
Refer to ER/Admit to hospital	1,354 (343)	931 (261)	-¶	68.8% (6.8%)
Other	39,231 (3,096)	23,619 (2,167)	15,612 (1,376)	60.2% (2.2%)
Missing	5,477 (897)	3,920 (712)	1,557 (347)	71.6% (4.7%)
Survey
NAMCS	264,766 (15,479)	191,973 (11,349)	72,792 (4,866)	72.5% (0.8%)
NHAMCS-OPD	27,157 (2,639)	19,160 (1,850)	7,997 (938)	70.6% (1.6%)
Survey year
2006	49,945 (3,747)	37,091 (2,781)	12,854 (1,408)	74.3% (1.8%)
2007	52,725 (4,387)	37,657 (3,258)	15,068 (1,673)	71.4% (2.0%)
2008	59,889 (5,113)	41,727 (3,733)	18,163 (1,820)	69.7% (1.7%)
2009	70,850 (6,441)	51,681 (5,154)	19,169 (1,844)	72.9% (1.7%)
2010	58,514 (4,529)	42,978 (3,588)	15,536 (1,596)	73.4% (1.9%)

^¶Fewer than 30 observations or greater than 30% of relative standard error. The NCHS considers such estimates to be unreliable.

S.E.: standard error; ER: emergency room.

Table 2.

Predicted prevalence ratio (PRR) of identifying diabetes using diagnosis codes and medication lists excluding biguanides (the ratio of the sensitivities) among outpatient visits by patients with diabetes identified by the checkbox, from the combined datasets of the National Ambulatory Medical Care Survey (NAMCS) and the National Hospital Ambulatory Medical Care Survey (NHAMCS), 2006–2010

	Univariate		Multivariate model A		Multivariate model B
	PRR (95% C.I.)	P-value	PRR (95% C.I.)	P-value	PRR (95% C.I.)	P-value
Age, years		<0.01		<0.01		<0.01
18–44	Reference		Reference		Reference
45–64	1.11 (1.03, 1.19) †		1.12 (1.04, 1.21) †		1.05 (0.99, 1.12)
65–74	1.13 (1.05, 1.22) †		1.14 (1.05, 1.24) †		1.06 (0.98, 1.14)
≥75	1.06 (0.98, 1.15) §		1.07 (0.98, 1.17) §		0.98 (0.91, 1.06) ‡§
Sex		<0.01		<0.01		<0.01
Female	Reference		Reference		Reference
Male	1.06 (1.03, 1.10) †		1.07 (1.03, 1.10) †		1.06 (1.02, 1.09) †
Race/ethnicity		<0.01		<0.01		<0.01
Non-Hispanic White	Reference		Reference		Reference
Non-Hispanic Black	1.09 (1.04, 1.15) †		1.10 (1.04, 1.16) †		1.09 (1.05, 1.14) †
Hispanic	1.11 (1.04, 1.17) †		1.10 (1.03, 1.17) †		1.11 (1.04, 1.19) †
Others	1.08 (1.01, 1.17) †		1.09 (1.01, 1.18) †		1.08 (1.00, 1.16) †
Expected sources of payment		<0.01		0.02		<0.01
Private	Reference		Reference		Reference
Medicaid	1.07 (1.02, 1.13) †		1.07 (1.01, 1.13) †		1.05 (0.99, 1.11)
Medicare	1.04 (0.99, 1.09)		1.04 (0.99, 1.09)		1.01 (0.97, 1.05)
Self-pay	1.10 (1.01, 1.19) †		1.10 (1.01, 1.19) †		1.12 (1.05, 1.20) †§
Others	1.12 (1.04, 1.20) †§		1.12 (1.05, 1.20) †§		1.10 (1.04, 1.16) †§
Region		0.90		0.65		0.64
Northeast	Reference		Reference		Reference
Midwest	0.98 (0.93, 1.03)		0.99 (0.94, 1.04)		0.98 (0.93, 1.04)
South	0.99 (0.94, 1.05)		0.98 (0.93, 1.04)		1.01 (0.95, 1.08)
West	0.99 (0.94, 1.04)		0.96 (0.91, 1.02)		0.98 (0.92, 1.04)
Number of clinic visits in the past 12 months		<0.01				0.07
0 (new patient)	Reference				Reference
1–2	0.93 (0.85, 1.01)				0.94 (0.87, 1.03)
3–5	0.97 (0.90, 1.05)				0.99 (0.92, 1.06)
≥ 6	0.91 (0.84, 0.98) †§				0.94 (0.87, 1.01) §
Number of diagnoses		<0.01				<0.01
1	Reference				Reference
2	1.34 (1.23, 1.47) †				1.23 (1.14, 1.32) †
3	1.49 (1.37, 1.62) †‡				1.29 (1.20, 1.38) †‡
Number of medications		<0.01				<0.01
0–2	Reference				Reference
3–5	1.28 (1.20, 1.38) †				1.19 (1.11, 1.26) †
7–8	1.41 (1.33, 1.51) †‡				1.33 (1.26, 1.41) †‡
Blood test for glucose ordered		<0.01				<0.01
No	Reference				Reference
Yes	1.19 (1.15, 1.24) †				1.10 (1.05, 1.16) †
Hemoglobin A1c ordered		<0.01				<0.01
No	Reference				Reference
Yes	1.26 (1.21, 1.30)†				1.19 (1.14, 1.25) †
Lipids/cholesterol ordered		<0.01				0.08
No	Reference				Reference
Yes	1.16 (1.12, 1.20)†				0.95 (0.90, 1.01)
Disposition		<0.01				0.07
Return at specific time	Reference				Reference
Refer to other physician	0.99 (0.94, 1.05)				0.98 (0.92, 1.03)
Refer to ER/Admit to hospital	0.93 (0.76, 1.12)				1.07 (0.94, 1.21)
Other	0.81 (0.75, 0.87)†‡				0.92 (0.87, 0.98) †
Survey		0.27		0.03		0.01
NAMCS	Reference		Reference		Reference
NHAMCS-OPD	0.97 (0.93, 1.02)		0.95 (0.90, 1.00) †		0.94 (0.89, 0.99) †
Survey year		0.32		0.40		0.57
2006	Reference		Reference		Reference
2007	0.96 (0.90, 1.03)		0.96 (0.89, 1.02)		0.97 (0.91, 1.04)
2008	0.94 (0.88, 1.00)†		0.94 (0.89, 1.00)		0.95 (0.90, 1.01)
2009	0.98 (0.92, 1.05)		0.98 (0.91, 1.05)		0.97 (0.91, 1.03)
2010	0.99 (0.93, 1.06)		0.99 (0.93, 1.05)		0.97 (0.91, 1.04)

Unknown/missing category for expected payment sources and for disposition were included in the models, although their predicted prevalence ratios are not shown in the table. The areas under the receiver-operating characteristic (ROC) curve were 0.571 and 0.698 for multivariate models A and B, respectively. P-values were estimated by Wald F-statistics for the overall significance of each variable. P-values for the pairwise comparisons for characteristics with more than two categories were estimated by t-statistics and are expressed with the following notations:

^†P<0.05 compared to the first category;

^‡P<0.05 compared to the second category;

^§P<0.05 compared to the third category.

S.E.: standard error; ER: emergency room; PRR: predicted prevalence ratio.

Multivariate model A adjusted only for basic demographic factors (age, sex, race/ethnicity, expected sources of payment, and region). The results from multivariate model A were similar to the results from the univariate models. The area under the receiver-operating characteristic (ROC) curve was 0.571. After adjusting for all potential confounders (multivariate model B), the alternative definition was less sensitive to identify diabetic patients who were 75 years of age or older, female, and Non-Hispanic White compared to patients of other race/ethnicity. Compared to patients with self-pay or other payment sources, the alternative definition was less sensitive to identify diabetic patients with private insurance or Medicare. The number of past clinic visits was not associated with the sensitivity of alternative definition, but having made six or more visits in the past 12 months was associated with a lower sensitivity. The alternative definition was more sensitive to identify patients with diabetes who had a greater numbers of diagnosis codes or medications listed, and patients for whom glucose or hemoglobin A1c measurements were ordered. The alternative definition was more sensitive to identify patients with diabetes who made office-outpatient visits than hospital-outpatient visits. The area under the ROC curve based on multivariate model B was 0.698. When we also included biguanides in the list of anti-diabetic medications, the patient characteristics that impacted the sensitivity of the alternative definition were essentially the same.

We also investigated the sensitivity of using only diagnosis codes (Tables 3) and only medication lists (Tables 4) to identify patients with diabetes. These additional analyses found that age, sex, and race/ethnicity remained significant factors associated with the likelihood of identifying patients with diabetes if we used only diagnosis codes or only medication lists. We also found that the differences in the likelihood of identifying diabetes across insurance types remained significant when diagnosis codes alone were used but not when medication lists alone were used.

Table 3.

Predicted prevalence ratio (PRR) of identifying diabetes using only diagnosis codes (the ratio of the sensitivities) among outpatient visits by patients with diabetes identified by the checkbox, from the combined datasets of the National Ambulatory Medical Care Survey (NAMCS) and the National Hospital Ambulatory Medical Care Survey (NHAMCS), 2006–2010

	Univariate		Multivariate model A		Multivariate model B
	PRR (95% C.I.)	P-value	PRR (95% C.I.)	P-value	PRR (95% C.I.)	P-value
Age, years		<0.01		<0.01		<0.01
18–44	Reference		Reference		Reference
45–64	1.11 (1.02, 1.22) †		1.13 (1.04, 1.24) †		1.08 (1.00, 1.17)
65–74	1.11 (1.00, 1.23) †		1.12 (1.01, 1.25) †		1.07 (0.95, 1.18)
≥75	0.98 (0.89, 1.08) ‡§		1.00 (0.90, 1.11) ‡§		0.94 (0.85, 1.04) ‡§
Sex		<0.01		<0.01		<0.01
Female	Reference		Reference		Reference
Male	1.08 (1.04, 1.13) †		1.09 (1.04, 1.14) †		1.07 (1.03, 1.11) †
Race/ethnicity		<0.01		<0.01		<0.01
Non-Hispanic White	Reference		Reference		Reference
Non-Hispanic Black	1.13 (1.06, 1.21) †		1.13 (1.06, 1.21) †		1.11 (1.05, 1.18) †
Hispanic	1.18 (1.09, 1.28) †		1.17 (1.08, 1.28) †		1.17 (1.08, 1.27) †
Others	1.17 (1.06, 1.29) †		1.20 (1.09, 1.33) †		1.16 (1.05, 1.28) †
Expected sources of payment		<0.01		0.03		<0.01
Private	Reference		Reference		Reference
Medicaid	1.09 (1.02, 1.16) †		1.08 (1.01, 1.16) †		1.08 (1.01, 1.15) †
Medicare	1.01 (0.96, 1.07) ‡		1.05 (0.99, 1.12)		1.02 (0.97, 1.09)
Self-pay	1.17 (1.05, 1.31) †§		1.16 (1.03, 1.29) †		1.21 (1.10, 1.33) †‡§
Others	1.17 (1.05, 1.32) †§		1.18 (1.05, 1.32) †		1.14 (1.03, 1.25) †
Region		0.37		0.08		0.17
Northeast	Reference		Reference		Reference
Midwest	0.95 (0.87, 1.03)		0.96 (0.88, 1.04)		0.98 (0.89, 1.07)
South	0.98 (0.90, 1.07)		0.96 (0.88, 1.05)		1.00 (0.90, 1.10)
West	0.93 (0.85, 1.02)		0.88 (0.80, 0.97) †		0.91 (0.83, 1.01)
Number of clinic visits in the past 12 months		<0.01				0.02
0 (new patient)	Reference				Reference
1–2	0.99 (0.88, 1.11)				1.02 (0.91, 1.15)
3–5	1.02 (0.92, 1.14)				1.06 (0.96, 1.18)
≥6	0.88 (0.79, 0.98) †‡§				0.96 (0.86, 1.08) §
Number of diagnoses		<0.01				<0.01
1	Reference				Reference
2	1.87 (1.61, 2.15) †				1.71 (1.50, 1.94) †
3	2.31 (1.99, 2.67) †‡				2.06 (1.81, 2.35) †‡
Blood test for glucose ordered		<0.01				<0.01
No	Reference				Reference
Yes	1.32 (1.25, 1.39) †				1.14 (1.06, 1.22) †
Hemoglobin A1c ordered		<0.01				<0.01
No	Reference				Reference
Yes	1.44 (1.37, 1.51) †				1.32 (1.24, 1.41) †
Lipids/cholesterol ordered		<0.01				0.11
No	Reference				Reference
Yes	1.27 (1.21, 1.34) †				0.94 (0.87, 1.02)
Disposition		<0.01				<0.01
Return at specific time	Reference				Reference
Refer to other physician	0.91 (0.83, 0.99) †				0.87 (0.80, 0.95) †
Refer to ER/Admit to hospital	0.67 (0.42, 1.08) †				0.90 (0.61, 1.33)
Other	0.73 (0.66, 0.81) †‡				0.87 (0.80, 0.95) †
Survey		0.57		0.05		0.17
NAMCS	Reference		Reference		Reference
NHAMCS-OPD	0.98 (0.92, 1.05)		0.94 (0.88, 1.00) †		0.94 (0.87, 1.02)
Survey year		0.40		0.58		0.69
2006	Reference		Reference		Reference
2007	0.98 (0.88, 1.09)		0.98 (0.88, 1.09)		0.98 (0.89, 1.08)
2008	0.94 (0.86, 1.01)		0.95 (0.88, 1.03)		0.95 (0.89, 1.02)
2009	0.99 (0.90, 1.10)		1.00 (0.90, 1.10)		0.98 (0.89, 1.08)
2010	1.00 (0.90, 1.10)		1.00 (0.90, 1.10)		0.99 (0.89, 1.09)

Unknown/missing category for expected payment sources and for disposition were included in the models, although their predicted prevalence ratios are not shown in the table. Multivariate models A and B did not include the number of medications as an independent variable. The areas under the receiver-operating characteristic (ROC) curve were 0.572 and 0.724 for multivariate models A and B, respectively. P-values were estimated by Wald F-statistics for the overall significance of each variable. P-values for the pairwise comparisons for characteristics with more than two categories were estimated by t-statistics and are expressed with the following notations:

^†P<0.05 compared to the first category;

^‡P<0.05 compared to the second category;

^§P<0.05 compared to the third category.

S.E.: standard error; ER: emergency room; PRR: predicted prevalence ratio.

Table 4.

Predicted prevalence ratio (PRR) of identifying diabetes using only medication lists excluding biguanides (the ratio of the sensitivities) among outpatient visits by patients with diabetes identified by the checkbox, from the combined datasets of the National Ambulatory Medical Care Survey (NAMCS) and the National Hospital Ambulatory Medical Care Survey (NHAMCS), 2006–2010

	Univariate		Multivariate model A		Multivariate model B
	PRR (95% C.I.)	P-value	PRR (95% C.I.)	P-value	PRR (95% C.I.)	P-value
Age, years		0.18		0.65		0.05
18–44	Reference		Reference		Reference
45–64	1.04 (0.93, 1.16)		1.05 (0.94, 1.17)		0.91 (0.84, 1.00) †
65–74	1.12 (1.00, 1.26) †		1.08 (0.95, 1.23)		0.90 (0.81, 1.01)
≥75	1.09 (0.95, 1.25)		1.04 (0.89, 1.21)		0.84 (0.74, 0.95) †
Sex		<0.01		<0.01		<0.01
Female	Reference		Reference		Reference
Male	1.10 (1.03, 1.18) †		1.11 (1.05, 1.19) †		1.11 (1.04, 1.17) †
Race/ethnicity		0.16		0.12		<0.01
Non-Hispanic White	Reference		Reference		Reference
Non-Hispanic Black	1.13 (1.01, 1.25) †		1.15 (1.02, 1.28) †		1.15 (1.06, 1.25) †
Hispanic	1.06 (0.96, 1.16)		1.07 (0.98, 1.18)		1.13 (1.03, 1.24) †
Others	1.08 (0.92, 1.27)		1.08 (0.90, 1.28)		1.12 (0.96, 1.31)
Expected sources of payment		0.22		0.39		0.99
Private	Reference		Reference		Reference
Medicaid	1.11 (0.99, 1.25)		1.11 (0.98, 1.25)		1.01 (0.91, 1.12)
Medicare	1.12 (1.01, 1.24) †		1.12 (0.99, 1.27)		1.03 (0.93, 1.13)
Self-pay	0.97 (0.80, 1.18)		0.98 (0.80, 1.20)		1.01 (0.86, 1.19)
Others	1.03 (0.86, 1.24)		1.02 (0.83, 1.25)		0.98 (0.83, 1.16)
Region		0.51		0.43		0.93
Northeast	Reference		Reference		Reference
Midwest	1.08 (0.94, 1.23)		1.08 (0.95, 1.24)		1.02 (0.89, 1.18)
South	1.00 (0.88, 1.14)		0.99 (0.87, 1.14)		1.03 (0.90, 1.18)
West	1.06 (0.93, 1.21)		1.05 (0.92, 1.20)		1.04 (0.91, 1.20)
Number of clinic visits in the past 12 months		0.01				0.17
0 (new patient)	Reference				Reference
1–2	0.87 (0.72, 1.05)				0.87 (0.74, 1.03)
3–5	1.00 (0.84, 1.19) ‡				0.95 (0.82, 1.11) ‡
≥6	0.99 (0.82, 1.19) ‡				0.92 (0.78, 1.07)
Number of medications		<0.01				<0.01
0–2	Reference				Reference
3–5	3.62 (3.08, 4.24) †				3.57 (3.06, 4.17) †
7–8	4.98 (4.30, 5.78) †‡				5.06 (4.37, 5.86) †‡
Blood test for glucose ordered		<0.01				0.02
No	Reference				Reference
Yes	1.14 (1.04, 1.24) †				1.09 (1.01, 1.18) †
Hemoglobin A1c ordered		<0.01				<0.01
No	Reference				Reference
Yes	1.19 (1.10, 1.28) †				1.14 (1.04, 1.24) †
Lipids/cholesterol ordered		0.09				0.01
No	Reference				Reference
Yes	1.08 (0.99, 1.17)				0.88 (0.80, 0.96) †
Disposition		<0.01				0.07
Return at specific time	Reference				Reference
Refer to other physician	1.03 (0.94, 1.14)				1.01 (0.93, 1.10)
Refer to ER/Admit to hospital	1.03 (0.62, 1.72)				1.22 (0.93, 1.60)
Other	0.72 (0.62, 0.84) †‡				0.88 (0.79, 0.98) †‡
Survey		0.79		0.98		0.24
NAMCS	Reference		Reference		Reference
NHAMCS-OPD	1.01 (0.93, 1.11)		1.00 (0.91, 1.09)		0.95 (0.87, 1.04)
Survey year		0.19		0.15		0.03
2006	Reference		Reference		Reference
2007	0.86 (0.76, 0.98) †		0.86 (0.75, 0.98) †		0.90 (0.81, 1.01)
2008	0.91 (0.80, 1.03)		0.91 (0.80, 1.04)		0.93 (0.84, 1.02)
2009	0.94 (0.82, 1.08)		0.93 (0.81, 1.07)		0.91 (0.81, 1.01)
2010	0.89 (0.78, 1.02)		0.88 (0.77, 1.00)		0.84 (0.76, 0.93) †

Unknown/missing category for expected payment sources and for disposition were included in the models, although their predicted prevalence ratios are not shown in the table. Multivariate models A and B did not include the number of diagnosis codes as an independent variable. The areas under the receiver-operating characteristic (ROC) curve were 0.539 and 0.580 for multivariate models A and B, respectively. P-values were estimated by Wald F-statistics for the overall significance of each variable. P-values for the pairwise comparisons for characteristics with more than two categories were estimated by t-statistics and are expressed with the following notations:

^†P<0.05 compared to the first category;

^‡P<0.05 compared to the second category;

^§P<0.05 compared to the third category.

S.E.: standard error; ER: emergency room; PRR: predicted prevalence ratio.

Among patients who had at least one diabetes-related diagnosis code, 56.2% listed a diabetes-related code as the primary diagnosis. In patients with diabetes defined by the checkbox and who had no diabetes-related diagnosis codes listed, the most common primary diagnosis codes were others (35.3%), diseases of the circulatory system (14.9%), symptoms, signs, ill-defined conditions (13.0%), and diseases of the respiratory system (11.7%).

DISCUSSION

We found that only approximately three-quarters of the office- or hospital-outpatient visits made by patients with diabetes are identified as such using diagnosis codes and medication lists. Based on this finding, previous estimates of outpatient health care utilization by people with diabetes derived from based on the NAMCS and NHAMCS-OPD are likely to be underestimated. For example, from 2006 through 2010, without the information from the diabetes checkbox, we would have estimated the number of outpatient visits by patients with diabetes in the United States to be 222 million rather 292 million. Furthermore, recognizing that the use of the checkbox does not guarantee complete ascertainment, the degree of underestimation may still be high.

Our study shows that diagnosis codes either alone or in combination with medication lists have a lower sensitivity, even when biguanides are included as anti-diabetic medications, to identify diabetes in ambulatory settings than was shown by several previous studies using similar sets of identification criteria. Identification criteria based on diagnosis codes with and without medication lists have been validated in various settings using other data sources, such as medical records, self-reports, prescription claims, or laboratory data, which were summarized in a meta-analysis (28). The results of six studies included in the meta-analysis showed sensitivities from 74.4% to 92.3% and specificities from 96.9% to 99.6% (28). These six studies included four from Canada and two from the United States. Two factors might have contributed to the higher sensitivities of the studies in the meta-analysis as compared to our study. First, the number of diagnosis codes allowed to be listed may not have been limited to three, as was the case with the NAMCS and NHAMCS. Second, NAMCS and NHAMCS did not consistently collect billing ICD codes and the Patient Record Forms were completed by trained surveyors instead of billing coders. On the other hand, the other studies required at least two claims in the outpatient setting to identify diabetes. This would lower sensitivity but increase specificity relative to one code. Therefore, the sensitivities estimated in the current study are likely related to the methods used in the NAMCS and NHAMCS, limiting these results generalizability.

As we hypothesized, the same alternative identification method using diagnosis codes and medication lists yielded a higher sensitivity in outpatient settings (72.3%) than in ED settings (20.5%) (18). This is likely because outpatient visits are more likely to address diabetes management than are visits to EDs, which are more likely to address injuries or acute illnesses. The likelihood of identifying patients with diabetes by diagnosis codes and medication lists differed according to demographic, medical, and survey characteristics. Compared to our previous analysis of ED visits, the results of this study were similar for some, but not for all characteristics (18). Both the NHAMCS-ED analysis and this analysis showed a differences in the likelihood of identifying patients with diabetes by payment source, number of diagnosis codes, and number of medications listed. We found that patients with private insurance or Medicare were less likely to have diabetes identified. In the previous study (18), we speculated that the higher likelihood of identifying diabetes through providers’ diagnosis codes and medication lists for those with self-pay or other sources of payment might be associated with visits to EDs to seek primary care, specifically diabetes care. However, the present study found these same trends for outpatient visits. Therefore, other explanations need to be considered. Based on our additional analyses (Tables 3 and 4) using only diagnosis codes and only medication lists to identify patients with diabetes, we found that differences in the likelihood of identifying diabetes was present across insurance types when diagnosis codes were used, but not when medication lists were used. It is possible that providers or surveyors are more aware of the diagnosis of diabetes in groups of patients with certain kinds of insurance coverage. It might also be possible that patients with certain kinds of insurance coverage make outpatient visits specifically for diabetes care, making it more likely for providers and surveyors to list a diagnosis of diabetes for these patients.

In this study, measuring glucose and hemoglobin A1c at outpatient visits was associated with a higher likelihood of identifying diabetes. This finding was similar to that of a previous analysis (18) which showed that measuring glucose or receiving IV fluids were associated with a higher likelihood of identifying diabetes. These results indicate that when providers specifically address a patient’s diabetes during the visit, the Patient Record Forms are more likely to list diabetes.

Our finding that age, sex, and race/ethnicity were associated with the likelihood of identifying patients with diabetes in the outpatient setting is consistent with our previous analysis in the ED setting (18). For example, in the current study, diabetes in patients 75 years of age or older were more likely to be missed compared with diabetes in younger patients, which is similar to the findings of our study in the ED setting (18). The lower sensitivity of identifying diabetes in elderly patients in NAMCS or NHAMCS may be related to the fact that the number of diagnosis codes is limited to a maximum of three. In elderly patients with multiple comorbidities, the diagnosis codes for diabetes may not be recorded. This speculation is supported by our finding that elderly patients are less likely to be identified only by diagnosis codes, but this trend was not clear using only medication lists.

We also found that women, compared to men, and non-Hispanic White patients, compared to other racial/ethnic groups, were less likely to have diabetes identified through the use of diagnosis codes and medication lists. In supplemental analyses using only diagnosis codes and only medication lists, both sex and race/ethnicity remained significant factors associated with the likelihood of identifying diabetes. These results suggest that female patients and non-Hispanic White patients diagnosed with diabetes not only have a lower likelihood of having diagnosis codes for diabetes listed, but also have a lower likelihood of reporting on anti-diabetic medication use or having anti-diabetic medications listed. Previously, lower intensity of medical treatment in diabetes was reported in non-Hispanic patients with diabetes (29; 30), but no differences in the intensity of medical treatment for diabetes has been reported between men and women (31). Because detailed clinical information is not available in the NAMCS and NHAMCS datasets, it is difficult to determine the likelihood or appropriateness of anti-diabetic medication use.

Diagnosis codes and medication lists are more likely to identify diabetes in office-outpatient visits than in hospital-outpatient visits; however, the reason for this is unclear. Future research regarding the difference between the two types of outpatient visits might be of interest.

Two limitations of this study should be mentioned. First, NAMCS and NHAMCS do not include identifiers for individual patients; therefore, some patients who made outpatient visits more than once may have had their visits counted independently, which would yield inaccurate estimates of the variance. However, the reporting periods of the NAMCS and NHAMCS are one week and four weeks, respectively, which are relatively short, decreasing the likelihood of repeated outpatient visits during the periods. Therefore, we believe that this limitation might affect our conclusion only to a small degree. Second, the NAMCS and NHAMCS included at most only three diagnosis codes and eight medications. A greater number of listed diagnosis codes and medications was associated with a higher likelihood of identifying diabetes. Thus, adding more diagnosis codes and medications to the NAMCS and NHAMCS Patient Record Forms might yield a higher sensitivity of identifying diabetes.

Based on the results of our study, we conclude that outpatient health care utilization and costs for people with diabetes have likely been underestimated. For example, the report of the American Diabetes Association on the economic burden of diabetes presented estimates of various costs associated with care for patients with diabetes. However, the report used an estimate of the number of outpatient visits made by patients with diabetes based on the primary diagnosis codes in the NAMCS and NHAMCS (32). Furthermore, a number of factors suggest that we will need to allocate more resources to outpatient health care for patients with diabetes in the near future. In the United States, the number of patients with diagnosed diabetes is predicted to increase by 165% from 2000 to 2050, mainly due to changes in demographic characteristics, population growth, and an increase in the prevalence of diabetes itself (33), and this does not account for the 19% of adults with undiagnosed diabetes (34). Only 43% of American patients diagnosed with diabetes receive three recommended preventive care services for diabetes (35). Approximately 511,000 diabetes-related hospitalizations, potentially preventable by appropriate outpatient care (36), occurred among U.S. adults in 2006 (37). Accounting for these factors, outpatient health care utilization and costs associated with diabetes management will likely increase. This study provides some of the information necessary to accurately estimate the outpatient care burden to the health care system from diabetes.

CONCLUSIONS

The alternative definition using providers’ diagnoses and medication lists in the NAMCS and NHAMCS-OPD are, at best, only 72% sensitive in identifying outpatient visits made by patients with diabetes. Accurate estimates of the numbers of outpatient visits made by patients with common chronic diseases are essential to develop strategies for allocating resources and to assess whether resources are being used appropriately. Further research is required to learn more about the nature of outpatient visits by patients with diabetes and to assess their future needs for outpatient care.