Adherence to outpatient epilepsy quality indicators at a tertiary epilepsy center

Abstract

Introduction

Quality indicators for the treatment of people with epilepsy were published in 2010. This is the first report of adherence to all measures in routine care of people with epilepsy at a level 4 comprehensive epilepsy center in the US.

Methods

Two hundred patients with epilepsy were randomly selected from the clinics of our comprehensive epilepsy center, and all visits during 2011 were abstracted for documentation of adherence to the eight quality indicators. Alternative measures were constructed to evaluate failure of adherence. Detailed descriptions of all equations are provided.

Results

Objective measures (EEG, imaging) showed higher adherence than counseling measures (safety). Initial visits showed higher adherence. Variations in the interpretation of the quality measure result in different adherence values. Advanced practice providers and physicians had different adherence patterns. No patient-specific patterns of adherence were seen.

Discussion

This is the first report of adherence to all the epilepsy quality indicators for a sample of patients during routine care in a level 4 epilepsy center in the US. Overall adherence was similar to that previously reported on similar measures. Precise definitions of adherence equations are essential for accurate measurement. Complex measures result in lower adherence. Counseling measures showed low adherence, possibly highlighting a difference between practice and documentation. Adherence to the measures as written does not guarantee high quality care.

Conclusion

The current quality indicators have value in the process of improving quality of care. Future approaches may be refined to eliminate complex measures and incorporate features linked to outcomes.

1. Introduction

There are 70 million people with epilepsy (PWE) worldwide [1]; 1/3 of those who have consistent access to appropriate and adequate doses of medications have medically refractory seizures [2]. People with epilepsy have reduced educational, employment, and financial outcomes; lower quality of life; and increased morbidity and mortality [3–5]. The 2012 report by the Institute of Medicine, Epilepsy Across the Spectrum [6], focused on nonseizure factors impacting people with epilepsy and also highlighted issues of access and appropriate care. Recommendations included developing better methods for assessment of quality of care and earlier referral of cases with refractory seizures to higher levels of epilepsy specialty care. There is increasing financial pressure to improve efficiency of care.

Nearly 18% of the US GDP is now directed to health care — higher than any comparable nation [7]. It is estimated that $10 billion is devoted to the care of PWE in the US annually [8,9], much of which is indirect costs [10]. Costs of epilepsy due to loss of employment surpass those of diabetes, anxiety, depression, and asthma combined [11]. The majority of both direct and indirect costs are accrued by people with refractory seizures [12]. Taken together, these data suggest that great benefits can be gained by improving the efficiency of care overall [13] and, specifically, by achieving seizure freedom whenever possible.

The National Association of Epilepsy Centers (NAEC) has outlined levels of care for epilepsy, from emergency settings to tertiary epilepsy surgical centers [14]. Generalist providers without epilepsy board certification necessarily manage many PWE. Improving quality of care depends upon providing guidelines for specific aspects of care, including when to refer to more specialized care. To this end, a set of performance measures for epilepsy care has been published [15].

Although CMS is using some of these measures to guide reimbursement, information is lacking regarding the best methods for assessing adherence and the typical adherence at NAEC epilepsy centers. To our knowledge, there are no publications documenting adherence levels to all eight measures for all patients during routine care in the US. Pugh et al. compared preliminary measures in tertiary and primary care settings [16]. There is a single report of adherence to the published measures in a pediatric setting [17], and two studies have evaluated adherence in the context of systematic changes intended to improve adherence, one of which used all eight published measures [18,19].

This study assessed adherence to the eight published epilepsy outpatient quality measures at a NAEC level 4 epilepsy center at an academic hospital. Additionally, factors affecting the quantification of adherence, as might be used for reimbursement purposes, were investigated.

2. Methods

2.1. Data collection

Two hundred subjects with epilepsy (ICD-9 345.xx) seen by epilepsy specialists at the Medical University of South Carolina (MUSC) between 1/1/2011 and 12/31/2011 were randomly selected for chart abstraction of all visits within the study time period. Measures were operationalized according to the Epilepsy Physician Performance Measurement Set https://www.aan.com/uploadedFiles/Website_Library_Assets/Documents/3.Practice_Management/2.Quality_Improvement/1.Quality_Measures/1.All_Measures/epilepsy.pdf. All nascent criteria for exclusion were noted and applied. Additional variables were abstracted as described below. The abstraction tool was constructed a priori and updated for specific issues that arose to ensure consistency moving forward. A wide variety of documentation styles were present within the clinical charts. For more subjective measures, any feature related to the measure was considered adherent (e.g., “folic acid recommended” would successfully meet measure 8). While not guaranteeing comprehensive care, this approach best represented the spirit of the performance measures, as published.

Abstractors and the principal investigator met frequently to maximize the systematic and consistent approach according to the predefined variable definitions. Physician documentation of each visit for each subject was abstracted into a REDCap database [20]. Visits were classified as initial or follow-up and abstracted identically. Results are reported for all visits and for initial visits because no measures apply strictly to follow-up visits. Each provider used their own approach, with no shared templates. This study was approved by the Internal Review Board of MUSC.

2.2. Quality measure adherence calculation

Adherence was operationalized for each quality measure according to the Epilepsy Physician Performance Measurement Set [21] as described in Table 1. Unless otherwise specified, adherence calculations for all measures used a numerator of all visits meeting criteria and the denominator of all visits. Factors impacting the measure design for this study are described in this section.

Table 1.

Quality measures and alternative quality measures.

#	Abbreviated title	Numerator	Denominator
1	All type, all freq	All types with quantified frequency (e.g., focal sz, 2/month)	All visits
1a	All type, sum freq	All types with overall summed frequency	All visits
1b	Sz type	All types, regardless of frequency	All visits
1c	Sz type only	All types, but no mention of any frequency	All visits
1d	Sz type, some freq	All types, some with specific frequency quantified	All visits
1e	Sz freq	Frequency in any manner, regardless of type	All visits
2	Etio or syndr	Documentation of specific causative etiology (including NOS) or epilepsy syndrome	All visits
2a	Etio	Etiology, regardless of syndrome	All visits
2b	Syndr	Syndrome, regardless of etiology	All visits
3	EEG	Documentation of review, request to review, or ordering of EEG	All visits
4	Imaging	Documentation of review, request to review, or ordering of brain imaging	All visits
4a	MRI	Documentation of review, request to review, or ordering of brain MRI	All visits
4b	Imaging in all subs	Documentation of review, request to review, or ordering of brain MRI	All visits- retaining diagnoses for which imaging is NOT indicated (e.g., PGE)
4c	Imaging in focal epi	Documentation of review, request to review, or ordering of brain MRI	Only visits with diagnosis for which imaging is indicated (e.g., focal epilepsy)
5	ASD side effects	Documentation of discussion about ASD side effects	All visits
5a	Starting new ASD	Documentation of discussion about ASD side effects	Only visits that document prescription for a new ASD
6	Refr + REC	All patients with refractory seizures targeted for presentation at REC within 3 years	All patients with refractory seizures
6a	Ongoing sz + ref	Patients with ongoing seizures documented as refractory	All patients with seizure frequency >0
6b	Refr foc epi + REC	All patients with refractory seizures with focal epilepsy targeted for presentation at REC within 3 years	All patients with refractory seizures with focal epilepsy
6c	Ongoing focal sz + ref	Patients with ongoing focal seizures documented as refractory	All patients with focal epilepsy with seizure frequency >0
7	Safety discussed	Documentation of discussion of context-specific safety issues within one year	All patients
7a	Bone health if injured	Bone health discussion in patients with history of injury from seizures	Patients with documentation of seizure-related injury
7b	Peds issues if peds	Safety discussion pertinent to children for patients <19 years of age	Patients <19 years of age
7c	Driving if > 15	Driving discussion in visits for patients >15 years of age	Patients >15 years of age
8	Preg disc age 12–44	Documentation of discussion of women's health-related issues for any woman of childbearing age (12–44 years)	Women of childbearing age (12–44 years)
8a	Preg disc on ASDs	Documentation of discussion of women's health-related issues for any woman of childbearing age (12–44 years) on ASDs	Women of childbearing age (12–44 years) on ASDs
8b	Fol acid age 12–44	Recommended to take or taking folic acid	Women of childbearing age (12–44 years)
8c	ASD disc on bc	Discussion of effects of ASDs on birth control	Women of childbearing age (12–44 years) taking systemic hormonal birth control

AAN recommended quality measures and alternative measures evaluated to determine adherence and factors that impact measurement of adherence. Numbers without letters indicate strict AAN recommended measures (e.g., 1 and 2). “Strict adherence” was considered successful for a visit when all described details for that measure were documented (1 added to both numerator and denominator). Absence of any of the features expected for a measure was considered failure of strict adherence (0 added to numerator, 1 added to denominator). Visits documenting a specific exclusion criterion were excluded (0 in both the numerator and the denominator). Alternate rules were generated to identify the impact of scoring factors on adherence. Each measure was applied to initial and all visits. (ASD = antiseizure drug, NOS = not otherwise specified, PGE = primary generalized epilepsy, REC = refractory epilepsy (surgical) conference.)

2.2.1. Measure 1: seizure type and current seizure frequency

Wide variability of documentation is possible for both type and frequency. Frequency may be provided without any types specified. When types are described, it may not be clear whether all types are included. Frequency may be relative without quantification (e.g., “better than last visit”) or may combine seizure types in one value. Alternative measures addressed these issues.

2.2.2. Measure 2: documentation of etiology of epilepsy or epilepsy syndrome

Epileptologists have long debated etiology [22,23], and lack of uniformity impacts measurement of adherence. Beyond taxonomy, for some patients, etiology is never definite, and excluding these patients from measurement is not simple. Does “cryptogenic” imply that quality care has been provided in searching for an etiology (i.e., should they be counted as etiologies)? This study considered such etiologies adherent.

2.2.3. Measure 3: EEG results reviewed, requested, or test ordered

Adherence was measured for initial and all visits. Adherence for this measure was assessed at all visits in recognition of the fact that ordering a study meets adherence criteria but does not ensure that result is integrated into future care for that patient. Continued documentation of prior results suggests that data are being considered as the care plan progresses.

2.2.4. Measure 4: MRI/CT scan reviewed, requested, or scan ordered

Adherence was measured for initial and all visits. Denominators were as follows: (4b) visits for all patients, regardless of epilepsy syndrome, and (4c) visits for patients with a documented focal syndrome.

2.2.5. Measure 5: querying and counseling about antiepileptic drug side effects

The numerator and the denominator for 5a corresponded to the definition.

2.2.6. Measure 6: surgical therapy referral consideration for intractable epilepsy

Strict adherence requires referral of patients with refractory seizures to more specialized providers, which automatically results in full adherence because all study physicians are tertiary epileptologists. This measure was, thus, operationalized using more stringent criteria (see Table 1). The numerators and denominators of the alternative measurements corresponded to their definitions. This measure is evaluated on a per-patient basis once per three years. For all patients with refractory seizures with no 2011 refractory epilepsy (surgical) conference (REC) presentation, all visits up to 3 years prior to or after their 2011 visits were evaluated, with an REC presentation in that timeframe considered adherent.

2.2.7. Measure 7: counseling about epilepsy-specific safety issues

This measure is evaluated on a per-patient basis once per year. For all patients with no visits meeting this measure in 2011, visits were abstracted up to one year prior to and after the 2011 visits. Any such adherent visit rendered the subject adherent. Alternative equations included visits with documentation as follows: (7a) denominator: patients with any history of injury due to seizures and numerator: bone health education or management among that group; (7b) denominator: patients <19 years old and numerator: safety discussion pertinent to children in that group; and (7c) denominator: patients >15 years of age and numerator: discussion about driving safety in that group.

2.2.8. Measure 8: counseling for women of childbearing potential with epilepsy

The time frame of evaluation was identical to that of measure 7, with equations as follows: (8) denominator: all women 12–44 years of age; (8a) denominator: women 12–44 years of age taking AEDs; (8b) denominator: women 12–44 years of age on AEDs and numerator: recommendation to take folic acid in that group; and (8c) denominator: women 12–44 years of age using systemic hormonal contraception and numerator: discussion of effects of AEDs on birth control among that group.

2.3. Assessment of adherence

The proportion of adherence was calculated for both all visits and initial visits for measures 1–5. Measures 6–8 were calculated by subject. Ninety-five percent Clopper–Pearson confidence intervals were calculated for all proportions. All analyses were conducted in R v.15.3 [24].

2.3.1. Consistency of adherence

To evaluate the impact of provider on adherence, the percentage of adherent visits was calculated for all measures for providers with >12 visits [25]. Chi-square tests and Fisher's exact test were used to compare the proportion adherence between physicians and advanced practice providers (APPs) across all measures for all visits, using Bonferroni adjustment for multiple comparisons (α = 0.0017). To evaluate the impact of patient factors, adherence to each measure was categorized for each patient with >1 visit as occurring at ‘no visits’, ‘some but not all visits’, and ‘every visit’, and the proportions of each were determined across patients for each measure.

3. Results

3.1. Descriptive data

Two hundred subjects were included (44% males) with a total of 350 visits (44 initial visits). One hundred patients had >1 visit, of whom 62 had 2 visits, 28 had 3 visits, 8 had 4 visits, and 2 had 5 visits. Among subjects with >1 visit, 60.2% saw the same provider for each visit. There were ten providers (7 MDs and 3 APPs), averaging 33.3 visits per provider (median: 40, range: 1–87). The age distribution was as follows: 3% with age of 0–2, 31.5% with age of 3–18, 61.5% with age of 18–65, and 4% with age >65 years. Seizure frequency was recorded as 0 at 103 visits and not reported at all for 86 visits, suggesting that at least 161 visits represented patients with active seizures. Among visits with a seizure frequency recorded, seizures per month per patient averaged 13.5 (median: 1, SD: 64, range: 0–660) (see Table 2).

Table 2.

Characteristics of the study population.

	Mean ± SD (median)	Range
Gender (% male)	44.0%
Age at time of visit (years)	30.5 ± 19.5	0.24–83.5
Age at epilepsy onset (years)	18.1 ± 17.8 (11.6)	0–82.3
Duration of epilepsy (years)	9.1 ± 12.2 (3.8)	0–60.2
Seizure frequency (per month)	13.5 ± 64.4 (1.0)	0–660

3.2. Quality measure adherence

3.2.1. Measure 1: seizure type and current seizure frequency

Strict adherence occurred in 42% of visits and in 55% of initial visits (see Table 3). Detailed description of at least one seizure type plus some quantification of frequency occurred in 51% of visits and in 64% of initial visits. Detailed documentation of at least one seizure type occurred in nearly 64% of visits and in nearly 73% of initial visits. Documentation of frequency in any manner occurred in 84% of visits, including over 90% of initial visits. Quantification of frequencies for some but not all seizure types occurred in <7% of all visits and in <5% of initial visits.

Table 3.

Adherence to epilepsy quality measures.

#	All observations					First visit only
	Measure	# adherent (N)	# total (D)	Percent adherent	95% CI	# adherent (N)	# total (D)	Percent adherent	95% CI
	Measure 1 (visits)
1	All type, all freq	145	348	41.7	36.4–47.0	24	44	54.6	38.9–69.6
1a	All type, sum freq	176	348	50.6	45.2–56.0	28	44	63.6	47.8–77.6
1b	Sz type	222	348	63.8	58.5–68.9	32	44	72.7	57.2–85.0
1c	Sz type only	45	348	12.9	9.59–16.8	2	44	4.55	0.06–15.5
1d	Sz type, some freq	23	348	6.61	4.24–9.75	2	44	4.55	0.06–15.5
1e	Sz freq	292	348	83.9	79.6–87.6	40	44	90.9	78.3–97.5
	Measure 2 (visits)
2	Etio or syndr	185	350	52.9	47.5–58.2	29	44	65.9	50.1–79.5
2a	Etio	98	350	28.0	23.4–33.0	14	44	31.8	18.6–47.6
2b	Syndr	80	350	22.9	18.6–27.6	7	44	15.9	6.64–30.1
	Measure 3 (visits)
3	EEG	255	350	72.9	67.9–77.5	43	44	97.7	93.3–100.0
	Measure 4 (visits)
4	Imaging	153	319	48.0	42.4–53.6	29	38	76.3	59.8–88.6
4a	MRI	164	350	46.8	41.8–52.0	30	44	68.2	52.4–81.4
4b	Imaging in all subs	165	350	47.1	41.8–52.5	31	44	70.5	54.8–83.2
4c	Imaging in focal epi	88	169	52.1	44.9–59.2	17	22	77.3	59.7–89.3
	Measure 5 (visits)
5	ASD side effects	218	319	68.3	62.9–73.4	25	33	75.8	57.7–88.9
5a	Starting new ASD	17	44	38.6	24.4–54.5	3	11	27.3	6.02–61.0
	Measure 6 (subject)
6	Refr + REC	17	41	41.5	26.3–57.9	1	7	14.3	0.36–57.9
6a	Ongoing sz and ref	39	115	33.9	25.4–43.3	7	32	21.9	9.28–40.0
6b	Refr foc epi + REC	9	28	32.1	18.6–48.7	0	0	NA	NA
6c	Ongoing focal sz + ref	25	49	51.0	38.3–63.7	4	16	25.0	11.0–45.7
	Measure 7 (subject)
7	Safety discussed	78	199	39.2	32.4–46.4	19	44	43.2	28.4–59.0
7a	Bone heath if injured	4	8	50.0	15.7–84.3	1	1	100.0	NA
7b	Peds issues if peds	6	72	8.33	3.12–17.3	1	12	8.33	0.21–38.5
7c	Driving if >15	47	145	32.4	24.9–40.7	15	34	44.1	27.4–60.8
	Measure 8 (subject)
8	Preg disc age 12–44	21	70	30.0	19.6–42.1	0	17	0.00	0.00–17.6
8a	Preg disc on ASDs	21	68	30.9	20.2–43.2	0	14	0.00	0.00–21.4
8b	Fol acid age 12–44	28	68	41.2	29.4–53.8	2	17	11.8	1.46–36.4
8c	ASD disc on bc	2	13	15.4	1.92–45.5	0	17	0.00	0.00–17.6

(N) numerator; (D) denominator. Measure abbreviations defined in Table 1.

3.2.2. Measure 2: documentation of etiology of epilepsy or epilepsy syndrome

Strict adherence was noted in 53% of all visits and in 66% of initial visits. Etiology (excluding “not otherwise specified”) was listed in 28% of visits, including 32% of initial visits. A specific epilepsy syndrome was identified in 23% of visits and in 16% of initial visits.

3.2.3. Measure 3: EEG results reviewed, requested, or test ordered

Strict adherence occurred in 98% of initial visits and in 73% of all visits.

3.2.4. Measure 4: MRI/CT scan reviewed, requested, or scan ordered

Strict adherence occurred in 76% of initial visits and in 48% of all visits. Of documented imaging, 98% were MRI. Among all subjects, regardless of diagnosis, there was documentation of imaging in 71% of initial visits and in 47% of all visits. Among patients with focal epilepsy, 52% of all visits and 77% of initial visits were adherent.

3.2.5. Measure 5: querying and counseling about antiepileptic drug side effects

Strict adherence occurred in 68% of all visits and in 76% of initial visits. Patients being started on a new seizure medication were presented with possible side effects in 39% of such visits and in 27% of initial visits.

3.2.6. Measure 6: surgical therapy referral consideration for intractable epilepsy

Among patients documented to have refractory seizures, 42% had an REC note within 3 years of their 2011 clinic visits. Among patients with documented seizure frequency >0, 34% were noted to have refractory seizures. Among patients with focal epilepsy with ongoing seizures, 51%were labeled as having refractory seizures. Of the group with refractory seizures, 32% had an REC note within the time window.

3.2.7. Measure 7: counseling about epilepsy-specific safety issues

Strict adherence occurred in 39% of subjects. Of patients with seizure-related injuries, half had a documented discussion about bone health and epilepsy (note, n = 8). Among patients <19 years old, child-specific safety discussions were documented for 8%. In patients >15 years old, discussion of driving safety was documented for 32%.

3.2.8. Measure 8: counseling for women of childbearing potential with epilepsy

Among women 12–44 years of age, 30% had strict adherence, with identical adherence among those on AEDs. Of all women 12–44 years of age, 41% of visits documented folic acid. Thirteen women were documented as using hormonal birth control, of which 2 (15%) had a documented discussion regarding the interactions with AEDs.

3.2.9. Exclusions

Measure 1 had a subject with exclusion at 2 of 3 visits: “cannot provide history” and “no reliable historian available”. Measure 4 had 20 subjects (with 31 visits) excluded for having PGE and, thus, no indication for imaging. There were 28 subjects (with 31 visits) excluded from measure 5 because they were not on AEDs. Measure 8 had 3 exclusions (10 visits): “physically unable to bear children”. Valid exclusions were documented in 27 subjects for measure 6, 12 with focal epilepsy. Seventeen subjects had measure 6 exclusions documented at all visits and were excluded from analyses. Subjects with exclusions documented at some visits (10 of 18 subjects with >1 visit) were not excluded. Exclusions included “MD deems not surgical candidate” (17 subjects), “prior surgery” (8 subjects, 2 with VNS), and “patient not interested” (3 subjects). No documentation exclusions were seen for measures 2–5 or 7.

3.3. Consistency of adherence

3.3.1. Consistency across providers (Table 4)

Table 4.

Per-visit adherence to quality measures by specific provider.

#	Quality measure	Prov 1 (n = 87)	Prov 2 (n = 54)	Prov 5 (n = 36)	Prov 8 (n = 44)	Prov 9 (n = 50)	Prov 10 (n = 45)
	Measure 1 (visits)
1	All type, all freq	28.7	55.8	52.8	36.4	42.0	55.6
1a	All type, sum freq	37.9	64.2	61.1	47.7	48.0	68.9
1b	Sz type	49.4	69.8	63.9	65.9	64.0	84.4
1c	Sz type only	12.6	9.62	8.33	25.0	12.0	11.1
1d	Sz type, some freq	6.90	7.69	2.78	18.9	6.00	4.44
1e	Sz freq	77.0	92.3	88.9	77.3	92.0	91.1
	Measure 2 (visits)
2	Etio or syndr	34.5	55.6	50.0	54.5	30.0	71.1
2a	Etio	18.4	22.2	22.2	25.0	42.0	46.7
2b	Syndr	9.20	35.2	13.9	20.5	34.0	37.8
	Measure 3 (visits)
3	EEG	66.7	74.1	72.2	93.2	64	80
	Measure 4 (visits)
4	Imaging	45.0	48.1	71.4	65.0	37.8	47.4
4a	MRI	43.7	46.3	69.4	61.4	36.7	53.3
4b	Imaging in all subs	43.7	46.3	69.4	62.8	36.7	53.3
4c	Imaging in focal epi	45.5	44.7	63.2	70.0	46.2	71.4
	Measure 5 (visits)
5	ASD side effects	47.5	74.5	39.3	63.2	87.8	86.7
5a	Starting new ASD	10.0	61.5	0.00	33.3	37.5	60.0
	Measure 6 — cannot be evaluated per visit
	Measure 7 (visits)
7	Safety discussed	43.8	35.2	13.9	56.8	34.0	48.9
7a	Bone heath if injured	66.7	50.0	100	0.00	100	75.0
7b	Peds issues if peds	0.00	0.00	0.00	11.3	0.00	4.54
7c	Driving if N15	44.1	25.9	13.9	100.0	16.2	39.3
	Measure 8 (visits)
8	Preg disc age 12–44	30.6	9.52	0.00	0.00	14.3	46.2
8a	Preg disc on ASDs	31.4	9.52	0.00	0.00	14.3	48.0
8b	Fol acid age 12–44	63.9	28.6	5.26	100	50.0	96.2
8c	ASD disc on bc	14.3	16.7	0.00	0.00	25.0	0.00

All providers with >5 visits were included. Adherence numerators and denominators were identical to those of Tables 2 and 3. In this table, the numerator was the number of adherent visits, and the denominator was all visits for that provider. Measure 6 cannot be evaluated per visit.

Level of adherence to strict and alternative measures varied by provider. The lowest variability across providers was seen for measures 1 and 3, while measure 8 showed the most (data not shown). Advanced practice providers showed significantly higher adherence to measures 2, 2a, 5, and 8 (p < 0.001 for all) (see Supplemental Table 1).

3.3.2. Consistency within patients

Adherence in all visits for a given measure occurred in a minority of subjects. The measures with highest adherence across visits for a given patient were 1b, 1e, 3, 5, 5a, and 5b, all of which were always performed in over 50% of multivisit patients (see Table 5).

Table 5.

Adherence across visits.

#	Quality measure	Never	Sometimes	Always
	Measure 1 (visits)
1	All type, all freq	35.0	41.0	24.0
1a	All type, sum freq	30.0	33.0	37.0
1b	Sz type	21.0	26.0	53.0
1c	Sz type only	75.0	22.0	3.0
1d	Sz type, some freq	86.0	14.0	0.0
1e	Sz freq	6.0	23.0	71.0
	Measure 2 (visits)
2	Etio or syndr	34.0	23.0	43.0
2a	Etio	62.0	14.0	24.0
2b	Syndr	71.0	11.0	18.0
	Measure 3 (visits)
3	EEG	12.0	29.0	59.0
	Measure 4 (visits)
4	Imaging	40.6	27.5	31.9
4a	MRI	40.2	27.2	32.6
4b	Imaging in all subs	40.2	27.2	32.6
4c	Imaging in focal epi	40.5	21.6	27.8
	Measure 5 (visits)
5	ASD side effects	18.0	27.0	55.1
5a	Starting new ASD	0.0	0.0	100.0a

Measures 6–8were excluded because they are not rated on a per-visit basis. Proportion of times patients with >1 visit never met the adherence criteria, sometimes (but not always) met the adherence criteria, and always met the adherence criteria.

^aNote: measure 5a included only six subjects.

The proportion of multivisit patients who never had specific measures met was lower than expected based on overall adherence to certain measures. Documentation of seizure type and frequency occurred in 43% of visits but in at least one visit in 65% of multivisit patients. Documentation of an etiology or syndrome occurred in 53% of visits overall but in at least one visit in 65% of multivisit patients. A similar trend was shown for measures 3–5 (see Table 5).

4. Discussion

This is the first report, to our knowledge, of adherence to all eight AAN quality indicators for care of PWE in typical practice at a tertiary medical center in the US. The primary findings were [1] assessment of adherence is highly dependent on the definitions of the numerator and the denominator and [2] adherence at our center appeared similar to that of prior reports [9–12].

This project has many strengths. Data were obtained from routine clinic notes, without specific guidance about quality indicators given to the providers a priori. All eight measures were assessed using strict operationalization of the published descriptions. There was no intervention applied to improve adherence and no template specifically geared towards the quality indicators. For quality measures to be applied for assessment of performance, a baseline level must be established. This study is the first to fully describe typical adherence to all eight published measures for the full age range of PWE at a tertiary center in the US.

Firm and rigorous criteria were explained for adherence to each measure, and alternate measures were included to both provide accurate assessment of adherence and highlight factors causing failure. For example, requiring multiple features for measure 1 led to underestimation of the quality of care provided. Although 64% of visits described at least one seizure type without reference to additional undescribed types and 84% provided some quantification of frequency, only 42% of visits provided quantified frequency for all described seizure types. For some patients, the presence of any ongoing seizures is sufficient to guide short-term care, and additional time spent detailing exact frequencies may not improve their seizure outcome. For other patients, precise quantification of each seizure type is essential for best management (e.g., prioritizing treatment of rare atonic seizure vs. frequent absence seizures). As currently written, this quality indicator does not account for these factors and results in a lower level of measured adherence because of the high level of detail required.

The ongoing controversy in taxonomy of epilepsy and seizures complicates assessment of measure 1. A simple example is whether to count (a) a focal seizure with loss of awareness and (b) a secondarily generalized seizure as the same or different types. While mechanistically and etiologically similar, patients will almost uniformly report these as separate seizure types, and they have different impact on patient safety. However, it is likely that different providers and different patients report these differently, resulting in inconsistency in adherence criteria.

As has been described [9], objective measures such as EEG and imaging tend to have higher adherence values. Of note, however, was the drop in adherence from initial to follow-up visits. The inclusive nature of measures 3 and 4 allows successful adherence even if an actual result is never observed by the provider (e.g., multiple attempts to obtain a result or a patient failing to schedule their MRI). It is, thus, possible that initial attempts fail and are abandoned. More likely, however, is the application of a test result in the formation of an assessment and treatment plan that is then carried forward without documentation of the diagnostic pieces at each subsequent visit. Electronic medical record (EMR) systems may improve the availability of prior studies and, thus, lead to improved inclusion of these results in active management. However, this may not translate into higher adherence because having access to the record through an EMR system is not the same as documentation in the visit note nor does documentation in each note, through automatic updating, guarantee the information is actively included in the ongoing assessment and plan.

For many of the measures, the true quality of care provided is not specifically delineated in the adherence assessment. For example, a head CT is likely inadequate for diagnosis of the majority of epilepsy syndromes reliant on brain imaging yet satisfies criteria for measure 4. Having reviewed a prior normal EEG result satisfies measure 3 but does not bring the patient closer to a definitive epilepsy diagnosis. It is notable that imaging was documented in about 47% of visits, regardless of whether the patient was known to have a diagnosis that rendered imaging nonindicated. The measures were written to avoid discrimination against providers without access to high-level technology. However, these details do affect the quality of care (e.g., the likelihood of detecting a focal abnormality and proceeding to surgical referral) and, perhaps, should not be completely excluded from performance assessment.

Variable denominators complicate the assessment of adherence. Measures 3 and 4 apply only to initial visits. Measures 6–8 have a 1- or 3-year time frame. Lack of standardization makes the calculation more difficult for self-assessment and creates opportunity for errors.

Several opportunities for improvement were identified by this study. As with other groups, our providers infrequently documented counseling regarding safety and pregnancy-related issues. This may reflect lack of documentation rather than omission from the care process (i.e., folic acid recommendation, a more objective measure, showed higher adherence). Given higher adherence to these measures by APPs, a mixed provider program may result in better quality of care.

There appeared to be underassessment of patients with refractory seizures for surgical consideration, with only 41% of patients with refractory seizures being presented at surgical conference. Furthermore, only 34% of patients with ongoing seizures were documented as having refractory seizures, which may reflect underrecognition of medication-unresponsive seizures in patients. Recognition of medication nonresponsiveness was better in the group with focal epilepsy, but a lower percentage of these patients had an REC note. Adding template features to trigger awareness has been shown to improve adherence [11] and may be applicable to this issue.

Similar rates of adherence to each measure across different providers suggest that the issues leading to failure are more likely system-based and less likely based on individual tendencies. However, the low rate of multivisit patients who always had the measure met suggests considerable inconsistency across visits. This may reflect different priorities at different visits (e.g., breakthrough seizures at one visit and medication side effects at the next). Improved systematization of quality indicators has been shown to increase adherence [12] and may also improve consistency of practice.

Of note, the proportion of multivisit patients who never had measures met appeared lower than that of the overall sample, suggesting that visit frequency may impact adherence (e.g., seizure frequency may be more easily quantifiable with shorter intervisit interval). Patients who require multiple visits may be more prone to active seizures or side effects, leading to higher likelihood of documentation.

A major challenge to accurate assessment of adherence based on clinical documentation is the tendency to omit absent data rather than to state its absence. For example, noting the description of multiple seizure types does not guarantee that all seizure types were described unless the provider explicitly states that the patient denied additional types, which is rarely done. Furthermore, if a patient is able to quantify some types of seizures (e.g., tonic–clonic) but not able to quantify others (e.g., absence), a provider may simply leave the unquantifiable type unquantified rather than state that it was not accurately quantifiable. Furthermore, there is typical emphasis on the “objective” aspects of the encounter, such as seizures, medications and diagnostic tests, and sparse documentation of counseling, such as safety and pregnancy issues. These tendencies may lead to underestimation of adherence and will require a culture change before documentation will more accurately reflect the clinical encounter.

This study has many design strengths, which may reduce bias. The patients were randomly selected from the overall outpatient roster based solely on billing code and calendar year. The visit sample reflects the intended clinic flow in that patients are seen initially by a physician and, in follow-up, primarily by advanced practice providers. Bias towards inflated adherence during abstraction cannot be completely excluded but was minimized by training medical students with no connection to the epilepsy clinic for primary data abstraction. Explicit presentation of adherence equations makes this project highly replicable.

Future work will be necessary to standardize the equations for calculation of adherence. Improved consistency of documentation through EMRs, templates, and guidance from national organizations will be essential for steady improvement in quality of care. Ultimately, studies focused on the impact of quality indicator adherence on patient outcomes will also be required, especially in the era of pay-for-performance reimbursement.

5. Conclusion

This is the first study, to our knowledge, to document adherence to the full set of performance indicators for epilepsy care for patients during typical practice at a tertiary epilepsy center. The primary conclusions are that adherence scores show room for improvement and may be greatly impacted by variation in documentation and methods of calculation. Adherence to some measures is likely reduced by the complexity of the requirements for success (measures 1 and 2), by unspecified definitions for the denominator (measures 3–5), or by the tendency not to document certain aspects of the clinical interaction (measures 7–8). Despite these issues, these quality indicators are a valuable step towards improving the standard of care and have definite value for highlighting areas for improvement. Future work will lead to refined measures with broader application to the care of PWE.