Patient assessment of physician performance of epilepsy quality-of-care measures

Summary

To identify gaps in physician practice of epilepsy care, an online survey was sent to members of a Web-based epilepsy community to ascertain whether their physician performed 8 quality measures for epilepsy care. A total of 221 of 348 recently active epilepsy patients (64%) completed the survey. More than 80% of patients agreed they knew their seizure type, epilepsy syndrome, current seizure frequency, and had an EEG and neuroimaging. Fewer (60%) recalled being asked about medication side effects at each visit and safety issues annually. Only 48% report referral to an epilepsy surgery specialist and only 46% of appropriate patients had discussed reproductive issues with epilepsy. This demonstrates some potential gaps in epilepsy care and these data have been submitted to the American Academy of Neurology and the National Quality Forum.

The American Academy of Neurology (AAN) and the American Medical Association–convened Physicians Consortium for Performance Improvement have developed a set of physician performance measures with the intent of recording and improving the quality of care provided to patients with epilepsy.¹ Quality of care can be assessed through a variety of means,² but most methods require time to have elapsed following the deployment of performance measures. We sought to utilize an online epilepsy community in order to establish a patient-perceived baseline before the implementation of quality-of-care measures for epilepsy to assist with the AAN’s submission to the National Quality Forum. We sought to investigate whether epileptologists and neurologists were already adhering to more of these measures than nonspecialist physicians.

METHODS

The PatientsLikeMe survey system was used to construct a set of self-report questions to assess whether patients recalled that their physician performed each epilepsy quality-of-care measure. PatientsLikeMe is a previously described Web site for patients with similar medical conditions collected into communities, including a community for epilepsy patients which has been shown to have patient-perceived benefits including connecting with others, learning more about their seizures, and being more adherent to their antiseizure medication.³ Profile data (resembling a social network) are collected via self-report questionnaire on demographics, conditions and comorbidities, treatments, symptoms, epilepsy syndrome type (if known), seizure frequency, side effects, and quality of life. The survey system is designed to collect additional information from members of the communities for research and other purposes. All members of the site have explicitly consented to be contacted for research purposes. Comparison of PatientsLikeMe against a widely used US insurance claims database⁴ suggests the population is, on average, of a similar age (around 36 years old), but with a different distribution. Members of PatientsLikeMe tend to be adults aged 20–50 who developed seizures in childhood or adolescence, with relatively low representation of children or seniors. They are also more likely to be female (PatientsLikeMe 70% vs claims database 54%) and to be on polytherapy for their seizures (PatientsLikeMe 53% vs claims 29%).

The specific questions in the survey are listed in table 1 and parallel the 8 accepted epilepsy quality performance measures and were developed by the co-chair of the committee (N.F.).¹ Response options provided for the questionnaire were graded as “strongly agree,” “agree,” “disagree,” “strongly disagree,” or “does not apply.” The approved epilepsy measures themselves include explicit detailed reasons why the measures may not apply to a given patient. However, the exclusions were not provided in the survey for simplicity and because patients may be unable to reliably document these, so “does not apply” was provided for these situations.

Table 1.

Patient assessment of quality measure performance

On September 15, 2010, invitations to participate in the survey were sent to all 2,362 epilepsy patients who had been members of the site for at least 30 days, which included inactive members, on the basis that they might return to complete the survey. Members received a private message on their account, for which they would have received an e-mail notification, though for privacy’s sake this did not mention epilepsy or the nature of the study. This date predates publication of the epilepsy quality-of-care measures, so allowing for a useful pretest.¹ Eligible patients had 14 days to log in to PatientsLikeMe and respond, and a reminder message was sent within a week to those who had not yet completed the survey. Only patients who answered all questions were analyzed. Data analysis was performed using Statistical Package for the Social Sciences (SPSS v 18.0).

Measure 8, counseling for women of childbearing potential, was assessed only for women between 12 and 44 years, using demographic data available to us from each patient’s existing profile data. Measure 6 was only assessed in patients with an affirmative response to “I have had a seizure in the past 7 days” or “an ER visit in the past 12 months for epilepsy” as a proxy measure of intractability since this question is contained in their online profile and is used as a surrogate for intractable epilepsy.

We created a score to determine the level of performance of the measures. The score is composed of the total number of measures with a positive endorsement. Selection of “strongly agree” or “agree” constituted positive endorsement. In order to define a physician performance score that applied to most patients, we excluded measures 6 and 8 since they only apply to subgroups of patients. Because measure 1 includes 2 items (1a, seizure type identification, and 1b, seizure frequencies), it contributes 2 points to the score. Therefore, the maximum possible score is 7. Measure performance was assessed based on the type of physician the patient sees, dividing physician specialty into epileptologist, general neurologist, primary care physician (PCP), and other specialties (including psychiatrists, internists, and all others).

RESULTS

At the time of survey completion, 348 members (15% of those invited) had logged on to PatientsLikeMe’s epilepsy community and so would have the opportunity to read the invitation. Of these, all survey questions were completed by 221 patients (64% of users logging in during this period), constituting the analyzed group, after excluding 21 patients who opted out of the survey and 61 who did not answer all questions. While 2,362 members were invited in total, this included users who had only logged in once and never returned to the site, therefore would not be representative of active site users.

Measure 6 (epilepsy surgery referral) was assessed in 140 patients with “intractable epilepsy” as defined above and measure 8 (women’s issues) was assessed in 61 women aged 12–44 years; these patients also had all other measures assessed.

To test for bias among the users who responded, we used the profile data of all other members of the community to compare against the responders. Mean age of respondents was 40 years (SD 12 years, range 17–72), which was 4 years older than nonresponders (F_3,1871 = 8.259, p < 0.001). Following on from this difference, respondents had experienced seizures for a longer period of time (mean time since first seizure: 23 years [SD 15]) than nonrespondents (19 years [SD 14], F_3,1806 = 5.757, p < 0.001). Women constituted 67% of respondents but this was not a statistically significant difference from nonresponders (χ² = 6.605, p = 0.086). Most respondents (84%) were Caucasian but we do not have data on race for nonresponders.

Among respondents, 49% self-reported that they had a localization-related epilepsy syndrome, 34% a generalized syndrome, and 16% unknown or miscellaneous. Among specific epilepsy syndromes, 32% of respondents had temporal lobe epilepsy, 15% an unspecified syndrome, 10% epilepsy with grand mal seizures on awakening, 8% other partial epilepsy syndrome, and 6% juvenile myoclonic epilepsy. Most patients (70%) were seen by a neurologist, 20% by an epileptologist, 7% by PCPs, and 3% by other specialties, e.g. internist, psychiatrist, or other. Profile data for nonrespondents were not available for race, epilepsy syndrome, or clinician specialist.

Table 2 shows the degree to which patients report their physician performed each measure, without regard to type of physician. More than 80% of patients “strongly agree” or “agree” that they know their seizure type, have been asked their current seizure frequency, know their epilepsy syndrome, and have had an EEG and neuroimaging. At least 60% recall being asked about medication side effects at each visit and safety issues annually. Only 48% of qualifying patients have been referred to an epilepsy surgery specialist and 46% of appropriate patients have discussed reproductive issues among women with epilepsy.

Table 2.

Epilepsy quality measure performance by type of physician provider

Table 2 shows differences of measure performance across physician specialties. The measure was considered performed if the patient scored it as “strongly agree” or “agree.” Epileptologist and neurologists consistently performed each measure more often. The group differences were significant only for seizure frequency, side effect assessment, epilepsy specialist referral, and discussion of safety issues. Difference in performance was also statistically significant for neuroimaging performance but the absolute difference was small.

The number of measures performed plotted by physician specialty are shown in the figure (F_3,217 = 10.551, p < 0.001). The mean number of measures performed for epileptologists was 6.3 (95% confidence interval [CI] 5.95–6.55), general neurologists 6.1 (95% CI 5.86–6.25), PCPs 4.9 (95% CI 3.96–5.90), and other specialists 4.1 (95% CI 2.91–5.34). Epileptologists performed more measures than neurologists but the difference is not significant. Post hoc Bonferroni testing reveals epileptologists adhered to a mean of 1.3 more measures than PCPs (p = 0.003, 95% CI 0.34–2.30) and 2.1 more measures than other specialties (p < 0.001, 95% CI 0.90–3.34).

Figure. Epilepsy quality measures by physician specialty.

Box plots of total number of measures performed (excluding contraception item) broken down by specialty of treating physician. Black line represents the median, box is the interquartile range (IQR), whiskers are 1.5× IQR, and circles are outliers (>1.5 × IQR). GP/PCP = general practitioner/primary care physician.

Patients seen in a medical center specialty clinic reported 0.7 more measures adhered to (mean = 6.3, SD 1.0) than those seen in a private office (mean = 5.6, SD 1.5, p < 0.001, 95% CI 0.24–1.20). We found no significant association between number of measures adhered to and sex, age, education, US census region, or self-reported epilepsy syndrome type (generalized vs partial vs unknown).

DISCUSSION

Overall, the survey demonstrates some gaps between recommended care and the care delivered to epilepsy patients. As the currently approved epilepsy quality-of-care measures were purposely designed to be minimum quality standards that can be universally implemented, it is not surprising that many are already adhered to and probably do not need further emphasis if the current results are borne out. Fortunately, the gaps in care identified should be easily remedied through physician and patient education. Specific educational information should be disseminated with regard to asking patients about medication side effects, reviewing safety issues, and counseling women of childbearing potential.

One of the most important identified gaps is in referral of refractory epilepsy patients to an epileptologist. Epilepsy surgery is the most effective treatment for intractable focal seizures in appropriately selected patients. A randomized controlled trial of epilepsy surgery vs standard medical therapy demonstrated that 58% of patients are seizure-free after surgery compared to only 8% on medical therapy alone.⁵ Our findings are consistent with the common report that patients who eventually do undergo surgery have usually had epilepsy for a mean of almost 20 years, presumably because they were not referred for epilepsy surgery in a timely manner.⁵ Clearly there are great gains to be made in terms of survival, quality of life, and seizure-free years even for patients who undergo successful surgery or are not surgery candidates.⁶ Even when patients are not good candidates for surgery, they are still likely to benefit from evaluation by an epilepsy specialist to confirm the diagnosis, maximize medications, consider alternative therapies, and provide education and access to social and other services.⁷

Epileptologists adhere to more of the quality-of-care measures than physicians from other specialties. However, only 20% of patients in our sample were seen by an epileptologist regularly; most patients in our sample (70%) were seen by a general neurologist. Areas in need of further attention by general neurologists include asking about side effects of medications at each visit; appropriate referral for nonpharmacologic treatments including surgery and vagus nerve stimulation; regular discussion of safety issues such as driving; and discussing issues for women of childbearing age. Among epileptologists there is room for improvement with regard to inquiring about medication side effects, safety issues, and counseling women with epilepsy.

Quality measures are often based on claims-based recording systems and emphasize documentation rather than care. This study asked patients whether they recall their physician performing the measure-specified activities, without regard to documentation. Of course, this could be subject to significant recall bias since it is possible that patients simply forget that their physician performed a specific measure. Therefore, the current study represents a potential “real world evidence” assessment of the care that patients perceive their physicians are delivering in practice, with some of the limitations that confers.

Items such as knowing the type of seizures and syndromes that the patient has may be subject to multiple influences including patient education, cognitive function, memory problems, and physician input. We included this item because although the quality measures themselves emphasize whether a seizure or syndrome has been identified in the medical record, it may be worth future studies to investigate whether this leads to improved patient knowledge of their condition and what impact this might have on outcomes.

It is challenging to assess via self-report the validity of the epilepsy surgery item. Our definition of “intractable epilepsy” was (at best) a proxy measure, and without more precise knowledge of the patient’s subtype and seizure history it is difficult to say what sort of conversations they should have been having with their clinician. However, we do know from other studies⁵ that patients experience long latency to surgery, so this might be explored in future studies. The high proportion of women reporting the contraception discussion was “not applicable” might also provide further area for future study.

This study has several limitations based on the study design. Given the self-reported nature of the system, it is impossible to verify that “patients” are indeed people diagnosed with epilepsy due to the anonymous nature of the participants. It may be that some of our respondents did not have seizures, or had primarily psychogenic nonepileptic seizures. However, given that there are no incentives for participating in the system, it seems unlikely that many respondents would have taken the time to register for the site and complete a survey if they did not at least believe they have epilepsy. Future work is under way collaborating with epilepsy specialist centers to verify and validate patient-reported outcomes to overcome this limitation and will be of use in follow-up studies. In terms of physician types, it is possible that patients cannot reliably distinguish between a neurologist and epileptologist, but future work using more traditional means might be able to explore this further. Such studies might also explore a larger sample of patients treated mainly in primary care, given that most of our sample (90%) saw an epileptologist or neurologist; we would hypothesize the quality situation will be poorer in the primary care setting.

Because patients must actively seek out the site and be capable of Internet use, it is likely that patients who are members of a Web forum of the type used in this study are relatively sophisticated and so may be more knowledgeable than patients in the wider community about epilepsy. This sophistication might provide more reliable answers to survey questions but also could bias the results toward a population that is more sophisticated than the average patient and thus be an underestimate of the real magnitude of gaps in care. Analysis of responders showed a clear bias toward responses from older patients.

Despite the limitations of this study, it clearly demonstrates some gaps in care, and our data have been submitted by the AAN to the National Quality Forum. We hope that by illuminating these that physicians will implement the quality measures to improve care for patients with epilepsy, especially consideration of referral to an epilepsy specialist for consideration of epilepsy surgery. It also can inform future changes in the approved epilepsy quality performance measures.

DISCLOSURES

P. Wicks is a paid employee of PatientsLikeMe and owns stock options in the company. The Patients-LikeMe R&D team has received funding from Abbott, Acorda, Avanir, Biogen, Genzyme, Johnson & Johnson, Merck, Novartis, Sanofi, and UCB. N. Fountain receives research support from NIH, UCB, Sepracor, Medtronic, and Neuropace, consulting fees from UCB, and serves on the National Association of Epilepsy Centers Board of Directors. Go to Neurology.org/cp for full disclosures.