Assessment of Parkinson disease quality measures on 12-month patient outcomes

Abstract

Background

Quality measures (QMs) exist to operationalize guidelines by measuring adherence to guidelines through documentation, ultimately leading to improved patient outcomes. Studies are rare looking at the relationship between adherence to Parkinson disease (PD) QMs and patient outcomes.

Methods

We assessed adherence of our movement disorders specialists (MDSs) to the American Academy of Neurology's 2010 PD QM set through chart review using the measure set work group's criteria of documentation. We then evaluated patient outcomes to see whether there was a correlation with adherence to these QMs.

Results

Ninety-seven consecutive patients met the inclusion criteria. The mean disease duration was 9.3 (5.8) years. All patients were assessed by 1 of 4 MDSs. A total of 68% of QMs were documented across all patients. There was a small positive correlation between the number of documented QMs the year before the index visit and the number of calls/emails both the year before and after the index visit (r = 0.20, p = 0.04 and r = 0.26, p = 0.01, respectively.) There was a small negative correlation between the number of documented QMs and the number of PD follow-up visits the year after the index visit (r = −0.19, p = 0.05.) No other outcome showed a statistically significant correlation with the adherence to documented QMs.

Conclusions

We found no clinically important improvement in patient outcomes with higher adherence levels. It is important that QM developers validate QMs to ensure that they fulfill the intended goal of improved patient outcomes.

Quality measures (QMs) are high priority for the Centers for Medicare & Medicaid Services to link reimbursement to health care quality. Quality care is “effective, safe, efficient, patient centered, equitable, and timely.”¹ QMs are tools to quantify provision of quality care based on available evidence and are designed to reimburse physicians more objectively, while promoting comprehensive patient care. QMs are also designed to help physicians identify priorities during busy clinical visits and to fulfill requirements for high-quality care for Medicare's “pay for performance” programs. The American Academy of Neurology (AAN) was an early developer of QMs for neurologic conditions including Parkinson disease (PD).

The original QMs for PD, published in Neurology in 2010,² were developed by 28 panel members representing a wide range of medical disciplines, patient organizations, and insurance plan representatives, among others. Two hundred fifty-eight recommendation statements from 8 guidelines and 1 consensus paper were used to create 10 PD QMs (table 1). These QMs were updated in 2016.³

Table 1.

American Academy of Neurology 2010 PD QMs

Although it is generally assumed that implementing QMs will improve quality of care and patient outcomes, there is limited research investigating this premise. The aims of this study were to (1) assess the frequency of documented care aligning with PD QMs at an academic movement disorders center and (2) assess whether documentation of QM adherence was associated with changes in outcomes. The hypothesis was that better QM adherence and documentation over a year of Parkinson office visits would be associated with better patient outcomes.

Methods

Setting

This was a retrospective study of patients with PD managed at the University of Maryland Movement Disorders Center. Consecutive patient encounters were reviewed for PD patients using an index follow-up visit between January 1, 2013, and March 31, 2013. For eligible patients, the visit date occurring before but closest to March 31, 2013, was identified as the index visit. The index visit and all PD-related visits occurring 1 year before that visit were reviewed for documentation compliant with the PD QMs. Office visits for procedures, e.g., deep brain stimulation programming or botulinum toxin injections, were excluded. The 1-year time frame was chosen because many PD QMs require documentation at least annually. Per our PD clinic protocol for patient visits, patients receive the Montreal Cognitive Assessment annually, and neurologists complete the full Unified PD Rating Scale (UPDRS) annually, including questions on cognition, psychosis, mood, and medication-related motor complications. Similarly, patients complete brief intake forms at every visit and expanded intake forms annually including Short Form-12v2 Health Status Survey (SF-12),⁴ depression, anxiety, and somatization assessed with the Brief Symptom Inventory, BSI-18,⁵ and disability with the Older Americans Research and Services (OARS) disability scale.

Standard protocol approvals, registrations, and patient consents

Institutional review board approval from the University of Maryland (#HP-00058712) was obtained before initiation of the project.

Patients

Inclusion criteria were patients with a diagnosis of idiopathic PD (International Classification of Diseases-9 code 332.0) who were seen both a year prior and a year after the index visit (to enable assessment of QMs and evaluation of 12-month outcomes).

Data extraction and definitions

Data were extracted from a chart review using a standardized spreadsheet. QM adherence was evaluated using the exact wording and description in the Performance Measurement Set Appendix.² For the purposes of this study, a QM was considered documented if the physician either used a clinical assessment scale or specifically documented a discussion regarding the QM. For Parkinson Disease Measure (PDM) 3 regarding annual cognitive assessments (table 1), care was considered compliant if at least 1 note within the year-long assessment period included either a cognitive assessment scale (e.g., the Montreal Cognitive Assessment) or a discussion of cognition within the note. Some measures use “or” or “and” to describe criteria for adherence. For PDM2, 4, 7, 8, 9, and 10, charts were considered compliant if any of the symptoms/inquiries were assessed because these use “or” wording. For example, PDM2: Psychiatric Disorders or Disturbances Assessment; All patients with a diagnosis of PD who were assessed for psychiatric disorders or disturbances (e.g., psychosis, depression, anxiety disorder, apathy, or impulse control disorder) at least annually. For this measure, assessment of any of these psychiatric symptoms was sufficient to fulfill the measure. However, for PDM1, charts were only compliant if they described review of current medications and review for the presence or absence of atypical PD features. The physician's note had to specifically note that there were no medications that might cause parkinsonism. For documentation of atypical features, the chart was considered compliant if the presence/absence of at least 1 atypical feature was documented (e.g., falls at presentation and early in the disease course, poor response to levodopa, symmetry at onset, rapid progression [to Hoehn and Yahr stage 3 in 3 years], lack of tremor, or dysautonomia.) For PDM6, a review of falls was captured if the physician's note described the history of falls or if review of the falls questionnaire was described at every visit within the year up to the index visit. Other QM adherence criteria can be found in table 1.

Outcomes

Prespecified outcomes of interest included death, number of PD follow-up visits, ED visits (reported in visit notes or described in the electronic health record), phone calls/patient emails, referrals (e.g., social work and therapy), and nursing home placement over the year following the index visit. Outcomes also included changes in ratings of quality of life, depression, disability, and disease severity that were obtained by linking the retrospective chart review to the University of Maryland PD and Movement Disorders Center longitudinal Health Outcomes Measurement Study with the visit closest to 1 year after the index visit. Quality of life was assessed with the SF-12,⁴ depression, anxiety, and somatization assessed with the BSI-18,⁵ disability with the OARS, and disease severity with the total and motor scores of the UPDRS.^6,7

Statistical analysis

The total number of documented QMs (possible range 0–10) was correlated with each outcome using a univariate linear regression for continuous outcome variables and logistic regression for categorical outcome variables (e.g., death and nursing home placement). Changes in disability, quality of life, depression, anxiety, somatization, and UPDRS were correlated between the earliest visit within the year before the index visit and the index visit. This allowed for the expected change in these outcomes to be reflected a year after QMs were reviewed. Changes in number of hospitalizations, mortality, patient PD visits, calls/emails, and nursing home placement were correlated between the year prior and the year after the index visit. These outcomes were reviewed a year after the index date as they are better captured over a full year than any single time point. If selected outcomes data were missing for a patient, he/she was not included in the analysis for the missing outcome. Significance was set at a p < 0.05 level.

Data availability

Any data not published within the article are available by request from any qualified investigator.

Results

Patient characteristics

The initial search identified 97 patients with a diagnosis of PD in the reviewed period. Thirty-eight patients were excluded because they only had 1 visit within the reviewed period or had a diagnosis other than PD on chart review. Further data were not collected on these patients because they did not meet the inclusion criteria. Three quarters of the patients were male. The mean age was 68.4 years (SD 8.2), and the mean disease duration was 9.3 (5.8) years. All patients were assessed by 1 of 4 movement disorder specialists (average 21.5 years of experience, range 6–32 years).

Visit characteristics

Patients had an average of 2.6 (0.7) visits the year before the index visit and 2.2 (0.8) visits the year after. Five patients had all 10 QMs documented, but a total of 68% of QMs were documented across all patients. Review of therapeutic options (PDM10) was the only measure documented 100% of the time. PDMs 2 (psychiatric symptoms), 3 (cognitive symptoms), and 5 (sleep symptoms) achieved 90% compliance. PDM1 (diagnosis review) had the lowest compliance rate at 29%. Review of falls was documented at every patient visit 30% of the time (figure).

Figure. Frequency of documentation per Parkinson Disease Measure.

Outcomes

There was only 1 death. Four patients transitioned to a nursing home in the year after the index date. There was a small positive correlation between the number of documented QMs the year before the index visit and the number of calls/emails both the year before and after the index visit (r = 0.20, p = 0.04 and r = 0.26, p = 0.01, respectively.) There was a small negative correlation between the number of documented QMs and the number of PD follow-up visits the year after the index visit (r = −0.19, p = 0.05.) No other outcome showed a statistically significant correlation with the adherence to documented QMs ( table 2.)

Table 2.

Statistical analysis of QM adherence with outcome measures

Discussion

Adherence with QM documentation at our movement disorder center was 68% for all measures. Six of the 10 measures achieved at least 77% compliance, and 4 had at least 90% compliance. In a 2017 study looking at improving adherence to PDMs, with a much larger cohort of movement disorders specialists' and patient visits, overall compliance rates were slightly lower than ours.⁸ Two measures achieving only about 30% adherence were more difficult measures to fulfill: the falls measure required documentation at every visit, and the diagnosis review measure required multiple parts including documentation of medication review and presence of any atypical features. Low rates of falls documentation were also identified in an abstract assessing frequency of PD QM documentation.⁹ However, that study, along with the 2017 study,⁸ documented high rates of diagnosis review, likely due to differences in study methodology (i.e., what constituted “compliance”). Complex measures also resulted in lower adherence in a study of compliance with AAN epilepsy measures.¹⁰ The variability in compliance across measures within a set is consistent with previous studies using the AAN epilepsy QMs, where variability across measures was reported by neurologist self-reported via survey,¹¹ patient recollection,¹² and chart review.^13,14 Safety counseling seems to be difficult across measure topics. Documentation of safety counseling, PDM8, occurred in only half of charts reviewed in this study and is also one of the QMs documented to have low compliance in epilepsy studies.^10,12–14 This likely represents both counseling rates and lack of documentation.

When analyzing for association between outcomes and adherence to the measures individually or as a complete set, the only statistically significant correlations in our study were small positive correlations with the number of calls before and after the index visit and a small negative correlation with the number of PD visits after the index visit. There was no effect on patient outcomes such as disability, depression, anxiety, ER visits, nursing home placement, or death. Although there are no other studies that investigate associations between PD QMs and outcomes, research linking AAN epilepsy QMs to outcomes revealed varying results. In 1 study, mean overall quality scores were associated with seizure control even when controlling for subspecialty involvement.¹³ Of individual QMs, only documentation of etiology, seizure type, or epilepsy syndrome (2014 QM2) was associated with higher degrees of seizure control.¹³ A separate study found that only documentation of seizure frequency at each visit (2014 QM1a) correlated with a lower risk of epilepsy-related adverse hospitalizations overall. For individuals with intellectual disabilities, MRI/CT review, request, or order (2011 QM4) was associated with a reported decreased risk of epilepsy-related adverse hospitalizations.¹⁵ Annual safety counseling (2011 QM7, 2014 QM4) was associated with a decreased risk of worsened seizure control for the whole cohort.¹⁵

Protocols including previsit questionnaires may increase the likelihood of QM compliance. The most consistently documented QMs reflected routine clinical protocols at the University of Maryland Movement Disorders Center. Although these policies may affect the generalizability of these results to other centers, they highlight the utility of intake questionnaires using patient-reported outcome measures to assess symptoms and enhance quality of care. Falls, therapy needs, and safety considerations—issues not captured in standard intake forms—were less frequently addressed in clinic documentation. The use of protocols to improve documentation of compliance was also demonstrated for the epilepsy measure set, where use of a progress note worksheet including quality-oriented fields resulted in greater documented adherence to AAN epilepsy QMs.¹⁶ Clinician education including a divisional presentation, note template, and providing copies of the measure set also resulted in improved documentation of epilepsy QMs.¹⁷ Practitioner education, electronic checklists, and automated tracking systems are suggested mechanisms to improve QM compliance.¹⁴ More broadly, it has also been shown that with more of a comprehensive approach from multiple specialties with more time that compliance with these measures can be more fully met.⁸

Although QMs should “improve or maintain health outcomes, patient safety, quality of life, cost of care, the patient experience, or coordination of care”,¹⁸ little is known about their effect on patient safety and outcomes. There are previous studies suggesting flaws in strict adherence to QMs,¹⁹ but this is the first study to correlate PD QM adherence directly with outcomes. The only correlations we found related to increased calls/emails in both the year before and after the index visit and slightly less PD visit follow-ups the following year. Overall, there were no clinically meaningful or statistical effects on patient outcomes based on QM adherence by our movement disorder specialists, including mortality, hospitalizations/ED visits, quality of life, depression scores, nursing home placement, disability, or disease severity.

There are several limitations to our study that may also explain this lack of effects on patient outcomes. First, there was the question about operationalization of how to track adherence within documentation language. For this study, this QM was counted as fulfilled only if the physician explicitly documented a PD diagnosis, that there are no concerns of medications causing parkinsonism, and an explanation regarding the presence or absence of atypical features. Such a thorough description may be less common at follow-up visits even if the clinician assessed for potential changes in diagnosis. Different approaches to this methodology would lead to different results. The updated PD QM set provides more specific requirements for each measure including diagnosis, but this study used the original 2010 measure set. We were also limited by the outcomes routinely collected at our center and those that could be discerned by chart review, such as the number or severity of falls or admissions to health systems within the University of Maryland Medical System. We chose to examine outcomes 1 year after the index date, but the effect of adherence to QMs may be better assessed over longer periods, particularly for rare outcomes such as death. The sample size may have resulted in limited precision to detect an important difference, particularly for uncommon outcomes. Compliance for several of the QMs was high; it is possible that an association with outcomes would be found if there were a greater compliance range for QMs. Last, it is difficult to generalize our results across other practices and practice types. Epilepsy studies also suggest that adherence rates may be different between general neurologists and specialists.^12–14 Study strengths include use of rigorous definitions of compliance, systematic collection of outcome measure, and that studies looking at this relationship between PD QM documentation and patient outcomes are rare.

This study showed that adherence to PD QMs did not affect 12-month outcomes at an academic movement disorders center. Future studies can clarify this issue further by prospectively identifying and tracking key outcomes, evaluating compliance and outcomes at various levels of care (e.g., primary care, general neurology, and referral centers), and evaluating patients over longer time frames. Registries such as the Axon Registry sponsored by the AAN will streamline QM documentation and make it more efficient, especially with over 2 million patient visits already and growing by 40–50 practices each year. Studies with larger sample sizes will be better powered to address the effect of individual measures along with the measure set as a whole. This study reinforces the importance of regular reassessment for the validity of individual QMs, something the AAN does through triennial measurement updates. Large scale data acquisition, like with Axon, will enable better validation of current and future QMs. QMs that do not demonstrate an effect on meaningful clinical outcomes can be retired. Future studies building on this preliminary work will lead to measure sets that are also both practical and useful for practicing neurologists and their patients.

Appendix. Authors

Study funding

No targeted funding reported.

Disclosure

J.P. Martello receives compensation for speaking for Lundbeck, AbbVie, Teva, and Acorda and serving on advisory boards for Lundbeck and AbbVie. L.M. Shulman receives research support from the NIH (1R01AG059651-01 and 1R21AG058118-01), Claude D. Pepper Older Americans Independence Center (3P30AG028747-13S30), University of Maryland Center for Health-Related Informatics and Bioimaging, Pfizer Pharmaceuticals (investigator-initiated study), Eugenia and Michael Brin, and The Rosalyn Newman Foundation. She receives royalties from Johns Hopkins University Press and Oxford University Press. E. Barr reports no disclosures. A. Gruber-Baldini receives research support from the NIH (R R01 AG059651; R21AG054143) and Gilead Science, Inc. M.J. Armstrong receives compensation from the AAN for work as an evidence-based medicine methodology consultant and serves on the level of evidence editorial board for Neurology and related publications (uncompensated). She receives research support from Agency for Healthcare Research & Quality (K08HS24159), a 1Florida ADRC (AG047266) pilot grant, and as the local PI of a Lewy Body Dementia Association Research Center of Excellence. She receives royalties from the publication of the book Parkinson's Disease: Improving Patient Care, and she has received honoraria for presenting at the AAN annual meeting (2017) and participating in Medscape CME. Full disclosure form information provided by the authors is available with the full text of this article at Neurology.org/cp.

TAKE-HOME POINTS

• → There were no clinically meaningful effects to patient outcomes based on PD QM documentation/adherence.

• → Adherence overall to the PD QM set for our movement disorder specialists was 68%.

• → There is room for improvement in the development of QMs and testing their validity on patient outcomes before they are widely adopted.