Collecting Patient-Reported Outcomes: Lessons from the California Joint Replacement Registry

Abstract

Context:

While patient-reported outcomes (PROs) have long been used for research, recent technology advancements make it easier to collect patient feedback and use it for patient care. Despite the promise and appeal of PROs, substantial barriers to widespread adoption remain—including challenges in interpreting privacy regulations, educating patients and physicians about the power that PRO collection can provide to patient-centered care.

Case Description:

This article describes lessons learned from the California Joint Replacement Registry’s (CJRR) five-year effort to collect PROs from patients undergoing total hip and total knee replacement surgeries. CJRR is a voluntary, multi-institutional registry in California that collects clinical and device information, as well as PROs from patients undergoing total hip arthroplasty (THA) and total knee arthroplasty (TKA) surgeries.

Proposed Solutions:

The CJRR encountered and developed solutions to overcome several key issues: (1) limitations of electronic PRO collection, (2) challenges in patient recruitment and tracking, (3) challenges in encouraging patients to complete PRO surveys, (4) real and perceived administrative burden to clinic and hospital staff, (5) surgeon engagement, and (6) survey costs.

Conclusion:

The CJRR’s field experience can inform growing numbers of providers and researchers who seek to more fully understand the impact of care from the patient’s perspective. In addition, the authors believe that these challenges can best be addressed through a combination of policy changes and increased incentives.

Context

Patient-reported outcomes (PROs) are reports of the status of a patient’s health condition, health behavior, or experience with health care that come directly from the patient, without interpretation of the patient’s response by a clinician.1 PROs are typically captured through patient surveys, which can assess a variety of factors, including symptom burden (e.g., depression or pain) and functional status (e.g., ability to perform activities of daily living). PROs have long been used to track patient outcomes longitudinally, but are less commonly used to provide immediate patient feedback to be used for patient care. Electronic and social media have made it easier to collect patient feedback, a trend reflected in growing efforts to include PROs in clinical practice.

There is now a renewed focus on the provision and measurement of patient-centered care. Examples include the establishment of the Patient-Centered Outcomes Research Institute (PCORI), the development of Patient-Reported Outcomes Measurement Information System (PROMIS) measures, and the emphasis in the Affordable Care Act (ACA) on patient-centered care. Additionally, national efforts are underway to link and leverage the work of these initiatives, including the National Patient-Centered Outcomes Research Network (PCORnet), and the National Quality Forum and policymakers’ interest in the use of patient-reported outcomes measures (PROMs) for performance measurement.

Despite the promise and appeal of PROs, substantial barriers to widespread adoption remain—including challenges in interpreting privacy regulations, educating patients and physicians about the power that PRO collection can provide to patient-centered care, and resources to support collection and use of electronic data. These challenges can best be addressed through a combination of policy changes and increased incentives.

This article describes lessons learned from the first five years of the California Joint Replacement Registry (CJRR), a voluntary, multi-institutional registry in California that collects clinical and device information, as well as PROs from patients undergoing total hip arthroplasty (THA) and total knee arthroplasty (TKA) surgeries. The CJRR’s field experience can inform growing numbers of physicians, hospitals, and researchers who seek to implement PRO collection to more fully include the patient’s perspective in the delivery of care.

Case Description and Methods

Although the learnings reflected in this article are based on a case study in orthopedics, findings are generalizable to others interested in collecting PROs. Many of the solutions developed at CJRR are applicable to others, regardless of the specialty, condition, or procedure, because the work to engage patients and clinicians, and to collect and use data across inpatient and outpatient settings, is universal. In addition, as described in more detail below, the CJRR collects both general health function information and disease-specific (orthopedic) information; the use of general health, function, and pain information is not limited to orthopedics.

More than one million joint replacement procedures are performed annually in the United States, with the fastest growth in patients less than 65 years old. The prevalence, cost, and projected growth of joint replacements have commanded the attention of payers and policymakers and has made it a primary concern in the delivery of health services. Of considerable note, the primary purpose of these procedures is to reduce pain and to improve the function of patients, and yet for many years these procedures have been performed without systematically measuring and understanding these key patient outcomes.

The lack of widespread collection and use of PROs in orthopedics is not due to lack of PRO survey tools. On the contrary, there are several well-validated PRO survey instruments for orthopedics: e.g., Western Ontario and McMaster Universities Arthritis Index (WOMAC), UCLA Activity Score, and the Oxford Hip and Knee Score.2 While these survey tools are widely available, the burden of collection and analysis has outweighed their widespread adoption.

In the last five years, several registries that collect and analyze clinical and device information about THA and TKA procedures have been established. A handful of these now collect PROs in addition to clinical and device information. The collaborations across these and other registries, through groups such as the American Medical Association’s National Quality Registries Network, and the International Society of Arthroplasty Registries, indicate that many challenges described in this article are shared.

Planning for the CJRR began in 2009, and three hospitals piloted the software solution in 2011. CJRR was established by the California HealthCare Foundation (CHCF), an independent philanthropy, the Pacific Business Group on Health (PBGH), a not for-profit coalition of 50 large employers, and the California Orthopaedic Association—a membership association representing the interests of California orthopedic surgeon members. The goal of their collaboration was to develop, pilot, and expand a statewide joint replacement registry that captures PRO data from patients and makes this information publicly available to encourage improvements in quality of care and to inform decision-making by patients, providers, payers, and policymakers. The CJRR is the only domestic registry the authors are aware of for THA and TKA with a commitment to publicly report PROs as a primary outcome. The CJRR began to publicly report outcomes by hospital in 2015.3

CJRR collects comprehensive information about total hip (THA and TKA replacements performed in California, including the surgical approach, device, patient demographics and clinical characteristics and, importantly, direct feedback from patients about their pain levels and functional status.

The 40 hospitals currently participating in CJRR perform over 40 percent of the annual volume of the TKA and THA procedures in California. CJRR hospitals include a mix of community hospitals, hospitals belonging to larger systems, and academic medical centers. They are geographically diverse and accept a variety of insurance types, including MediCal, Medicare, and private insurance. The CJRR collects information on TKA and THA procedures regardless of payer type. The CJRR captures TKA and THA procedures for those surgeons within these hospitals who elect to participate in the CJRR.

The CJRR interfaces with participating hospitals, surgeons, and patients through the CJRR website. CJRR works with hospitals, surgeons, and their patients to set up automated extractions of electronic data on a periodic basis. For PRO collection, during preoperative surgeon office visits, patients are provided with information about the CJRR. For patients interested in participating, their email contact information is uploaded to the CJRR. The CJRR then sends patients links to survey questions at preidentified intervals, and also displays the status of the patient response on a dashboard that is visible to staff at participating surgeon and hospital offices. The CJRR uses a combination of disease-specific and general functional status PRO survey instruments2 to survey patients before surgery and at specified intervals (three months; one year, and semiannually thereafter) following the surgery.

Findings and Major Themes

Following its 2011 pilot, the CJRR began to expand, with the goal of capturing half of THA and TKA surgeries in California within five years. By 2015, there were 47 hospitals and affiliated surgeons participating in the voluntary registry, representing over 40 percent of the procedures performed. Major findings fall into the following categories, discussed below: (1) limitations of electronic PRO collection, (2) challenges in patient recruitment and tracking, (3) challenges in encouraging patients to complete PRO surveys, (4) real and perceived administrative burden to clinic and hospital staff, (5) need for strong surgeon engagement to collect PROs, and (6) survey costs and alignment of surveys.

Limitations of Electronic PRO Collection

The CJRR aspired to collect all data, including PROs, using electronic means in order to minimize patient and administrative burden and to reduce errors in the data. While CJRR is able to collect all clinical and implant data from electronic sources, the CJRR found that collecting all PRO data electronically was not fully feasible. This finding is consistent with the experience of similar initiatives as described in the literature.4,5,6

Hospital and CJRR staff found challenges in collecting accurate email addresses for patients (early results found 5 percent of email addresses supplied were erroneous), avoiding having CJRR’s messages caught in patients’ email SPAM filters (estimated in 2013 at up to 40 percent), and reminding patients to complete follow-up surveys. Several solutions were developed, the most impactful of which were the following:

• Reformatting the email reminders to come from the patient’s surgeon, rather than the CJRR;

• Creating a dashboard so that office staff can see if patients have completed their questionnaires and, if not, staff can re-send the questionnaire, provide a paper version to the patient, and call the patient to follow up;

• Giving patients an option to complete paper surveys in either English or Spanish; and

• Creating an outbound calling program, where a medical assistant (MA), employed by CJRR, places outbound calls to patients who have not completed follow-up surveys.

Challenges in Patient Recruitment and Tracking

Lack of unique patient identifier

A unique patient identifier is needed to aggregate information about each patient over time and across sites of care, especially in orthopedics where patients may have surgery at one location and, if the surgery does not go well, have it redone (known as a “revision”) elsewhere. Because there is not a national patient identifier, the CJRR created a unique identifier by using a hashing algorithm to transform patients’ social security numbers. At the time that the patient is asked to participate in the CJRR, they are asked for permission to use a hashed version of their social security number in order to develop a unique patient tracking number. Either CJRR or the participating hospital encrypts all social security numbers before any information is entered into the registry. The transformation is a one-way hash, and cannot be decrypted. This process was approved by the CJRR’s institutional review board (IRB).

Privacy concerns—patient and hospital

Even with the hashing process described above, CJRR encountered patient concerns about sharing their social security numbers. CJRR developed a campaign to equip hospital and office staff to proactively educate patients about the value of completing PROs. This included developing patient information sheets and brochures that can be stocked at physician offices, and the CJRR website includes a patient information section, including videos. In addition, the CJRR training process now includes coaching on how to engage patients in discussions about the registry, privacy concerns, and other issues.

Hospital privacy concerns about PROs were also a key barrier to the early growth of CJRR. CJRR was designed to comply with the Health Insurance Portability and Accountability Act (HIPAA) and other patient privacy regulations. A lack of clear interpretation of HIPAA and the Common Rule (an ethics rule for biomedical and behavioral research involving human subjects in the United States) about the definitions of research and human subjects research as they pertain to PROs and patient registries have led CJRR and other domestic registries to invest extensive time and resources to work with participating surgeons and hospitals and their attorneys to clarify protections in place and to develop agreements. AcademyHealth’s Electronic Data Methods Forum’s brief6 summarizes these issues, which have caused similar challenges to other registries.

CJRR also found that each hospital had its own approach to oversight of HIPAA and privacy regulations. Larger hospitals and hospital systems had their own IRBs, but in most cases, still required each participating hospital within the system to seek individual approval to participate in the CJRR. Smaller, independent hospitals tended not to have their own IRBs or resources to advise them. In response, CJRR developed a standard protocol and templates that can be used for hospital internal IRBs, and also established a relationship with a commercial IRB—the Western IRB—that has an approved protocol for CJRR. Twenty current CJRR participants have used Western IRB and joined under this common, multicenter protocol. The other hospitals continue to use their own IRB approval processes.

Challenges in Encouraging Patients to Complete PRO Surveys

Language barriers

Given the diverse population of California, CJRR’s participating hospitals have patients for whom English is a second language (information gathered in 2012 indicated up to 18 languages spoken). CJRR participants had between 2 and 14 percent of their patients who spoke Spanish as their primary language. To address this barrier, CJRR made available Spanish versions of the same PRO surveys for use by all CJRR participating hospitals.

Lack of email address or access to computer

As described above, some patients prefer to complete surveys on paper. For this reason, CJRR makes paper versions of the surveys available in English and Spanish. These paper versions are faxed to the CJRR, where they are optically scanned and uploaded. In addition, several participating sites have developed ways to offer electronic access to patients at the surgeon office or hospital. For example, several surgeons offer tablets to patients in waiting rooms and many hospitals offer computers to patients before, during, or after presurgical classes.

Reminders

In addition to the email reminders CJRR automatically sends to patients, prompting them to complete surveys, the CJRR has an outbound calling program. For patients who have not completed surveys, a medical assistant calls patients who have not completed surveys to remind them to do so and to offer assistance. The CJRR piloted this program in 2014. Survey completion rates were compared between targeted patients and nontargeted patients at the same sites (controls). Results of the pilot program showed that 51.3 percent of patients who had not completed surveys and who were reminded through phone calls to complete them ultimately completed their surveys, compared to 34.4 percent for patients in the control group.

Real and Perceived Administrative Burden to Clinic and Hospital Staff

The fact that CJRR hospital participants vary in size and structure translates into varied staff resources to encourage patients to participate in the registry and explain the importance of PROs. While academic medical center participants may have dedicated research staff in the clinic that routinely interact with patients, few community hospitals were staffed for such interactions. At a minimum, a few pieces of information, such as email address, must be collected from the patient at the time the patient is registered for surgery in order to enroll them in the registry. In addition, it is important for both the surgeon and the office staff to encourage the patient to complete the PRO survey. A 2014 internal survey of CJRR participants indicated that, depending on the number of patients at the participating CJRR hospital or surgeon’s office, it takes from 10 to 20 minutes of administrative staff time per patient to initially register and encourage patients to participate in the registry.

Here again, training and sharing of best practices has been critical to patient participation. The CJRR staff identified best practices and used them to develop model workflows that are shared with other hospitals. These workflows provide details about which staff member speaks to patients, when they have these conversations, how technology is used, what written materials are helpful, and how surgeons can be engaged to communicate to patients the importance of PRO survey completion.

Need for Strong Surgeon Engagement to Collect PROs

CJRR found that surgeons who talk to their patients about the importance of PRO survey completion have stronger patient participation. The CJRR has employed a multipronged approach to educate surgeons about the importance of PRO collection, what role PROs play in value-based payment and recognition programs, and how surgeons can easily and consistently communicate to patients the importance of completing surveys. CJRR holds regular surgeon webinars led by other surgeons, and surgeons receive quarterly, confidential reports, which display their PRO collection rate relative to other surgeons. Finally, some surgeons are using a “prescription pad” developed by CJRR. This suggestion came from one of CJRR’s pilot sites, where staff printed prescription-sized sheets, each of which has information about the CJRR, the importance of PROs, the Web address of the CJRR, and a signature line. During the patient visit, the surgeon discusses with the patient the importance of participating in the registry, signs a “prescription,” and gives it to the patient to take home, such as they would a prescription for medication or physical therapy. CJRR made these sheets available to all participating sites.

Survey Costs and Alignment of Surveys

During the CJRR pilot, its research committee recommended the use of the WOMAC, SF-12 and UCLA Activity score instruments. Two of the surveys initially selected, the SF-12 and the WOMAC, had licensing fees. The licensing costs for the small scale of the pilot period were funded by the initial grant funds; however, as CJRR grew, the licensing costs quickly became a budgetary burden. Therefore, in 2014, the CJRR switched to using surveys that are available in the public domain without a licensing fee, the Veterans RAND 12 Item Health Survey (VR-12), the WOMAC questions in the Hip dysfunction and Osteoarthritis Outcome Score (HOOS), and Knee dysfunction and Osteoarthritis Outcome Score (KOOS) and continued with the UCLA Activity Score.

Another issue in the selection of survey instruments was the CJRR’s desire to align with other orthopedic registries, both domestic and international. Anticipating that PRO measurement would become more widely accepted, CJRR wanted to select survey tools that would allow the development of combined data sets for safety and research. CJRR achieved this through direct outreach to other registries, and has advocated for a more formal process to align data definitions and elements across orthopedic registries.

CJRR’s Current Agenda for PROs

Performance reporting

CJRR is the first domestic orthopedic registry to publicly report PRO results by hospital, with its first reports in 2015. As discussed above, CJRR participants encompass a range of sizes and organizational models, with accompanying variation in their patient populations. While the survey instruments employed by the CJRR have been validated for specificity and reliability, and CJRR employs exclusion criteria that are aligned with those used by the Center for Medicare and Medicaid Services (CMS) for its value-based payment programs, it was important to have a robust risk adjustment methodology to adjust for these varying hospital models. At this time, the CJRR is reporting meaningful changes in the HOOS and KOOS WOMAC scores. WOMAC uses 24 questions to measure pain and function. CJRR also reports the percent of all patients who had surgery and completed a survey (response rate) for each participating hospital. These results are risk adjusted in order to control for the diseases, conditions, and other patient characteristics that could cause PRO data to vary due to circumstances outside of a provider’s control. The risk-adjustment model compares each hospital’s risk-adjusted PRO rate to all participating hospitals’ overall rate to identify whether each hospital is better, as expected, or worse than expected.

Making PRO results easily accessible during patient visits

Ideally, PROs would be collected before patient visits, integrated into the electronic health record (EHR), and made available to clinicians to use in discussions with patients during the visit. Due to technology issues (i.e., not able to do two-way data exchange with multiple EHR systems at CJRR’s 40 participating hospitals), the CJRR cannot yet offer this automated functionality. To date, we are aware of some surgeons whose staff access a CJRR data abstract—which includes individual patient results, print individual patient results, and attach them to the patient chart for surgeon’s review.

Discussion

In many settings, including orthopedic surgery, cancer treatment, and elective surgical procedures, the inclusion of PROMs is seen as a critical component of high value care. The Consumer-Purchaser Alliance’s recent issue brief is an example of this, and other examples are described below.7 However, as discussed above, those engaged in implementation face significant challenges: incorporating PROs into the workflow, overcoming privacy concerns, minimizing patient and office staff burden as well as costs associated with PRO data collection, and training for staff and surgeons to encourage patients to complete PROs. The root causes of many of these challenges can be addressed by a combination of policy changes and incentives.

Policy Changes

Clarify HIPAA and Common Rule

As described above, competing interpretations of HIPAA and the Common Rule regarding the privacy protections for longitudinal collection of PROs for registries and quality improvement are a barrier to adoption and scaling of PRO collection. Numerous representatives from the research community, as well as a coalition of specialty-based registries, have met with the Office for Human Research Protections (OHRP) and the Office of Civil Rights (OCR)over the past four years to request clarification and coordination between the two regulatory bodies. Encouragingly, during the writing of this article, on September 8, 2015, a Notice of Proposed Rulemaking (NPRM)8 was issued that has the potential to address some of these issues. The NPRM includes some exceptions that will exempt researchers, including registries, from complying with the Common Rule if they are complying with HIPAA and other applicable privacy rules. The NPRM does not require OHRP to issue more general guidance on the application of the Common Rule to clinical data registries, but the authors are hopeful that the Medicare Access and CHIP Reauthorization Act of 2015 (MACRA) legislation will result in this in the future.

Encourage solutions to the lack of a universal patient identifier

An additional challenge to PRO collection over time is posed by the lack of a universal patient identifier. There may be solutions to this issue other than use of social security numbers or other government sponsored identifier. Efforts are needed to coalesce researchers and clinicians around common solutions that will allow tracking over time of patients who receive care at different institutions.

Continue to invest in public domain PRO surveys and sponsor measure alignment

As described above, there are a growing number of parallel initiatives to collect PROs. There is a need for regular forums to share and align survey tools and research methods, and to map questions and data elements so that they can be consistent across studies and settings. For example, in orthopedics, the Food and Drug Administration (FDA) sponsored the International Consortium of Orthopaedic Registries (ICOR) project, which involved several leading international orthopedic registries in a yearlong project to map and align data elements for procedures and devices. A similar initiative, both within and across specialties, is needed for PRO tools. As mentioned earlier, NIH has funded the PROMIS measures, which has resulted in several promising generic and domain-specific tools that are available in the public domain. Validation of these tools in orthopedic conditions will help encourage their broader use. Ultimately, a valid and widely used tool in the public domain would ease the pathway for providers to incorporate PROs in their daily practice.

Incentives

Build stronger incentives

Payers are beginning to recognize PROs as important inputs to quality measurement. Currently, PRO measures are included in addition to other process and quality measures, adding to the burden of collection for providers, and not yet clearly rewarding the extra effort involved in PRO collection. Several initiatives are underway that, if strengthened, will encourage continued collection of PROs. Both private and public payers are interested in PRO and outcomes measures, and there have been initial moves toward incorporating these measures into value-based payment programs. For example, the PBGH Negotiating Alliance requires PRO collection for participation in its Center of Excellence (COE) travel surgery program. Similarly, the Wisconsin Business Coalition requires PRO collection for inclusion in its COE program. The Massachusetts Alternative Quality Contract also requires the collection of PROs for participation in its 2015 contract. Several registries have had initial discussions with insurance carriers about the feasibility of requiring a PRO survey prior to surgery, similar to how a lab test may be required as part of the preoperative clearance. Some health plans have also included PRO collection in their scoring for inclusion of hospitals and surgeons in “Center of Excellence” programs; however, no direct financial incentives for the collection of PROs are currently offered. In addition, work to develop performance measures—PROMs—from PROs is underway. In 2015, CMS began to require the collection of PROs for patients in medical homes who are undergoing cancer care. Similarly, CMS has commissioned the development of PRO-based performance measures (one that is hospital-based, and one that is physician-practice-based) for Medicare patients who undergo hip or knee replacement. Finally, the Office of the National Coordinator for Health Information’s Meaningful Use program has sought to reward practices that invest in EHRs. This has resulted in many EHR vendors including PRO capability in their software.

Conclusion

This article reports challenges and solutions related to collecting PROs developed during the first five years of the California Joint Replacement Registry. Planning began in 2009, and between 2011 and 2015, the CJRR collected clinical, device, and PRO information from approximately 13,000 patients undergoing THA or TKA in California. The experience of the CJRR is generalizable to others collecting any type of PROs in ambulatory or inpatient settings, since the changes necessary to culture, physician-patient interaction, workflow, and technology are generally applicable to anyone collecting PROs. We describe several of the obstacles encountered, as well as a variety of successful strategies to improve patient participation. We also suggest policy changes, and encourage the development of further incentives as critical steps toward promoting widespread collection of PROs.