CORR Insights®: What Factors Are Associated with Patient-reported Outcome Measure Questionnaire Completion for an Electronic Shoulder Arthroplasty Registry?

Where Are We Now?

The Health Information Technology for Economic and Clinical Health Act, part of the American Reinvestment and Recovery Act, requires eligible providers to demonstrate meaningful use of electronic health records, and has gone through various stages of implementation during the past decade [8]. The Merit-based Incentive Payment System, an important component of this program, calls for public health and clinical data registry reporting to support the goals of quality, safety, and reduction of health disparities [4]. The Qualified Clinical Data Registries list includes the Centers for Medicare & Medicaid Services-approved registries that are authorized to submit reports from eligible clinicians to the Merit-based Incentive Payment System program. Specialty societies, collaboratives, health informational technology vendors, and individual hospitals can create and maintain these registries [5]. The American Academy of Orthopaedic Surgeons Registry Program is one such example. More than 20 years ago, Berry et al. [3] described the difficulties of maintaining an institutional registry for hip arthroplasties at the Mayo Clinic, citing high costs, the need for dedicated personnel, and difficulties maintaining adequate follow-up.

The conversion of paper-based patient-reported outcome measurement (PROM) tools to electronic or online programs promised increased efficiency for clinicians and a decreased burden on patients [11, 14]. Reducing hurdles for patients is crucial because patient participation is required to ensure that clinical outcome tools yield valid and reliable outcome measurements. Despite widely available outcome tools on various electronic platforms in the modern era, it remains unclear whether this conversion has lived up to the hype. Further, the costs (in terms of dollars and administrative burdens) of the conversion and implementation of electronic PROMs have not been thoroughly described. Finally, we are just beginning to determine the return on our investment, measured in quality and reduction of disparities.

Ling et al. [10] conducted a retrospective study of an institutional shoulder arthroplasty registry after converting to an electronic platform and determined low follow-up proportions, a high resource burden in terms of the need for manual assistance, and most concerning, differences in the completion rate of questionnaires based on race, gender, level of education, and preoperative function. Their findings of selection bias in those completing PROMs is worrisome because conclusions derived from such a registry are not generalizable to the population at large, and therefore do not meet the goals of the conversion to electronic health records and required registry reporting. Equally troubling is that evidence-based registries inform national benchmarking, risk-adjustment, and value payment models, which may drive further issues of disparities in care [1, 9, 15].

Where Do We Need To Go?

To harness the full power of registries and electronic records for all stakeholders, we must ensure that PROM instruments are appropriate and consistently collected with minimal bias. Although the disparities reported by Ling et al. [10] may not be generalizable beyond their own institution, I believe they probably are; the difficulties in execution noted for a well-resourced, well-reputed, high-volume institution are likely to exist in most healthcare systems. To address low questionnaire completion rates in general, as well as potentially important differences in questionnaire completion in different sociodemographic groups, additional resources—in terms of money and personnel—are necessary.

Attempts to improve questionnaire completion, such as by incorporating multiple PROMs into one, have resulted in the need to validate the new, condensed tools [7, 13]. Computer adaptive tests can minimize the number of necessary questions, but additional technology might be required to administer these tools [2]. The Single Assessment Numerical Evaluation has demonstrated validity in measuring clinical outcomes after shoulder arthroplasty and rotator cuff repair, with moderate correlations to other longer PROMs such as the American Shoulder and Elbow Surgeons score [6, 12].

Identifying the minimal dataset to yield the data we need must be balanced against the burden of acquiring those data. Presently, this burden is disproportionately placed on clinicians and patients, as a prior study [3] and the current work [10] have suggested. Further work is needed to determine whether we can further reduce the questionnaire burden while maintaining quality validated, reliable, sensitive, and generalizable PROM instruments. We must now also work to remove disparities in questionnaire completion rates in order to have registries that yield useful clinical outcome information. The fruits of this labor can then be applied to our diverse population, and not simply to patients with the greatest means or those of a particular race.

How Do We Get There?

To determine whether a particular registry is achieving equal response rates across various demographic groups, ongoing analyses are needed to assess for selection bias. With routine monitoring of a registry, a dashboard could be developed to track particular groups of people who are not responding adequately, which could then trigger additional efforts to enhance completion rates. All registries and platforms should perform this type of assessment as a form of internal quality control, to ensure the instruments can be generalized to the general population or determine whether corrective actions are needed to improve response rates from underrepresented groups. Studies based on registries should also be required to publish these data so their conclusions can be interpreted in the appropriate population context.

Although the Centers for Medicare & Medicaid Services has introduced incentive programs for hospitals to enhance registry participation [4], perhaps direct patient incentives can further improve questionnaire completion. Because quality research is the primary purpose of registries, and payers also benefit from the results of registry-based research, rebates or copay reductions from payers could serve as direct financial incentives for patients, similar to paying research participants for completing various milestones. This approach, of course, may raise ethical concerns pertaining to excessive inducement, and the resultant potential of biased enrollment. Introducing a financial consideration or motivation to provide informed consent, which is critical for ethical research, may compromise a truly autonomous and independent decision. Large financial incentives may disproportionately affect socioeconomically underprivileged patients, potentially leading to biased enrollment and adversely affecting generalizability. Exploitation is also an ethical concern if a financially disadvantaged group bears a greater burden of the research without adequate compensation.

Beyond the national mandates that some countries use or other payer incentives, additional efficiencies are feasible for the individual surgeon or practice. For example, at my institution, we elected to use our electronic medical record to directly house PROMs instead of relying on a third-party system. Although there were some modest costs for building the PROM questionnaires, the added embedded functionality has enhanced our ability to collect outcome data routinely and efficiently. We have aligned this data collection with our routine clinical practice; at routine visits as well as at other timepoints, patients complete the American Shoulder and Elbow Surgeons and Single Assessment Numerical Evaluation PROMs along with questions typically asked about the patient’s history of present illness. Because the patient portal to our electronic medical record is popular with patients, and participation is encouraged for clinical purposes, patients are already familiar with the interface and can complete questionnaires from their home before their appointments, in the waiting room on a tablet, or in the clinic examination room directly with the help of a medical assistant or clinician. Email and text reminders are simple to send and again come from the electronic medical record portal that patients are accustomed to seeing. Questionnaire responses are directly integrated into clinical notes supporting typical required documentation, improving clinician efficiency as well. Additional research and cost analyses directly comparing this approach with a third-party approach will inform which method provides the greatest value.