Patient Report Improves Posthospital Discharge Event Capture in Total Joint Replacement: A Novel Approach to Capturing All Posthospital Event Data

Abstract

Introduction:

Current approaches to quantifying total posthospital complications and readmissions following surgical procedures are limited because the United States does not have a single health care payer. Patients seek posthospital care in varied locations, yet hospitals can only quantify those returning to the same facility. Seeking information directly from patients about health care utilization following hospital discharge holds promise to provide data that is missing for surgeons and health care systems.

Background:

Because total joint replacement (TJR) is the most common and costly elective surgical hospitalization, we examined the concordance between patients’ self-report of potential short-term complications and their readmissions and our review of medical records in the initial hospital and surrounding facilities.

Methods:

Patients undergoing primary total hip or knee replacement from July 1, 2011, through December 3, 2012, at a large site participating in a national cohort of TJR patients were identified. Patients completed a six-month postoperative survey regarding emergency department (ED), day surgery (DS), or inpatient care for possible medical or mechanical post-TJR complications. We reviewed inpatient and outpatient medical records from all regional facilities and examined the sensitivity, specificity, and positive- and negative predictive values for patient self-report and medical records.

Findings:

There were 413 patients who had 431 surgeries and completed the six-month questionnaire. Patients reported 40 medical encounters (9 percent) including ED, DS or inpatient care, of which 20 percent occurred at hospitals different from the initial surgery. Review of medical records revealed 9 additional medical encounters that patients had not mentioned including five hospitalizations following surgery and four ED visits. Overall patient self-report of ED, DS, and inpatient care for possible complications was both sensitive (82 percent) and specific (100 percent). The positive predictive value was 100 percent and negative predictive value 98 percent.

Discussion:

Patient self-report of posthospital events was accurate. Substantial numbers of patients required care at outlying hospitals (not where the TJR occurred).

Conclusion:

Methods that directly engage patients can augment current posthospital utilization surveillance to assure complete data.

Introduction

Evaluating complications following medical and surgical procedures is challenging due to fragmentation of care. Patients who experience complications may seek care from new providers and different hospitals rather than return to the original health care team and hospital. On a local level, physicians and hospitals are usually unaware of patients who seek care from others. Also, there is no single national data source to identify all patients and track their outcomes over time. Analysis of Centers for Medicare and Medicaid Services (CMS) data, which does have total capture of health care services, is limited to Medicare eligible patients—thus excluding those under 65 years of age. Due to this lack of comprehensive 30- and 90-day posthospital data for all patients, health facilities have limited ability to assess their patients’ care and, thus, are constrained in their quest to implement quality improvement activities.

Total joint replacements (TJR), specifically total knee (TKR) and total hip (THR) replacement, are the most common inpatient procedures, with over 1 million surgeries performed annually in the United States.1 While most patients have marked improvement in pain reduction and restoration of function following surgery, short-term complications occur in 7–8 percent of patients.2 Given the sheer volume of surgeries in the United States, reducing the occurrence of postsurgical complications is an important public health priority. In particular, hospital readmissions following elective procedures are the focus of efforts to control health care costs and improve quality of care.

Unfortunately, achieving timely and accurate reporting of these adverse events has proven difficult—whether the insurer, hospital, or institution does the surveillance (see Figure 1).3–12 CMS is currently reporting hospitals where knee- and hip-surgery patients have high readmission and complication rates, but orthopedic surgeons and hospitals are concerned as the Medicare data used for those analyses covered surgeries that occurred up to three and a half years before the report was released. Medicare databases also exclude approximately 40 percent of patients under the age of 65 who undergo TJR.13 In addition, analysis relying on electronic health records, and institutional databases at the hospital performing the surgeries, may not capture the full spectrum of the care patients receive if patients seek care from other facilities. The impact of this care pattern will vary based on the hospital type. However, this is clearly a concern for specialty hospitals or high-volume hospitals, where patients often have to travel great distances and thus might seek local care for urgent postsurgical concerns.14–16

Figure 1.

Data That Is Captured (and Not Captured) by Insurer and Institutional Surveillance Approaches

Note: ^*Members denote those patients who are members of the specific insurer.

Prior work has shown that patients are able to accurately report their joint replacement surgery as well as the indication for their surgery.17,18 In addition, they are able to accurately report some postoperative complications in terms of positive predictive value.19 However, there are several limitations to those studies. For instance, none have looked at the sensitivity and specificity of patient self-report, specifically, none of the studies examined medical records (electronic or paper) to verify the complications reported or explored false negatives responses (e.g., how many patients who do not report a complication actually had an event).19

We proposed (1) to quantify the amount of “missing post-hospital event data” due to patients seeking post-THR or TKR care from new providers and hospitals; and (2) to evaluate patients’ self-report of potential complications, as a novel approach to addressing missing data. More specifically, we examined the medical records (electronic or paper) of patients who did and did not report a complication, which allowed us to assess the sensitivity and specificity of patient self-report and the correlation with medical record documentation. In addition, we examined the patients’ location of care to estimate the proportion of events at outlying hospitals, thus quantifying “missing data” for health care systems. These data will inform the potential need for patient self-report of complications and the additional information it provides following TJR to fill this gap.

Materials and Methods

This is based on a large national cohort of THR and TKR patients designed to evaluate comparative effectiveness of TJR, funded by the Agency of Healthcare Research and Quality (Function and Outcomes Research in Comparative Effectiveness Registry: FORCE-TJR, P50 HS018910-01).13 The FORCE-TJR cohort is a nationally representative group of patients undergoing TKR and THR surgeries, with 4–7 percent being African Americans, which is consistent with other national cohorts reflecting the disparity of TJR use in this patient population. With respect to ethnicity, 2.3 percent characterized themselves as Hispanic or Latino, of whom one-third reported their first language to be Spanish. The FORCE-TJR study has two Spanish native language recruiters and coordinators, and all study materials have been translated into Spanish. The FORCE-TJR study is approved by the Institutional Review Boards at the University of Massachusetts Medical Study and at the participating sites. At the time of enrollment, patients provide consent for retrieval of medical records, allowing the assessment of potential complications.

We examined the location of reported care (hospital where the TJR occurred versus other hospital) for the entire FORCE-TJR cohort. However, we used a subcohort to assess agreement between self-reported complications in the patient questionnaire and data from medical records for the first 18 months of FORCE-TJR enrollment, and to evaluate the proportion of care that occurred at outlying hospitals and was missing in institutional data records. These analyses were based on data from one of the core sites within the FORCE-TJR network that enrolled patients between July 1, 2011, and December 31, 2012. This approach was chosen because we needed to ascertain all medical records from a geographic region (from the TJR hospital and the surrounding hospitals).

We evaluated possible events that were reported on the patient six-month follow-up questionnaires. Patients were asked to report all hospitalizations (“Since your discharge from the hospital after your knee/hip surgery, have you been hospitalized?”), emergency department (ED) visits (“Since your discharge from the hospital after your hip/knee surgery, have you had to seek medical care at an Emergency Room?”), returns to the operating room (“Since your discharge from the hospital after your hip/knee surgery, have you had any day surgery related to your hip/knee surgery?”), or nonelective outpatient visits for concerns about their surgical knee or hip (“Since your discharge from the hospital after your hip/knee surgery, have you had a not regularly scheduled follow-up visit at your primary care provider, orthopedic surgeon or urgent care center for problems related to your hip/knee surgery?”). We encouraged patients to be inclusive when completing the form. We designed the questionnaires to capture the wide range of complications possibly related to the surgery as outlined by the National Quality Forum (NQF), specialty societies, and CMS, which include medical complications, mechanical complications,20 and hospitalizations for any reason within 30 days of discharge from the surgical procedure. Specifically, inclusion and exclusion criteria were based on complication definitions used by the NQF, the Knee Society,19 and CMS, and are presented in Table 1. We excluded patient-reported events that occurred prior to joint surgery (n=1). We excluded events not considered complications, including elective surgery (n=11) involving a different joint, unrelated elective procedures such as colonoscopies (n=17), management of medical comorbidities not related to surgery such as cancer treatment (n=26). In addition, eight patients had more than one visit associated with the same diagnosis within 30 days of each other. An example is that of a patient presenting to the emergency room (ER) with abdominal pain and diarrhea and, a few days later, being hospitalized for abdominal pain and diarrhea; this was considered to be one event in the analyses.

Table 1.

Listing of Inclusion and Exclusion Criteria for Events Occurring in the Total Joint Replacement (TJR) Study Cohort

Criteria	Event Description	Number of Events
Inclusion	1. All cause readmission within 30 days of TJR surgery	26
	2. Health care utilization following TJR for diagnoses consistent with a complication*	33
Exclusion	1. Events occurring prior to TJR surgery	1
	2. Elective orthopedic procedures	11
	3. Elective procedures unrelated to joint replacements	17
	4. Treatment of unrelated medical comorbid conditions	36
	5. Multiple health care encounters with the same diagnosis within 30 days of each other were collapsed into a single event**	8

Source:

^*Based on the National Quality Forum, Knee Society and CMS diagnosis listings.

Notes:

^**8 patients had a collective 18 health care encounters (6 patients had 2 encounters and 2 patients had 3 encounters) within 30 days associated with the same diagnosis that was collapsed into 1 event per patient; thus to total number of included events equal all health care utilization that met inclusion criteria (26 + 33), subtracting the second and third encounters for the same diagnosis ([26 + 33]−8=49).

For patients who did report an adverse event, as part of the validation process, we reviewed available outpatient and inpatient records at a FORCE-TJR core site (institution where the TJR occurred) as well as the medical records from the institutions where the patients reported they sought care. When appropriate, and to support the diagnoses, we reviewed orthopedic clinic notes, ED notes, admissions- and physical examinations history, hospital discharge summaries, operation reports, anesthesia notes, and orthopedic consultations.

For patients who did not report any potential adverse events, we established a three-stage process to identify potential false negative reports. First, we conducted a review of all orthopedic outpatient notes during the first (nine) months following TJR at a FORCE-TJR core site. Then, we searched for ED records, day surgery (DS), and inpatient hospitalizations for all the patients at our study site (where the TJR occurred). The core site institution was the closest hospital to the patient’s home for 40 percent of the cohort (n=166). For the remaining cohort (n=265), we attempted to request records from the nearest hospital to the patient. We mailed 263 medical record releases to these hospitals nearest the patients’ homes (the signed medical-record release forms were missing for two patients). Of this total, 33 were declined. All other medical record request release forms (n=230) were submitted and received by the outlying hospitals.

We reviewed the medical record information to confirm that patient self-report of receiving health care following TJR for a potential complication actually occurred and to identify a diagnosis for this care. For the purpose of validity, documentation of an ED, DS, or inpatient hospitalization in the medical records was considered the “gold standard.” Based on our ascertainment and review process of available records in patients who did and did not report care following TJR for a possible complication, we calculated the sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) for patient self-report of care. We defined the term “sensitivity” of a patient’s report of post-TJR care as the proportion of patients who correctly reported care (meaning there was medical record evidence of health utilization) based on all events identified that met inclusion criteria. “Specificity” was the proportion of the study population that did not report seeking care among those with no post-TJR events. The “PPV” was defined as the proportion of true positives (seeking care) correctly identified by the patient survey responses. The “NPV” was defined as the proportion of true negatives (did not seek care) correctly identified by the questionnaires. We calculated reliability measures of percent agreement and kappa statistics.21 The kappa statistic measures the extent of exact agreement adjusting for chance agreement, with <0.4 representing poor agreement, 0.4–0.75 intermediate agreement and > 0.75 excellent agreement. We assessed validity and reliability. Finally, to assess generalizability we compared the core site institutional cohort to the larger FORCE-TJR cohort with data obtained from the same period using t tests, chi square, and Fisher’s exact tests, as appropriate.

Findings

There were 9,513 patients who enrolled in the national FORCE-TJR cohort and underwent TJR between July 1, 2011, and December 3, 2012. Based on patient report at the time of the six-month surveys, 31 percent of the self-reported care for potential complications occurred at facilities other than where the TJR was performed (25 percent of hospitalizations, 33 percent DSs, and 37 percent EDs). Across FORCE-TJR sites, between 0 and 80 percent of the readmissions in the 30 days following surgery occurred at hospitals other than where the TJR had been performed.

During this period, 458 patients underwent TJR at one of the FORCE-TJR core sites, of which 413 (90 percent) enrolled in FORCE and comprise the subcohort for these analyses. The 413 patients underwent a collective 431 surgeries and completed the six-month follow-up questionnaires responding to questions concerning seeking care for potential complications requiring an ED visit, DS, or inpatient hospitalization. Overall, the majority of patients were female (59.2 percent), with a mean age of 64 years (± 9.8), and were educated beyond high school (Table 2). When compared to the whole FORCE-TJR cohort over the same period, patients undergoing hip procedures in the institutional subcohort had more severe disease—with worse pain, worse stiffness, and worse function scores. Among those undergoing knee procedures, patients from the core site institutional subcohort had more impairment in joint- and global functions.

Table 2.

Baseline Characteristics for the AE Cohort and All FORCE Patients

	AE Cohort		All Other FORCE Cohorts		P value
	N	% or SD	N	% or SD
Female (%)	255.00	59.16	5324.00	58.62	0.823
Age (mean ± SD)	64.03	9.81	65.47	10.13	0.004
Education					0.210
High school or less (%)	104.00	26.13	2553.00	30.28
More than high school (%)	284.00	71.36	5675.00	67.32
Other (%)	10.00	2.51	202.00	2.40
Comorbidity Index					0.318
Zero (%)	229.00	56.27	4557.00	53.71
One (%)	81.00	19.90	1791.00	21.11
Two–five (%)	52.00	12.78	966.00	11.39
Six or more (%)	45.00	11.06	1170.00	13.79
FOR HIP
Operative joint*
Pain score (mean ± SD)	42.83	21.01	48.15	19.81	0.001
Stiffness score (mean ± SD)	34.54	22.37	37.87	21.62	0.061
Function score (mean ± SD)	38.96	20.02	44.83	19.31	0.000
Baseline SF-36**
MCS (mean ± SD)	49.68	12.99	50.06	12.61	0.694
PCS (mean ± SD)	29.84	9.49	31.71	8.77	0.006
FOR KNEE
Operative joint*
Pain score (mean ± SD)	53.15	18.28	51.21	18.96	0.139
Stiffness score (mean ± SD)	41.35	19.80	43.04	22.13	0.277
Function score (mean ± SD)	54.24	17.43	51.63	18.52	0.042
Baseline SF-36**
MCS (mean ± SD)	52.50	10.96	51.22	12.37	0.120
PCS (mean ± SD)	33.60	8.12	32.53	8.35	0.056

Notes:

^*Estimated operative joint scores using the HOOS for hips and KOOS for knees; lower scores represent more pain or stiffness, and poorer function.

^**Lower MCS and PCS scores represent poorer health.

For 35 surgeries, 40 events were reported on the six-month questionnaire, and all were associated with a documented ED visit, DS, or inpatient hospitalization (Figure 2). For 396 surgeries, no events were reported. In addition to the events reported by patients, we identified nine health care encounters after review of the medical records, including five hospitalizations and four ED visits. Of the hospitalizations not reported by patients, four out of the five occurred within seven days of surgery. One hospitalization was for an opiate overdose in a patient with a prior history of opiate abuse, one was for pain control, another was a hospitalization for postoperative fever but no source was identified, and one was diagnosed as cellulitis due to skin breakdown from psoriasis in an upper extremity. The fifth hospitalization occurred six weeks following TJR and had a diagnosis of “leg pain.” Review of the records revealed that the patient was newly diagnosed with spinal stenosis during that hospitalization, based on MRI findings. For the four ED visits that were not reported, all occurred within two weeks of surgery and were diagnosed as typical postoperative surgical joint pain. For the 40 events reported by patients, 17 were related to medical conditions surrounding surgery and 32 to symptoms related to the surgical joint.

Figure 2.

Results

Overall, the sensitivity for patient self-report was 82 percent, specificity 100 percent, PPV 100 percent, and NPV 98 percent, with excellent reliability (98 percent) and agreement (kappa 0.98) (Table 3). Sensitivity was highest for DS (100 percent) and lowest for hospitalizations (75 percent). No matter the location of care, specificity and NPV were 100 percent since there were no instances where patients reported care that did not occur.

Table 3.

The Validity of Patient Self-Report of Complications by Location of Care

	Reported Care		Did Not Report Care		Sens*	Spec*	PPV*	NPV*	Agree*	Kappa*
Location of Visit	True Pos*	False Pos*	True Neg*	False Neg*
Inpatient hospitalization	15	0	416	5	0.75	1.00	1.00	0.99	0.99	0.85
Emergency department	21	0	411	4	0.84	1.00	1.00	0.99	0.91	0.99
Day surgery	4	0	432	0	1.00	1.00	1.00	1.00	1.00	1.00
All locations	40	0	387	9	0.82	1.00	1.00	0.98	0.89	0.98

Notes:

^*True Pos= True positive, False Neg=False Positive, True Neg= True Negative, False Neg= False Negative, Sens=Sensitivity, Spec= Specificity, PPV = Positive Predictive Value, NPV= Negative Predictive Value, Agree=Agreement, Kappa= Kappa statistic.

Discussion

To our knowledge, this is the first study to address the problem of missing data to health care institutions and providers due to patients seeking care elsewhere following TJR and to evaluate patient self-report as a novel approach to fill this gap. Our work reveals the breadth of this issue, given that almost one-third of patients sought care from providers other than the initial hospital where the TJR was performed. Because the FORCE-TJR network includes both regional referral centers and community hospitals, our analyses demonstrate that all hospital settings are at risk of underestimating postdischarge readmissions and surgical complications. Overall, we found patient self-report of symptoms and health care utilization—including ED, DS, or inpatient hospitalization—to be both sensitive and specific with high positive and negative predictive values. This suggests that directly approaching patients for potential complications can augment current surveillance approaches to gather postoperative outcomes.

Interestingly, all patients (four) who had DS procedures were able to recall them. In contrast, some of the patients with ED visits and hospitalizations did not report them. There are several potential reasons patients failed to report these encounters. Patients may not have believed the care warranted reporting because, in the majority of instances, a complication was ruled out or it was the result of an unrelated condition (e.g., spinal stenosis, cellulitis related to psoriasis). Alternately, patients may not have reported hospitalizations in the first few weeks following surgery because of a lack of recall due to narcotic pain medications and distractions with rehabilitation and the new demands of the recovery period. We are currently investigating whether contacting patients earlier (at two months postdischarge) results in a more complete ascertainment of health care utilization.

Our results are similar to others showing high positive predictive values for patient-reported, short-term complications following surgery.19 However, our approach was different from prior reports as we did not ask patients to provide diagnoses. Rather, we asked patients to report any care they sought for the surgical knee or hip and the associated symptoms. We then performed chart reviews based on the symptoms described by the patient. For example, if a patient reported a hospitalization for chest pain four days following surgery, the chart was reviewed; however, if a patient had reported these symptoms four months following surgery, no chart would have been requested. For this reason, we did not evaluate the PPV of patient-reported postoperative diagnoses.

There are substantial practical implications to these results for practicing surgeons and hospitals that perform TJR. In December 2013, CMS began publicly reporting hospital-level 30-day all-cause readmission rates following elective THR/TKR and hospital-level complication rates. The CMS data are limited to patients 65 years and older, or who are disabled. To anticipate these reports, orthopedic surgeons and hospital leaders are expressing a desire to better understand the postdischarge outcomes their patients experience, specifically the patients who seek care elsewhere that vary from practice to practice. Many surgeons are unsure about the proportion of their patients who receive nonroutine post-TJR care at outlying hospitals. Use of patient-report can identify complications regardless of where patients seek care. For example, one specialty hospital reported that two-third of patients who require hospitalizations for complications do so at outside institutions.19 Therefore, surgeons whose patients come from a large geographic catchment area may find patient self-report to be a valuable tool to better understand their clinical outcomes. In addition, this information may be useful to confirm or refute potential complications, as others have shown that professional hospital coders significantly overcode complications based on a review of hospital and clinic charts.22 Novel ways to engage and sustain partnerships with patients to identify post-TJR events are needed. Providing a variety of reporting mechanisms for patients—such as prestamped postcards, toll-free telephone numbers, websites, or phone apps—will likely play a role. A few of the FORCE sites place a phone call to patients at 30 days post-TJR to assess any events. Given the likely financial implications based on CMS reimbursement, hospitals will be encouraged to develop better care coordination and to follow up with patients to learn of potential complications before they require hospitalization through monitoring of patients during the recovery process. It is typical for hospitals to survey postdischarge patients about their satisfaction with their care. In parallel, it would be possible to use these surveys to screen for postdischarge events.

Using patient self-report and specifically developing long-term relationships with patients can greatly advance our understanding of postoperative outcomes and overcome some of the limitations of other data sources. For example, when assessing longer-term outcomes such as revision, patient-report is a feasible process to identify complications among patients who move to new geographic areas or who change insurance or health care providers. It is particularly important to quantify complications among the almost 50 percent of patients undergoing TJR who are under 65 years of age and therefore have not reached Medicare eligibility. In addition, patient self-report of complications, e.g., deep vein thrombosis and pulmonary embolus, has been shown to more likely reflect the clinical record than ICD-9-CM codes would.19,23 Undercoding is a concern for the Medicare-age TJR population. We are currently in the process of comparing our chart-confirmed, patient-reported complications among patients over 65 years of age to those reported by Medicare claims to define the level of completeness of coded complication rates.

A significant strength of this study was the ability to examine the medical records of both patients who did and patients who did not report a potential complication. No prior study defined both the positive and negative predictive values. However, there are a few limitations to this study. First, the study was completed at a single high-volume TJR site, due to feasibility. However, we have no reason to believe that patients treated at this site would have greater or poorer recall of post-TJR complications. Second, the outcomes of interest occurred infrequently, so despite the review of more than 400 charts, the estimates are based on only 49 outcomes of interest. Third, recall bias by patients, especially among early hospitalizations, remains a concern. Lastly, the validity of patient self-report will likely vary from one condition to another, so these results may not be applicable to other surgeries or procedures.

However, our approach is an advance compared to previously published estimates that were based on questioning patients up to two years following surgery.19 In addition, the negative and positive predictive values were strong across all postdischarge settings, and those reported fill the void of data among patients under 65 years of age where no single health care administrative database exists.

This study has demonstrated the value of patient self-report of post-TJR care. The next question is how best to efficiently and effectively implement this approach into current clinical practice. One possible mechanism is to utilize current hospital staff and resources. Presently, many hospitals participate in the American College of Surgeons National Surgical Quality Improvement Program (ACS NSQIP).24 A trained surgical clinical reviewer collects pre- and postoperative data derived from medical records and contacts patients assessing 30-day clinical outcomes. This model could be tailored for the orthopedic population, such as including patient-reported outcomes so that evaluation of pain relief and functional improvement following surgery can be assessed. Additionally the derived data can be linked with the electronic health records to facilitate communication with the patient’s surgeon as well as the primary care provider and associated specialists involved in the patient’s care.

Conclusion

In summary, within a national network of orthopedic surgeons and their TJR patients, 20–30 percent of postoperative ED, DS, and inpatient hospitalizations, in the first six months following surgery, occurred at facilities other than where the TJR occurred. The varied settings for post-TJR care impede quality improvement efforts by surgeons and hospitals, as hospital-specific databases do not include comprehensive tallies of postdischarge events and health care utilization. In subsequent analysis, it was determined that patient self-report is a valid and potentially very efficient approach to augment current postdischarge surveillance efforts in TJR. Patient self-report is a novel method of reducing missing data from patients seeking care from outside hospitals and can improve the ability of surgeons to self-monitor potential complications; missing data may occur when patients change providers, which may be related to distance to the TJR hospital, patients moving to new geographic regions, or patients changing insurance.