Validation of Pre-operative Patient Self-Assessment of Cardiac Risk for Non-Cardiac Surgery: Foundations for Decision Support

Abstract

Objectives:

To better characterize patient understanding of their risk of cardiac complications from non-cardiac surgery and to develop a patient driven clinical decision support system for preoperative patient risk management.

Methods:

A patient-driven preoperative self-assessment decision support tool for perioperative assessment was created. Patient’ self-perception of cardiac risk and self-report data for risk factors were compared with gold standard preoperative physician assessment to evaluate agreement.

Results:

The patient generated cardiac risk profile was used for risk score generation and had excellent agreement with the expert physician assessment. However, patient subjective self-perception risk of cardiovascular complications had poor agreement with expert assessment.

Conclusion:

A patient driven cardiac risk assessment tool provides a high degree of agreement with expert provider assessment demonstrating clinical feasibility. The limited agreement between provider risk assessment and patient self-perception underscores a need for further work including focused preoperative patient education on cardiac risk.

INTRODUCTION

Surgical interventions provide opportunities for patients to alleviate potential morbidity and mortality. However, these procedures frequently result in cardiac, pulmonary, bleeding and infectious complications. Surgery is a frequent health care intervention with an estimated 6 million non-cardiac surgical procedures performed every year in the United States, with progressive growth in procedures noted each subsequent year (1,2). Approximately 25% of these procedures include major intra-abdominal, thoracic, vascular, and orthopedic procedures known to be associated with significant perioperative cardiovascular morbidity and mortality (3).

The American College of Cardiology/American Heart Association (ACC/AHA) issued perioperative guidelines for non-cardiac surgery in 2007(2), as well as a focused update on preoperative beta blockade in 2009 (3), and endorsed guidelines to direct clinicians to estimate the risk of major adverse myocardial events. These guidelines modified recommendations on the Revised Cardiac Risk Index (RCRI) (4,5) to highlight a set of conditions associated with higher post-operative morbidity and mortality for patients undergoing surgery. The primary risk factors of the RCRI include ischemic heart disease, compensated or prior congestive heart failure, cerebrovascular disease, diabetes mellitus requiring insulin, and renal insufficiency typically with a creatinine level of 2.0 or above (5).

Although the RCRI guidelines have been an important attempt to simplify preoperative risk assessment, it remains a significant challenge for clinical practitioners to provide proper evidence-based preoperative evaluations given the constantly changing clinical evidence and the broad realm of specialty literature pertinent to preoperative testing and risk management (6–11). As surgical interventions continue to evolve with increasingly complex and costly procedures, there is a critical need to improve the preoperative assessment to effectively identify clinical risk factors and manage existing co-morbidities. In the perioperative window, from one month before to one month after the surgical intervention, targeted risk mitigation can be implemented to reduce surgical complication risk. In the context of growing surgical work volumes, there is a paucity of well-trained primary providers and preoperative assessment clinics to address patient needs with ongoing growth in the number of preoperative assessments (12,13), (14). An important partner for the surgical team is the patient, who ultimately has the most at stake from the surgical benefits and potential complications (15). Unfortunately, few tools are available for patients to self-identify surgical risk and empower them to work in tandem with multi-disciplinary surgical teams. Such tools could help patients become better informed of their surgical risk and address an important knowledge gap since most patients have limited recall of the risks and benefits of surgical interventions after completing the pre-operative clinical workup (16,17). In addition to the potential educational and clinical benefits to the patients, patient driven decision support tools could also be a cost-effective adjunct tool for surgical quality efforts including Accountable Care Organizations (ACO), medical homes, and other efforts to enhance clinical quality. Having effective patient driven clinical assessments can provide surgical providers a greater appreciation of surgical risks prior to their planned procedures and facilitate optimized multi-specialty care delivery.

Though patients may not fully understand all the details discussed with their providers prior to their surgical procedures, the informed consent process has been established to insure discussions of risk and benefits do take place. The preoperative care process makes it possible for patients to better understand any planned procedures by asking their providers targeted questions and to facilitate their own information gathering. Prior studies on patient understanding of clinical risk focused on patient perceptions of risks and benefits, (16–20) but there is limited data on whether patient risk perceptions have a significant correlation with provider risk assessment. Such mutual understanding by both patients and providers is important to better manage potential complications that can occur during surgical procedures. When gaps in the patient and provider perceptions are present, this creates an important education opportunity to help patients fully appreciate the implications of any planned procedure.

BACKGROUND

The identification and management of clinical risk prior to surgery occurs in the context of significant time constraints. Typically, the clinical workup prior to surgery is a full history and physical examination completed 30 days prior to surgery to fulfill regulatory requirements and to maintain clinical care standards. These preoperative visits focus on clinical conditions that may affect anesthesia and surgery and are usually completed by a primary care provider or in an anesthesiology clinic. Frequently, these medical evaluations are provided in conjunction with surgery specific patient education and assessment and coordinated with surgical, anesthesia and medical providers. In this 30 day preoperative window, the patient is at risk of having exacerbations of their chronic medical conditions contributing to operative morbidity and mortality. As a result, it is important that patients have their assessments completed as close as possible to their surgery, yet it is important to allow enough time to mitigate any clinical conditions which may place the patient at increased surgical risk. Frequently, diagnostic testing and specialty consultation are required prior to the surgical date to insure appropriate care. Unfortunately, no clinical guidelines provide guidance on how to optimize pre-operative assessment timing. Patients with complicated medical histories often require a complicated evaluation and treatment regimens prior to surgery that may involve multiple preoperative assessments in order to optimize their clinical conditions (17). The effectiveness of these visits is clinically important since the decompensation of chronic conditions just prior to surgery may lead to late cancellations resulting in unfilled operating room slots. Late surgical cancellations can have substantial adverse effects on hospital revenue since many operating rooms costs are fixed expenditures and surgical procedures generating substantial proportions of hospital revenue (21). To optimally deliver surgical care, preoperative planning and risk management capacity must be enhanced for those patients at risk of clinical decompensation prior to and after surgery (22).

Patient self-assessment software can potentially mitigate time pressures in the 30-day preoperative window by adding important triage data to better identify the at-risk population. Such tools would optimally be patient-driven since they are most aware of any new changes that occur prior to surgery. However, to use such patient-driven information collection systems, it is important to establish validity and reliability. The ability of patients to self-identify clinical risk factors has been previously established in many studies of chronic disease (21,23–26). However, tools for patient self-identification of clinical risk factors have not translated efficiently from large research studies for use in clinical care. There are strategic advantages to having patients identify their own clinical risk factors both in terms of patient understanding of their own clinical conditions and to facilitate accurate clinical history data. In addition, a patient driven approach could potentially be used to reduce treatment costs by minimizing provider mediation to obtain medical information. Although personal health records have the potential to facilitate pertinent patient driven clinical data, the lack of effective direct connections to the patient’s electronic medical record remains a major limitation. An additional question is whether patient self-identification of risk factors is valid for focused clinical assessments such as cardiac risk (27). In order to address some of the critical needs of preoperative assessment, a continuous improvement project was undertaken to assess if patient self-report data on cardiac risk could be validated against the current standard of clinical practice. Patient self report data on cardiac risk perception and objective cardiac risk factors were compared with the provider mediated revised cardiac risk index assessment. The RCRI was selected for use since it is the most widely used clinical assessment that has been validated for use in planning pre-operative clinical interventions and is well established in existing care guidelines (28–30). The authors were motivated to use a widely used risk scale with minor modifications of the descriptive disease elements and symptomatic verbiage to facilitate patient comprehension. The primary project goal is to create a valid and evidence-based foundation for a patient driven preoperative cardiac risk decision support tool for preoperative clinical decision making.

METHODS

Study Design – A prospective, single center, hospital based observational study to evaluate an institutionally approved continuous quality improvement initiative was conducted at VA Medical Center (VAMC) in Minneapolis, Minnesota.

Site Description - A total of 309 patients, who visited the preoperative medicine clinic in the Department of General Medicine VAMC Minneapolis during the study period were included in the study. The Minneapolis Veterans Administration Hospital is a major referral site within the VA system and serves as a surgical multi-specialty site for the VISN 23 region. The VISN 23 clinical health care network serves more than 400,000 enrolled Veterans residing in the states of Iowa, Minnesota, Nebraska, North Dakota, South Dakota and portions of Illinois, Kansas, Missouri, Wisconsin and Wyoming. The preoperative medical clinic operated from the Minneapolis clinical site is a large preoperative medicine clinic site with 10 clinicians providing preoperative medical assessments.

Patient Selection – All patients, referred to the Pre-operative Medicine Clinic during the study period from 1 Dec 2011 to 28 February 2013 were eligible for the study. After excluding 5 patients who were not able to complete surveys, a total of 304 patient surveys and medical records were analyzed for this study.

Cardiac Risk Tool Development– The survey tool used for this study was developed with a goal of enabling patients to complete this survey online through secured messaging at remote sites. The implementation of the survey with electronic messaging can help facilitate remote care and potentially enhance the ability to aggregate patient history and medical information with direct links to electronic patient medical records. (Figure 1)

Figure 1:

Overview of Methods

To accurately assess and capture patient medical and surgical history, exercise tolerance, and cardiovascular risk perceptions, a survey instrument was developed as a 25-question assessment tool. The instrument was developed by mapping the American Heart Association Guidelines for Pre-operative risk assessment of cardiac complications of Non-Cardiac Surgery (Figure 2). Specific questions were developed to identify each of the six revised cardiac risk index factors and a mapping algorithm was established to generate a RCRI score with expert provider review. Where applicable, questions from established and validated patient instruments like Rose Questionnaire for ischemic heart disease; Questionnaire to Verify Stroke Free Status (QVSFS) for cerebrovascular disease; and Compendium of Physical Activity for exercise capacity, were modified and used to generate survey questions. In addition to the objective cardiac risk factor assessment, questions were developed to identify patient cardiac risk perception on a graded scale. These results were scored with a mapping algorithm to identify relative levels of risk perceived by the patients that reflected current clinical guidelines.

Figure 2:

Overview of Survey Development

The development included iterative pilot surveys assessed by practicing preoperative medical providers and patients. The survey content was tailored for patient use with an approximate reading level of grade 7 by Flesch-Kincaid Readability analysis to facilitate patient use.

Questions on important surgical contraindications including certain high risk surgical preclusion criteria, preexisting conditions, cardiovascular risk perception, exercise tolerance, and elements of the 6 revised cardiac risk index risk factors were adapted for patient use and incorporated into the instrument. The validity assessment in this paper focuses on the patient perception of their cardiac risk as well as patient self-reported cardiac history to assess tool reliability when compared to gold standard provider evaluation.

Statistical Analysis:

Provider notes for each patient’s preoperative medical visit were examined by assessing provider risk scoring and their objective cardiac risk assessment. Study data was analyzed using the SAS statistical package (Version 9.3, Cary, N.C.). A p-value of less than 0.05 was considered to be significant.

RESULTS

In Table 1, the characteristics of the study patients included self-reported cardiac risk tool responders. Exercise tolerance was included in the table as it is an important predictor of cardiac risk with surgery and provides an important surrogate measure of the group cardiac risk.

Table 1:

Study Patient Characteristics

Age mean (± S.D.) {Range}	63 years (±12) {25 – 94 years}
Males n (percent)	290 (95%)
Exercise Tolerance (> 4 Mets) n (percent)	276 (93%)

Patient cardiac risk was evaluated using patient self-assessment tool responses, which were mapped and aggregated to create a revised cardiac risk index score. These results were compared with a clinical chart review of the final medical consultation on cardiac risk completed by each patient’s clinical provider during their preoperative visit. Patient self-assessment was completed just prior to the planned preoperative clinical visit providing a very low likelihood of a change in clinical status between the time of the patient self-report and the medical provider’s assessment. The risk factor results with aggregated scores are noted in table 2.

Table 2:

Comparison of Calculated RCRI Scores: Calculated from Clinician notes versus Patient Self-Reported Cardiac Risk Factors

Calculated from Provider Notes, Chart Review n (%)	Revised Cardiac Risk Index Score	Calculated from Patient Surveys n (%)
209 (69)	0	211 (70)
64 (21)	1	60 (20)
16 (5)	2	23 (8)
7 (2)	3	4 (1)
3 (1)	4	2 (0.6)
2 (0.6)	5	1 (0.3)

The patient self-report of their estimated cardiac risk with surgery is listed in Table 2. These results were compared with the clinical provider’s estimate of operative cardiac risk documented in the clinical chart after the patient’s routine preoperative medical visits was completed. The results were assessed for inter-rater reliability by calculating a kappa statistic (31). The Kappa Statistic (Weighted) for percent agreement between RCR Score generated from patient survey instrument and Chart review is 0.78.(95% C.I.0.72–0.85). These results demonstrated a substantial level of agreement between the two estimates of reported cardiac risk.

After excluding surveys with missing data elements for patient self-assessment and provider perceptions of cardiovascular risk, 282 surveys were used for comparing risk perceptions. As described in Table 3, the aggregate frequencies of cardiac risk perceptions were similar but demonstrated substantial disparities for inter-rater agreement between patient and providers. The calculated kappa showed that the patient’s self risk perception had poor level of overall agreement with physician perception of cardiac risk with a weighted Kappa score of only 0.18 (95% C.I. 0.04–0.31) with individual kappa for low, intermediate and high risk comparisons being 0.17. 0.08 and 0.21 respectively.

Table 3:

Comparison of Perceived Cardiac Risk-Clinician Assessment Versus Patient Self-Risk Estimate Survey

From Provider Report n (%)	Perceived Cardiac Risk	From Patient Report n (%)
236 (84)	Low	242 (86)
41 (15)	Intermediate	36 (13)
5 (2)	High	4 (1)

There were only 19 high risk procedures planned among the participating patients and most major surgical subspecialties were included in study with 105 of the planned surgeries deemed to have low estimated cardiac risk and 178 of the procedures designated as intermediate (Table 4). Tables 5, identifies the types of procedures for which the patients were being evaluated. Cardiac surgeries were excluded as they received their preoperative assessment from the cardiothoracic clinical service lines at the clinical site of the study and practice guidelines assess risk for cardiac surgery differently than non-cardiac surgery.

Table 4:

Risk categories for performed surgeries

Risk Category	n (%)
Low Risk	107 (35)
Intermediate Risk	178 (59)
High Risk	19 (6)

Table 5:

List of Surgical Procedures

Procedure Type	n (%)
Orthopedic	89 (29)
Neurosurgery	38 (13)
Otolaryngology	34 (11)
General Surgery	32 (11)
Ambulatory	31 (10)
Urology	27 (9)
Ophthalmology	23(8)
Vascular	15(5)
Podiatry	7 (2)
Colorectal	2 (0.6)
Dental	2 (0.6)
Cardio-Thoracic	1 (0.3)
OB-GYN	1 (0.3)

DISCUSSION

The results of the assessment of this quality improvement initiative provide two key insights on patient self-assessment of cardiac risk. The patient’s self-assessment of their perceived pre-operative cardiac risk had poor agreement with expert clinical providers. In contrast, the patient self-report of their primary cardiac risk factors showed substantial inter-rater reliability with the provider assessment. In addition, the patient self-reported risk data mapped well to the existing clinical standards of preoperative cardiac risk (Revised Cardiac Risk Index) demonstrating the feasibility of a patient driven approach. These findings provide important evidence that patients can adequately provide self-report data that can accurately estimate cardiac risk with surgical interventions. The ability of patients to self-report risk factors with similar results to the gold standard provider clinical assessment indicates that the development of decision support tools with patient driven interfaces may potentially be used for preoperative cardiac assessment. In contrast, patients appear less able to predictably provide an accurate perception of their pre-operative cardiac risk, which matches the gold standard assessment of their clinical providers.

There are a number of limitations to this evaluation. The sample size is relatively modest, which makes it difficult to ensure that the patient self-report of cardiac risk factors and self-perception data is a true representation of Veterans Administration preoperative patients. The data collected in the study focuses on a regional Veterans Administration pre-operative patient population and the data is generated from a single clinical practice site. It is unclear if these results could be extended to non-VA clinical sites given the predominant male distribution of the sample and mandates further study. Another limitation is the relative lack of high cardiac risk surgical procedures in the study population. It is unclear how patients perceive the risk of these procedures and correspondingly uncertain whether that perception can be detected in the objective risk data related to procedures. Further assessment, including patients undergoing high-risk procedures, can be used to assess this aspect of the survey in future work.

The future work for the project, based on the findings to date, will focus on developing patient friendly computer interfaces in the secure messaging application. In addition, the survey tool may also be used to pre-screen patients with high-risk characteristics to potentially receive high-risk assessments, which have been developed at the clinical site of this research project (21,32). Patient prescreening can potentially be matched with protocols to preorder cardiac assessments when indicated. This could help improve clinical care coordination, particularly for those patients who have to travel hundreds of miles for their preoperative assessment as part of the national VA health care system.