Provider Survey on Automated Clinical Decision Support for Cardiovascular Risk Assessment

Abstract

Objective

To investigate provider opinions regarding a clinical decision support (CDS) system for cardiovascular risk assessment and for the creation of a replacement system.

Methods

From March to April 2018, an invitation letter with a link to a self-administered web-based survey was sent via e-mail to 279 providers with primary appointment in the Department of Cardiovascular Medicine, Mayo Clinic, Rochester. The e-mail was sent to providers on March 8, 2018 and the survey closed on April 16, 2018.

Results

One hundred providers responded to the survey yielding an overall response rate of 35.8%. Of these, 52 (52%) indicated they had used the cardiovascular (CV) risk profile CDS system and were classified as users and prompted to continue the survey. Among users, 42 (80.8%) indicated use of the CDS was either important (25; 48.1%) or very important (17; 32.7%) in their clinical practice; 45 (86.5%) responded that the system was very easy (17; 32.7%) or easy (28; 53.8%) to use. In addition, 48 (96.0%) users indicated that the CV risk profile supported their thought process at the point-of-care; 47 (97.9%) users indicated similar functionalities should be implemented into the new electronic health record system and 41 (85.4%) users reported new functionalities should also be incorporated.

Conclusions

For most users, the CDS system was easy to use and supported clinical thought process at the point-of-care. Users also felt their practice was supported and should continue to be supported by CDS systems providing individualized patient information at the point-of-care.

In the United States, cardiovascular diseases account for 1 of 3 deaths and are a leading cause of death for both men and women.1, 2 Cardiovascular diseases are related to risk factors including smoking, hypertension, high blood cholesterol, diabetes, and obesity. While progress has been made in establishing strategies for primary and secondary prevention of cardiovascular diseases, many studies have identified gaps in implementation and adherence to guideline recommended strategies.3, 4, 5, 6 Additionally, it takes an average of 17 years for 14% of guidelines to be integrated into practice.⁷ Thus, there is urgent need to find solutions to address these gaps in prevention via increased implementation of guideline-recommended cardiovascular preventive strategies.

Prior studies have used clinical decision support (CDS) systems to promote individualized cardiovascular prevention.8, 9, 10 CDS provides timely information at the point-of- care to inform patient care decisions.¹¹ Our institution implemented a CDS system in 2008 to standardize and support guideline-based preventive cardiovascular care. This CDS system, termed the cardiovascular (CV) risk profile, displayed individualized patient information in the following categories: risk factors, body composition, vascular health, metabolic syndrome, heart disease risk analysis, lifestyle factors, recommendations, and follow-up. Pertinent data elements were included in each category, including high-density lipoprotein and blood glucose levels within the risk factor category and patient reported diet and exercise within the lifestyle factor category (Supplemental Figure 1, available online at http://mcpiqojournal.org). In addition, the system also provided a mortality risk score calculated by Framingham, Reynolds, or pooled cohort equations. Results were categorized as low-risk, intermediate-risk, and high-risk for cardiovascular disease.12, 13 Each data element retrieved was also classified as “low,” “intermediate,” or “high” risk using cut-points defined by published medical literature. These data elements were displayed in a color-coded categorization as follows: low-risk results displayed in green, intermediate risk in yellow, and high risk in red (Supplemental Figure 1, available online at http://mcpiqojournal.org). Recommendations for care were individualized and based on patient characteristics. The CV risk profile CDS system was built primarily for use in the Division of Preventive Cardiology in the Cardiovascular Health Clinic at Mayo Clinic Rochester. This system was linked to the Mayo Clinic Rochester electronic health record (EHR) for 10 years until transition to a new EHR at which time the CDS system became nonfunctional. The aim of this study was to investigate provider opinions regarding the CV risk profile system and the creation of a new CDS system compatible with the new EHR.

Methods

A provider survey with 9 questions was developed in conjunction with the Mayo Survey Research Center. The first question determined whether providers would continue with completion of the survey or not. If their answer was “no,” then they were not prompted to answer further. However, if they responded “yes,” they were prompted to continue with survey completion. The survey questions are displayed in Supplemental Table 1 (available online at http://mcpiqojournal.org). Survey questions 2 to 7 were focused on characteristics related to usability of the CV risk profile system.

The Survey Research Center sent an invitation letter by e-mail with a link to a self-administered web-based survey for 279 providers including all staff physicians, fellows, nurse practitioners, and physician assistants with a primary appointment in any Division of the Department of Cardiovascular Medicine at the Mayo Clinic Rochester. The assignments to these Divisions are based on the expertise of each provider. There are 10 separate divisions as follows: Preventive Cardiology, Cardiovascular Ultrasound, Circulatory Failure, Community Cardiology, Comprehensive Cardiology, Heart Rhythm Services, Interventional Cardiology, Ischemic Heart Disease and Critical Care, Structural Heart Disease, and Vascular Cardiology. Cardiology fellows rotate in all divisions during their training. Nurse practitioners and physician assistants are assigned to primary appointments for inpatient or outpatient practices. Staff physicians rotate in inpatient or outpatient assignments. PhD exercise physiologists are assigned to outpatient practices in the Division of Preventive Cardiology. When the survey was sent, the Preventive Cardiology outpatient clinic (named Cardiovascular Health Clinic) was staffed by 14 physicians, 2 nurse practitioners, and 4 PhD exercise physiologists. The email was initially sent to providers on March 8, 2018 and the survey closed on April 16, 2018 after 5 reminder emails. To compare rates of CDS use between different provider roles, the Pearson chi-square test was used. Due to small numbers in some subgroups, Fisher's exact test was used to test for differences in rates of CDS use between primary assignments. This study was approved by the Institutional Review Board.

Results

One hundred providers responded to the survey yielding an overall response rate of 35.8%. Among the 100 providers who responded to the survey, 52 (52%) indicated they had used the CV risk profile system (responded “yes” to question 1) and were classified as users and prompted to continue the survey. The remaining 48 (48%) indicated they had not used this system (responded “no” to question 1) and were classified as “nonusers” and not asked to continue to complete the survey. Among users, there were 29 staff physicians, 14 cardiology fellows, 7 nurse practitioners or physician assistants, and 2 PhD exercise physiologists. Among nonusers there were 26 staff physicians, 9 fellows, and 13 nurse practitioners or physician assistants. There was no difference of provider role and CDS use response (P=.21; Pearson χ² test; Figure 1).

Figure 1.

Distribution of question-1 responses by provider role. The number of provider respondents for each category are displayed at the top of each column. CDS = clinical decision support system; MD = medical doctors; NP/ PA = nurse practitioners or physician assistants; PhD = Doctor of Philosophy degree in exercise physiology.

All respondents with primary appointment in the Division of Preventive Cardiology were users of the system (Figure 2). All respondents with primary assignment in heart rhythm services, research, and vascular cardiology were nonusers of the system (Figure 2). As shown in Figure 2, there were definite differences between primary assignments (P<.001) and CDS use responses. Providers with primary appointment in divisions other than Preventive Cardiology used the CDS during patient encounters when risk assessment was indicated. However, 48 (48%) respondents did not require the CDS because their clinical practices addressed other aspects of patient cardiovascular health.

Figure 2.

Distribution of question 1 responses by primary assignment. The number of provider respondents for each category are displayed at the top of each column. CDS = clinical decision support system; CV = cardiovascular; MD = medical doctors; NP/ PA = nurse practitioners or physician assistants; US = ultrasound.

Among the 52 users, 8 (15.4%) reported they used the system daily. The majority (18, 34.6%) reported using the CV risk profile system less than once per month. Eleven (21.2%) providers indicated they used the system several times per week and 6 (11.5%) indicated they used the system once per month (Supplemental Figure 2, available online at http://mcpiqojournal.org). When asked how important users considered the system for their clinical practice, 42 (80.8%) indicated it was either important (25, 48.1%) or very important (17, 32.7%). Only 8 (15.4%) providers reported the system as very unimportant (Supplemental Figure 3, available online at http://mcpiqojournal.org). Forty-five (86.5%) users also reported that the system was either very easy (17, 32.7%) or easy (28, 53.8%) to use in clinical practice (Figure 3). In addition, 48 (96.0%) of users reported that the CV risk profile supported their thought process at the time of patient encounters (Figure 3).

Figure 3.

User opinions on CV risk profile showing the percentage of users that reported finding CV risk profile easy or very easy to use (86.5%) or not easy to use (13.5%) and supportive of their clinical thought process (96.0%) or not supportive of their clinical thought process (4.0%). CV = cardiovascular; % = percentage of users.

When asked if similar functionalities to the CV risk profile system should be added into the new EHR system, 47 (97.9%) users indicated these functionalities should be implemented and 1 (2.1%) responded that there was no need (Figure 4). 41 (85.4%) users reported new functionalities should be incorporated into the new EHR and 7 (14.6%) responded that there was no need for new functionalities (Figure 5).

Figure 4.

User-reported need for similar CDS functionalities in the new EHR showing percentages of users that reported need for similar functionalities in the new EHR (97.9%) or no need for similar functionalities (2.1%). CDS = clinical decision support system; EHR = electronic health record system; % = percentage of users.

Figure 5.

User-reported need for new CDS functionalities in the New EHR showing percentage of users that reported need for new functionalities (85.4%) or no need for new functionalities (14.6%). CDS = clinical decision support system; EHR = electronic health record system; % = percentage of users.

The majority of users were staff physicians 29 (58.0%) (Supplemental Figure 4, available online at http://mcpiqojournal.org). The next largest group of users was fellows (13, 26%) followed by nurse practitioners and physician assistants (6, 12%). Among the users, there were also 2 (4.0%) PhD exercise physiologists. The responses for years in practice had a U-shaped distribution (Supplemental Figure 5, available online at http://mcpiqojournal.org). The majority of users were in practice for 5 years or less (18, 36.0%). The next largest user group was providers who had completed more than 20 years in practice (16, 32.0%).

Discussion

The major findings of this study were the high support for both similar and new CDS functions to be implemented into the new Mayo Clinic EHR. As 97.9% (47 of 48) of users surveyed indicated that similar functionalities present in the CV risk profile system should be retained for the new EHR, it appears their practice was supported and should continue to be supported by CDS systems that provide individualized patient information at the point-of-care. Importantly, 96% (48 of 50) of users reported that the CV risk profile supported their clinical thought processes at the point-of-care and 86.5% (45) of users found the CDS system easy to use.

In comparison to other similar studies, the provider survey response rate of 35.8% (100 of 279) was greater than the response rate of 27% (207 of 760) reported by Chaudhry et al¹⁴ investigating provider opinions about CDS for provision of evidence-based care to patients with peripheral artery disease at the point-of-care. Additionally, the response rate of the study herein was greater than the reported rates of 10% (10 of 100) to 15% (15 of 100) from a random sample of physicians across specialties (primary care, obstetrics/gynecology, and cardiology) in a national survey of guideline-recommended strategies for cardiovascular disease prevention.¹⁵

Our results showed providers favor implementation of a similar CDS system as well as new functionalities. Most (41 of 48, 85.4%) providers surveyed indicated new functionalities should be included in the updated CDS system. New functionalities could further prioritize and filter information provided, refining the system to suit provider needs. Our observations suggest builders of the next CDS system for cardiovascular risk assessment involve a multidisciplinary team composed of clinicians, informaticians, and technologists for development of a replacement CDS tool incorporating both established and new functionalities.

Though the majority of providers found the system easy to use, 13.5% (7 of 52) of providers suggested they found the current CV risk profile system difficult indicating there is an opportunity to improve usability. In the present study, most providers reported the CV risk profile supported their clinical thought processes at the point-of-care. A new CDS system will be designed in collaboration with clinicians to continue to achieve this purpose. In prior reports, physicians have decreased usage rate of CDS systems if information was not provided at the point-of-care,¹⁶ underscoring the importance of system design that supports clinician thought process. Additionally, physicians are more likely to use CDS systems when data entry is not required and information provided by the CDS system is accurate.¹⁶ The new CDS system will extract data elements from the EHR automatically and manual data entry will not be required.

In the present study the response rate of 35.8% (100 of 279) to the web-based survey was low. While low, this rate is similar to rates of prior web-based surveys of physicians.¹⁷ Additionally, a prior meta-analysis has also shown that response rates of web-based surveys by health professionals were similar to those reported herein.¹⁸ To mitigate the low response rate to our web-based survey, we conducted analysis of user logs of the CV risk profile CDS system. This approach has been previously reported for workflow analysis as a component of models for design, development, implementation, use, and evaluation of health information technology for CDS.19, 20, 21 This user log analysis showed that over a 12-year interval (from 2006 to 2018) the CV risk profile CDS system generated 39,396 reports by 282 users including 211 physicians (staff or fellows), 9 nurse practitioners or physician assistants, 19 registered nurses, 2 certified exercise specialists, 5 PhDs, and 36 other users. This observation indicates high use of the risk profile CDS system in our clinical practice, despite the small number of user respondents to our web-based survey.

The present study reports only demographic information of nonusers. However, survey questions 2 to 7 were focused on characteristics related to usability of the CV risk profile system and nonuser respondents did not qualify to respond to questions regarding a system they did not use. The question on which functionalities providers wanted added to the CDS was not included in this present survey. However, functionalities for the new CDS will be investigated by both a future provider survey and focus group discussions to design the replacement CDS tool. A similar approach was used by Hasnie et al,²² to design CDS for familial hyperlipidemia. The shortfalls of the old system will also be examined using these methodologies.

Conclusion

In ongoing efforts to reduce cardiovascular disease related morbidity and mortality, a CDS system similar to the previous CV risk profile system should be developed and refined to support provider needs for clinical practice. The new CDS system will display risk factors, results of risk calculations with individualized recommendations for guideline-based patient care, and address CDS challenges including factors that have prevented successful implementation of clinical practice guidelines for cardiovascular disease risk prevention. Based on the results of the survey conducted in this study, the CV risk profile system was easy for most providers to use and supported thought process in assessment of cardiovascular risk factors. This study also showed that providers favor continued refinement of the CV risk profile after implementation into the new EHR.

Supplemental Online Material

Supplemental material can be found online at http://mcpiqojournal.org. Supplemental material attached to journal articles has not been edited, and the authors take responsibility for the accuracy of all data.