Physician Perceptions and Beliefs about Generating and Providing a Clinical Summary of the Office Visit

Summary

Background

A core measure of the meaningful use of EHR incentive program is the generation and provision of the clinical summary of the office visit, or the after visit summary (AVS), to patients. However, little research has been conducted on physician perceptions and beliefs about the AVS.

Objectives

Evaluate physician perceptions and beliefs about the AVS and the effect of the AVS on workload, patient outcomes, and the care the physician delivers.

Methods

A cross-sectional online survey of physicians at two academic medical centers (AMCs) in the northeast who are participating in the meaningful use EHR incentive program.

Results

Of the 1 795 physicians at both AMCs participating in the incentive program, 853 completed the survey for a response rate of 47.5%. Eighty percent of the respondents reported that the AVS was easy (very easy or quite easy or somewhat easy) to generate and provide to patients. Nonetheless, more than three-fourths of the respondents reported a negative effect of generating and providing the AVS on workload of office staff (78%) and workload of physicians (76%).

Primary care physicians had more positive beliefs about the effect of the AVS on patient outcomes than specialists (p<0.001) and also had more positive beliefs about the effect of the AVS on the care they delivered than specialists (p<0.001).

Conclusions

Achieving the core meaningful use measure of generating and providing the AVS was easy for physicians but it did not necessarily translate into positive beliefs about the effect of the AVS on patient outcomes or the care the physician delivered. Physicians also had negative beliefs about the effect of the AVS on workload. To promote positive beliefs among physicians around the AVS, organizations should obtain physician input into the design and implementation of the AVS and develop strategies to mitigate its negative impacts on workload.

1. Introduction

The Medicare and Medicaid Electronic Health Record (EHR) Incentive Program, also widely referred to as the EHR Meaningful Use Program, incentivizes eligible providers (EPs) to adopt and meaningfully use EHRs [1–2]. The objectives of the EHR Meaningful Use Program are to increase the adoption of EHRs as well the use of EHRs to improve delivery of care, decrease medical errors, improve efficiency of care, and enhance patient centeredness of care [2]. The program was designed to take place over three stages from 2011 through 2016, with core measures for achieving meaningful use of EHRs becoming more rigorous with each stage. EPs who meet the measures of the program and become meaningful users could receive up to $44 000 (Medicare) or $63 750 (Medicaid). Beginning in 2016, EPs who failed to achieve Meaningful Use by 2014 would see a reduction of their Centers for Medicare and Medicaid (CMS) payments beginning at 1% of total payments, with penalties increasing by 1% per year afterward.

A core measure to achieve meaningful use of the EHR is the provision of the clinical summary of the office visit, or the after visit summary (AVS). CMS has defined the AVS as a clinical summary that “provides a patient with relevant and actionable information and instructions” [3]. Information in the AVS consists of elements such as the patient name, date and location of visit, an updated medication list, updated vitals, reason(s) for visit, procedures and other instructions based on clinical discussions that took place during the office visit, any updates to a problem list, and immunizations or medications administered during visit, summary of topics covered/ considered during visit, and time and location of next appointment/testing if scheduled. Stage 1 of the EHR Meaningful Use Program specified that the AVS should be provided to patients for more than 50 percent of all office visits within three business days. Stage 2 of the program made this core measure more rigorous by specifying that the AVS should be provided to patients or patient-specified representatives for more than 50 percent of all office visits within one business day.

The provision of the AVS as a core measure in the EHR Meaningful Use Program is aimed at enhancing patient access to their health information as well as improving patient engagement in their care [2, 4]. A pilot study that examined patient perceptions of the AVS found that 93% of the respondents agreed that the AVS was helpful, 96% found the problem list to be correct, and 93% found the medication list to be correct [5]. Almost three-fourth of the respondents (74%) agreed that the AVS helped them better understand their plan of care. A study on patient satisfaction with the AVS found high levels of satisfaction across different content conditions of the AVS [6]. However, it is not clear whether such positive perceptions of the AVS carry over to the physicians who are generating and providing the AVS. Additionally, the provision of the AVS poses a significant challenge from a physician standpoint as physicians need to ensure that all the relevant information is entered into the AVS and that workflows are established for the patient to receive the AVS either in the office at the end of the visit or through some other mechanism such as a patient portal [4].

2. Objective

There is almost no literature on physician perceptions about the AVS that they generate and provide to patients as part of the EHR Meaningful Use Program. How easy is it for physicians to generate and provide an AVS? Do physicians perceive that the AVS is effective in providing a summary of the office visit? How satisfied are physicians with the AVS that they provide to patients? In this paper, we evaluate physician perceptions of the AVS and assess the relationship of these perceptions with physician characteristics such as age, gender, and specialty. We also evaluate physician beliefs about the effect of the AVS on workload, patient outcomes, and the care the physician delivers. As we had previously noted, there is a growing need in the field of applied clinical informatics to assess physician perceptions and beliefs about the impact of the use of the EHR in general as well as use of specific functionality of the EHR to meet requirements of the EHR Meaningful Use Program such as the AVS [7]. As this is one of the first studies on this topic it is exploratory in nature.

3. Methods

3.1 Participants

The participants in this study were physicians, or eligible professionals with respect to the Meaningful Use of EHR program, at two academic medical centers (AMCs) in the northeast. Eligible Professionals (EPs) are physicians, excluding fellows and residents, who participate in Medicare or Medicaid, and whose eligible billing comes from office visits. With respect to the AMCs, AMC1 is a 777-bed medical center with 57 000 admissions per year and AMC2 is a 907-bed medical center with 48 000 admissions per year. AMC1 has 101 affiliated outpatient practices with 170 primary care physicians, 1 423 specialists, and 769 000 outpatient visits a year. AMC2 has 175 affiliated outpatient practices, with 259 primary care physicians, 1 737 specialists, and 1.5 million outpatient visits a year. Overall, 1 795 physicians at the two AMCs were identified as participants in the Meaningful Use of EHR program and were considered subjects for this study. At both AMCs, a common home-grown outpatient EHR has been in use since 1998. The homegrown EHR is an internally developed, web-based, fully functioning EHR that includes notes from primary care and subspecialty clinics; hospital discharge summaries; ICD-9 coded problem lists; health maintenance lists; medication prescribing; coded allergies; lab and radiographic results; and results management. The EHR has clinical decision support in the form of reminders for preventative services and management of chronic problems, medication prescribing alerts, and decision support during results management. The AVS is a functionality of the EHR and can be provided through one of several mechanisms: as a print out at the office, mailed to the patient home, and available via the patient portal. The adoption and use rate of the outpatient EHR by physicians is 100% across both AMCs as it is a requirement for affiliation with the AMCs.

3.2 Instrument Development

We developed a survey instrument to assess physician perceptions and beliefs about the AVS. Our development of the survey items drew upon several themes identified in the literature on the AVS and in physician evaluation of EHRs. Given the workflow challenge and time required to generate the AVS [4] one of the items focused on physician perception of the ease of generating and providing the AVS. Physician satisfaction has been evaluated with respect to EHRs as well as performing clinical functions with the EHR [8]. We developed an item to assess physician satisfaction with the AVS. We also created two items, one focusing on physician perception of the effectiveness of the AVS and the other on physician expectation of the AVS. Effectiveness of the AVS captures the physician’s overall rating of the AVS for providing a clinical summary of the office visit. Expectation of the AVS is the perception of the degree to which the AVS provides a clinical summary of the office visit. In the case of physician beliefs, we drew upon our previous work on beliefs about the impact of the EHR to develop items about the beliefs of the effect of the AVS on physician workload, patient outcomes, and the care the physician personally delivers [7]. For example, on the care the physician personally delivers we included items on quality of care, patient-centered care, and adherence to care. All items were captured as Likert scales. An additional measure we captured was the physician’s personal innovativeness in the domain of information technology (PIIT). PIIT is the willingness of the individual to try out new information technology [9]. PIIT is captured through four Likert items (coded Strongly Disagree to Disagree) as for example: “Among my peers, I am usually the first to try out new information technologies, and I like to experiment with new information technologies.” We had successfully used PIIT as a measure to distinguish levels of innovativeness in a patient population [10]. We wanted to explore whether the concept of PIIT can be applied to a physician population. Specifically, we wanted to assess whether physicians with greater levels of PIIT perceive that the AVS was easier to generate, effective, and have greater expectation of the AVS in capturing a clinical summary. Finally, through our survey instrument we gathered data on physician specialty (primary care, medical specialty, and surgical specialty), race, and number of outpatient hours worked per week. After developing the survey instrument we conducted a test of the survey with the four physician investigators on the study.

3.3 Data Collection

As described previously, our subjects were physicians who were eligible professionals at two academic medical centers (AMCs) in the northeast. We administered the survey using the Survey Monkey tool. We sent the initial survey followed by three reminders. To enhance the response rate, we offered an iPad to each of three randomly drawn respondents.

3.4 Statistical Analysis

For key variables of interest in our study such as perceptions of AVS and beliefs about the effect of the AVS, we present descriptive statistics using percentages. We and others have previously reported that primary care physicians and specialists differ in their perceptions of the impact of the EHR on domains like patient outcomes and the care the physician delivers [7,11]. To explore this finding in this study we present descriptive statistics for the overall population as well as categorized by physician specialty. In the case of beliefs about the effect of AVS, we created scales from the individual variables that captured the domains and report Cronbach’s Alpha for the scales. For example, we created a belief scale called Effect of the AVS on workload from the following individual belief items: workload of office staff, workload of physicians, time for an office visit, and workflow associated with office visit. We created three belief scales: effect of AVS on workload, effect of AVS on patient outcomes, and effect of AVS on care physician personally delivers. Our scales were an average of the scores of the items comprising that domain. We conducted bivariate analyses relating perceptions and belief scales with physician socio-demographic characteristics, physician specialty, and PIIT using the Pearson chi-square test and the t-test. We also conducted bivariate associations between perceptions and belief scales. Finally, we employed multiple regression analysis using a forced entry method to assess the relationship between socio-demographics, physician specialty, perceptions and belief scales. In this analysis we defined belief scales as the dependent variables and socio-demographics, physician specialty, and perceptions as the explanatory variables. As there are three belief scales we ran three regression models. A p-value less than 0.05 was considered statistically significant. Since this was an exploratory study, we did not adjust for multiple comparisons. Thus, the p-values should be interpreted cautiously.

3.5 Institutional Review Board Approval

Approval for the study was obtained from the Partners Healthcare Institutional Review Board.

4. Results

4.1 Response Rates

Of the 1 795 eligible providers at the two AMCs, 853 completed the survey for an overall response rate of 47.5%. Respondents and non-respondents did not differ on age but differed on gender and specialty. 46% of respondents and 54% of non-respondents were male physicians (p=0.04). 55% of respondents and 45% of non-respondents were primary care physicians (p=0.002).

Of the 853 respondents, 56.7% were male and 43.3% were female; 69% were less than 55 years of age and 31% were 55 years or older; and 72% were Caucasian. 22% of respondents were primary care physicians and 78% were specialists. 53.5% of respondents worked 20 hours or less per week and 46.5% worked more than 20 hours per week.

4.2 Physician perceptions of the AVS

Eighty percent of the respondents reported that the AVS was easy (very easy or quite easy or somewhat easy) to generate and provide to patients. One-fifth (20%) reported that the AVS was quite difficult or very difficult to generate and provide to patients. Ease of generating and providing the AVS (dichotomized as very easy to somewhat easy versus quite difficult to very difficult) was not associated with physician socio-demographic characteristics (gender, age, and race), specialty, or number of hours worked per week. With respect to physician personal innovativeness in the domain of information technology (PIIT), we created a scale with the four items that captured the measure which we labeled PIIT (Cronbach’s Alpha=0.88). Ease of generating and providing the AVS was not associated with PIIT.

With respect to physician expectation of the AVS, 61.4% reported that the AVS was a little below or far below their expectation of providing a clinical summary of the office visit. 31.6% reported that the AVS was where expected or a little above or far above their expectation. Expectation of the AVS (dichotomized as a little below or far below versus other) was not associated with physician gender, race, specialty, or hours worked per week. Expectation about the AVS was associated with age: 64% of physicians <55 years of age reported that the AVS is a little below or far below in expectation compared to 55% who are 55 and older (p=.02). Expectation was also significantly associated with PIIT: physicians who reported the AVS was a little below or far below expectation had a mean PIIT of 3.42 compared to mean PIIT of 3.28 for physicians in the other group (p=.03).

Over a third of physicians (35.6%) reported that the AVS was Poor in terms of effectiveness for providing a clinical summary of the office visit. 39.2% reported effectiveness was Fair, and 25.2% reported it as Other (good, very good, or excellent). Effectiveness (categorized as Poor, Fair, or Other) was not associated with socio-demographic characteristics nor hours worked per week. Effectiveness was associated with specialty: 38.4% of specialists reported that the effectiveness of the AVS is Poor compared to 24.9% of primary care physicians (p=.004). Effectiveness was associated with PIIT: physicians who reported Fair effectiveness had a mean PIIT (3.26) that was lower than physicians who reported Poor effectiveness (3.46) (p=.006). Additionally, in the Poor category, younger physicians had a greater mean PIIT (3.54) than older physicians (3.27) (p=.026), and male physicians had a greater mean PIIT (3.59) than female physicians (3.27) (p=.003). Similarly, in the Other category (Good/Very Good/Excellent), younger physicians and male physicians had a greater mean PIIT than older physicians and female physicians respectively.

Two-thirds of the physicians (66.2%) reported “A Little” or “Not At All” with respect to their overall satisfaction with the AVS. Satisfaction (dichotomized as A Little or Not At All versus Other) was not associated with socio-demographics nor hours worked per week. Satisfaction was also not associated with PIIT. Satisfaction was associated with specialty: 59% of primary care physicians reported A Little or Not At All in terms of satisfaction with the AVS compared to 68% of non-primary care physicians (p=.02).

4.3 Physician beliefs about the effect of generating and providing the AVS on workload, patient outcomes, and the care they personally deliver

More than three-fourths of the physicians reported a negative effect of generating and providing the AVS on workload of office staff (78.4%) and workload of physicians (76.1%) (►Table 1). Eighty one percent of physicians reported that generating and providing the AVS will negatively affect the work flow associated with the office visit. 46.6% reported there would be no effect and 43.4% reported a positive effect of the AVS on patient satisfaction with the office visit. 57.5% of the physicians reported that there would be no effect of generating and providing the AVS on patient-physician interaction during the office visit.

Table 1.

Physician beliefs about the effect of generating and providing the AVS on workload and patient outcomes

Response categories	Specialty	Beliefs*
		Workload of office staff	Workload of physicians	Time for an office visit	Workflow associated with office visit	Patient satisfaction with office visit	Patient harm and potential liability	Patient-physician interaction during office visit
Negative effect (-1 or-2)	Specialty	80.0%	75.2%	67.6%	81.1%	11.9%	27.9%	24.6%
	Primary Care	72.7%	79.2%	73.1%	81.5%	3.5%	18.6%	14.1%
	All	78.4%	76.1%	68.8%	81.2%	10.1%	25.9%	22.3%
No effect	Specialty	16.6%	19.1%	28.4%	14.2%	50.7%	56.4%	59.2%
	Primary Care	23.8%	17.9%	22.8%	15.0%	31.8%	55.2%	51.2%
	All	18.2%	18.8%	27.2%	14.4%	46.6%	56.2%	57.5%
Positive effect (+1 or+2)	Specialty	3.4%	5.7%	3.9%	4.7%	37.4%	15.7%	16.3%
	Primary Care	3.5%	2.9%	4.1%	3.5%	64.7%**	26.2%***	34.7%**
	All	3.4%	5.1%	4.0%	4.5%	43.4%	18.0%	20.3%
Mean (SD)	Specialty	-1.01 (0.75)	-0.95 (0.84)	-0.79(0.77)	-1.03(0.78)	0.29(0.83)	-0.18(0.83)	-0.14(0.79)
	Primary Care	-0.87 (0.76)	-1.01 (0.76)	-0.84(0.72)	-0.98 (0.72)	0.73 (0.76)	0.09(0.88)	0.23(0.81)
	All	-0.97 (0.76)	-0.97 (0.82)	-0.80 (0.75)	-1.02(0.77)	0.38(0.83)	-0.12(0.85)	-0.08(0.81)

*Individual items on the survey

**p=.000 (specialty vs primary care)

***p=.002 (specialty vs primary care)

In the case of specialty, 79.2% of primary care physicians reported a negative effect of the AVS on physician workload compared to 75.2% of specialists but the difference was not significant (►Table 1). However, primary care physicians and specialists differed on all belief items related to patient outcomes. 64.7% of primary care physicians reported that the effect of AVS on patient satisfaction will be positive compared to 37.4% of specialists (p<.001).

We created two scales of the belief items in ►Table 1. The first scale was called Effect of AVS on workload (Cronbach’s Alpha = 0.84) and consisted of the first four items: workload of office staff, workload of physicians, time for an office visit, and workflow associated with office visit. The second scale was called Effect of AVS on patient outcomes (Cronbach’s Alpha=0.70) and consisted of the last three items in ►Table 1: patient satisfaction with office visit, patient harm and potential liability, and patient-physician interaction during office visit.

Effect of AVS on workload was significantly associated with gender: mean effect of generating and providing the AVS on workload was –1.0 for female physicians compared to –0.89 for male physicians (p=.02). Effect on workload was also significantly associated with number of hours worked per week: physicians who worked more than 20 hours had a mean effect of –1.0 compared to –0.80 for physicians who worked 20 hours or less per week (p=.04). Effect of AVS on patient outcomes was significantly associated with only item, specialty of the physician: mean effect of generating and providing the AVS on patient outcomes for primary care physicians was 0.35 compared to –0.01 for non-primary care physicians (p=.000).

In terms of the effect of the AVS on the care the physician personally delivers, two-thirds of physicians believed that generating and providing the AVS will have no effect on outcomes of care (66.2%) and continuity of care (66.3%) (►Table 2). 58.7% believed that generating and providing the AVS will have no effect on quality of care and 54.4% believed that there will be no effect on patient adherence to care. Primary care physicians differed significantly from specialists on all belief items related to the effect of the AVS on the care they deliver except for efficiency of care. For example, 45.1% of primary care physicians reported a positive effect of AVS on quality of care compared to 23.6% of specialists (p=.000).

Table 2.

Physician beliefs about the effect of generating and providing the AVS on care the physician personally delivers

Response categories	Specialty	Beliefs*
		Effectiveness	Patient-centered care	Outcomes	Quality	Continuity	Patient adherence	Efficiency
Negative effect (-1 or -2)	Specialty	19.9%	18.1%	13.0%	14.7%	10.5%	9.2%	56.6%
	Primary Care	12.1%	9.8%	4.6%	6.9%	2.9%	4.7%	56.3%
	All	18.1%	16.3%	11.2%	13.0%	8.8%	8.2%	56.5%
No effect	Specialty	58.6%	52.8%	68.6%	61.7%	68.1%	58.8%	35.8%
	Primary Care	43.1%	35.3%	57.8%	48.0%	59.6%	39.0%	32.2%
	All	55.2%	48.9%	66.2%	58.7%	66.3%	54.4%	35.0%
Positive effect (+1 or +2)	Specialty	21.5%	29.2%	18.4%	23.6%	21.4%	32.0%	7.6%
	Primary Care	44.8%**	54.9%**	37.6%**	45.1%**	37.4%**	56.4%**	11.5%
	All	26.6%	34.8%	22.6%	28.3%	24.9%	37.4%	8.5%
Mean (SD)	Specialty	-0.04(0.85)	0.06 (0.88)	0.01 (0.72)	0.05 (0.79)	0.09(0.72)	0.22 (0.76)	-0.64 (0.86)
	Primary Care	0.37 (0.86)	0.55(0.89)	0.39(0.72)	0.46 (0.78)	0.41 (0.71)	0.60 (0.76)	-0.59 (0.94)
	All	0.05(0.86)	0.16(0.90)	0.09(0.72)	0.14(0.80)	0.16(0.72)	0.30 (0.77)	-0.63 (0.88)

*Individual items on the survey

**p=.000 (specialty vs primary care)

We computed a scale of all the items in ►Table 2 and labeled it Effect on care physician personally delivers (Cronbach’s Alpha = 0.93). Effect on care physician personally delivers was significantly associated with specialty: mean effect of generating and providing the AVS on the care they personally deliver was 0.31 for primary care physicians compared to –0.03 for non-primary care physicians (p<.001).

4.4 Relationships between perceptions of AVS and belief scales about the effect of generating and providing the AVS

Physician perceptions of the AVS were significantly related to belief scales about the effect of generating and providing the AVS on workload, patient outcomes, and the care the physician personally delivers (►Table 3). The relationships were also in the expected direction. For example, mean effect of AVS on workload was –0.81 when the AVS was perceived as somewhat easy or quite easy or very easy to generate and provide compared to the mean effect of –1.45 when the AVS was perceived as quite difficult or very difficult to generate and provide. Similarly, mean effect of AVS on patient outcomes was 0.53 when satisfaction with AVS was somewhat or quite a bit or very much compared to –0.18 when satisfaction with AVS was a little or not at all. Finally, mean effect of AVS on care physician personally delivers was –0.44 when physicians perceive the effectiveness of the AVS to be poor or fair compared to 0.60 when physicians perceive the effectiveness of the AVS to be good or very good or excellent (p=.000).

Table 3.

Relationship between perceptions of AVS and belief scales about the effect of generating and providing the AVS on workload, patient outcomes, and care physician personally delivers*

Perceptions	Response categories	Belief scales about the effect of generating and providing the AVS on
		Workload			Patient Outcomes			Care physician personally delivers
		Specialty	Primary Care	All	Specialty	Primary Care	All	Specialty	Primary Care	All
Ease of generating and providing the AVS	Quite Difficult or Very Difficult	-1.46(0.47)	-1.46(0.47)	-1.45(0.47)	-0.33 (0.69)	0.01 (0.65)	-0.26 (0.69)	-0.48 (0.73)	-0.17(0.69)	-0.41 (0.73)
	Somewhat Easy or Quite Easy or Very Easy	-0.82 (0.63)	-0.79(0.55)	-0.81 (0.61)	.07 (0.62)	0.45 (0.58)	0.15(0.63)	0.07(0.61)	0.45 (0.57)	0.15(0.61)
Expectations of the AVS	A Little Below or Far Below	-1.11 (0.61)	-1.09 (0.56)	-1.10(0.60)	-0.20 (0.63)	0.19(0.61)	-0.13(0.65)	-0.24 (0.66)	0.07(0.55)	-0.18(0.65)
	Where Expected or A Little Above or Far Above	-0.65 (0.62)	-0.71 (0.60)	-0.68 (0.62)	0.33 (0.53)	0.55(0.55)	0.37 (0.54)	0.31 (0.52	0.60 (0.65)	0.37 (0.56)
Effectiveness of the AVS	Poor	-0.48 (0.64)	-1.30(0.59)	-1.25(0.59)	-0.45 (0.60)	-0.02 (0.65)	-0.38 (0.62)	-0.48 (0.64)	-0.24 (0.54)	-0.44 (0.63)
	Fair	0.05 (0.47)	-0.94 (0.53)	-0.89(0.56)	0.08 (0.46)	0.23 (0.49)	-0.12(0.47)	0.05 (0.47)	0.22 (0.43)	0.09 (0.46)
	Good or Very Good or Excellent	0.52 (0.49)	-0.59(0.54)	-0.59(0.62)	0.56 (0.47)	0.78 (0.47)	0.60 (0.49)	0.52 (0.49)	0.87(0.55)	0.60(0.52)
Satisfaction with the AVS	A Little or Not at All	-1.12(0.60)	-1.15(0.54)	-1.12(0.58)	-0.24(0.59)	0.08 (0.56)	-0.18(0.60)	-0.28 (0.62)	-0.04 (0.48)	-0.23 (0.60)
	Somewhat or Quite a Bit or Very Much	-0.58(0.61)	-0.59(0.54)	-0.57 (0.60)	0.48 (0.48)	0.72 (0.50)	0.53 (0.50)	0.47 (0.46)	0.80(0.52)	0.55(0.50)

*Perceptions of AVS are individual items on the survey; Belief scales about the effect of generating and providing the AVS are scales derived from individual belief items on the survey (see Section 4.3); Belief scales range from -2 to +2; Cells in the table represent mean (SD) for the belief scale; p=.000 for all results shown in the table

To further explore predictors of belief scales we fitted a series of multiple linear regressions (forced entry method) with sociodemographics, physician specialty, PIIT, and perceptions as the explanatory variables and belief scales as the dependent variables. We fitted three regression models, one for each of the three belief scales (►Table 4). In the case of effect of AVS on workload, perception of the ease of generating and providing the AVS was the most significant predictor followed by satisfaction with the AVS. In the case of effect of AVS on patient outcomes and the care the physician personally delivers, satisfaction with the AVS was the most significant predictor followed by perception of the effectiveness of the AVS. Physician specialty remained a significant predictor in the multivariate models in the case of effect of AVS on patient outcomes and effect of AVS on care physician personally delivers.

Table 4.

Results of multiple regression*

Predictors	Dependent variables: Belief scales about the effect of generating and providing the AVS
	Model 1: Workload	Model 2: Patient outcomes	Model 3: Care physician personally delivers
Ease of generating and providing the AVS	0.319 (0.000)	0.055 (0.084)	0.108 (0.000)
Expectations of the AVS	0.076 (0.066)	0.063 (0.106)	0.071 (0.051)
Effectiveness of the AVS	0.111 (0.022)	0.270 (0.000)	0.242 (0.000)
Satisfaction with the AVS	0.218 (0.000)	0.289 (0.000)	0.368 (0.000)
Gender	-0.052 (0.104)	-0.037 (0.226)	0.034 (0.221)
Age	0.008 (0.796)	0.004 (0.885)	0.002 (0.948)
Race	0.010 (0.736)	0.061 (0.037)	0.022 (0.411)
Hours worked per week	-0.075 (0.017)	-0.109 (0.000)	-0.081 (0.003)
Specialty	-0.010 (0.759)	0.168 (0.000)	0.135 (0.000)
PIIT	-0.044 (0.164)	0.010 (0.738)	0.020 (0.465)
R Square (Sig. F Change)	0.343 (0.000)	0.414 (0.000)	0.499 (0.000)

*Cells in table represent Standardized Beta Coefficients (Sig.)

5. Discussion

Four of five physicians studied reported that the AVS was easy to generate and provide to the patient. At the same time, many physicians had low expectation and satisfaction with the AVS. Additionally, most physicians believed that generating and providing the AVS had a negative effect on their workload and had no effect on outcomes of care or continuity of care. In ►Table 5, we present a sampling of comments from physicians to further illustrate these beliefs.

Table 5.

Sampling of physician comments on the effect of AVS on workload, patient outcomes, and care delivered

This does not improve patient care in the least. In a high volume practice setting, reconciling a number of different medications that have no relevance to their visit that day is a waste of resources and can be incredibly time consuming, particularly when dealing with elderly patients. It is also confusing for the patient. There have been several instances where patients have complained to us about info that is on the AVS that they took issue with; in each case, that info related to problem list items and/or meds that were entered by another clinician in another specialty. I have not seen a single instance where the AVS improved patient care; it certainly does not improve office efficiency as it increases the paperwork burden.

The AVS seems to simply be a way for the institution to get federal money. It has VERY little to do with the quality of care delivered.

I do not think the AVS improves patient care and can be actually harmful to my patient population (listing depression, obesity, prior abortion / termination) can be hurtful to a patient when this information was provided to you in confidence but not for everyone to see easily on a print out.

AVS is a total and utter waste of time, energy and money. It does not add to quality of care or patient safety.

It doesn’t include ANYTHING recurrent visit except newly prescribed medications. It takes a lot of time to generate and I don’t think it’s very effective. Further, to meet MU criteria, we have to hand it out even when patients don’t speak or read English. Thus, it’s time consuming and I don’t think it improves quality of care or patient safety.

It takes too long to generate the EOV and AVS. I end up spending more time facing the computer screen than the patient. Takes me about 3 extraminutes per visit to generate the EOV AVS which in a session of 14 patients, time adds up.

Our findings point to a dichotomy between a core measure of the EHR Meaningful Use program, the provision of a clinical summary of the office visit to patients, and physician perceptions and beliefs about the effect of generating and providing the AVS. The intent of the proponents of meaningful use with respect to the provision of the AVS was to promote patient access to health information and enhance patient engagement in their health care [2]. However, the negative impacts of generating and providing the AVS on workload from a physician perspective were not addressed in the meaningful use regulation. Furthermore, the AVS measure in meaningful use does not specify any requirements related to patient characteristics. To meet the meaningful use measure patients are likely to receive the AVS without consideration of issues related to language, illiteracy, and limited cognitive abilities that may influence patient ability to interpret the information in the AVS. In this context it is perhaps not surprising that physicians in our study believed that there was no effect of generating and providing the AVS on patient outcomes and care delivery. As Rogers has pointed out, innovations can be associated with positive and negative consequences [12]. Our findings suggest the need to study both of these types of consequences associated with implementing the requirements of the EHR Meaningful Use Program [13].

In the case of socio-demographic characteristics, we generally did not find physician gender, age, or race to be associated with perceptions or beliefs. These findings are similar to our study on physician beliefs about the impact of the EHR on quality of care and efficiency of care [7]. Additional research is needed to determine the generalizability of our findings and assess whether there is an association between socio-demographic characteristics and physician perceptions and beliefs about the effect of specific measures of the meaningful use of EHR program.

We also did not find that physician personal innovativeness in information technology (PIIT) was associated with perceptions and beliefs with the exception of two perception items: expectation of the AVS and effectiveness of the AVS. Physicians with a higher level of PIIT rated the AVS as a little below or far below in expectation of providing a clinical summary of the office visit. Physicians with a higher level of PIIT also rated effectiveness of the AVS as poor. These findings seem to suggest that physicians with a higher level of innovativeness are seeking a better designed AVS, one that would improve both expectation and effectiveness. These findings may also be linked to the role of PIIT as a moderator of the consequence of perceptions. According to Agarwal and Prasad [9], in this moderating role of PIIT individuals with higher PIIT could be associated with poorer positive perceptions than individuals with lower PIIT assuming the same level of usage intentions (in our case the same level of AVS use intention to meet the requirements of meaningful use). There is a need for additional empirical and theoretical studies on the relationship between the level of innovativeness in physicians and their perceptions of the AVS as well as other requirements of the EHR Meaningful Use program.

Physician specialty was associated with effectiveness of the AVS, satisfaction with the AVS, and effect of generating and providing the AVS on patient outcomes and the care the physician personally delivers. Primary care physicians had more positive perceptions and beliefs than specialists in all four areas. We and others had previously reported that primary care physicians had more positive beliefs about the impact of the meaningful use of the EHR on quality of care and efficiency of care than specialists [7,11]. It is possible that these global beliefs about the impact of the meaningful use of the EHR carry over to specific measures such as the provision of the AVS. Particularly with respect to the AVS, the negative beliefs held by specialists may be driven by the opinions of specialists that they should not be responsible for the generation and provision of the AVS and its associated tasks such as medication and problem reconciliation and that these are best left to primary care physicians. As one specialist in our study commented: “There is nothing wrong with the VSR (visit summary report) process (generating or delivery). However, there is no PURPOSE served by the VSR to my practice or to my patients. If the VSR was actually meaningful, then it would be worth the extra work and infrastructure to deliver it. I think there might be a role for the VSR / reconciled med list in the primary care setting. I don’t think there is a role for it as it exists in the surgical subspecialty”. Similarly another specialist noted: “Eliminate the requirement, at least for specialty care. The relative value of spending over half an office visit reconciling primary care meds at a specialty visit is extremely low”.

Our findings on the influence of perceptions on beliefs make intuitive sense. In particular, physician satisfaction with the AVS was a significant predictor of all the beliefs. Satisfaction was the most important predictor of the effect of the AVS on the care the physician personally delivered and it was a significant predictor of the effect of the AVS on patient outcomes. This suggests the importance of ensuring physician satisfaction with the AVS.

Our study is one of the first to assess physician perceptions about the AVS and beliefs about the effect of the AVS, and was exploratory in nature. In terms of generalizability, we suspect that the negative beliefs about the effect of the AVS on physician and staff workload are likely to be found in other settings using vendor EHRs. A qualitative study of physicians using a vendor EHR found similar concerns about the amount of time required to keep elements of the AVS such as the medication list current [6]. However there is a need for additional studies in non-AMC and private practice settings and those with vendor EHRs. There is also a need to apply theoretical models such as the diffusion of innovations to the study of AVS. One area that could be particularly useful is the concept of “reinvention” suggested by Rogers in his diffusion of innovation model [12]. When innovations have negative consequences they are sometimes reinvented to mitigate the consequences. In the case of the AVS, reinvention could take place through alternative workflows such as nurses providing the AVS to patients. This could address the negative beliefs among physicians, especially specialists, to the AVS. Finally, while this study focused on physician perceptions there is a need for studies on patient perceptions of the AVS. For example, how do patients perceive the effectiveness of the AVS and how satisfied are patients with the AVS? Are there differences in perceptions and satisfaction when patients receive an AVS at a primary care visit compared to a specialist vist?

6. Conclusions

This is one of the first studies to report physician perceptions and beliefs about generating and providing the AVS as required by the Meaningful Use of EHR Program. We found that while achieving the core measure requirement of generating and providing the AVS was easy for physicians it did not necessarily translate into positive beliefs about the effect of the AVS on patient outcomes and the care the physician personally delivered, though most had probably not assessed the extent to which their patients find it helpful. It may take time for physicians to realize the effect of the AVS on patient outcomes and the care they deliver. One study reported that it took a number of years before physicians in the United Kingdom developed positive beliefs that a computerized record system improved quality of care and helped them practice better medicine [14]. While characteristics of patients receiving the AVS such as language and illiteracy contribute to the lack of positive beliefs among physicians, the design of the AVS could also be a factor in the beliefs that physicians hold. Simply providing a laundry list of elements on the AVS to meet the meaningful use measure may not be sufficient. One study noted that when graphical trends of response to therapy were presented in the AVS it motivated patients to adhere to their therapy [15]. Physician input can be sought into ways in which the AVS can be re-designed to provide better summary information from the office visit while at the same meeting the meaningful use requirement. This could in turn promote positive beliefs among physicians about the effect of the AVS on patient outcomes and care delivered.

Our study found significant negative beliefs among physicians about the effect of generating and providing the AVS on physician workload. Other studies have reported similar negative beliefs on workflow with respect to physician use of EHRs, even among physicians considered as super-users of the EHR [16–18]. The factors contributing to the negative beliefs in these studies included additional documentation burden on physicians in the EHR environment and a failure to address physician needs with respect to workflow and practice support. In our study, a factor influencing the negative beliefs of physicians to the AVS, especially among specialists, was that generating and providing the AVS and associated tasks such as medication reconciliation was the responsibility of the primary care physician. It remains to be seen whether specialist perceptions and beliefs about the AVS change if they receive feedback on the extent to which the AVS is helpful to their patients.