Provider Attitudes Toward Public-Private Collaboration to Improve Reminder/Recall: A Mixed-Methods Study

Abstract

Objective

To assess primary care providers’ current reminder/recall practices, preferences for collaboration with health departments in reminder/recall efforts, attitudes toward practice-based and population-based reminder/recall, and experiences with a population-based reminder/recall intervention.

Methods

Providers responsible for making decisions about immunization delivery at all primary care practices that participate in the Colorado Immunization Information System were surveyed. Data collection was preceded by an intervention in which half of 14 counties received a population-based reminder/recall intervention conducted by the health department. Practice staff involved in immunization activities were then selected for semi-structured telephone interviews based on the location of their practice within specified strata, including whether they were in the intervention counties, urban/rural location, and practice type.

Results

282/383 (73.6%) of providers responded to the survey and 253 who administered vaccines to children 19-35 months were retained; 82 staff members at 36 practices were interviewed. Providers’ preferences for who should conduct reminder/recall were almost evenly split, with slightly more indicating that it should be conducted by the health department. Cost and feasibility issues were perceived barriers to conducting practice-based recall, particularly among urban practices. Support for population-based reminder/recall was highest among rural practices. Concern about perceived inaccuracies in immunization registry data was the major barrier to conducting population-based reminder/recall. The population-based intervention did not create an undue burden on practices.

Conclusions

A collaborative approach to reminder/recall involving both providers and health departments is preferable for many providers and may be a viable solution to the barriers of practice-based reminder/recall.

Reminder or recall messages (reminder/recall) are an effective method of improving timeliness and completion of recommended immunizations to prevent disease, typically increasing immunization coverage rates 5-20%.¹ All types of reminder/recall methods have been found to be effective among different age groups and within a variety of settings.^2-6 Reminder/recall is sometimes conducted with the help of immunization information systems (IIS), or immunization registries, which contain immunization records of all children within a geographic area.⁷ Use of such registries improves the effectiveness of reminder/recall and significantly improves immunization delivery.^4,5,8

Despite ample evidence of the effectiveness of reminder/recall, relatively few private providers initiate and sustain reminder/recall activities. This is unfortunate given that the vast majority of immunization-related activities now take place in the private, rather than the public, sector.^8-10 In one national study, less than 20% of private providers reported using a reminder/recall system;⁸ in another study, only 16% of private pediatricians routinely used reminder/recall messages, compared to 51% of public clinics.¹¹ There are significant barriers to practice-based reminder/recall, including time constraints,^7,11,12 cost,^7,11 need for additional training and/or staff time,⁷ concerns about ease and functionality of the reminder/recall system,^7,13 concerns about confidentiality and Health Insurance Portability and Accountability Act (HIPAA) privacy and security,⁷ lack of coordination between the practice’s clinical and administrative systems,^7,11 and distrust in the accuracy of immunization registry data.¹³

One solution to overcoming some of the barriers to practice-based reminder/recall may be collaboration between the public and private sectors. A recent Institute of Medicine report emphasized the need for collaborative efforts between primary care and public health in order to improve population health.¹⁴ Such collaboration could include improving the safety, development, and/or delivery of vaccines.¹⁵ Public health departments or other public entities might include patients served by private providers within a larger population-based reminder/recall effort. One study has suggested that providers might view population-based reminder/recall as an acceptable alternative to practice-based reminder/recall, given the significant barriers to practice-based efforts.¹³ Private practices could collaborate by providing public health departments with updated demographic and vaccination data for their patients to make population-based reminder/recall more effective. Such collaboration would benefit efforts to improve contact with patients, given the high number of invalid or undeliverable mailing addresses encountered in reminder/recall efforts..³

Little is known about providers’ attitudes toward public-private collaboration to improve reminder/recall. This study uses a mixed-methods approach in order to build on previous findings that providers view population-based reminder/recall to be generally acceptable.¹³ We employ survey data and interview data to assess primary care providers’ current reminder/recall practices, preferences for collaboration with public health departments in reminder/recall efforts, attitudes toward practice-based and population-based reminder/recall, and experiences with a population-based reminder/recall intervention.

Methods

Overview

This study draws on quantitative and qualitative data gathered via survey and key informant interviews. Data collection was preceded by an intervention in which half of the 14 counties in Colorado received a population-based reminder/recall intervention conducted by the Colorado health department. The Colorado Multiple Institutional Review Board approved this protocol.

Intervention

Seven (three urban, four rural) of the 14 counties in this study received a population-based reminder/recall intervention conducted by the Colorado state public health department in summer 2010.¹⁹ All practices in these counties received a joint letter from the Colorado Department of Public Health and Environment (CDPHE), the Colorado Immunization Information System (CIIS), and the Children’s Outcomes Research group notifying them of the intervention. The health department used the CIIS to identify children aged 19-35 months within the designated counties who were overdue for immunizations, then mailed a reminder/recall letter to the parents of those children. The letters included the logo of the local county health department. Up to 3 mailings (1 letter, 2 postcards) were sent to parents across a three month period; children who became up-to-date between mailings were removed from the mailing list.

Study Population and Data Collection

Survey administration was conducted in October and November 2010. A paper-based, self-administered survey and a $10 incentive were mailed to the providers who self-identified as being responsible for making decisions about immunization delivery at all primary care practices in Colorado. Practices were drawn from a 2009 CIIS survey of all primary care practices in the state. Survey questions were developed by the study team based on previous immunization survey instruments and were pilot-tested with pediatric and family medicine providers. These questions asked for information about the practice county; respondents’ position within the practice; practice participation in CIIS; practice specialty, type, and size; characteristics of patient population; previous reminder/recall practices to parents of children in need of immunizations; and beliefs about how reminder/recall should be conducted. Practices in counties included in the population-based intervention were also asked questions about their experience with the intervention. Using a modified Dillman methodology, an approach incorporating follow-up with non-responders, a reminder postcard was mailed to each practice five days after the survey was mailed; non-responders received up to two additional surveys.

Following recommendations for effective mixed methods research,^16-18 quantitative and qualitative data collection were purposively integrated. Interviewees were drawn from the surveyed population. Practice staff were selected for interviews based first on whether their county had been involved in the population-based recall. Those in counties participating in the population-based recall were then stratified by whether they were an urban family medicine practice, an urban pediatric practices, or a rural practice family medicine practice. Within each strata, practices were then randomly sampled and recruited for interviews. Semi-structured interview guides were designed to complement and expand survey data. Telephone interviews were conducted in October and November 2010 with 2-3 staff members at each sampled practice who self-identified as being involved in immunization policy or activities. At each practice, interviewees included at least one senior managing physician/partner as well as an office manager or practice administrator. Interview questions were designed to probe practices’ previous reminder/recall practices to parents of children in need of immunizations, and beliefs about and preferences for how reminder/recall should be conducted. Practices in counties included in the population-based intervention were also asked questions designed to probe their experiences with and perceptions of the intervention, including patient response and impact on the practice. Each interview lasted approximately 25-30 minutes. All interviews were digitally recorded and transcribed verbatim.

Data Analyses

Characteristics of respondents and survey questions were reported using descriptive statistics. To assess whether responses differed depending on rural/urban status, chi-squared analyses were performed. All analyses were done using SAS Version 9.2 (SAS, SAS Institute, Cary, North Carolina).

Analysis of the key informant interviews was an iterative process involving established qualitative content methods and reflexive team analysis.^20-22 Four members of the study team read interview transcripts multiple times in order to achieve immersion, then engaged in initial coding using an emergent rather than a priori approach.²¹ After initial codes were established, the team met regularly to discuss emergent new codes and themes, confirm inter-coder reliability, and assess the preliminary results of the analysis process.^23,24 The qualitative data software program ATLAS.ti v.6.0 was used for data organization and management.

Results

Sample Characteristics

282 of 383 (73.6%) of primary care providers, each representing one primary care practice, responded to the survey. Of those, 253 indicated that their practice administered vaccines to children in the age group relevant to this study (ages 19-35 months) and were retained in the final cohort. Rural providers comprised 13.8% of the cohort. Approximately one-third of all responding practices had conducted some type of reminder/recall in the previous 12 months, although this percentage was significantly higher among rural practices than among urban practices. Table 1 depicts the characteristics of the surveyed practices.

Table 1.

Characteristics of Surveyed Practices

Question	Total N=253 (%)	Rural Col N=35 (%)	Urban Col N=218 (%)
Intervention Practices	128 (50.6%)	18 (51.4%)	110 (50.5%)
Physician Specialty* (p=0.008)
Pediatrics	56 (22.1)	6 (17.1)	50 (22.9)
Family Practice	170 (67.2)	20 (57.1)	150 (68.8)
RHC / IM / CH	27 (10.7)	9 (25.7)	18 (8.3)
Practice Type
Private practice	196 (77.8)	23 (67.6)	173 (79.4)
Group/HMO/MCO	18 (7.1)	2 (5.9)	16 (7.3)
CHC	20 (7.9)	5 (14.7)	15 (6.9)
PH Clinic	3 (1.2)	1 (2.9)	2 (0.9)
Other	15 (6.0)	3 (8.8)	12 (5.5)
Provider Gender
Female	152 (60.6)	17 (48.6)	135 (62.5)
Estimated proportion of patients <3 years old
0%	2 (0.8)	0 (0.0)	2 (0.9)
1-9%	123 (48.8)	14 (40.0)	109 (50.2)
10-24%	81 (32.1)	11 (31.4)	70 (32.3)
25-49%	40 (15.9)	9 (25.7)	31 (14.3)
<=50%	6 (2.4)	1 (2.9)	5 (2.3)
Estimated proportion of patients with Medicaid or CHP+*
< 10% (less than, not less than or equal to)	131 (55.3)	9 (26.5)	122 (60.0)
≥10%	106 (44.7)	25 (73.5)	81 (39.9)
Estimated proportion of patients uninsured
< 10%	170 (69.1)	21 (63.6)	149 (70.0)
≥10%	76 (30.9)	12 (36.3)	64 (30.0)
Estimated proportion of patients who are Black/African American (8% missing)* (M- H)
0%	9 (3.9)	7 (26.9)	2 (1.0)
1-9%	172 (74.1)	19 (73.1)	153 (74.3)
≥10%	51 (22.0)	0 (0.0)	51 (24.8)
Estimated proportion of patients who are Hispanic/Latino* (M-H)
1-9%	67 (27.2)	8 (23.5)	59 (28.4)
10-24%	79 (32.6)	4 (11.8)	75 (36.1)
25-49%	61 (25.2)	12 (35.3)	49 (23.6)
50-74%	30 (12.4)	8 (23.5)	22 (10.6)
75-100%	5 (2.1)	2 (5.9)	3 (1.4)
Participate in Colorado Immunization Information System (CIIS)*
Yes	171 (69.2)	30 (85.7)	141 (66.5)
In the past 12 months, sent out reminder/recall letters, postcards or made telephone calls to parents of children in your practice who were in need of immunizations
Yes	95 (37.8)	18 (51.4)	77 (35.6)
Method usually used to generate a reminder/recall list to contact parents about needed childhood immunizations
CIIS	20 (21.1)	5 (27.8)	15 (19.5)
An Electronic Medical Record database	36 (37.9)	6 (33.3)	30 (39.0)
An administrative billing database	9 (9.5)	3 (16.7)	6 (7.8)
EMR OR billing	45 (47.4)	9 (50.0)	36 (46.8)
A paper-based or “tickler file”	33 (34.7)	2 (11.1)	31 (40.3)
Other	10 (10.5)	2 (11.1)	8 (10.4)
Checked more than one	11 (11.6)	0 (0.0)	11 (14.3)

^*Significant difference between rural and urban providers, P < 0.05

M-H denotes Mantel-Haenszel chi-square test.

Key informant interviews were conducted with 82 staff members at 36 primary care practices across fourteen counties. Interviews were conducted with 21 physicians, 13 nurses, 16 office managers, and 32 other staff members, including immunization coordinators, clinical managers, regional managers, medical assistants, clinical directors, clinic supervisors, clerical assistants, directors, and executive directors.

Preferences for Collaboration

Providers indicated an almost even split in their preferences for who should conduct reminder/recall. When asked to take into account feasibility, financial issues, privacy issues and beliefs about provider responsibility, almost half (47.2%) of providers indicated that their practice should conduct reminder/recall, while slightly more (49.2%) indicated that the health department should conduct it. However, salient differences emerged between rural and urban practices (Figure 1). Though preferences among urban practices were divided relatively evenly, a much higher percentage of rural practices preferred the public health department conduct reminder/recall. Among the 49.2% of providers overall who preferred that health departments be involved in conducting reminder/recall, the preferred type of collaboration varied. The highest percentage preferred to provide updated immunization and contact information to the health department and for the health department letterhead to send out notices on their letterhead, without the practice’s name. Roughly one-third preferred to provide updated immunization data and contact information to the health department and then have the health department send out the reminder/recall letter on the practice’s letterhead. The remaining providers preferred that their practice have no involvement in the reminder/recall effort at all. Only 13% of rural practices preferred no involvement at all, compared to 29.9% of urban practices.

Figure 1.

How should reminder/recall be conducted?

Our ability to further tease out differences within rural and urban practices by practice type was constrained by the limitations of the data. Most rural practices in Colorado are family medicine practices; only a very few (n=6) in this study were pediatric practices (see Table 1). Because this small number is not sufficient for meaningful statistical analysis, we combined all (i.e., rural and urban) pediatric practices into a single category and compared this category to rural and urban family medicine practices. Compared to our reference category of urban family medicine practices, rural family medicine practices were less likely to indicate that their office should conduct reminder/recall (OR 0.60, 95% CI 0.21-1.69), while pediatric practices were more likely to indicate that their office should conduct reminder/recall (OR 2.05, 95% CI 1.07-3.90). While these data do not allow us to determine whether the rural-urban difference is driven by practice type, they clearly show that there are significant differences between pediatric, rural family medicine and urban family medicine practices with regard to reminder/recall preferences and that rural-urban differences remain salient within family medicine practices.

The interview data indicated support for a variety of public-private collaboration types, including those discussed above. Collaboration was seen by many interviewees, particularly those located in rural areas, as a welcome opportunity to better their practice. As one physician in a rural family medicine practice put it,

“I think [collaboration] would be great. Because sometimes we don’t necessarily have all the resources or we don’t exactly know how best to run something, so when you are collaborating with the state…[the state] knows what works and what doesn’t, and [they] can give suggestions as to how best to make something like this work.”

Interviewees also suggested additional ways to collaborate. One suggested possibility was for the health departments to send the reminder/recall list for practices to review prior to conducting the reminder/recall to improve data accuracy. Another idea was a staggered approach, in which practices send an initial reminder/recall letter to patients, followed by a second letter by the health department.

A particularly striking finding emergent in the interview data was the difference in enthusiasm for public-private collaboration between rural and urban practices. While some practices in both rural and urban counties were open to collaboration with the health department for reminder/recall efforts, rural practices were much more likely to be open about combining efforts. Rural interviewees indicated more support for collaborative opportunities with the health department on reminder/recall efforts in part because their practices had often worked with the health departments in the past to achieve other health goals. As one rural physician said,

“Our health department, they also give immunizations and it gives the families an option to either do it at the health department or through us. It’s actually good. It enhances our different practices and immunization rates.”

In the words of a medical director at another rural practice,

“Our parents and kids seem to respond to our local public health [department] very well. I think just because they are a big part of our community. [We collaborate with them a lot.] We will have meetings and [health department staff] will come here to our facility or we’ll meet collectively, with everybody in the county. They are very active here.”

In contrast, most urban providers said relatively little about collaboration and reported little to no past collaborative experience with health departments or other public entities. Though most did not express opposition to the possibility of collaboration per se, the lack of existing care relationship with the health department created a barrier to envisioning and pursuing collaborative possibilities.

Attitudes Toward Practice-Based Reminder/Recall

As shown in Table 2, a majority of all surveyed practices indicated providers should be responsible for ensuring that children get immunized, and a similarly high percentage indicated the belief that their practice already does a good job of reminding parents when their child is in need of immunizations. Nonetheless, most providers did not prefer to send out reminder/recall notifications to their patients. Cost and feasibility issues were perceived barriers to conducting practice-based recall, particularly among urban practices.

Table 2.

Survey Responses for Practices Currently Administering Vaccines to Children Aged 19-35 Months

	Agree strongly N (%)	Agree somewhat N (%)	Disagree strongly/somewhat N (%)
It is the responsibility of a provider to make sure children get immunized	53 (21.2)	110 (44.0)	87 (34.8)
Our practice does a good job of reminding parents when their child is in need of immunizations	72 (28.7)	111 (44.2)	68 (27.1)
I would prefer to send out reminder/recall letter to our own patients about which immunizations they need rather than having the health department do this	29 (11.6)	80 (32.0)	141 (56.4)
It costs too much money for our practice to conduct reminder/recall for children in our practice in need of immunizations*	39 (15.7)	106 (42.7)	103 (41.5)
It is not feasible for our practice to send out reminder/recall notices to children in our practice who are in need of immunizations	28 (11.2)	90 (36.1)	131 (52.6)
It is the health department’s responsibility to make sure children are immunized	10 (4.0)	72 (29.1)	165 (66.8)
I would be bothered if/that the health department sent out reminder/recall letter and postcards to families in our practice	12 (4.8)	26 (10.4)	211 (84.7)
I support the health department sending reminder/recall letters to patients in our practice*	103 (41.4)	102 (41.0)	44 (17.6)
I think it is a violation of privacy for the public health department to contact parents regarding their children needing an immunization	6 (2.4)	25 (10.0)	219 (87.6)
It is NOT a good use of public money for the health department to send reminder/recall letters about immunizations to parents	18 (7.3)	45 (18.1)	185 (74.6)
I think reminder/recall letters sent by the health department would not be as effective in bringing patients in for immunizations as a letter from our practice	30 (12.0)	112 (45.0)	107 (42.9)
I think reminder/recall letters sent by the health department are the best way to reach the most people	42 (16.8)	121 (48.4)	87 (34.8)
The health department does not have the most accurate information about our patients’ immunizations*	43 (17.6)	119 (48.6)	83 (33.8)

^*Significant difference between rural and urban providers, P < 0.05

Interviews with practice staff provided deeper insight into the barriers to conducting practice-based reminder/recall. These data indicate that cost, time, and staffing were formidable barriers, as was some practices’ lack of training on how to conduct reminder/recall. Expense was perhaps the primary barrier for many interviewees. As one rural pediatrician put it,

“The drawback for my practice is the expense… it is very expensive for us to do on a regular basis.”

Others emphasized the intertwined nature of cost, time, and staffing issues. In the words of one urban family medicine physician,

“It’s not like we can afford to have a whole bunch of people working on extra projects… vaccines are not necessarily something that we make money on. I think if [reminder/recall] could be done without a tremendous amount of, you know, hours spent, then I think it would actually be a nice thing.”

Because of such barriers, and given the resources needed, many viewed practice-based reminder/recall as a “low return on investment,” as one provider put it. However, the presence of these barriers did not mean that reminder/recall was not viewed as important by primary care practices. As one nurse in a rural family medicine practice said,

“I think in a smaller practice you don’t have the manpower or time management gets to be an issue, but I do think it is necessary and really should be made a priority.”

Attitudes Toward Population-Based Reminder/Recall

Providers’ attitudes toward population-based reminder/recall are also depicted in Table 2. Most indicated that they would not be bothered if the health department conducted reminder/recall and a similarly high percentage indicated support for health departments sending reminder/recall letters to patients in their practice. Support among rural practices was particularly high. Providers also rejected potential arguments against population-based reminder/recall, for example that health department involvement would constitute a violation of privacy or would not be a good use of public funds.

Although a slight majority of practices believed that reminder/recall letters sent by the health department would be less effective than letters sent by the practice, a larger majority believed that reminder/recall letters sent by the health department are the best way to reach the most people and might help increase immunizations among those without a usual source of care. However, a majority of surveyed providers, particularly those in urban practices, also indicated doubts about the accuracy of health departments’ immunization data.

Interviewees also indicated that perceived inaccuracies in immunization status/history and patient contact information in CIIS data were significant barriers to conducting population-based reminder/recall. As one urban pediatrician noted,

“I’ve gotten letters from state [health departments]… telling me this child is deficient [regarding immunizations] and it turns out I don’t even have that patient, or the patient isn’t [missing immunizations]. So I just think the accuracy is what’s missing. I think if we could get an accurate picture, it would probably be very helpful.”

Such inaccuracies often made for difficult interactions with parents. As one rural family medicine physician described,

“One major problem I see with [health departments] sending out information unless they’ve checked with the provider first… it can become a real hassle as far as all the parents calling and saying ‘no, no, no, I don’t need this’ and then they call us and say ‘what’s going on?’ and ‘why are they hassling me?’”

Several other barriers to population-based recall also emerged from the interview data. Some interviewees were concerned that population-based reminder/recall would conflict with practices’ reminder/recall efforts and cause confusion among patients. Others were concerned that a letter from the health department might encourage parents to bring their children to the health department, rather than the practice, for the immunization(s) needed, causing them to miss an opportunity for a well check. The following quote, by a pediatrician in an urban practice, captures both concerns:

“I don’t want parents just coming in for immunizations when they need to come in here for well checks also. They need to do the two at the same time…. [if health departments issue reminder/recall notices to patients in my practice], it might confuse them. Like, ‘okay, are we supposed to go to the health department now for our shots instead of our doctor’s office?’”

Still other interviewees expressed the belief that population-based reminder/recall would not respect or work well with patients’ communication or vaccination preferences, thus angering and alienating them. One physician in an urban family medicine practice predicted that,

“Some families would find it offensive… if they started immunizations and then decided to stop for some reason and then when they get the reminder they are like ‘well, I’m working with my doctor about this so why is the health department involved?’”

Experiences with Population-Based Reminder/Recall

Among the 128 surveyed practices in the counties that experienced population-based reminder/recall during the intervention, such concerns were only partially supported. Indeed, the survey data indicate that the population-based recall did not create an undue burden on practices. Only slightly more than half (56.3%) of practices in population-based reminder/recall counties indicated they were even aware of the effort. Of those, most (78.6%) were not aware of parents from their practice who were angry or upset about receiving reminder/recall letters from the health department, and most (67.2%) reported that the reminder/recall letter sent by the health department did not result in a high volume of patients. However, roughly half (49.2%) reported that at least some of their patients expressed confusion about why they received reminder/recall notices from the health department rather than from the practice.

The qualitative data further supports the finding that population-based recall did not create an undue burden on most practices. Most interviewees from practices in the intervention counties reported experiences ranging from neutral to positive. Staff reported that the population-based reminder/recall experience seemed to be effective and did not substantially increase their workload. Interviewees from rural practices were especially enthusiastic in their support for population-based reminder/recall and their belief in its effectiveness. As one rural physician put it, health department involvement meant additional voices “trying to advocate for the idea of the kids needing these immunizations. I think the more parents can hear it from different angles the better our acceptance rate will be.”

Most practices reported receiving no patient feedback about the population-recall efforts, suggesting that patients did not find the experience to be controversial or problematic. However, two urban practices reported receiving negative comments from parents, largely due to inaccuracies in the CIIS database. These practices found the experience upsetting.

Discussion

These data both support and extend previous work on provider- and population-based reminder/recall efforts. Taken together, they suggest the need to further explore the effectiveness and feasibility of alternative approaches to traditional, practice-based reminder/recall, including efforts such as centralized, population-based reminder/recall conducted by public health entities. Our findings regarding the low number of private providers conducting reminder/recall and the barriers to practice-based reminder/recall support those from previous studies. A minority of providers in our survey was conducting reminder/recall, and most were not using an immunization registry to do so. This is consonant with prior research documenting that 20% or fewer of private providers typically conduct reminder/recall for their patients.^8,11 Similarly, many of the barriers to practice-based reminder/recall reported by our survey respondents and interviewees have been reported elsewhere, including perceived cost,^7,11 feasibility,^7,13 time,^7,11,12 staffing issues,⁷ and distrust in the accuracy of immunization registry data.¹³

However, many of the findings emergent in our data offer unique contributions to the literature on provider attitudes toward reminder/recall. Rather than relying solely on hypothetical scenarios to elicit providers’ opinions about this alternative to practice-based reminder/recall, we offer evidence that an actual population-based reminder/recall effort did not create a significant burden on practices. Though Dombowski and colleagues³ have recently assessed a population-based reminder/recall effort in Michigan, the primary outcome measures in that study were feasibility of notification (address validity and address deliverability), rather than providers’ perceptions of intervention impact. The practice staff in this study did not perceive an overwhelming number of patients recalled to practices; indeed, our data indicate that the intervention did not significantly impact practice operations, at least as perceived and reported by practice staff. Though some parents were confused about why they received a letter from the health department, few were angry or upset by this. The vast majority of practice staff reported a neutral to positive experience with the population-based reminder/recall effort, finding it to be an effective method of reaching their patients. Providers in rural practices were especially enthusiastic about the experience.

Second, this study provides information on providers’ feelings about and preferences for methods of public-private collaboration. Such information is necessary if effective reminder/recall is to be accomplished, as it provides insight into ways to move forward that would be acceptable to providers. The data suggest that a collaborative approach to reminder/recall involving both providers and health departments is preferable to many providers and thus may be a viable solution to the barriers of practice-based reminder/recall. Roughly half of providers surveyed preferred that reminder/recall involve the health department in some way. Preferences for the format of the recall notification varied but, taken together, our data indicate that having the notification appear to come from both the health department and the practice might alleviate some of the problems raised, including patient confusion and physician concerns that families would be more likely to go the health department for immunization. The need to collaborate in updating patient contact data was also generally endorsed. Overall, the idea of collaboration was viewed positively by many practices, particularly those in rural areas, as an opportunity to improve their services to patients.

Finally, this study indicates significant differences between urban and rural practices’ attitudes toward public-private collaboration. Among other differences, both the idea and the experience of public-private collaboration were consistently viewed more positively and enthusiastically by rural providers. Our interview data indicate the existence of a long and established history of collaboration between rural private practices and health departments to accomplish a variety of health outcomes. Public and private entities in rural areas often already have systems in place for communication and collaboration, and thus rural practices are already more likely to refer patients to health departments when the practice is unable to provide care or when a less expensive option is needed. This is supported by previous research finding that children in rural areas are more likely to utilize both public and private sectors.^25,26 In contrast, there is no similar tradition of a collaborative care relationship among public and private entities in urban areas. Urban practices were more likely to prefer no practice involvement with population-based reminder/recall efforts, were more likely to indicate distrust of CIIS data and, during the interviews, said relatively little about collaboration. While they did not indicate opposition to collaboration per se, urban interviewees clearly did not have a vision for what successful public-private collaboration could look like. It may be that the specialization that has developed in urban counties as a result of higher population density and demand for services itself serves as a barrier to public-private collaboration. These data suggest caution may be warranted when designing collaborative public-private reminder/recall interventions; a “one size fits all” approach may be neither effective nor advisable. Instead, regional considerations should be included when designing and implementing collaborative reminder/recall efforts. Such considerations should include regional history, social and economic constraints, and degree of specialization within and across health service organizations.

Finally, the mixed methods utilized in this study offer a unique contribution to the literature on provider attitudes toward immunization reminder/recall. To our knowledge, this is the first study that has employed both quantitative and qualitative approaches in the examination of this issue. The use of survey as well as interview data permits both breadth and depth, and allows for more nuanced understanding of the multiple facets relevant to public-private collaborative efforts.

There are several limitations to this study. Data were collected only from practices in Colorado, limiting generalizability. Although our response rate was high, we are unable to determine if/how the attitudes and practices of survey respondents may have differed from those of non-respondents. Further, our survey data reflect only providers’ reported, rather than observed, immunization practices. It is possible that some survey respondents misrepresented their practices’ behaviors regarding or attitudes toward practice-based and population-based reminder/recall. Finally, the key informant interview data reported here offer insight into only 36 primary care practices and thus cannot be considered representative of all primary care practices.

In its call for the development of improved means of engagement and integration to benefit population health, the 2012 Institute of Medicine report has emphasized the need for approaches designed to facilitate the integration of primary care and public health.¹⁴ This study offers deeper insight into providers’ perceptions of practice- and population-based reminder/recall and preferences for public-private collaboration. The survey and interview data paint a clearer picture of the barriers to, support for, and experiences of practice- and population-based reminder/recall, as well as relevant patterns of urban/rural difference that have meaning for the development of future interventions.

What’s New.

Providers’ attitudes toward public-private collaboration to improve reminder/recall were assessed. Many providers, particularly those in rural areas, preferred an approach involving both practices and health departments. A population-based reminder/recall intervention did not create a significant burden on practices.