Skip to main content
Canadian Family Physician logoLink to Canadian Family Physician
. 2010 Dec;56(12):1322–1331.

Facilitators and barriers to implementing quality measurement in primary mental health care

Systematic review

Facteurs qui favorisent ou qui gênent la mise en place de mesures de la qualité des soins primaires en santé mentale

Donald Addington 1,, Tania Kyle 2, Soni Desai 2, JianLi Wang 3
PMCID: PMC3001932  PMID: 21375065

ABSTRACT

OBJECTIVE

To identify facilitators and barriers to implementing quality measurement in primary mental health care as part of a large Canadian study (Continuous Enhancement of Quality Measurement) to identify and select key performances measures for quality improvement in primary mental health care.

DATA SOURCES

CINAHL, EMBASE, MEDLINE, and PsycINFO were searched, using various terms that represented the main concepts, for articles published in English between 1996 and 2005.

STUDY SELECTION

In consultation with a health sciences research librarian, the initial list of identified references was reduced to 702 abstracts, which were assessed for relevance by 2 coders using predetermined selection criteria. Following a consensus process, 34 articles were selected for inclusion in the analysis. An additional 106 citations were identified in the references of these articles, 14 of which were deemed relevant to this study, for a total of 57 empirical articles identified for review. Most articles described implementation of health care innovations and clinical practice guidelines, 5 focused on quality indicators, and 1 examined mental health indicators.

SYNTHESIS

Content analysis of the 57 articles identified 7 common categories of facilitators and barriers for implementing innovations, guidelines, and quality indicators: indicator characteristics, promotional strategies, implementation strategies, resources, individual-level factors, organizational-level factors, and external factors. Implementation studies in which these factors were addressed were more likely to achieve successful outcomes.

CONCLUSION

The overlap in facilitators and barriers across implementation of mental health indicators, health care innovations, and practice guidelines is not surprising, as they are often related. The overlap strengthens the findings of the limited number of studies of quality indicators. The Continuous Enhancement of Quality Measurement process for identification and selection of indicators has attended to some of these issues by using a rigorous scientific approach and by engaging a range of stakeholders in selecting and prioritizing the indicators.


One in 5 Canadians will experience a mental illness during his or her lifetime.1 Most of those who use mental health services will seek mental health care in Canada’s primary health care system. A general population survey found that among patients who consulted health care professionals for mental health purposes, more than 35% saw FPs only, 25% saw FPs and other mental health care providers (eg, psychiatrists, psychologists, social workers), and 40% saw other mental health care providers only.2 From the perspective of FPs, 1 in 4 people visiting an FP has a clinically significant mental health condition.3,4 While there is a high demand for mental health services in primary care, there are considerable gaps between the quality of services provided and optimal care.5 The use of evidence-based measures (indicators) has been suggested as part of the process of quality improvement.6

Promotion of quality measurement activities related to primary mental health care has taken place in Australia,7 the United Kingdom,8 and the United States.9 In Canada, the Continuous Enhancement of Quality Measurement (CEQM) in Primary Mental Health Care: Closing the Implementation Loop project was launched in 2004. The goal of CEQM was to improve the quality of mental health care for all Canadians by fostering quality measurement in primary mental health care. It aimed to achieve this goal through building pan-Canadian consensus on a small set of quality measures. The set of health measures for Canadian primary care mental health services was to reflect a multistakeholder perspective and be suitable for facilitating quality improvement. A 3-stage process led to a final set of 30 consensus measures. The results and reports are available online at www.ceqm-acmq.com.

A systematic review of the current literature on the facilitators and barriers to implementing quality measures in primary care was conducted as a subproject of the CEQM. This paper presents the results of that review.

DATA SOURCES

The review focused on 4 main concepts: primary care, mental health, quality indicators, and innovation or change. Four electronic databases (CINAHL, EMBASE, MEDLINE, and PsycINFO) were searched, using various terms that represented the main concepts, for articles published in English between 1996 and 2005. The database search yielded 89 555 citations for all of the combinations of the 4 main concepts (Table 1). The number of citations was reduced to 75 063 citations by eliminating the searches that did not include quality indicators as a search term. A health sciences research librarian was consulted regarding a method for reducing the number of citations, and she recommended that searches with more than 300 articles be removed, because these searches were not successful in targeting a manageable number of articles. This resulted in 733 citations and, after removing duplicates, it was reduced to 702 citations.

Table 1.

Results for different combinations of search concepts by database

CONCEPTS SEARCHED MEDLINE PSYCINFO EMBASE CINAHL
Primary care
Mental health
Quality indicators
Innovation or change
6* 0 281* 0
Primary care
Quality indicators
Innovation or change
225* 0 1852 13*
Primary care
Quality indicators
5376 163* 13 255 5*
Primary care
Innovation or change
1043 41 3855 1
Mental health
Quality indicators
Innovation or change
28* 4* 2523 6*
Mental health
Quality indicators
3668 772 46 885 1*
Mental health
Innovation or change
231 312 9009 0
*

Citations were retained and assessed for relevance.

The abstracts of the 702 citations were printed and assessed for relevance by 2 coders with master’s-level research training (T.K. and S.D.) using predetermined selection criteria. The 2 coders rated 50 abstracts and compared their answers to assess whether they were applying the criteria in the same manner.10 They discussed the discrepancies until they agreed upon a rating. Some of the criteria were rewritten to increase clarity. To be selected as relevant, an abstract had to focus on primary health care (or primary mental health care) and refer to a quality improvement tool or the process of implementing quality measurement, quality indicators, or quality improvement. In the first round, coders rated the articles as yes, no, or unsure, and their ratings showed agreement for 533 abstracts (20 yes and 513 no) and did not agree or were rated unsure for 169 abstracts (κ = 0.540). The 169 abstracts were rated again by the same 2 people, but this time they had to make a forced choice of yes or no. After the second round, the assessment for relevance yielded 671 abstracts with identical ratings (62 yes and 609 no) and 31 abstracts with mixed ratings (κ = 0.775). A professor of psychiatry (D.A) with expertise in performance measurement took the role of a third coder for the 31 tied ratings, and the final result was 83 yes and 619 no.

The 83 agreed-upon articles were retrieved and read, and the 44 articles reporting findings of original research were selected for review. During the first reading, 3 types of articles were identified: those that specifically addressed quality indicators, those that addressed clinical practice guidelines, and those that addressed health care innovations in a broader sense. Quality ratings are a key step in systematic reviews10; however, with so few empirical articles on implementing quality indicators, we included all of them as long as they contained findings of original research. One of the researchers searched the reference lists of the most relevant articles for secondary references, uncovering 106 additional references. After 2 rounds of coding for relevance by the same 2 researchers using the original criteria, 34 abstracts were selected, the corresponding articles were read, and 14 additional empirical articles were added to the review.

Content analysis was used to abstract any text mentioning implementation facilitators or barriers from the selected articles.11 This process was completed by one of the master’s-trained researchers who consulted the second researcher as needed. A separate list of facilitators and barriers was created for the 3 groups of articles: quality indicators, clinical practice guidelines, and health care innovations. Two researchers independently examined each list of facilitators and barriers and grouped them by topic or recurring idea. The 2 researchers then compared their groupings and agreed upon broad categories to fit the data. Category development is a process of understanding and explaining the data.12

SYNTHESIS

The selected articles are summarized in Table 2.6,1368 The authors of most of the articles were based in the United States (n = 29), discussed broad health care innovations (n = 32), and used solely qualitative research methods (n = 35). Twenty articles about implementing clinical practice guidelines were also found, even though the search was intended to focus on articles about implementing quality indicators and clinical practice guidelines was not a search term. There were only 5 empirical studies of the specific process of implementing quality indicators. The settings of 4 of these studies were clearly primary care, yet the indicators were not mental health–related, and 1 study implemented mental health quality indicators in a community-based mental health clinic. The fifth study was retained, even though the clinic might have offered both primary and secondary care, because we wanted to glean information about implementing mental health–specific indicators. The 5 articles included 1 quasi-experimental study, 1 case study, 1 retrospective audit, and 2 qualitative studies published between 2000 and 2004. Table 3 lists facilitators and barriers to implementing quality indicators, as listed in these specific articles, and Table 4 presents facilitators in the form of a checklist for readiness to implement clinical practice guidelines and other health care innovations.

Table 2.

Summary of the empirical articles reviewed: N = 57.

STUDY LOCATION SETTING STUDY PURPOSE METHOD AND SAMPLE
Quality indicator studies
Ballard,13 2003 US PC Describe elements of change involved in implementing QI Qualitative case study; 1 health care system
Exworthy et al,14 2003 UK PC Examine effect of PI on GPs’ clinical autonomy Interviews; 52 GPs, nurses, and managers
Gorrell et al,15 2004 Australia MH Measure change in service after introducing specialized teams Audit; 47 in control group, 70 in treatment group
McColl et al,6 2000 UK PC Test feasibility of deriving QI in all practices in a PCG Retrospective audit; 18 practices in 1 PCG
Wilkinson et al,16 2000 UK PC Study PC clinicians’ reactions to the use of PIs Interviews; 29 GPs, 11 practice managers, 12 nurses
Clinical practice guideline studies
Brown et al,17 2000 US PC, MH Evaluate effectiveness of 2 CPG implementation methods RCT, quasi-experimental; physicians, nurses, assistants
Cabana et al,18 2000 US PC Describe barriers to successful use of asthma CPGs Focus groups; 22 participants
Cabana et al,19 2001 US PC Identify barriers to GP adherence to asthma CPGs Cross-sectional survey; 829 random GPs
Cranney et al,20 2001 UK PC Identify what impedes use of hypertension CPGs Interviews; 34 random GPs
Eccles et al,21 2002 UK PC Evaluate computerized support system for implementing CPGs RCT; 60 GPs
Goldberg et al,22 1998 US PC Determine effectiveness of interventions on compliance with CPGs RCT; 15 small group practices in 4 PC clinics
Gupta et al,23 1997 Australia PC Determine GPs’ views on and recall of CPGs Questionnaire; 286 random GPs
Gupta et al,24 1997 Australia PC Examine GPs’ views on CPGs Survey; 286 GPs
Hermens et al,25 2001 Netherlands PC Assess the key elements of a successful implementation Prospective cohort study; 1586 random GPs
Kaner et al,26 1999 UK PC Evaluate training and support in implementation of intervention RCT; 128 GPs
Karuza et al,27 1995 US PC Develop and evaluate dissemination intervention RCT; 13 group practices and 5 PC GPs
Lin et al,28 1997 US PC, MH Examine whether education has an effect on treatment Quantitative, quasi-experimental
Nutting et al,29 2002 US PC, MH Examine why GPs and nurses were not using depression CPGs Cluster analysis, interviews; 6 nurses, 12 GPs
Picken et al,30 1998 US PC Assess level of modification of CPGs by PC physicians Quantitative modified Delphi approach; 68 random GPs
Puech et al,31 1998 Australia PC Examine care patterns and strategies for local implementation of CPGs Questionnaire; 83 random GPs
Rashidian and Russell,32 2003 UK PC Develop model for implementing CPGs Interviews; 25 GPs and PC academics
Rollman et al,33 1999 US PC, MH Describe steps of disseminating CPGs by EMRs Qualitative, descriptive; PC physicians
Silagy et al,34 2002 Australia PC Determine effects of local adaptation of CPGs Survey; 400 random GPs from 2 practice divisions
Smith et al,35 2004 UK PC, MH Examine GPs’ views on using CPGs Interviews; 11 GPs
Thompson et al,36 2000 UK PC, MH Assess effectiveness of educational program RCT; 60 PC physicians
Health care innovation studies
Aupont,37 2001 US PC, MH Analyze how quality improvement affects provider behaviour Quantitative, qualitative survey; 26 providers, 30 000 patients
Benjamin and Seaman,38 1998 Bahrain PC Discuss lessons learned from implementing an innovation Qualitative case study; 1 health centre
Bentz et al,39 2002 US PC Pilot-test 2 tobacco tracking systems Qualitative case study; 2 PC offices
Brown et al,40 2003 US PC Describe the redesign process using a quality model Qualitative case study; 1 service area
Byrne et al,41 2004 US PC, MH Determine effects of clinical service lines on inpatient and urgent care use Survey; 140 directors of health facilities
Campbell et al,42 2002 UK PC Investigate the concept of clinical governance Qualitative case study; 12 PC groups or trusts
Cohen et al,43 2004 US PC Develop a model of practice change Qualitative comparative case study; 15 practices
Cox et al,44 1999 UK PC Improve the repeat-prescribing process Qualitative case study; 1 practice of 7 GPs
Earnest et al,45 1998 US PC Describe various strategies of quality and performance improvement Qualitative case study; 1 urban health care system
Feifer et al,46 2003 US PC Discuss the self-managing team concept and illustrate its application Qualitative case study; 1 practice centre
Fischer et al,47 1998 US PC Analyze case studies from a quality improvement project Qualitative case study; 6 random clinics
Frijling et al,48 2003 Netherlands PC Test transferability of observations from research to real-life situations Controlled before-and-after trial; 617 general practices
Gillespie et al,49 2004 UK PC Explore how the term patient-centred care is understood Interviews; 47 individuals
Hermann et al,50 2001 US PC, MH Describe an implementation process and its outcomes Qualitative descriptive study; 11 health centres, 1599 patients
Hogg et al,51 2002 Canada PC Understand the role of a facilitator in preventive performance Qualitative case study; 7 practices
Holden,52 2002 UK PC Evaluate a PC audit group Qualitative case study; 1 audit group
Jackson and Bircher,53 2002 UK PC Examine how a new model transformed a general practice Qualitative case study; 1 medical centre
John et al,54 2001 UK PC Describe a benchmarking project Qualitative prospective longitudinal cohort study; 8 service trust sites
Knox et al,55 2001 US PC Identify strategies for improving staff performance Qualitative, quantitative in-depth cross-case analysis; 8 practices
Korsen et al,56 2003 US PC, MH Implement a multifaceted intervention Qualitative case study; 2 practices
Ledlow and Bradshaw,57 1999 US PC Improve practice using an animated simulation tool Qualitative case study; 1 family practice
Lövgren et al,58 2001 Sweden PC Explore physicians’ and nurses’ views on adopting a new care policy Qualitative interview; 50 physicians and nurses
Magnan et al,59 1997 US PC Examine turmoil during a quality improvement project Survey; 44 clinics
Marshall et al,60 2002 UK PC Investigate importance of culture in implementing clinical governance Qualitative case study; 12 PC groups or trusts, 50 senior PC managers
Nemeth,61 2003 US PC Analyze a quality improvement intervention Qualitative fieldwork; 6 cases
Rubenstein et al,62 2002 US PC Assess how quality improvement teams affect a program Interviews, observation; 2 large PC practices, 5 QI teams
Solberg et al,63 1999 US PC, MH Assess need for and acceptability of depression care system Focus groups and survey
Tobin and Norris,64 1998 Australia PC, MH Report on a project to implement total quality management strategies Interviews; 100 MH staff and GPs
Townes et al,65 2000 US PC Develop and implement a change Qualitative case study; 1 health centre
Willcocks,66 2003 UK PC Explore early experiences of a new PC organization Qualitative longitudinal case study; 1 PC group
Zapka et al,67 2004 US PC Present a process evaluation Qualitative case study, randomized trial; 6 community health centres
Zomalt,68 1997 US PC, MH Study staff perspective of doing continuous quality improvement Qualitative case study; day rehabilitation program staff

CPG—clinical practice guideline, EMR—electronic medical record, GP—general practitioner, MH—mental health, PC—primary care, PCG—primary care group, PI—performance indicator, QI—quality indicator, RCT—randomized controlled trial.

Table 3.

Facilitators and barriers to implementing quality indicators: N = 5 articles.

CATEGORY FACILITATORS BARRIERS
Indicator characteristics
  • Well-recognized definitions

  • Clear definitions

  • Evidence-based

  • Inclusive (cover important areas)

  • Reflect current knowledge

  • Based on reliable, complete data

  • Represent an “open” agenda

  • Lack of precision

Promotional strategies
  • Focus on services endorsed by a government task force

  • Can be used to demonstrate clinical competence

  • Credible indicators

  • Indicators linked to performance incentives

  • Existence of a “product” champion to enthuse and educate

  • Develops capacity to monitor care

  • Viewing indicators as a threat to autonomy

  • Viewing indicators as not credible

  • Viewing indicators as tools to penalize bad performance

  • Financial penalties based on performance areas beyond the scope of professional control

Implementation strategies
  • Use assessors with medical or research expertise

  • Indicators selected sparingly

  • Indicator information is part of documenting care

  • Documentation method is user-friendly and guides care delivery

  • Documented data are computer-ready

  • Use previously developed tools

  • Use of an audit tool

  • There is an implementation plan

  • Create a multidisciplinary quality improvement team

  • Using government-associated assessors

  • Lack of definitive diagnoses in charts

  • Difficult-to-define intervention thresholds

Resources
  • Information technology is current or resources available for upgrading

  • Time is available for interpreting and acting on indicator data

  • Practical support for data entry

  • Capital available for extra costs

  • Incompatible computer systems

  • Generating indicators is costly

  • Labour intensive

  • Requirement for external staff

  • Lack of computer training

  • Increased workload

Individual-level factors
  • Staff are competent in documentation

  • Staff have good communication and collaboration skills

  • Indicators are a personal interest or responsibility

  • Staff have good computer skills

  • Staff have limited computer skills

  • Difficulties understanding indicator-related terminology or concepts

  • Confusion in applying Read codes

  • Reduced professional autonomy and trust

  • Short-term expectations of improved quality of care

Organizational-level factors
  • Multistakeholder involvement

  • Board members are aligned with implementation plan

  • Team agreement on purpose, benefits, and importance of indicators

  • Some services not recorded in medical record or are difficult to find

  • No clear responsibility for data entry

  • Perceived lack of time to plan

  • Lack of team approach to change

  • Limited interprofessional communication

External factors
  • Access to expert advice about clinical issues

  • Indicators “fit in” with local initiatives and policies

  • Competition between practices

Table 4.

Readiness to implement quality measurement checklist*: Read each statement and indicate your response with a check mark. Complete the checklist based on your perspective in the organization. Try to respond to every statement.

QUESTIONS AND STATEMENTS YES NO NOT APPLICABLE
What are your organization’s plans regarding quality measurement?
  1. Has no plans to implement quality measurement
  2. Intends to implement quality measurement in the next 6 months
  3. Intends to implement quality measurement in the next 30 days
  4. Has been using quality measures for a short time (less than 6 months)
  5. Has been using quality measures for 6 months or longer
If you checked YES for statements 2 or 3, please complete the remainder of the checklist. Otherwise, you may stop now.
What are the characteristics of the quality measures you wish to implement?
  A1. The measures are evidence-based
  A2. The terms comprising the measures have recognized definitions
  A3. The measures have recognized norms or benchmarks
How are the quality measures being promoted?
  B1. The measures are published in a respected source
  B2. The measures are endorsed by a credible source, such as physician licensing body or professional association
  B3. Measures are promoted as an efficient solution to quality assurance
  B4. Quality measurement is promoted through the use of incentives
  B5. The measures are championed by a leader
  B6. Local stakeholders participated in adapting measures to local circumstances
What implementation strategies are available to your organization?
  C1. Collecting measurement data is part of documenting care
  C2. The measures are kept to the minimum number necessary
  C3. There is an implementation plan to follow
  C4. Academic detailing or outreach by a trained professional
  C5. Practice-based group learning with a facilitator and a specialist
  C6. A consultant is available to help the staff to implement the measures
Which statements BEST describe your organization’s resources?
  D1. The office has Internet access
  D2. The office computer system can support an EMR
  D3. Documentation is compatible with the EMR
  D4. There is a budget for quality improvement activities
  D5. There is a staff member with quality measurement skills
  D6. Using quality measures does not add extra time or workload to staff
Which statements BEST describe the individuals in your organization?
  E1. Staff comply with the current documentation method
  E2. Staff have good computer skills
  E3. Physicians adhere to practice protocols
  E4. Physicians think measures could be used to monitor and reward good performance
  E5. Physicians believe implementing measures will lead to improved practice
  E6. Quality measurement is a personal interest of a staff member or physician
Which statements BEST describe the current operation of your organization?
  F1. There is positive leadership in the organization
  F2. The decision-making authority is clear
  F3. Organization leaders understand the effects of their decisions on patient care
  F4. Clinicians from different professional groups work as a team
  F5. Physicians are able to allocate time for quality measurement activities
  F6. Front-line staff are involved in planning for change or innovation
  F7. There is team agreement on the purpose and benefits of quality measures
  F8. There is a staff member who is responsible for data entry
Which of these external factors affect your organization?
  G1. There is a shortage of specialists for timely mental health referrals
  G2. The political environment is open to new health care innovations

EMR—electronic medical record.

*

Completing the checklist will provide practical information about your organization’s readiness to perform quality measurement. The outcome is not a “readiness score,” but rather a starting point for discussion and planning. The checklist statements are organized into categories that prompt you to assess your organization in terms of its stage of planning for quality measurement, the characteristics and promotion of the quality measures, implementation strategies, available resources, staff readiness, operational readiness, and external factors. Quality measures (or quality indicators) are norms, criteria, standards, and other direct qualitative and quantitative measures used in determining the quality of health care. Examples include “the percentage of mental health clinicians who have appropriate skills in cognitive behavioural therapy” and “the percentage of patients being treated for depression who receive the appropriate dosage and duration of treatment of antidepressants.”

The 2 master’s-trained researchers agreed upon 7 broad categories to represent the facilitators and barriers to implementing quality indicators. The categories that fit the quality indicator data were similar to the categories chosen for the clinical practice guideline and health innovation data. These similarities suggest that the same facilitators and barriers apply across quality indicators and clinical practice guidelines as 2 types of the broader class of health innovations. The 7 broad categories that represent the facilitators and barriers to implementing quality indicators include measure characteristics (key attributes), promotional strategies, implementation strategies, resources, individual-level factors, organizational-level factors, and factors external to the organization. The articles about implementing quality indicators are the focus of this paper. A number of articles reviewed provided both their results and rich, detailed information about the experience of implementation.13,15,16,69

DISCUSSION

The facilitators and barriers to implementing quality indicators in primary care are discussed by category and in relation to the innovation and change literature.

Characteristics

The stakeholders’ perceptions of the importance of what is being measured and the ease with which it can be measured are important considerations for selecting quality measures. Much of the variance in adoption rates for both processes and quality measures can be explained by a measure’s key attributes, as perceived by potential adopters. Attributes that are positively related to adoption include perceived benefit to patients, fit with existing skills and resources, ease of testing, face validity, and level of change required to implement the process and its measure. The perceived complexity of an innovation and its quality measure is negatively related to its adoption rate.70,71 Indicators were also more likely to be adopted if they reflected current knowledge, were evidence-based, covered important areas, used reliable and complete data, and represented an “open” versus a “hidden” agenda. An important barrier to implementing clinical practice guidelines in primary care, which reflects many of the above issues, was faced when the measures were originally developed for secondary or another level of care.

Promotion and implementation

Innovations are spread by influences that range from passive diffusion to active dissemination. Diffusion has been characterized as unplanned, informal, decentralized, and often mediated by peers, while dissemination is described as planned, formal, often centralized, and more likely to occur through vertical channels.71 Rogers described 5 main steps that take place before new measures are fully adopted: knowledge, persuasion, decision, implementation, and confirmation.70

Endorsement of measures by credible organizations, such as a government task force for quality indicators or publication in a respected journal, was shown to facilitate adoption. In contrast, the belief that quality measurement is a threat to professional autonomy or a tool to penalize bad performance is seen as a barrier. This highlights the importance of involving individuals in the early stages of planning the measurement process. Interpersonal communication between 2 or more similar individuals is more effective than mass communication in persuading someone to adopt an innovation.70 Linking quality indicator use to performance incentives was reported to be useful in some contexts. However, the use of financial penalties based on performance areas beyond the scope of professional control is a barrier. Ensuring that all stakeholders share the same perception of incentives is critical for successful implementation.

Resources

The PRECEDE-PROCEED model outlines the steps for planning, implementing, and evaluating innovations that are directed toward improving the health of individuals, populations, or organizations.72 Assessing the need for and availability of resources is vital to the implementation planning process. Dedicated resources such as time, funding, and skilled personnel are enabling factors that make desired change possible.71,72

Individual-level factors

The knowledge, attitudes, beliefs, values, and perceptions of individuals are predisposing factors that can facilitate or hinder motivation for change. In turn, the innovation decision is influenced by this motivation as well as by individuals’ needs, goals, skills, learning styles, and social networks.71,72 The implementation of quality indicators is facilitated when quality measurement is a personal interest or responsibility of a physician, staff member, or office manager.

Organizational-level and external factors

Organizational structures, culture, and resources are important for supporting the adoption of new processes and measures.71,72 In the context of primary care in Canada, there are obvious differences across provinces in the relationships between government funders, primary care providers, and specialty care providers. Relationships and funding arrangements that support information systems integration and collaboration between providers are more easily linked to quality measures. More specific factors within organizations that were reported to facilitate the implementation of indicators were multistakeholder involvement, board member support, and team agreement on the purpose, importance, and benefits of indicators. Reported implementation barriers included a perceived lack of time to plan and limited communication among professions. Generating indicators was difficult when some services were not documented in the medical record and the responsibility for data entry was not clear.73

Conclusion

The CEQM is an innovative Canadian project developing a consensus set of quality measures for primary mental health care. This review has demonstrated that successful implementation of quality measures can occur but will depend on the interaction of multiple factors, including measure characteristics, promotional messages, implementation strategies, resources, the intended adopters, and the intraorganizational and interorganizational contexts. As we undertake pilot projects to implement our quality measures, it will be important to gather data about the process and the outcomes.

Acknowledgments

The authors developed the checklist as a part of the Continuous Enhancement of Quality Measurement: Closing the Implementation Loop project, which is funded by Health Canada’s Primary Health Care Transition Fund.

EDITOR’S KEY POINTS

  • While there is a high demand for mental health services in primary care, there are considerable gaps between the quality of services provided and optimal care.

  • Promotion of quality measurement activities related to primary mental health care has taken place in many countries. In Canada, the Continuous Enhancement of Quality Measurement in Primary Mental Health Care project was launched in 2004. This paper presents the results of a systematic review of the current literature on the facilitators and barriers to implementing quality measures in primary care, conducted as part of this larger project.

  • The authors found that successful implementation of quality measures can occur, but that success depends on the interaction of multiple factors, including measure characteristics (key attributes), promotional messages, implementation strategies, resources, the intended adopters, and the intraorganizational and interorganizational contexts.

POINTS DE REPÈRE DU RÉDACTEUR

  • Alors qu’il existe une forte demande pour les services primaires en santé mentale, la qualité des services prodigués est loin d’être idéale.

  • Dans plusieurs pays, on a favorisé la mise en place de systèmes de mesure de la qualité des services primaires en santé mentale. Le projet Amélioration continue de la mesure de la qualité des soins et services de première ligne en santé mentale a été lancé au Canada en 2004. Cet article présente les résultats d’une revue systématique de la littérature effectuée à l’occasion de ce projet, revue portant sur les facteurs qui favorisent ou qui gênent la mise en place de mesures de la qualité des soins primaires.

  • Les auteurs ont observé qu’il est possible d’instaurer des mesures de la qualité, mais que le succès repose sur l’interaction entre plusieurs facteurs, incluant les caractéristiques de l’instrument de mesure (attribut clés), les messages promotionnels, la mise en place de stratégies, les ressources, les intervenants visés et les contextes intra et inter-organisationnels.

Footnotes

This article has been peer reviewed.

Cet article a fait l’objet d’une révision par des pairs.

Contributors

Ms Kyle and Ms Desai performed the literature search and coded the selected articles. All authors contributed to the concept and design of the study; data gathering, analysis, and interpretation; and preparing the manuscript for submission.

Competing interests

None declared

References

  • 1.Health Canada Editorial Board Mental Illness in Canada . A report on mental illness in Canada. Ottawa, ON: Health Canada; 2002. [Google Scholar]
  • 2.Lesage AD, Goering P, Lin E. Family physicians and the mental health system. Report from the Mental Health Supplement to the Ontario Health Survey. Can Fam Physician. 1997;43:251–6. [PMC free article] [PubMed] [Google Scholar]
  • 3.Tiemens BG, Ormel J, Simon GE. Occurrence, recognition, and outcome of psychological disorders in primary care. Am J Psychiatry. 1996;153(5):636–44. doi: 10.1176/ajp.153.5.636. [DOI] [PubMed] [Google Scholar]
  • 4.Spitzer RL, Kroenke K, Williams JB. Validation and utility of a self-report version of PRIME-MD: the PHQ primary care study. Primary Care Evaluation of Mental Disorders. Patient Health Questionnaire. JAMA. 1999;282(18):1737–44. doi: 10.1001/jama.282.18.1737. [DOI] [PubMed] [Google Scholar]
  • 5.Von Korff M, Goldberg D. Improving outcomes in depression. BMJ. 2001;323(7319):948–9. doi: 10.1136/bmj.323.7319.948. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6.McColl A, Roderick P, Smith H, Wilkinson E, Moore M, Exworthy M, et al. Clinical governance in primary care groups: the feasibility of deriving evidence-based performance indicators. Qual Health Care. 2000;9(2):90–7. doi: 10.1136/qhc.9.2.90. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7.Australian Government Department of Health and Ageing . Future directions: your toolkit for implementation: implementing a national quality and performance system for the Divisions of General Practice network. Canberra, Australia: Government of Australia; 2005. [Google Scholar]
  • 8.Shield T, Campbell S, Rogers A, Worrall A, Chew-Graham C, Gask L. Quality indicators for primary care mental health services. Qual Saf Health Care. 2003;12(2):100–6. doi: 10.1136/qhc.12.2.100. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9.Chisholm M, Howard PB, Boyd MA, Clement JA, Hendrix MJ, Reiss-Brennan B. Quality indicators for primary mental health within managed care: a public health focus. Arch Psychiatr Nurs. 1997;11(4):167–81. doi: 10.1016/s0883-9417(97)80024-6. [DOI] [PubMed] [Google Scholar]
  • 10.Adair CE, Simpson L, Birdsell JM, Omelchuk K, Casebeer AL, Gardiner HP, et al. Performance measurement systems in health and mental health services: models, practices and effectiveness: a state of the science review. Edmonton, AB: Alberta Heritage Foundation for Medical Research; 2003. [Google Scholar]
  • 11.Neuendorf KA. The content analysis guidebook. London, Engl: Sage Publications; 2002. [Google Scholar]
  • 12.Morse JM, Richards L. Readme first for a user’s guide to qualitative methods. Thousand Oaks, CA: Sage Publications; 2002. [Google Scholar]
  • 13.Ballard DJ. Indicators to improve clinical quality across an integrated health care system. Int J Qual Health Care. 2003;15(Suppl 1):i13–23. doi: 10.1093/intqhc/mzg080. [DOI] [PubMed] [Google Scholar]
  • 14.Exworthy M, Wilkinson EK, McColl A, Moore M, Roderick P, Smith H, et al. The role of performance indicators in changing the autonomy of the general practice profession in the UK. Soc Sci Med. 2003;56(7):1493–504. doi: 10.1016/s0277-9536(02)00151-x. [DOI] [PubMed] [Google Scholar]
  • 15.Gorrell J, Cornish A, Tennant C, Rosen A, Nash L, McKay D, et al. Changes in early psychosis service provision: a file audit. Aust N Z J Psychiatry. 2004;38(9):687–93. doi: 10.1080/j.1440-1614.2004.01450.x. [DOI] [PubMed] [Google Scholar]
  • 16.Wilkinson EK, McColl A, Exworthy M, Roderick P, Smith H, Moore M, et al. Reactions to the use of evidence-based performance indicators in primary care: a qualitative study. Qual Health Care. 2000;9(3):166–74. doi: 10.1136/qhc.9.3.166. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17.Brown JB, Shye D, McFarland BH, Nichols GA, Mullooly JP, Johnson RE. Controlled trials of CQI and academic detailing to implement a clinical practice guideline for depression. Jt Comm J Qual Improv. 2000;26(1):39–54. doi: 10.1016/s1070-3241(00)26004-5. [DOI] [PubMed] [Google Scholar]
  • 18.Cabana MD, Ebel BE, Cooper-Patrick L, Powe NR, Rubin HR, Rand CS. Barriers pediatricians face when using asthma practice guidelines. Arch Pediatr Adolesc Med. 2000;154(7):685–93. doi: 10.1001/archpedi.154.7.685. [DOI] [PubMed] [Google Scholar]
  • 19.Cabana MD, Rand CS, Becher OJ, Rubin HR. Reasons for pediatrician nonadherence to asthma guidelines. Arch Pediatr Adolesc Med. 2001;155(9):1057–62. doi: 10.1001/archpedi.155.9.1057. [DOI] [PubMed] [Google Scholar]
  • 20.Cranney M, Warren E, Barton S, Gardner K, Walley T. Why do GPs not implement evidence-based guidelines? A descriptive study. Fam Pract. 2001;18(4):359–63. doi: 10.1093/fampra/18.4.359. [DOI] [PubMed] [Google Scholar]
  • 21.Eccles M, McColl E, Steen N, Rousseau N, Grimshaw J, Parkin D, et al. Effect of computerised evidence based guidelines on management of asthma and angina in adults in primary care: cluster randomised controlled trial. BMJ. 2002;325(7370):941. doi: 10.1136/bmj.325.7370.941. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 22.Goldberg HI, Wagner EH, Fihn SD, Martin DP, Horowitz CR, Christensen DB, et al. A randomized controlled trial of CQI teams and academic detailing: can they alter compliance with guidelines? Jt Comm J Qual Improv. 1998;24(3):130–42. doi: 10.1016/s1070-3241(16)30367-4. [DOI] [PubMed] [Google Scholar]
  • 23.Gupta L, Ward JE, Hayward RS. Clinical practice guidelines in general practice: a national survey of recall, attitudes and impact. Med J Aust. 1997;166(2):69–72. doi: 10.5694/j.1326-5377.1997.tb138723.x. [DOI] [PubMed] [Google Scholar]
  • 24.Gupta L, Ward J, Hayward RS. Future directions for clinical practice guidelines: needs, lead agencies and potential dissemination strategies identified by Australian general practitioners. Aust N Z J Public Health. 1997;21(5):495–9. doi: 10.1111/j.1467-842x.1997.tb01741.x. [DOI] [PubMed] [Google Scholar]
  • 25.Hermens RP, Hak E, Hulscher ME, Braspenning JC, Grol RP. Adherence to guidelines on cervical cancer screening in general practice: programme elements of successful implementation. Br J Gen Pract. 2001;51(472):897–903. [PMC free article] [PubMed] [Google Scholar]
  • 26.Kaner EF, Lock CA, McAvoy BR, Heather N, Gilvarry E. A RCT of three training and support strategies to encourage implementation of screening and brief alcohol intervention by general practitioners. Br J Gen Pract. 1999;49(446):699–703. [PMC free article] [PubMed] [Google Scholar]
  • 27.Karuza J, Calkins E, Feather J, Hershey CO, Katz L, Majeroni B. Enhancing physician adoption of practice guidelines. Dissemination of influenza vaccination guideline using a small-group consensus process. Arch Intern Med. 1995;155(6):625–32. [PubMed] [Google Scholar]
  • 28.Lin EH, Katon WJ, Simon GE, Von Korff M, Bush TM, Rutter CM, et al. Achieving guidelines for the treatment of depression in primary care: is physician education enough? Med Care. 1997;35(8):831–42. doi: 10.1097/00005650-199708000-00008. [DOI] [PubMed] [Google Scholar]
  • 29.Nutting PA, Rost K, Dickinson M, Werner JJ, Dickinson P, Smith JL, et al. Barriers to initiating depression treatment in primary care practice. J Gen Intern Med. 2002;17(2):103–11. doi: 10.1046/j.1525-1497.2002.10128.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 30.Picken HA, Greenfield S, Teres D, Hirway PS, Landis JN. Effect of local standards on the implementation of national guidelines for asthma: primary care agreement with national asthma guidelines. J Gen Intern Med. 1998;13(10):659–63. doi: 10.1046/j.1525-1497.1998.00200.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 31.Puech M, Ward J, Hirst G, Hughes AM. Local implementation of national guidelines on lower urinary tract symptoms: what do general practitioners in Sydney, Australia suggest will work? Int J Qual Health Care. 1998;10(4):339–43. doi: 10.1093/intqhc/10.4.339. [DOI] [PubMed] [Google Scholar]
  • 32.Rashidian A, Russell I. Towards better prescribing—a model for implementing clinical guidelines in primary care organizations in the NHS. Clin Govern Int J. 2003;8(1):26–32. [Google Scholar]
  • 33.Rollman BL, Gilbert T, Lowe HJ, Kapoor WN, Schulberg HC. The electronic medical record: its role in disseminating depression guidelines in primary care practice. Int J Psychiatry Med. 1999;29(3):267–86. doi: 10.2190/7CCY-D8JB-53XK-PPTK. [DOI] [PubMed] [Google Scholar]
  • 34.Silagy CA, Weller DP, Lapsley H, Middleton P, Shelby-James T, Fazekas B. The effectiveness of local adaptation of nationally produced clinical practice guidelines. Fam Pract. 2002;19(3):223–30. doi: 10.1093/fampra/19.3.223. [DOI] [PubMed] [Google Scholar]
  • 35.Smith L, Walker A, Gilhooly K. Clinical guidelines of depression: a qualitative study of GPs’ views. J Fam Pract. 2004;53(7):556–61. [PubMed] [Google Scholar]
  • 36.Thompson C, Kinmonth AL, Stevens L, Peveler RC, Stevens A, Ostler KJ, et al. Effects of a clinical-practice guideline and practice-based education on detection and outcome of depression in primary care: Hampshire Depression Project randomised controlled trial. Lancet. 2000;355(9199):185–91. doi: 10.1016/s0140-6736(99)03171-2. [DOI] [PubMed] [Google Scholar]
  • 37.Aupont O. Exploring the relative importance of practice characteristics on adoption of a quality improvement program in primary care: identification of conditions accelerating dissemination of guideline concordant care for depression. 2001. [doctoral thesis]. Boston, MA: Brandeis University;
  • 38.Benjamin S, Seaman M. Applying continuous quality improvement and human performance technology to primary health care in Bahrain. Health Care Superv. 1998;17(1):62–71. [PubMed] [Google Scholar]
  • 39.Bentz CJ, Davis N, Bayley B. The feasibility of paper-based tracking codes and electronic medical record systems to monitor tobacco-use assessment and intervention in an Individual Practice Association (IPA) Model health maintenance organization (HMO) Nicotine Tob Res. 2002;4(Suppl 1):S9–17. doi: 10.1080/14622200210128036. [DOI] [PubMed] [Google Scholar]
  • 40.Brown DS, Church L, Heywood T, Hills JF, McCarthy S, Serway C. Lessons in redesigning a quality program across the continuum. J Healthc Qual. 2003;25(1):36–42. 50. doi: 10.1111/j.1945-1474.2003.tb01031.x. [DOI] [PubMed] [Google Scholar]
  • 41.Byrne MM, Charns MP, Parker VA, Meterko MM, Wray NP. The effects of organization on medical utilization: an analysis of service line organization. Med Care. 2004;42(1):28–37. doi: 10.1097/01.mlr.0000102493.28759.71. [DOI] [PubMed] [Google Scholar]
  • 42.Campbell SM, Sheaff R, Sibbald B, Marshall MN, Pickard S, Gask L, et al. Implementing clinical governance in English primary care groups/trusts: reconciling quality improvement and quality assurance. Qual Saf Health Care. 2002;11(1):9–14. doi: 10.1136/qhc.11.1.9. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 43.Cohen D, McDaniel RR, Jr, Crabtree BF, Ruhe MC, Weyer SM, Tallia A, et al. A practice change model for quality improvement in primary care practice. J Healthc Manag. 2004;49(3):155–68. [PubMed] [Google Scholar]
  • 44.Cox S, Wilcock P, Young J. Improving the repeat prescribing process in a busy general practice. A study using continuous quality improvement methodology. Qual Health Care. 1999;8(2):119–25. doi: 10.1136/qshc.8.2.119. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 45.Earnest MP, Grimm SM, Malmgren MA, Martin BA, Meehan M, Potter MB, et al. Quality improvement in an integrated urban healthcare system: a necessary journey. Clin Perform Qual Health Care. 1998;6(4):193–200. [PubMed] [Google Scholar]
  • 46.Feifer C, Nocella K, DeArtola I, Rowden S, Morrison S. Self-managing teams: a strategy for quality improvement. Top Health Inf Manage. 2003;24(1):21–8. [PubMed] [Google Scholar]
  • 47.Fischer LR, Solberg LI, Kottke TE. Quality improvement in primary care clinics. Jt Comm J Qual Improv. 1998;24(7):361–70. doi: 10.1016/s1070-3241(16)30387-x. [DOI] [PubMed] [Google Scholar]
  • 48.Frijling B, Hulscher ME, van Leest LA, Braspenning JC, van den Hoogen H, Drenthen AJ, et al. Multifaceted support to improve preventive cardiovascular care: a nationwide, controlled trial in general practice. Br J Gen Pract. 2003;53(497):934–41. [PMC free article] [PubMed] [Google Scholar]
  • 49.Gillespie R, Florin D, Gillam S. How is patient-centred care understood by the clinical, managerial and lay stakeholders responsible for promoting this agenda? Health Expect. 2004;7(2):142–8. doi: 10.1111/j.1369-7625.2004.00264.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 50.Hermann RC, Erickson P, Joseph R, Bor DH. Improving the detection of mental disorders by primary care clinicians. In: Dickey B, Sederer LI, editors. Improving mental health care: commitment to quality. Washington, DC: American Psychiatric Publishing Inc; 2001. pp. 285–95. [Google Scholar]
  • 51.Hogg W, Baskerville N, Nykiforuk C, Mallen D. Improved preventive care in family practices with outreach facilitation: understanding success and failure. J Health Serv Res Policy. 2002;7(4):195–201. doi: 10.1258/135581902320432714. [DOI] [PubMed] [Google Scholar]
  • 52.Holden J. St Helens and Knowsley MAAG 1991–2001: were we effective? J Clin Governance. 2002;10(3):139–49. [Google Scholar]
  • 53.Jackson S, Bircher R. Transforming a run down general practice into a leading edge primary care organisation with the help of the EFQM excellence model. Int J Health Care Qual Assur Inc Leadersh Health Serv. 2002;15(6–7):255–67. doi: 10.1108/09526860210442029. [DOI] [PubMed] [Google Scholar]
  • 54.John A, Enderby P, Hughes A, Petheram B. Benchmarking can facilitate the sharing of information on outcomes of care. Int J Lang Commun Disord. 2001;36(Suppl):385–90. doi: 10.3109/13682820109177916. [DOI] [PubMed] [Google Scholar]
  • 55.Knox AB, Underbaake G, McBride PE, Mejicano GC. Organization development strategies for continuing medical education. J Contin Educ Health Prof. 2001;21(1):15–23. doi: 10.1002/chp.1340210104. [DOI] [PubMed] [Google Scholar]
  • 56.Korsen N, Scott P, Dietrich AJ, Oxman T. Implementing an office system to improve primary care management of depression. Psychiatr Q. 2003;74(1):45–60. doi: 10.1023/a:1021193606120. [DOI] [PubMed] [Google Scholar]
  • 57.Ledlow GR, Bradshaw DM. Animated simulation: a valuable decision support tool for practice improvement. J Healthc Manag. 1999;44(2):91–101. [PubMed] [Google Scholar]
  • 58.Lövgren G, Aström G, Engström B. A care policy and its implementation. Int J Nurs Pract. 2001;7(2):92–103. doi: 10.1046/j.1440-172x.2001.00247.x. [DOI] [PubMed] [Google Scholar]
  • 59.Magnan S, Solberg LI, Giles K, Kottke TE, Wheeler JW. Primary care, process improvement, and turmoil. J Ambul Care Manage. 1997;20(4):32–8. doi: 10.1097/00004479-199710000-00006. [DOI] [PubMed] [Google Scholar]
  • 60.Marshall M, Sheaff R, Rogers A, Campbell S, Halliwell S, Pickard S, et al. A qualitative study of the cultural changes in primary care organisations needed to implement clinical governance. Br J Gen Pract. 2002;52(481):641–5. [PMC free article] [PubMed] [Google Scholar]
  • 61.Nemeth LS. Implementing change for effective outcomes. Outcomes Manag. 2003;7(3):134–9. [PubMed] [Google Scholar]
  • 62.Rubenstein LV, Parker LE, Meredith LS, Altschuler A, dePillis E, Hernandez J, et al. Understanding team-based quality improvement for depression in primary care. Health Serv Res. 2002;37(4):1009–29. doi: 10.1034/j.1600-0560.2002.63.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 63.Solberg LI, Korsen N, Oxman TE, Fischer LR, Bartels S. The need for a system in the care of depression. J Fam Pract. 1999;48(12):973–9. [PubMed] [Google Scholar]
  • 64.Tobin M, Norris G. Mental health and general practice: improving linkages using a total quality management approach. Aust Health Rev. 1998;21(2):100–10. doi: 10.1071/ah980100. [DOI] [PubMed] [Google Scholar]
  • 65.Townes PG, Jr, Benson DS, Johnston P. Apollo 11: an experiential model for team-based transformational change. J Healthc Qual. 2000;22(5):23–8. doi: 10.1111/j.1945-1474.2000.tb00149.x. [DOI] [PubMed] [Google Scholar]
  • 66.Willcocks S. Developing the effectiveness of primary care organisations in the UK National Health Service. A case study. J Health Organ Manag. 2003;17(3):194–209. doi: 10.1108/14777260310480749. [DOI] [PubMed] [Google Scholar]
  • 67.Zapka J, Goins KV, Pbert L, Ockene JK. Translating efficacy research to effectiveness studies in practice: lessons from research to promote smoking cessation in community health centers. Health Promot Pract. 2004;5(3):245–55. doi: 10.1177/1524839904263713. [DOI] [PubMed] [Google Scholar]
  • 68.Zomalt GD. Implementation of continuous quality improvement in a community mental health care centre [dissertation] San Diego, CA: University of San Diego; 1997. [Google Scholar]
  • 69.McColl A, Roderick P, Gabbay J, Smith H, Moore M. Performance indicators for primary care groups: an evidence based approach. BMJ. 1998;317(7169):1354–60. doi: 10.1136/bmj.317.7169.1354. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 70.Rogers EM. Diffusion of innovations. Toronto, ON: The Free Press; 1995. [Google Scholar]
  • 71.Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O. Diffusion of innovations in service organizations: systematic review and recommendations. Milbank Q. 2004;82(4):581–629. doi: 10.1111/j.0887-378X.2004.00325.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 72.Green LW, Kreuter MW. Health promotion and planning: an educational and environmental approach. 2nd ed. Mountain View, CA: Mayfield; 1991. [Google Scholar]
  • 73.Rashidian A, Eccles MP, Russell I. Falling on stony ground? A qualitative study of implementation of clinical guidelines’ prescribing recommendations in primary care. Health Policy. 2008;85(2):148–61. doi: 10.1016/j.healthpol.2007.07.011. Epub 2007 Sep 4. [DOI] [PubMed] [Google Scholar]

Articles from Canadian Family Physician are provided here courtesy of College of Family Physicians of Canada

RESOURCES