Development of a self-reported reflective tool on advanced access to support primary healthcare providers: study protocol of a mixed-method research design using an e-Delphi survey

Abstract

Introduction

Timely access is one of the cornerstones of strong primary healthcare (PHC). New models to increase timely access have emerged across the world, including advanced access (AA). Recently in Quebec, Canada, the AA model has spread widely across the province. The model has largely been implemented by PHC professionals with important variations; however, a tool to assess their practice improvement within AA is lacking. The general objective of this study is to develop a self-reported online reflective tool that will guide PHC professionals’ reflection on their individual AA practice and formulation of recommendations for improvement. Specific objectives are: (1) operationalisation of the pillars and subpillars of AA; (2) development of a self-reported questionnaire; and (3) evaluation of the psychometrics.

Methods and analysis

The pillars composing Murray’s model of AA will first be reviewed in collaboration with PHC professional and stakeholders, patients and researchers in a face-to-face meeting, with the goal to establish consensus on the pillars and subpillars of AA. Leading from these definitions, items will be identified for evaluation through an e-Delphi consultation. Three rounds are planned in 2020–2021 with a group of 20–25 experts. A repository of recommendations on how to improve one’s AA practice will be populated based on the literature and enriched by our experts throughout the consultation. Median and measures of dispersions will be used to evaluate agreement. The resulting tool will then be evaluated by PHC professionals for psychometrics in 2021–2022.

Ethics and dissemination

The Centre Intégré de Santé et de Services Sociaux de la Montérégie-Centre Scientific Research Committee approved the protocol, and the Research Ethics Board provided ethics approval (2020-441, CP 980475). Dissemination plan is a mix of community diffusion through and for our partners and to the scientific community including peer-reviewed publications and conference presentations.

Strengths and limitations of this study.

• Provides a revisited and operationalisation of the pillars and subpillars of the advanced access (AA) model developed 25 years ago.

• Provides a rigorously developed up-to-date tool on AA based on the literature and on the experiences of various primary healthcare stakeholders in response to their expressed needs.

• Involves the participation of multiple stakeholders with different roles and attached to diverse organisation originating from multiple environments including urban, rural and remote regions.

• The Delphi method allows the experts to express their thoughts independently, while encouraging pragmatism, honesty and creativity.

• The developed tool may not be transferable without cultural adaptation to other settings where the principles of AA are implemented.

Introduction

Advanced access model

Timely access is widely recognised as a cornerstone of effective primary healthcare (PHC). The advanced access (AA) model, developed to increase timely access, has been promoted and adopted in primary care settings in various countries. It is the most commonly used organisational model to reduce wait times for primary care appointments.¹ Timely access is one of the guiding principles of the patient-centred medical home (PCMH).² The pillars of AA complement the PCMH model (timely access, comprehensiveness, continuity, interprofessional collaboration) and emphasise organisational components. AA has been defined according to five pillars^{3 4}: (1) balance supply and demand; (2) reduce the backlog of previously scheduled appointments; (3) review the appointment system; (4) integrate interprofessional practice; and (5) develop contingency plans (see figure 1). The AA model was developed in the USA in 2001 and implemented in North America and Europe, with many studies in the USA, the UK and Canada and its effectiveness demonstrated in various healthcare systems.^5–9 Benefits of AA include reduced wait times,5 6 9–11 fewer missed appointments^{5 10} and improved professional and^{8 12} patient satisfaction,⁵ and provider productivity.⁹

Figure 1.

The five pillars of advanced access. The figure shows the advanced access model with the original five pillars and guiding principles. This is used as a starting point for this proposal.

The AA model aligns with the organisational guiding principles for a high-quality high-performance primary care organisation as put forward by the College of Family Physicians of Canada.^{13 14} That said, even if the concept of AA was developed more than 20 years ago by Murray et al remains current, it needs to be adapted to the contemporary context.¹⁵ This study contributes to refine AA based on more interdisciplinary-based team and the need to improve PHC practice with a quality improvement approach.

Evolution of AA and state of research in the province of Quebec

In Quebec, AA was first introduced in 2012, and since then it has been widely promoted by the Quebec College of Family Physicians, as well as by the Ministry of Health and Social Services.^{16 17} Family physicians are strongly encouraged to implement an AA model based on the five pillars proposed by Murray et al.³ Over the past 6 years, the majority of PHC family physicians in Quebec have introduced AA in their organisations at varying levels of implementation.^{4 18}

An expressed need for a reflective tool

Several guides have been developed in Canada,^18–20 the USA^21–23 and Europe²⁴ to assist PHC professionals and/or organisations to plan and implement AA. These guides offer recommendations to plan supply, reduce demand and organise appointment management in order to achieve and maintain a balance between supply and demand, thus enabling timely responses to patient requests. The guides generally present principles of AA, along with how to implement initial changes and some measurement tools. They also offer strategies to support the introduction of AA, but lack information and guidance to improve and sustain AA or to troubleshoot issues over time.

There is no tool even to evaluate the status of AA in a professional’s practice, let alone to guide its continuous improvement and sustainability. Inspired by the principles of the Model for Improvement,²⁵ such a tool could be used to align metrics across multiple PHC providers, while operationalising the complex process of providing access to care in a daily practice. One could use the developed tool to reflect on one’s practice and plan for modifications to improve patient access. This project was therefore developed in response to a clearly expressed ministerial desire to meet the needs of PHC professionals to be supported in the clinical integration and improvement of AA in PHC settings. This study will provide an online reflective tool that will be used as requested by PHC professionals seeking to improve their AA practice.

Self-reported tools are useful reflective strategies to support quality improvement as they are easily accessible and available when needed, regardless of location. These tools also provide an effective way to promote self-reflection and identification of strengths and areas in need.²⁶ Some tools available online, provide diagnosis, document or assess a level of practice or alignment with goals such as those of the tool developed by the College of Family Physicians in Canada to assess the PCMH²⁷ or the Universal Health Coverage Primary Health Care Self-Assessment Tool.²⁸ Online tools may also offer a reflective perspective to provide actionable advice,²⁷ or immediate results and guidance.²⁹

There is a need to develop a self-reported online reflective tool to support AA implementation and improvement by PHC providers. To achieve this objective, it is important to ensure that there is consensus on the underlying model. Differences in definition or interpretation of the pillars of AA could lead to operationalisation difficulties, that can and should be avoided.

Objectives

The main objective of this study is to develop a self-reported reflective tool to support PHC providers to improve their AA practice.

The specific objectives are to:

1. Operationalise the pillars and subpillars of the AA model.

2. Develop a self-reported questionnaire on their practice in AA.

   1. Develop a questionnaire to assess the level of implementation of the AA model.

   2. Identify key recommendations to improve AA.

3. Evaluate the psychometrics of the tool.

The main deliverable of our study will be a self-reported reflective tool on AA that will be combined with a repository of recommendations to improve AA, available on an electronic platform easily accessible to PHC providers and teams. This includes physicians, nurses and nurse practitioners, social workers, pharmacists, nutritionists, psychologists and so on. Users will receive, in one place, an evaluation of their AA practice and personalised recommendations to support improvement.

Methods

Study design overview

A modified Delphi methodology will be used, to develop a reflective tool and identity strategies to improve AA. A literature review and analysis of selected articles will be used to identify conceptual constructs, followed by an iterative consensus achievement process among key experts including a face-to-face meeting and an online survey tool (e-Delphi).³⁰ Using an iterative process, a Delphi consultation is an effective technique designed to obtain the most reliable consensus within a group of experts regardless of their geographical spread.^31–33 A group of experts from the province of Québec will include diverse providers such as family physicians, nurse practitioners, nurses, front-desk and administrative staff and policymakers. Grounding the initial Delphi round in concepts derived from literature and based on initial experts’ input during the face-to-face meeting will both be efficient, and will stimulate participation in the following steps of the tool development.

The process entails three sequential phases, with phase 1 being qualitative, while phases 2 and 3 being including quantitation. Table 1 briefly presents all three phases with their specific objectives. Phase 1 consists of a face-to-face meeting. This pre e-Delphi consultation aims to establish common bases in the operationalisation of a revised AA model (objective 1). Phase 2 will consist of a three-round consultation to identify the content of the self-reported reflective tool. Phase 3 will follow to assess the developed reflective tool and its applicability to different PHC professionals and in different PHC environments.

Table 1.

The AA reflective tool development in brief

Phase 1 Pre-Delphi consultation Establishing common bases Operationalise the AA model	Research team identifies AA pillars and definitions from the literature Identification and recruitment of AA experts Consensus building on pillars and brainstorming about subpillars of AA through a facilitated face-to-face meeting
Phase 2 e-Delphi consultation Creation of the AA reflective tool (list of essential items to be assessed in the reflective tool by PHC practitioners/clinicians)	First round of consultation Panel expert agreements scores (from 1 to 9) on subpillars and definitions Suggestions/comments for modification or addition to subpillars
	Second round of consultation Global and individual feedback report from round 1 (Level of consensus achieved, global and individual expert panel scores) Panel expert agreement scores (from 1 to 9) on new propositions and modification emerging from round 1 Panel expert agreement scores (from 1 to 5) on the importance of each subpillar and the list of items to measure their level of implementation Panel expert agreement (yes/no) on suggested response scales Suggestions/comments for modification or addition of items
	Third round of consultation Global and individual feedback report from round 2 (Level of consensus achieved, global and individual expert panel scores) Consensus building on items by pillar and subpillar Suggestions for practical recommendations by item to provide to clinicians to improve their AA practice
Phase 3 Piloting, adjustment and development of a repository of recommendations To assess the tool and its applicability in different PHC environments	Questionnaire completion by PHC clinicians in different PHC settings Psychometric properties analyses Focus groups to receive feedback and improvement tips Final adjustments Development of a repository of recommendations with actionable guidance

AA, advanced access; PHC, primary healthcare.

The self-reported reflective tool aims to evaluate the processes associated with each AA pillar while allowing its users to grasp their strengths and weaknesses with respect to their level of implementation.

Study management

The research team includes researchers with AA and methodological expertise, and PHC professionals including family physicians and nurses. The team will oversee the development and ongoing processes of the study, as well as major decisions regarding the selection of AA experts to invite to the face-to-face meeting (Phase 1) and to the e-Delphi consultation (Phase 2). They will also be involved in piloting material and instruments ahead of consultations.

The expert panel

To maximise the acceptability and usefulness of the reflective tool in Quebec’s contemporary context, the research team will establish an AA expert panel comprised of provincial and local decision-makers, family physicians, practitioner and clinical nurses, continuous quality improvement officers, front-desk and administrative staff, as well as patients and researchers working in the field of AA. We will seek to bring together experts with diverse expertise based on their role in their own organisation as well as at the local, regional or provincial level. Purposive and snowball sampling techniques will be used to identify eligible participants. Forty potential participants will first be approached and invited by the principal investigators by email to join the expert panel and to be part of the pre-Delphi meeting with the hope of recruiting and maintaining a sample of 20–25 experts across all three e-Delphi rounds. This is above the target of 10–18 individuals, to ensure the development of productive group dynamics and to maximise chances of reaching consensus among experts.³⁰ Participants will be considered for the panel if they are working in PHC or belong to an organisation working closely with PHC professionals, and have an extensive experience with AA (5+ years) as a practitioner or manager. Practitioners and managers who were involved in the development of the training sessions provided by the Quebec College of Family Physicians will also be invited.³⁴ Strategies to maximise the retention rate include personalised reminders from one of the principal investigators, with the goal of not losing more than 30% of the participants over the three expected rounds.³⁰

Phase 1: operationalisation of the AA model

Building a consensus around the AA model will first entail agreeing on the pillars and subpillars that are essential to integrate in an AA practice in PHC. The starting point will be the conceptual framework of the five guiding principles of AA developed by Murray and Berwick in 2003³: (1) balance supply and demand; (2) reduce the backlog of previously scheduled appointments; (3) review the appointment system; (4) integrate interprofessional practice; and (5) Develop contingency plans (see figure 1).

Phase 1 will consist of a literature review to conceptualise a revised AA model. Search terms such as ‘advanced access,’ ‘open-access,’ ‘same-day scheduling,’ ‘timely access’ and ‘AA implementation’ will be used. The literature review will include scientific studies and grey literature reports at local, national and international levels. This scan will allow us to delineate the pillars and subpillars as defined in models of AA in the contemporary literature, as well as constructs that need to be measured within each subpillar.

Following revision by the research team, the revised AA model will be submitted to the AA expert panel in a face-to-face meeting for discussions to build consensus while refining the pillars and definitions. This in-person meeting will be highly interactive and use facilitation techniques for group consultations, to encourage everyone’s participation. The meeting will be organised in two steps. First, a variation of a ‘World Café’ will be used to initiate and lead a collective reflection around the AA pillars and definition identified by the research team. A World Café is a simple yet powerful method to enable meaningful conversations driven by participants and the topics that are relevant and important to them,^{35 36} to lay the groundwork for common understandings. Building on the results of the World Café, a ‘carousel brainstorming’ technique will be used with AA experts to brainstorm on important components to be included in each pillar. A Carousel Activity is a communicative and interactive opportunity for participants to get up and move around a room in a circular fashion, stopping intermittingly to comment, discuss or respond in writing to probing headings/questions/topics/themes posted by a facilitator.³⁷ This technique allows for small group discussion, followed by whole-group collective reflection.³⁸ Facilitation techniques ensure that everyone is able to participate equally and is able to express his/her viewpoint freely, while ignoring hierarchical concerns.³⁹ The results of this meeting will be analysed by the research team to come to a consensus on the conceptual model of AA including the name, number of pillars and definition of each identified pillar. This will serve as groundwork for the e-Delphi consultation.

Phase 2: creation of the AA reflective tool through e-Delphi consultation

Phase 2 will be conducted on an online survey tool (Survey Monkey platform). After each round, a personalised report will be sent to each AA expert providing an overall view of responses as well as their own, in an anonymised format. A list of processes to operationalise the various pillars and subpillars developed during phase 1 will be reviewed and adapted by the research team before submission to the expert panel. The mandate of the panel will be to set the importance of the suggested processes for each of the subpillars and to achieve a consensus on a final list of processes, considered to be very important or essential for assessment of an AA practice. Experts will also be surveyed regarding the adequacy of suggested response scales for each item of the reflective AA tool.

Round 1

An individualised link to a personalised questionnaire will be sent to each AA expert. The first round of consultation will propose subpillars that have emerged from phase 1. For each of the proposed subpillars, AA experts will be asked to rate their level of agreement (on a scale of 1–9) regarding the relevance of this subpillar to the concept of AA. The median as well as measures of dispersion will be used as indicators of the level of consensus. There is no commonly defined rule to determine achievement of consensus, so a pre-hoc decision was made to consider 75% agreement to be consensus. Based on applied methods to determine consensus in a Delphi,^{40 41} we define the following three zones: a median between 7 and 9 indicates high relevance, a median between 1 and 3 indicates low relevance and a middle zone relevance of a median between 4 and 6 where the relevance is uncertain. These subpillars will be retained for a second round of consultation. A 9-point evaluation scale was chosen for this phase to allow participants to express their perception of relevance of a subpillar using a wide range of possibilities. Experts will also be asked to provide their definition, to comment on the subpillar, to indicate if this subpillar is associated with the appropriate pillar and to add any subpillars they think are missing along with suggested definitions.

Round 2 and 3 surveys

A feedback report from the previous round will accompany each new round of questionnaires, including the level of consensus achieved along with global and individual relevance scores for each item. In rounds 2 and 3 (and further if needed), the importance of the suggested items will be rated using 5-point Likert scale. The specification of each element of the response scale (1=Not important at all, 2=Little important, 3=Somewhat important, 4=Very important and 5=Essential) is intended to simplify the respondents' burden of response while adding clarity to the responses obtained. Consensus will be attained if 75% of respondents rate an item ‘Very important’ or ‘Essential.’ More specifically, consensus will be reached with a median rating of 4 or more, with an IQR of less than 1. If an item is rated below 4 by more than 25% of respondents, this be interpreted to be non-consensus. AA experts will be given the opportunity to modify their initial response in light of the answers provided from the group, so as to facilitate the group evolution towards consensus.^{33 42} They will also be asked to score new propositions and modifications emerging from the previous round. The process will continue with further rounds until a consensus on the importance of each item is reached or not.^{33 42} The e-Delphi rounds will cease when an acceptable degree of consensus is reached.^{43 44} Particular attention will be given following each round to assess whether consensus has been reached over a particular round, or rather evolved throughout the process, and whether the group’s opinion has changed over the rounds.⁴⁵ Phase 2 will result in the creation of the online reflective tool based on the list of items for which consensus was achieved, to rigorously assess the processes required in an AA practice.

Phase 3: assessment and applicability of the newly developed AA reflective tool and development of a Repository of recommendations

A subgroup of 5–10 AA experts who participated in phases 1 and/or 2 will be consulted to formulate and prioritise recommendations for an optimal AA practice.

AA reflective tool refinement

Survey completion sessions will be organised with PHC professionals and staff from five different PHC clinics, who will not have been involved in the previous phases of the study. These survey completion sessions will include feedback discussions on the completion of the tool, and will be led by the research team. Following cognitive testing techniques,⁴⁶ these sessions are intended to identify items that are not clear or need to be reformulated, as well as any difficulties encountered during completion of the questionnaire. Following this piloting and development period, the AA reflective tool will be considered to be final, and ready to be evaluated by a larger number of potential users.

Development of a repository of recommendations

A repository of recommendations aligned with the different components of the revised AA model will be created, in relation to the result obtained after the completion of the tool. This repository of recommendations will be made available through the electronic platform, on completion of the evaluation using the AA reflective tool. Recommendations will be personalised according to the professional ‘portrait’ based on their score on each pillar, providing them with actionable avenues. Such an approach was inspired by the primary care quality improvement tool of the College of Family Physicians of Canada.²⁷

The repository of recommendations will be inspired by systematic collation of best practices, by reviewing the literature related to each of the components of AA (eg, improving interprofessional collaboration, optimisation of telephone reception, managing escalation of emergencies) and using feedback generated by the expert panel on the final e-Delphi round. Implementation guides as well as locally developed help tools will serve as sources of recommendations for the repository and will be expanded with experiences of AA experts and their close collaborators. The repository will be expanded and refined during the third round of the e-Delphi and during the survey completion sessions. If discussion of recommendations cannot be addressed in the third round of the e-Delphi, we will bring the discussion to experts in an additional face-to-face or virtual meeting.’

Assessing the psychometric properties of the reflective tool

The final step of development of the tool will consist of the evaluation of some of its psychometric properties. To do so, we plan to recruit a minimum of 150–200 PHC professionals in at least 10 PHC clinics that have not been involved in the development of the tool.⁴⁷ The family physicians, nurses and other professionals working in those PHC clinics will be asked to complete the newly developed tool and comment on its content. Following qualitative feedback, the first analysis will be at the item level: after excluding items with more than 4% missing values, we will do an item discrimination analysis to learn how well an item can discriminate between high and low AA performers. Items with a lack of variation in responses—that is, that are either too easy to or too difficult to attain—will be reviewed for content and adequacy of the response scale. Other properties tested will be the tool’s reliability (repeatability and intra-reliability and inter-reliability in different contexts or between different types of healthcare professionals), and validity (construct validity with a confirmatory factor analysis). Finally, internal consistency will be analysed for each pillar and subpillar.

Patient and public involvement

At least two patients will be invited to the face-to-face meeting in phase 1 as well as the e-Delphi survey. We also intend to consult the patient partners’ group related to our research infrastructure at the end of the tool development to discuss issues that may have arisen and could require a patient point of view. This group is composed of five patient partners involved in different research project on AA.

Ethics and dissemination

Ethics and consent for participation will be sought at each phase of the study. Participation in a meeting or completion of an electronic survey will be considered to be consent for participation and use of anonymised data; as in focus groups or individual interviews, a written consent will be sought.

The results of dissemination plan includes communications through the PHC community including our partner organisations, and to the scientific community via the peer-reviewed literature and conferences. We will attempt to reach many PHC professionals to let them know of our findings through professional organisations and by organising a symposium that will bring together our expert participants as well as colleagues from their organisations or from other organisations and PHC clinics. As so, elements and lessons learnt from this study will be shared through multiple community media resources, including presentations and webinars, newsletters and a symposium on AA initiated by the research team.

Discussion

Scientific articles on the foundations of AA have been published over the past 20 years. During this time interdisciplinary collaborative practice has evolved within PHC family practices, with several healthcare professionals now being part of the PHC team. This study will help to redefine the foundations of the AA model by integrating an interdisciplinary team-based focus, while considering changes that have been put in place in PHC practices. The inclusion of various PHC stakeholders in the tool development process will allow the tool and its content to reflect realities experienced in the field. Participation of AA champions in the overall development process of the AA reflective tool will benefit the community’s acceptance of the tool.

This study will provide a rigorously developed up-to-date AA reflective tool based on the literature and on the experiences of various PHC stakeholders, while providing a response to their expressed needs for a reflective tool. The newly developed reflective tool will be helpful as the Ministry of Health and Social Services and professional medical organisations are currently promoting AA in PHC settings and other professional associations across Canada. Timely access is a key pillar of PCMH, a well-known model to guide the development of PHC around the world. The reflective tool on AA developed in this study will be available for use and adaptation in different countries.

The repository of recommendations developed in parallel with the tool will provide the AA tool users with advice in relation with their own AA evaluation, and provide pragmatic and personalised recommendations to improve their AA practice. This will all be conveniently available online.

Ethics statements

Patient consent for publication

Not applicable.