Abstract
Introduction
Compared with heterosexual, cisgender populations, sexual and gender minority (SGM) people are more likely to suffer from serious health conditions and insufficient access to health services. Primary care is at the frontlines of healthcare delivery; yet, few clinics have resources or mechanisms in place to meet SGM patient needs. This developmental study protocol focuses on reducing health disparities among SGM patients by identifying, adapting and developing SGM practice guidelines/recommendations and implementation strategies for primary care clinics in urban and rural New Mexico. Using input from patients, healthcare advocates and providers, and researchers, the study will pilot a practice parameter and implementation toolkit to promote SGM-specific cultural competence at multiple service delivery levels.
Methods and analysis
We will recruit providers/staff from four Federally Qualified Health Centers (FQHCs) serving ethnically and geographically diverse communities. Incorporating the Implementation of Change Model and an intersectionality perspective, data collection includes a systematic review of SGM-specific practice guidelines/recommendations, focus groups and semistructured interviews, quantitative surveys and the Nominal Group Technique (NGT) with providers/staff. We will categorise guidelines/recommendations identified through the review by shared elements, use iterative processes of open and focused coding to analyse qualitative data from focus groups, interviews and the NGT, and apply descriptive statistics to assess survey data. Findings will provide the foundation for the toolkit. Focus groups with SGM patients will yield supplemental information for toolkit refinement. To investigate changes in primary care contexts following the toolkit’s pilot, we will undertake systematic walkthroughs and document review at the FQHCs, analysing these data qualitatively to examine SGM inclusiveness. The structured data-informed Plan-Do-Study-Act method will enable further revision of the toolkit. Finally, focus groups, interviews and quantitative surveys with providers/staff will highlight changes made in the FQHCs to address SGM patient needs, barriers to sustainment of changes, satisfaction, acceptability, usability and feasibility of the toolkit.
Ethics and dissemination
The study has been reviewed and approved by the Pacific Institute for Research and Evaluation Institutional Review Board. Informed consent will be obtained from all participants before their involvement in research activities begins. Study results will be actively disseminated through peer-reviewed journals, conference presentations, social media and the internet, and community/stakeholder engagement activities.
Keywords: cultural competency, health status disparities, implementation science, minority health, primary health care, sexual and gender minorities
Strengths and limitations of this study.
This developmental study addresses healthcare disparities among sexual and gender minority (SGM) populations as guided by a Scientific Advisory Board of SGM patients, healthcare advocates and providers, and researchers.
The study facilitates an examination and prioritisation of organisational and clinical practice guidelines, resulting in a triangulated and analysed set of guidelines approved by a diverse group of stakeholders representing SGM communities and healthcare advocates and providers.
The prioritised guidelines and practical implementation strategies will be integrated into a comprehensive user-friendly toolkit.
The study will explore implementation strategies to introduce the toolkit into primary care practices, and advance pragmatic recommendations for improving services for SGM people from the perspectives of FQHC providers, staff and patients.
The study is limited to four FQHCs in a single state, which may limit generalisability of findings and the toolkit; the small sample sizes also preclude implementation of a randomised controlled trial design to assess organisational and practice changes enabled by the toolkit.
Introduction
Reducing health disparities for sexual and gender minority (SGM) populations, including persons who are lesbian, gay, bisexual, transgender and/or queer (LGBTQ), is a public health priority in the USA.1–3 Compared with heterosexual, cisgender people, SGM individuals are more likely to suffer from poorer mental health, substance misuse, inadequate diet and exercise, and sexually transmitted infections that are often first identified in primary care.4–7 They are also less likely to access preventive services, cancer screening and treatment for cardiovascular disease, diabetes, hypertension and other serious conditions.1 5 6 8 9 Many experience ‘minority stress’ from chronic exposure to stigma and discrimination.10 11 Intersecting minority identities may compound these effects, disproportionately impacting gender-diverse persons,12 ethnic and racial minorities,13 individuals of low income or educational attainment,14 15 and rural residents.16 17
Health disparities for SGM people are deepened by ongoing provision of suboptimal services in healthcare systems with histories of promoting stigma around sexuality and gender atypicality (eg, denying services to persons with HIV/AIDS or who are transgender, conversion therapies).1 6 18–22 Stigma denigrating sexual/gender difference enables discriminatory attitudes and behaviour among healthcare providers/staff that contribute to minority stress.23 Persons suffering from minority stress may internalise anti-SGM sentiment, accept discrimination and microaggressions, and anticipate recurrence of negative experiences.10 11 24 Minority stress may lead to perceptions of provider bias or incompetence, inhibiting patients from revealing SGM status and health risk behaviours.25
Primary care, particularly in Federally Qualified Health Centers (FQHCs), is an ideal target for SGM healthcare intervention due to its person-centred approach, the access it offers to patients of varied social backgrounds,26 and the prevention, screening and treatment services it affords to patients across their lifespan.27–29 Yet, primary care often lacks sufficient resources or mechanisms to ensure that practice settings and service delivery are attentive to SGM patients.30 31 Environmental/structural elements (eg, décor, forms, mission statements) contribute to SGM invisibility, and staff attitudes, language and behaviours may exacerbate feelings of marginalisation.30–34 Insufficient SGM-specific competence among providers inhibits disclosure of SGM status in clinical encounters, undermining patient satisfaction.35 This invisibility can underpin provider beliefs that SGM status is unimportant to patients. Failure among providers to ask relevant questions in attempts to present neutral attitudes towards SGM patients36 37 may also factor into misdiagnoses of health concerns, ineffective treatment and subpar care.38 39 Adequate medical education/training on SGM care is also wanting.40 41
Implementing practice guidelines for SGM competent care that draw from national policies, recommendations for SGM-inclusive medical education curricula42–45 and organisational contexts is imperative to rectify these gaps at provider/staff, practice and service-system levels.8 31 33 46 Although current guidelines/recommendations (henceforth ‘guidelines’) contain critical information about SGM patient-centred clinical environments and interactions, they are fragmented, not based in primary care research, and neglect population-based intersectional attributes (eg, race/ethnicity, culture, rurality) and input from both service providers and SGM patients.31 While a 2010 systematic review identified six philosophically and practically consistent guidelines for SGM patient care (table 1), they lack sufficient evidence and mechanisms for implementation in primary care.31 Such findings for patient care were reproduced in a 2018 review,47 and a 2017 review found no articles evaluating organisational change for care of SGM people.42 This study responds to these gaps, as its goals include (1) developing and triangulating clinical and organisational SGM practice guidelines that can be feasibility implemented in primary care; (2) curating a practice parameter and implementation toolkit by collaborating with providers/staff and SGM patients; and (3) creating measurable implementation strategies and resources to integrate guideline-specific and tool-specific innovations to enable organisational and practice change in primary care for SGM individuals.48
Table 1.
Synthesised recommendations for primary care from existing guidelines appraised by McNair and Hegarty
| 1. Creating inclusive environments | Overt signs/displays; sensitive language/attitudes among staff; inclusive intake forms; optional self-identification; non-discrimination policies; procedures addressing complaints. |
| 2. Standards for clinician–patient communication | Non-judgemental and affirming attitudes; assuring confidentiality; gender-neutral language; use of patient’s language; open, inclusive questioning; complete sexual history; responding to disclosure. |
| 3. Sensitive documentation of SGM identity/orientation | Medical notes (documenting SGM identity/orientation and informing patients of what is written), electronic medical records, referral letters and decision-makers/next of kin/emergency contact. |
| 4. Special knowledge for SGM awareness | Impact of discrimination on health; mental health/substance misuse; reproductive health; safer sex; higher risks for specific diseases; coming out; referrals to support groups and health professionals. |
| 5. Staff training | Confidentiality; use of intake forms; identifying/addressing SGM-negativity; support visibility of SGM employees; inclusive hiring practices supporting SGM recruitment. |
| 6. Addressing population health issues | Marketing services to SGM communities; engaging in SGM-targeted health promotion; performing community outreach and forging relationships with SGM agencies; advocacy. |
SGM, sexual and gender minority.
Study aims
Participatory methods are critical to evaluating guidelines and implementation strategies to improve primary care for SGM people.49 This developmental study attempts to reduce SGM disparities by partnering with FQHCs in the majority–minority state of New Mexico (NM). We will employ the Nominal Group Technique (NGT), an efficient participatory priority-setting process,50 51 to ground SGM practice guidelines in primary care and advance theory-based implementation strategies to promote guideline adherence. Providers/staff from four FQHCs will deploy the Plan-Do-Study-Act (PDSA) approach to pilot the toolkit.52–55 The Implementation of Change Model (IoCM)56 and an intersectionality lens1 57–59 will assist in developing implementation strategies that are optimally relevant to local communities.60 Both perspectives thus comprise the conceptual basis for data collection, guideline adoption and implementation strategy development and testing. This study has three specific aims:
Prioritise SGM practice guidelines and adapt and develop implementation strategies for primary care settings with attention to the intersections of race/ethnicity, rurality and socioeconomic conditions.
Develop/refine a comprehensive toolkit of SGM practice guidelines and implementation strategies to provide FQHCs with resources to promote and evaluate SGM-specific competence at multiple service delivery levels.
Evaluate toolkit implementation at (1) individual provider/staff, (2) social/practice setting and (3) organisational context levels in supporting SGM-specific primary care in FQHCs.
This study responds to national calls to address SGM health disparities by spearheading an approach to implement critical and feasible primary care practice guidelines to promote the well-being of SGM patients with intersecting minority identities.1–3 It also responds to research priorities in the USA to enhance SGM health in under-resourced, understaffed primary care clinics that are stretched to form a crucial safety net.2 61–63 Finally, the study is an essential start for continued research using a type 2 hybrid effectiveness-implementation design for dual testing of the effectiveness of SGM guidelines and implementation strategies specific to primary care.64 65
Methods
Study design and overview
This study features a systematic review of guidelines for SGM-inclusive culturally competent primary care, focus groups and semistructured interviews, quantitative surveys and use of the NGT to facilitate uptake of SGM practice guidelines in primary care. Our 10-person Scientific Advisory Board (SAB), a panel of SGM patients, healthcare advocates and providers, and researchers, will play critical roles in interpreting data from these sources and creating the toolkit. Our study has two phases. Phase 1 engages the SAB and providers/staff from the participating FQHCs in prioritising/assessing guidelines and implementation strategies (aim 1) for the toolkit (aim 2). Phase 2 (aim 3) pilots the toolkit in FQHCs to obtain feasibility, acceptability, usability, fidelity and satisfaction data. The iterative nature of study findings allows for ongoing feedback from participants and accuracy checks to increase internal validity and credibility, reducing possibilities of biassing results.66 Participatory methods will enhance the toolkit’s relevance to a diverse clientele. Finally, implementation experts emphasise selecting or tailoring implementation strategies based on theory, barrier assessments or other rationale.67 This study will generate insights into implementation strategies to overcome barriers to toolkit adoption for different settings and stakeholders.67 68 A timeline of study activities appears in table 2.
Table 2.
Timeline of study activities by quarter
| Study activities timeline by quarter | 1.1 | 1.2 | 1.3 | 1.4 | 2.1 | 2.2 | 2.3 | 2.4 | 3.1 | 3.2 |
| Perform systematic literature review, recruit sites and conduct document reviews, walkthroughs and focus groups/interviews with surveys | ||||||||||
| Develop list of practice guidelines and implementation strategies from data collection and research evidence derived from systematic review | ||||||||||
| Undertake Nominal Group Technique and develop toolkit | ||||||||||
| Hold focus groups with SGM patients and revise toolkit | ||||||||||
| Organise implementation meetings and convene implementation resource teams | ||||||||||
| Engage in toolkit piloting via Plan-Do-Act-Study cycles with coaching support | ||||||||||
| Conduct final document review, walkthroughs and focus groups/interviews with surveys | ||||||||||
| Analyse and draft results and develop a follow-up study featuring a hybrid type 2 effectiveness-implementation experimental design |
SGM, sexual and gender minority.
Patient and public involvement
This study protocol emerged through a lengthy SGM patient and participant engagement process initiated in 2014 with funding from the Patient-Centred Outcomes Research Institute. With this funding, we conducted a series of town hall meetings with SGM people in ethnically and geographically diverse regions of NM regarding their health and healthcare needs. We then developed a statewide SGM health collaborative of SGM patients, healthcare advocates and providers, and researchers to analyse findings from these meetings. This collaborative next created a research agenda for improving SGM healthcare and organised a series of now annual SGM health summits that allow for broader patient and public input into this agenda.
Findings from the town halls and the collaborative’s deliberations led to the identification of primary care as a key site for research-based intervention,35 particularly in rural and otherwise medically underserved communities, and to development of participatory procedures for conducting health-related research with SGM populations.69 The collaborative also identified two major barriers limiting the capacity of primary care clinics to improve services for SGM patients: (1) lack of comprehensive sets of guidelines based in primary care research; and (2) insufficient implementation supports (eg, access to education, training, data on SGM patients) that might assist providers/staff in bustling yet under-resourced clinics in taking part in organisational change efforts to advance quality care for SGM patients. Of note, patients in the SGM health collaborative and/or attending the summit have also provided critical feedback into the design of this study protocol, sharing their ideas for recruitment and the overall conduct of this research. Convening the SAB represents step 1 of our conceptual model; patients on the SAB will continue to offer feedback into study instrumentation, interpretation of findings and dissemination strategies. We will share research results with patients and study participants through online briefs, the annual summit and on-site presentations in communities where the participating FQHCs are located.
Conceptual framework
Implementing innovations, including guidelines and toolkits, in primary care is complex. We will draw from the IoCM56 and use an intersectionality lens1 57–59 to prioritise/assess guidelines and incorporate targeted implementation strategies to aid their translation into everyday clinical work. The IoCM (figure 1) is a systematic approach to plan, organise and implement change, and considers a range of factors impacting implementation.56 For example, an FQHC’s climate and organisational capacity can affect the willingness of providers/staff to engage in new practices,70 as do their individual characteristics (eg, job tenure, professional development level).71 Leadership is also key.72 Persons leading implementation must be effective change agents; their ability to motivate and interact with employees shapes provider/staff attitudes towards new practices.73 Addressing readiness to change, provider/staff attitudes (eg, SGM-negativity) and misinformation,74 and engaging FQHC workers as agents of change via the IoCM will allow them to emerge as champions in deepening capacity to improve primary care for SGM patients.
Figure 1.
Adapted implementation of change model by study phase, aim and step. FQHC, Federally Qualified Health Center; NGT, Nominal Group Technique; SAB, Scientific Advisory Board; SGM, sexual and gender minority.
Given our focus on intersectionality, we recognise that gender and sexuality are only two of several factors affecting the social identities, circumstances and health/healthcare outcomes of SGMs.1 58 59 Data collection, analysis and toolkit planning must thus consider the racial/ethnic, socioeconomic and geographical diversity found in places like NM, where structures of oppression and privilege beget unequal healthcare opportunities for specific populations.58 75 By integrating the IoCM and intersectionality theory, this study is among the first to move beyond assessing SGM healthcare needs and barriers to developing and testing strategies based on understanding the particular experiences of provider/staff and patients of multiple minority statuses, and both organisational and worker capacity to implement innovations in primary care.76
In addition to the IoCM and an intersectionality perspective, we turn to theories of change in public health (table 3)77 and data from the systematic review, qualitative focus groups/interviews, surveys and the NGT to design implementation strategies targeting multiple healthcare levels: (1) individual provider/staff (eg, knowledge, attitudes); (2) social/practice setting (eg, teamwork, opinion leaders, leadership); and (3) organisational context (eg, administrative, structural and cultural factors shaping the workplace).77 78 With the SAB and FQHC stakeholders, we will consider relevant change theories to articulate both rationale and processes by which the strategies will lead to greater SGM competence and higher quality care for SGM patients.
Table 3.
Sample theories of change based on the IoCM to inform SGM practice guideline implementation strategies
| Type and example of theory | Description |
| Sample theories pertinent to individual provider/staff | |
Cognitive
|
Provision of a convincing argument as to why it is worth the time and cost to make services more appropriate for SGM patients. |
Motivational
|
Determine expectations of outcomes from implementing SGM practice guidelines and assess whether the expected outcomes are desirable to stakeholders. Increase perceived social norms for guideline adherence while supporting providers/staff. |
| Sample theories pertinent to social and practice setting | |
| Social Network and Influence | Opinion leaders, formal/informal leaders and significant peers share views and model implementation of SGM practice guidelines (also see Social Learning Theory). |
| Theories on Teamwork | Encourage team collaboration to create a better environment for SGM populations. The team sets goals and targets and reviews process together regularly. |
| Theories on Professionalism | Appeal to sense of professional identity/standards (eg, use recommendations from American Medical Association for physicians and from the American Nursing Association for nurses). |
| Sample theories pertinent to organisational context | |
| Theory of Quality Management | Assumes inadequate performance is an organisational failure requiring strong leadership and organisational changes. Organisations set improvement goals and collaborate to reach goals. |
| Theories of Organisational Culture | Recognise organisational cultures shape work performance and can be altered to achieve an innovation-centred culture to improve performance and stimulate improvements in patient care. |
IoCM, Implementation of Change Model; SGM, sexual and gender minority.
Study context
Our setting is NM, a state ranking 47th in median household income,79 with the second largest percentage of residents below the poverty level (19.7%).80 Hispanic/Latinx and Native American people are 60% of residents.81 About 3% of adults82 and 15.1% of high-school students identify as sexual minorities83; 0.75% of adults84 and 3.4% of high-school students identify as gender minorities.85 Access barriers and cultural competence deficits in care contribute to SGM health disparities.35 82 86 Aims 1 and 2 involve participants from two rural and two urban FQHCs serving racial/ethnic minority communities. Because numerous health disparity populations (eg, Hispanic/Latinx, Native American, socioeconomically disadvantaged, rural) are key FQHC consumers, our study’s FQHC context supports wider applicability to intersectional SGM people.
FQHC samples and recruitment
We will use purposive sampling to represent the range of views/experiences of individual and organisational factors related to prioritising/assessing and implementing guidelines.87 We will include six to eight providers/staff per focus group at each FQHC; one to three clinic administrators per FQHC will take part in interviews. We will work closely with clinic administrators to recruit FQHC employees for the on-site focus groups/interviews. Clinic administrators will advertise focus groups/interviews on FQHC listservs and in employee common areas. Our team will present the study purpose and design at staff meetings. Recruitment may attract persons already sensitive to issues in SGM care; however, these sensitivities may also heighten their ability to perceive and discuss issues in SGM care. Thus, such sensitivities will neither negate their advice for introducing and enacting the guidelines in primary care nor perceptions of implementation barriers/facilitators. Eligible providers/staff must have worked at the FQHC for one or more year(s) for an average of at least 20 hours per week to ensure familiarity with clinical procedures and context-specific healthcare needs. Eligible administrators include persons responsible for professional leadership and the overall management and operation of the FQHC. We will recruit a subset of this sample of FQHC personnel for the NGT, as described below. Following the NGT, we will work with the SAB to develop our toolkit, which will be presented to two additional focus groups (one rural, one urban) of six to eight SGM patients recruited from the FQHC catchment areas. Inclusion criteria include being age ≥18, self-identifying as SGM, and service utilisation at the FQHC in the past 5 years. The challenges of research with SGM people include lack of identification with externally imposed social categories (eg, gay, transgender), and the problem of recruiting ‘hidden’ populations for studies on sensitive topics.88 89 We will deploy purposive sampling methods to overcome these challenges: (1) snowballing (members of the population of interest link researchers with candidates); (2) outcropping (soliciting candidates at places they are known to frequent); and (3) advertising (newspapers, websites).88
Data collection
Document reviews/systematic walkthroughs (phase 1; aim 1; IoCM step 2)
We will analyse documentation to assess for changes in organisational context related to SGM inclusiveness at baseline and on piloting the toolkit (aim 3). Documents of interest are derived from the Healthcare Equality Index (HEI), a national benchmarking tool used in over 1600 healthcare facilities to evaluate policies/practices related to equity and inclusion of SGM patients, visitors and employees.90 Documents requested of clinic administrators will be compiled into an inventory, and analysed using HEI scoring criteria that centre on (1) employment non-discrimination/staff training, (2) patient services/support, (3) employee benefits/policies and (4) patient/community engagement.91 Two researchers will also apply a checklist based on the criteria of the Gay and Lesbian Medical Association during systematic walkthroughs of the FQHCs to observe evidence of visual clues, or décor, suggesting the site is safe for SGM patients, that is, public display of non-discrimination statements and SGM-oriented brochures, educational materials and posters.92 The walkthroughs will also address whether visual clues pertain to patients of intersecting identities in FQHC catchment areas.
Systematic review of SGM-specific guidelines (phase 1; aim1; IoCM step 2)
Our systematic review of the literature will lend insight into current guidelines for culturally competent primary care for SGM patients. We will consult with academic librarians and the SAB to identify appropriate terms and databases for the review. The databases will likely include: CINAHL, PsycARTICLES/PsycINFO, Mental Measurements, SPORTDiscus, SocINDEX, PubMed/MEDLINE, Web of Science and Cochrane Collaboration. We will employ two searches, each consisting of three sections of keywords (see online supplementary file for a detailed list of possible search terms). For the first search, the sections will comprise keywords identifying SGM populations, keywords pertaining to primary care medical services and keywords concerning guidelines and recommendations. We will limit the keyword search to only abstracts and keywords, exclusively English-language results and without restriction of publication date. After this first search, we will conduct a second search, expanding to all healthcare settings, rather than narrowly focus on primary care, as helpful practice guidelines for culturally competent care for SGM patients may present in other service milieus. We will undertake this second search using the same three sections of keywords and criteria used in the first search. We will perform the first search over a 2-month period; the second search will occur over 1 month. On completing the searches in each database, we will import all results into EndNote X8 and cull duplicates.93
bmjopen-2019-032787supp001.pdf (53.1KB, pdf)
We will review the titles/abstracts, then full texts, of the publications iteratively, removing those not meeting inclusion criteria and inputting the remaining texts into an Excel worksheet. Multiple content experts will independently review each guideline to identify major thematic areas. They will convene regularly to agree on the content of thematic areas, assigning each guideline into these areas. The full study team will review the exhaustive list of guidelines, eliminating redundancies for condensation purposes. Each included publication will be rated on the extent to which it meets criteria across multiple domains encompassing scope/purpose, stakeholder involvement, rigour of development, conflict of interest, external review and clarity of presentation.31 47 The shortened list will be presented to the SAB wherein we will gather member perceptions regarding the importance of, and feasibility of, implementing the items it contains.
FQHC focus groups/interviews with surveys (phase 1; aim 1; IoCM step 2)
We will assess current practices/experiences of FQHC stakeholders related to primary care for SGM people. Participants will complete brief (20 min) surveys prior to focus groups/interviews on individual, social/practice setting and organisational factors relevant to implementing guidelines.94 The measures include: Attitudes toward Lesbians and Gays Scale (α>0.80)95; Bisexualities: Indiana Attitudes Scale (α=0.91)96; Attitudes toward Transgender Individuals Scale (α=0.95)97; Lesbian, Gay, Bisexual, and Transgender Development of Clinical Skills Scale (a=0.86)98; Context (α=0.85, eg, culture, opinion leaders) and Facilitation (α=0.95, eg, senior leadership, leadership implementation) modules of the Organisational Readiness to Change Assessment 99; Implementation Climate Scale (α=0.91)100; and the Evidence-Based Practice Attitude Scale (α=0.76).101 102 The focus groups/interviews will pose open-ended questions to study in-depth organisational attributes of FQHCs and attitudinal factors, behaviours and experiences at varying levels affecting SGM care.78 Questions will centre on general knowledge/experience with SGM patients, adopting guidelines in primary care, and implementation barriers/facilitators. By tapping into provider/staff and administrator perspectives, we can understand how different levels (eg, provider/staff, social and practice setting, organisation) align to ensure optimal care for SGM people, enabling us to identify targets for and potential impediments to practice innovation. The 60-min to 90-min focus groups/interviews will be digitally recorded, transcribed and reviewed for accuracy.
Qualitative/quantitative data analysis (phase 1; aim 1; IoCM step 2)
We will import transcripts into a password-protected NVivo V.12 database for iterative analysis, first using open coding to locate themes/issues, assign codes to segments of text based a priori on topics in the focus group/interview guides, and identify and define new codes.103–105 We will also create codes based on key sensitising concepts from intersectionality theory (eg, intersecting identities, structural factors)57–59 106 and implementation science (eg, leadership, climate)94 107 that help establish ‘a general sense of reference’ for analysis.66 Second, we will use focused coding to discern codes that recur or represent unusual issues.104 105 We will cross-reference statements of interest (eg, text coded with ‘welcoming environment’ and ‘discrimination exemplar’”) to ascertain relationships in data both in and across FQHCs, and group codes with similar content into broad themes linked to retrievable text segments.104 108 We will enter the survey data into the Statistical Package for the Social Sciences V.25(SPSS) for descriptive analyses aggregated at the FQHC level,109 comparing qualitative and survey data across organisations to ascertain areas of strength and weakness regarding factors likely to affect guideline implementation in primary care. Products will include a summary of key issues to consider in prioritising/assessing guidelines and jumpstarting implementation strategy development via the NGT.
Nominal Group Technique (phase 1; aim 1; IoCM step 3)
After developing a list of guidelines and implementation strategies from the empirical literature and focus groups/interviews, we will use the NGT to prioritise them. The NGT has been fruitfully applied in direction setting in health services research and implementation science.50 110 111 The NGT convenes small groups of diverse stakeholders to generate ideas, develop consensus and set priorities for standards or guidelines, particularly in situations where the research base is inconclusive.51 110 While the NGT occurs in groups, emphasis is less on sample size and more on involving people of different roles/social locations to ensure heterogeneity of viewpoints.50 111 We will use the NGT to prioritise strategies to implement the guidelines, inviting a subset (n=8–12) of focus group/interview participants to a 2-hour NGT session held in a central location. Participants will be given the list and preprinted ‘Nominal Group Task Statement Forms’ specifying exploratory questions resembling: (1) “What are likely the most impactful and feasible guidelines or recommendations to improve care for your SGM patients?” (2) “List the strategies or steps that would best help your organisation implement and sustain guidelines and recommendations to improve care for SGM patients.” Participants will have the opportunity to select, adapt and suggest additional guidelines or implementation strategies for toolkit inclusion. They will first independently strategise in silence, then engage in a serial discussion of each idea, group ranking and vetting of priorities, and re-ranking until reaching consensus using the 70/30 consensus voting procedure that entails respectful conversation of dissenting opinions.112
Toolkit development and refinement (phase 1; aim 2; IoCM step 4)
Focus group/interview, survey and NGT data will inform the integration of existing guidelines with implementation strategies into the toolkit (table 4), with increased attention to issues of organisational context and intersectionality.1 58 59 We will work with the SAB to refine the toolkit and develop fidelity measures by reviewing outlines for each module and arriving at agreement via the 70/30 consensus method.112 We will draft easy-to-follow materials and procedures to promote change in healthcare and policies concerning SGM patients of multiple minority statuses using accessible language and drawing on examples from the above research to illustrate potential barriers/facilitators to change at the individual, social/practice setting and organisational levels. For each module, we will include tools to assess attitudes, practices and competencies; select implementation strategies to match the local context; develop feasible priorities and goals; create action plans; and then evaluate progress towards goals.
Table 4.
Preliminary outline of the comprehensive toolkit
| Module | Description |
| 1. SGM guideline overview | Underlying rationale of relevant guidelines and key issues to consider when implementing them. |
| 2. Creating an IRT | How to identify and engage providers/staff in the FQHC to lead implementation of the guidelines. |
| 3. Engaging SGM patients | How to identify, recruit and involve SGM patients of multiple minority statuses in implementation. |
| 4. Assessing organisational barriers and facilitators | How to perform a localised problem analysis of current care practices and policies related to SGM patients and identify factors likely to impact implementation of SGM practice guidelines. |
| 5. Selecting practice guidelines based on organisational assessment | How to use data from an organisational self-assessment to develop statement of practices/policies requiring change, identify barriers and facilitators, and prioritise SGM practice guidelines to implement (or improve implementation of) in the FQHC social/practice setting. |
| 6. Choosing theory-based implementation strategies | How to apply an intersectionality lens and match a theory of change at the individual, social/practice setting and organisational levels with specific SGM practice guidelines. |
| 7. Obtaining support from leaders/champions/staff | How to garner ‘buy in’ from leaders of FQHCs at various levels and actively involve physicians and other key staff as opinion leaders or champions in the change process. |
| 8. Creating action plans | How to develop action plans to guideline implementation drawing on the organisational assessment. |
| 9. Developing evaluation plans | How to select fidelity and impact measures for guidelines and implementation strategies. |
| 10.Using action plans | How to determine roles; review accomplishments, deadlines and budget; and provide feedback. |
| 11.Planning for the future | How to use evaluation data to refine implementation; recruiting new members to the FQHC implementation team; long-term strategic planning to better care for SGM people in the FQHCs. |
| Appendix: Measures and tools | Example measures, policies, documentation, intake forms, brochures, mission statements and so on. |
FQHC, Federally Qualified Health Center; IRT, implementation resource team; SGM, sexual and gender minority.
SGM patient focus groups (phase 1; aim 2; IoCM step 5)
Patient input is essential to interventions to improve primary care for SGMs.69 Community perspectives and community-identified competencies, such as being comfortable with SGM patients and shared medical decision-making between providers and patients, improve care by ensuring that community member priorities are not neglected. Community input is also crucial to determining the expertise that providers/staff may require to best care for diverse SGM patients, many of whom can articulate their experiences of minority stress in healthcare encounters.113 For this study, two focus groups (one rural, one urban) of six to eight SGM patients of varying races/ethnicities from the FQHC catchment areas will provide feedback into the toolkit’s validity and refinement. Participants will be given a copy of the toolkit to review prior to the focus group. During the first 20 min of the group, they will draft a list of toolkit gaps, acceptability and strengths/limitations. Questions asked subsequently will centre on these issues, experiences with primary care and the extent to which the toolkit addresses issues of race/ethnicity, culture, rurality and other intersections of SGM population attributes. We will analyse transcripts using the procedures described above, sharing results with the SAB to update the toolkit prior to piloting. Finally, we will share the toolkit with FQHC participants for final input before testing begins.
Toolkit pilot test (phase 2; aim 3; IoCM steps 5–7)
The FQHCs will implement the refined toolkit with ongoing coaching and assessment over 1 year. An SGM-specialist coach who is well-versed in the toolkit will meet with each FQHC’s leadership to develop an implementation resource team (IRT) that meets monthly to develop goals and action plans and monitor progress in carrying out the prioritised guidelines and implementation strategies using the PDSA method, a four-stage cyclic, iterative learning approach to test a change implemented in a clinical milieu.52–55 The IRTs are small stakeholder groups of three to five persons that will lead integration of guidelines into routine care, and may include clinic administrators, providers/staff and patient advocates. Per the first cycle—Plan—the IRT drafts a concise statement regarding a guideline to put into practice, and then an action plan describing the goal/outcome to accomplish via this guideline and associated measures. The IRT articulates the implementation strategies or steps to promote adoption of the guideline, while establishing a relatively short-term timeline for completion. For the second cycle—Do—the IRT sets the action plan into motion, observing, collecting data and documenting what happens when the strategies are executed. During this cycle, the IRT asks, “Did everything go as planned?” and determines whether the plan must be modified.54 During the third cycle—Study—the IRT examines the results of its efforts, identifying lessons learnt, whether the goal/outcome was attained with fidelity to the action plan and how well the implementation strategies worked. For the fourth cycle—Act—the IRT delineates its conclusions regarding the success of the change, clarifying what worked and did not work, and what it may do differently to facilitate productive implementation, as well as potential adaptations and next steps for scale up or a new cycle.54 55
We do not expect FQHCs to move forward with all guidelines in the toolkit at once, but to evaluate its content and proceed to implement guidelines incrementally via the PDSA method, improving on efforts to advance SGM practice changes with each successive cycle. The IRTs will also examine implementation needs and troubleshoot barriers using PDSA planning templates included in the toolkit. Thus, for instance, an IRT wanting to include SGM data in an FQHC’s electronic health record system might focus on empowering hesitant providers/staff to ask relevant questions of patients or revise patient intake forms with non-stigmatising elicitation terminology. The toolkit will include guidance and model examples related to these and other topics.
The IRTs will facilitate team collaboration to instantiate new practices and will benefit from the implementation strategy of coaching when applying the toolkit.114 115 The SGM-specialist coach will strive to build confidence in IRT members during the PDSA process, emphasising how to motivate positive behaviour change among FQHC stakeholders to foster successful implementation and fidelity or adherence to guidelines included in the toolkit.116–119 For action planning, the coach can advise on prioritising guidelines and using theory-based implementation strategies via toolkit materials (eg, assessments, checklists and examples).120
We will evaluate guideline implementation progress by undertaking walkthroughs in each FQHC and collecting and analysing minutes from (1) IRT meetings, (2) copies of completed action plans and fidelity measures in the toolkit and (3) and other organisational context documentation (eg, intake forms, brochures, policies at start and when changed). We will administer a final round of focus groups/interviews with providers/staff and administrators of each FQHC using the same sample sizes and procedures described earlier, focusing on toolkit implementation at (1) individual provider/staff, (2) social/practice setting and (3) organisational context levels. A complementary set of small group interviews with IRT members will examine changes made to address SGM patient needs, barriers to sustainment, as well as toolkit satisfaction, acceptability, usability and feasibility.121 More specifically, questions asked in focus group/interview formats will centre on how use of the toolkit influences care for SGM patients, its contributions to patient and provider/staff satisfaction, difficulties involved in applying the toolkit in real-world practice, constraints experienced by the organisation and providers/staff during implementation, overall utility and ease of employing featured implementation strategies, and the range of positive and negative factors ultimately affecting the toolkit’s uptake and perceived impacts. These data will inform final revisions to the toolkit to be agreed on by the SAB.
Limitations
The study is limited to four FQHCs in a single state, which may limit generalisability of findings and the toolkit. The purposeful sampling strategy may lead to an over-representation of clinic personnel concerned about care for SGM patients, or with vested interests in portraying themselves and the FQHCs positively. The small sample sizes also preclude implementation of a randomised controlled trial design to assess organisational and practice changes resulting from the toolkit.
Ethics and dissemination
The Pacific Institute for Research and Evaluation Institutional Review Board has reviewed and approved the study’s procedures. All FQHC employees who take part in data collection activities, such as a focus group, interview, or an NGT session, will sign an informed consent form prior to their participation. The consent form will explain that the information they share will remain confidential and will not be disclosed with anyone outside the research team. Participants will be randomly assigned identification numbers to assure confidentiality and reminded that they can leave the study at any time and for any reason without penalty. Results and products from this study will be disseminated via local/state/national presentations and peer-reviewed publications, and through social media and community/stakeholder engagement activities.
Next steps and dissemination
After advancing SGM practice guidelines, implementation strategies and indicators of guideline/implementation fidelity, we are planning a future study with a hybrid type 2 effectiveness-implementation experimental design and a larger number of FQHCs.64 65 We will assign FQHCs to (1) guidelines without implementation support, (2) guidelines with implementation support, (3) services as usual without implementation support and (4) services as usual with implementation support. Both studies will provide methods to transform how FQHCs care for racially, ethnically, socioeconomically, and geographically diverse SGM patients.
Supplementary Material
Acknowledgments
We thank the members of the New Mexico LGBTQ Health Collaborative for making this research possible, namely, Molly Adler, Cameron Crandall, Rebecca Dakota, Paige Duhamel, Edward Fancovic, Jamie Finkelstein, Jonathan Flores, Gregory Gomez, Beverly Gorman, Adrien Lawyer, Benjamin Moser, La Tisha Rico, Crystal Romney, Amber Royster, Denise Ruybal, Nathaniel Sharon and Alma Rosa Silva-Bañuelos. We also thank Patricia Hokanson for assistance with formatting the references.
Footnotes
Contributors: CW, MK, AEG and RS initiated and conceived of the study in partnership with the New Mexico LGBTQ Health Collaborative. CW, MK and AEG drafted the research protocol and overall study design. MS developed the systematic review protocol. RS, SD and KE reviewed and critically revised the protocol. All authors approved the final version of this protocol.
Funding: This work was supported by a grant from the US National Institute of Minority Health and Health Disparities (R21MD011648), and two Pipeline to Proposal awards from the Patient-Centered Outcomes Research Institute.
Competing interests: None declared.
Patient consent for publication: Not required.
Provenance and peer review: Not commissioned; externally peer reviewed.
References
- 1. Institute of Medicine The health of lesbian, gay, bisexual, and transgender people: building a foundation for better understanding. Washington, DC: National Academies Press, 2011. [PubMed] [Google Scholar]
- 2. National Institutes of Health LGBT Research Coordinating Committee Consideration of the Institute of medicine (IOM) report on the health of lesbian, gay, bisexual, and transgender (LGBT) individuals. Bethesda, MD: National Institutes of Health, 2013. [Google Scholar]
- 3. National Institutes of Health Sexual and Gender Minority Research Coordinating Committee NIH FY 2016-2020 strategic plan to advance research on the health and well-being of sexual and gender minorities. Rockville, MD: National Institutes of Health, 2016. [Google Scholar]
- 4. Fredriksen-Goldsen KI, Cook-Daniels L, Kim HJ, et al. Physical and mental health of transgender older adults: an at-risk and underserved population. Gerontologist 2014;54:488–500. 10.1093/geront/gnt021 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5. Molina Y, Lehavot K, Beadnell B, et al. Racial disparities in health behaviors and conditions among lesbian and bisexual women: the role of internalized stigma. LGBT Health 2014;1:131–9. 10.1089/lgbt.2013.0007 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6. Mayer KH, Bradford JB, Makadon HJ, et al. Sexual and gender minority health: what we know and what needs to be done. Am J Public Health 2008;98:989–95. 10.2105/AJPH.2007.127811 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7. Cochran SD, Mays VM, Sullivan JG. Prevalence of mental disorders, psychological distress, and mental health services use among lesbian, gay, and bisexual adults in the United States. J Consult Clin Psychol 2003;71:53–61. 10.1037/0022-006X.71.1.53 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8. Hutchinson MK, Thompson AC, Cederbaum JA. Multisystem factors contributing to disparities in preventive health care among lesbian women. J Obstet Gynecol Neonatal Nurs 2006;35:393–402. 10.1111/j.1552-6909.2006.00054.x [DOI] [PubMed] [Google Scholar]
- 9. Law M, Mathai A, Veinot P, et al. Exploring lesbian, gay, bisexual, and queer (LGBQ) people's experiences with disclosure of sexual identity to primary care physicians: a qualitative study. BMC Fam Pract 2015;16:175 10.1186/s12875-015-0389-4 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10. Meyer IH, Schwartz S, Frost DM. Social patterning of stress and coping: does disadvantaged social statuses confer more stress and fewer coping resources? Soc Sci Med 2008;67:368–79. 10.1016/j.socscimed.2008.03.012 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11. Meyer IH, Prejudice MIH. Prejudice, social stress, and mental health in lesbian, gay, and bisexual populations: conceptual issues and research evidence. Psychol Bull 2003;129:674–97. 10.1037/0033-2909.129.5.674 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12. Bockting WO, Miner MH, Swinburne Romine RE, et al. Stigma, mental health, and resilience in an online sample of the US transgender population. Am J Public Health 2013;103:943–51. 10.2105/AJPH.2013.301241 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13. Meyer IH. Identity, stress, and resilience in lesbians, gay men, and bisexuals of color. Couns Psychol 2010;38:442–54. 10.1177/0011000009351601 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14. Ross LE, O'Gorman L, MacLeod MA, et al. Bisexuality, poverty and mental health: a mixed methods analysis. Soc Sci Med 2016;156:64–72. 10.1016/j.socscimed.2016.03.009 [DOI] [PubMed] [Google Scholar]
- 15. Wilson EC, Chen Y-H, Arayasirikul S, et al. Differential HIV risk for racial/ethnic minority trans*female youths and socioeconomic disparities in housing, residential stability, and education. Am J Public Health 2015;105:e41–7. 10.2105/AJPH.2014.302443 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16. Barefoot KN, Rickard A, Smalley KB, et al. Rural lesbians: unique challenges and implications for mental health providers. J Rural Ment Health 2015;39:22–33. 10.1037/rmh0000014 [DOI] [Google Scholar]
- 17. Whitehead J, Shaver J, Stephenson R. Outness, stigma, and primary health care utilization among rural LGBT populations. PLoS One 2016;11:e0146139 10.1371/journal.pone.0146139 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18. Bidell MP, Stepleman LM. An interdisciplinary approach to lesbian, gay, bisexual, and transgender clinical competence, professional training, and ethical care: introduction to the special issue. J Homosex 2017;64:1305–29. 10.1080/00918369.2017.1321360 [DOI] [PubMed] [Google Scholar]
- 19. Wright J. Only your calamity: the beginnings of activism by and for people with AIDS. Am J Public Health 2013;103:1788–98. 10.2105/AJPH.2013.301381 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20. Haldeman DC. The practice and ethics of sexual orientation conversion therapy. J Consult Clin Psychol 1994;62:221–7. 10.1037/0022-006X.62.2.221 [DOI] [PubMed] [Google Scholar]
- 21. Dreger AD. A history of intersexuality: from the age of gonads to the age of consent. J Clin Ethics 1998;9:345–55. [PubMed] [Google Scholar]
- 22. Baker K, Beagan B, assumptions M. Making assumptions, making space: an anthropological critique of cultural competency and its relevance to Queer patients. Med Anthropol Q 2014;28:578–98. 10.1111/maq.12129 [DOI] [PubMed] [Google Scholar]
- 23. Sabin JA, Riskind RG, Nosek BA. Health care providers’ implicit and explicit attitudes toward lesbian women and gay men. Am J Public Health 2015;105:1831–41. 10.2105/AJPH.2015.302631 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24. Sue DW, Microaggressions and marginality: Manifestation, dynamic, and impact. Hoboken, NJ: John Wiley and Sons, Inc, 2010. [Google Scholar]
- 25. Hollenbach AD, Eckstrand KL, Dreger A, Implementing curricular and institutional climate changes to improve health care for individuals who are LGBT, gender nonconforming, or born with DSD: A resource for medical educators. Washington, DC: Association for American Medical Colleges Advisory Committee on Sexual Orientation, Gender Identity and Sex Development, 2014. [Google Scholar]
- 26. Davy Z, Siriwardena AN. To be or not to be LGBT in primary health care: health care for lesbian, gay, bisexual, and transgender people. Br J Gen Pract 2012;62:491–2. 10.3399/bjgp12X654731 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27. Donaldson MS, Yordy KD, Lohr KN, et al. Primary care: America's health in a new era. Washington, DC: National Academies Press, 1996. [PubMed] [Google Scholar]
- 28. Starfield B. Primary care: balancing health needs, services, and technology. New York, NY: Oxford University Press, 1998. [Google Scholar]
- 29. Starfield B, Shi L, Macinko J. Contribution of primary care to health systems and health. Milbank Q 2005;83:457–502. 10.1111/j.1468-0009.2005.00409.x [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30. Eliason MJ, Schope R. Does "don't ask don't tell" apply to health care? Lesbian, gay, and bisexual people's disclosure to health providers. J Gay Lesbian Med Assoc 2001;5:125–34. 10.1023/A:1014257910462 [DOI] [Google Scholar]
- 31. McNair RP, Hegarty K. Guidelines for the primary care of lesbian, gay, and bisexual people: a systematic review. Ann Fam Med 2010;8:533–41. 10.1370/afm.1173 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32. Wilkerson JM, Rybicki S, Barber CA, et al. Creating a culturally competent clinical environment for LGBT patients. J Gay Lesbian Soc Serv 2011;23:376–94. 10.1080/10538720.2011.589254 [DOI] [Google Scholar]
- 33. McClain Z, Hawkins LA, Yehia BR. Creating welcoming spaces for lesbian, gay, bisexual, and transgender (LGBT) patients: an evaluation of the health care environment. J Homosex 2016;63:387–93. 10.1080/00918369.2016.1124694 [DOI] [PubMed] [Google Scholar]
- 34. Rossi AL, Lopez EJ. Contextualizing competence: language and LGBT-based competency in health care. J Homosex 2017;64:1330–49. 10.1080/00918369.2017.1321361 [DOI] [PubMed] [Google Scholar]
- 35. Kano M, Silva-Bañuelos AR, Sturm R, et al. Stakeholders' recommendations to improve patient-centered "LGBTQ" primary care in rural and multicultural practices. J Am Board Fam Med 2016;29:156–60. 10.3122/jabfm.2016.01.150205 [DOI] [PubMed] [Google Scholar]
- 36. Dahan R, Feldman R, Hermoni D. Is patients' sexual orientation a blind spot of family physicians? J Homosex 2008;55:524–32. 10.1080/00918360802345321 [DOI] [PubMed] [Google Scholar]
- 37. Willging CE, Salvador M, Kano M. Brief reports: unequal treatment: mental health care for sexual and gender minority groups in a rural state. PS 2006;57:867–70. 10.1176/ps.2006.57.6.867 [DOI] [PubMed] [Google Scholar]
- 38. Johnson MJ, Nemeth LS. Addressing health disparities of lesbian and bisexual women: a grounded theory study. Womens Health Issues 2014;24:635–40. 10.1016/j.whi.2014.08.003 [DOI] [PubMed] [Google Scholar]
- 39. Petroll AE, Mosack KE. Physician awareness of sexual orientation and preventive health recommendations to men who have sex with men. Sex Transm Dis 2011;38:63–7. 10.1097/OLQ.0b013e3181ebd50f [DOI] [PMC free article] [PubMed] [Google Scholar]
- 40. Korpaisarn S, Safer JD. Gaps in transgender medical education among healthcare providers: a major barrier to care for transgender persons. Rev Endocr Metab Disord 2018;19:271–5. 10.1007/s11154-018-9452-5 [DOI] [PubMed] [Google Scholar]
- 41. Streed CG, Davis JA. Improving clinical education and training on sexual and gender minority health. Curr Sex Health Rep 2018;10:273–80. 10.1007/s11930-018-0185-y [DOI] [Google Scholar]
- 42. Eckstrand KL, Lunn MR, Yehia BR. Applying organizational change to promote lesbian, gay, bisexual, and transgender inclusion and reduce health disparities. LGBT Health 2017;4:174–80. 10.1089/lgbt.2015.0148 [DOI] [PubMed] [Google Scholar]
- 43. U.S. Department of Health and Human Services Better health and well-being: making improvements for lesbian, gay, bisexual, transgender (LGBT) Americans. Washington, DC: U.S. Department of Health and Human Services, 2012. [Google Scholar]
- 44. Society of Teachers of Family Medicine Group on LGBT Health Recommended curriculum guidelines for family medicine residents: lesbian, gay, bisexual, transgender health. Leawood, KS: American Academy of Family Physicians, 2013. [Google Scholar]
- 45. The Joint Commission Advancing effective communication, cultural competence, and patient-and-family centered care for the lesbian, gay, bisexual, and transgender (LGBT) community: a field guide. Oak Brook, IL: The Joint Commission, 2011. [Google Scholar]
- 46. Schilder AJ, Kennedy C, Goldstone IL, et al. "Being dealt with as a whole person." Care seeking and adherence: the benefits of culturally competent care. Soc Sci Med 2001;52:1643–59. 10.1016/S0277-9536(00)00274-4 [DOI] [PubMed] [Google Scholar]
- 47. Klein DA, Malcolm NM, Berry-Bibee EN, et al. Quality primary care and family planning services for LGBT clients: a comprehensive review of clinical guidelines. LGBT Health 2018;5:153–70. 10.1089/lgbt.2017.0213 [DOI] [PubMed] [Google Scholar]
- 48. Meyers DC, Durlak JA, Wandersman A. The quality implementation framework: a synthesis of critical steps in the implementation process. Am J Community Psychol 2012;50:462–80. 10.1007/s10464-012-9522-x [DOI] [PubMed] [Google Scholar]
- 49. Grol R, Ouwens M, Wollersheim H. Planning and organizing the change process : Grol R, Wensing M, Eccles M, Improving patient care: the implementation of change in health care. 2nd edn Hoboken, NJ: John Wiley & Sons, 2013: 64–76. [Google Scholar]
- 50. Van de Ven AH, Delbecq AL. The nominal group as a research instrument for exploratory health studies. Am J Public Health 1972;62:337–42. 10.2105/AJPH.62.3.337 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 51. Rankin NM, McGregor D, Butow PN, et al. Adapting the nominal group technique for priority setting of evidence-practice gaps in implementation science. BMC Med Res Methodol 2016;16:110 10.1186/s12874-016-0210-7 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 52. Speroff T, O'Connor GT. Study designs for PDSA quality improvement research. Qual Manag Health Care 2004;13:17–32. 10.1097/00019514-200401000-00002 [DOI] [PubMed] [Google Scholar]
- 53. Gillam S, Siriwardena AN. Frameworks for improvement: clinical audit, the plan-do-study-act cycle and significant event audit. Qual Prim Care 2013;21:123–30. [PubMed] [Google Scholar]
- 54. Agency for Healthcare Research and Quality Plan-do-study-act (PDSA) directions and examples. Available: http://www.ahrq.gov/professionals/quality-patient-safety/quality-resources/tools/literacy-toolkit/healthlittoolkit2-tool2b.html [Accessed Nov 2019].
- 55. Taylor MJ, McNicholas C, Nicolay C, et al. Systematic review of the application of the plan-do-study-act method to improve quality in healthcare. BMJ Qual Saf 2014;23:290–8. 10.1136/bmjqs-2013-001862 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 56. Wandersman A, Chien VH, Katz J. Toward an evidence-based system for innovation support for implementing innovations with quality: tools, training, technical assistance, and quality assurance/quality improvement. Am J Community Psychol 2012;50:445–59. 10.1007/s10464-012-9509-7 [DOI] [PubMed] [Google Scholar]
- 57. Choo HY, Ferree MM. Practicing intersectionality in sociological research: a critical analysis of inclusions, interactions, and institutions in the study of inequalities. Sociological Theory 2010;28:129–49. 10.1111/j.1467-9558.2010.01370.x [DOI] [Google Scholar]
- 58. Crenshaw K. Demarginalizing the intersection of race and sex: a black feminist critique of antidiscrimination doctrine, feminist theory and antiracist politics. Univ Chic Leg Forum 1989;1989:139–67. [Google Scholar]
- 59. Crenshaw K. Mapping the margins: intersectionality, identity politics, and violence against women of color. Stanford Law Rev 1991;43:1241–99. 10.2307/1229039 [DOI] [Google Scholar]
- 60. Bauer GR. Incorporating intersectionality theory into population health research methodology: challenges and the potential to advance health equity. Soc Sci Med 2014;110:10–17. 10.1016/j.socscimed.2014.03.022 [DOI] [PubMed] [Google Scholar]
- 61. Coulter RWS, Kenst KS, Bowen DJ, et al. Research funded by the National Institutes of Health on the health of lesbian, gay, bisexual, and transgender populations. Am J Public Health 2014;104:e105–12. 10.2105/AJPH.2013.301501 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 62. Coulter RWS, Kenst KS, Bowen DJ, et al. Coulter et al. respond. Am J Public Health 2015;105:e2–3. 10.2105/AJPH.2014.302316 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 63. Voyles CH, Sell RL. Continued disparities in lesbian, gay, and bisexual research funding at NIH. Am J Public Health 2015;105:e1–2. 10.2105/AJPH.2014.302265 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 64. Curran GM, Bauer M, Mittman B, et al. Effectiveness-implementation hybrid designs: combining elements of clinical effectiveness and implementation research to enhance public health impact. Med Care 2012;50:217–26. 10.1097/MLR.0b013e3182408812 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 65. Bernet AC, Willens DE, Bauer MS. Effectiveness-implementation hybrid designs: implications for quality improvement science. Implement Sci 2013;8:S2 10.1186/1748-5908-8-S1-S2 [DOI] [Google Scholar]
- 66. Patton MQ. Qualitative research & evaluation methods: integrating theory and methods. 4th edn Thousand Oaks, CA: Sage Publications, 2015. [Google Scholar]
- 67. Gagliardi AR, Alhabib S. Trends in guideline implementation: a scoping systematic review. Implement Sci 2015;10:54 10.1186/s13012-015-0247-8 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 68. Grol R. Implementing guidelines in general practice care. Qual Health Care 1992;1:184–91. 10.1136/qshc.1.3.184 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 69. Kano M, Sawyer KP, Willging CE. Fostering CBPR to promote equitable healthcare for LGTBQ+ people : Wallersten N, Duran B, Oetzel J, CBPR for health: engaging for social and health equity. 3rd edn San Francisco, CA: Jossey-Bass, 2017: 345–50. [Google Scholar]
- 70. Aarons GA, Sawitzky AC. Organizational climate partially mediates the effect of culture on work attitudes and staff turnover in mental health services. Adm Policy Ment Health 2006;33:289–301. 10.1007/s10488-006-0039-1 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 71. Aarons GA, Glisson C, Green PD, et al. The organizational social context of mental health services and clinician attitudes toward evidence-based practice: a United States national study. Implementation Sci 2012;7:56 10.1186/1748-5908-7-56 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 72. Aarons GA, Ehrhart MG, Farahnak LR, et al. Aligning leadership across systems and organizations to develop a strategic climate for evidence-based practice implementation. Annu Rev Public Health 2014;35:255–74. 10.1146/annurev-publhealth-032013-182447 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 73. Aarons GA. Transformational and transactional leadership: association with attitudes toward evidence-based practice. PS 2006;57:1162–9. 10.1176/ps.2006.57.8.1162 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 74. Thaker S, Steckler A, Sanchez V, et al. Program characteristics and organizational factors affecting the implementation of a school-based indicated prevention program. Health Educ Res 2008;23:238–48. 10.1093/her/cym025 [DOI] [PubMed] [Google Scholar]
- 75. Hankivsky H, Cormier R. Intersectionality: moving women's health research and policy forward. Vancouver, Canada: Women's Health Research Network, 2009. [Google Scholar]
- 76. Lehman WEK, Greener JM, Simpson DD. Assessing organizational readiness for change. J Subst Abuse Treat 2002;22:197–209. 10.1016/S0740-5472(02)00233-7 [DOI] [PubMed] [Google Scholar]
- 77. Grol R, Wensing M, Bosch M. Theories on implementation of change in healthcare : Grol R, Wensing M, Eccles M, Improving patient care: the implementation of change in health care. 2nd ed Hoboken, NJ: John Wiley & Sons, 2013: 18–39. [Google Scholar]
- 78. Grol R, Wensing M. Effective implementation of change in healthcare: A systematic approach : Grol R, Wensing M, Eccles M, Improving patient care: the implementation of change in health care. 2nd ed Hoboken, NJ: John Wiley & Sons, 2013: 40–63. [Google Scholar]
- 79. U.S. Census Bureau Survey 1-year estimates ranking table: median household income (in 2017 inflation-adjusted dollars), 2019. Available: https://www.census.gov/acs/www/data/data-tables-and-tools/ranking-tables/
- 80. U.S. Census Bureau Amercian community survey 1-year ranking table: percent of people below poverty level in the past 12 months (for whom poverty status is determined), 2017. Available: https://www.census.gov/acs/www/data/data-tables-and-tools/ranking-tables/ [Accessed 25 Jun 2019].
- 81. U.S. Census Bureau State and county quick facts - New Mexico 2018. Available: https://www.census.gov/quickfacts/ [Accessed 25 Jun 2019].
- 82. Padilla J. Overview of New Mexico LGBT health data for the legislative health and human service Committee. Santa Fe, NM, 2014. [Google Scholar]
- 83. Tomedi LE, Oglesbee S, Padilla J, et al. The health and well-being of lesbian, gay, and bisexual youth in New Mexico: data from the 2015 New Mexico Youth Risk & Resiliency Survey. Santa Fe, NM: New Mexico Department of Health, New Mexico Public Education Department, and University of New Mexico Prevention Research Center, 2017. [Google Scholar]
- 84. Flores A, Herman J, Gates G. How many adults identify as transgender in the United States? Los Angeles, CA: The Williams Institute, 2016. [Google Scholar]
- 85. NM YRRRS Connections Facts and figures from 2017 the new Mexico youth risk and Resiliency survey: gender identity. Santa Fe, NM: New Mexico Department of Health, Public Education Department, and University of New Mexico Prevention Research Center, 2018. [Google Scholar]
- 86. Tomedi LE, Padilla JL. Health inequities by sexual orientation in New Mexico 2005-2011. New Mexico Department of Health: Santa Fe, NM, 2013. [Google Scholar]
- 87. Aarons GA, Palinkas LA. Implementation of evidence-based practice in child welfare: service provider perspectives. Adm Policy Ment Health 2007;34:411–9. 10.1007/s10488-007-0121-3 [DOI] [PubMed] [Google Scholar]
- 88. Gay and Lesbian Medical Association Healthy people 2010: companion document for lesbian, gay, bisexual, and transgender (LGBT) health. San Francisco, CA: Gay and Lesbian Medical Association, 2001. [Google Scholar]
- 89. Willging C. "I'm not that innocent": Recruiting sexual and gender minorities for rural mental health research. Pract Anthropol 2007;29:10–14. 10.17730/praa.29.2.3t6r78t4366563q1 [DOI] [Google Scholar]
- 90. Human Rights Campaign Healthcare equity index. Available: https://www.hrc.org/hei [Accessed Nov 2019].
- 91. Human Rights Campaign HEI scoring criteria. Available: https://www.hrc.org/hei/hei-scoring-criteria [Accessed Nov 2019].
- 92. Gay and Lesbian Medical Association Guidelines for care of lesbian, gay, bisexual, and transgender patients. San Francisco, CA: Gay and Lesbian Medical Association, 2006. [Google Scholar]
- 93. Endnote X8 2018.
- 94. Aarons GA, Horowitz JD, Dlugosz LR. The role of organizational processes in dissemination and implementation research : Brownson RC, Colditz GA, Proctor EK, Dissemination and implementation research in health: translating science to practice. New York, NY: Oxford University Press, 2012: 128–53. [Google Scholar]
- 95. Herek GM. Attitudes Toward Lesbians and Gay Men Scale : Davis CM, Yaber WL, Bauserman RL, Handbook of sexuality-related measures. Thousand Oaks, CA: Sage Publications, 1998: 392–3. [Google Scholar]
- 96. Dodge B, Herbenick D, Friedman MR, et al. Attitudes toward bisexual men and women among a nationally representative probability sample of adults in the United States. PLoS One 2016;11:e0164430 10.1371/journal.pone.0164430 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 97. Walch SE, Ngamake ST, Francisco J, et al. The Attitudes Toward Transgendered Individuals Scale: psychometric properties. Arch Sex Behav 2012;41:1283–91. 10.1007/s10508-012-9995-6 [DOI] [PubMed] [Google Scholar]
- 98. Bidell MP. The Lesbian, Gay, Bisexual, and Transgender Development of Clinical Skills Scale (LGBT-DOCSS): establishing a new interdisciplinary self-assessment for health providers. J Homosex 2017;64:1432–60. 10.1080/00918369.2017.1321389 [DOI] [PubMed] [Google Scholar]
- 99. Helfrich CD, Li Y-F, Sharp ND, et al. Organizational Readiness to Change Assessment (ORCA): development of an instrument based on the Promoting Action on Research in Health Services (PARIHS) framework. Implementation Sci 2009;4:38 10.1186/1748-5908-4-38 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 100. Ehrhart MG, Aarons GA, Farahnak LR. Assessing the organizational context for EBP implementation: the development and validity testing of the Implementation Climate Scale (ICS). Implementation Sci 2014;9:157 10.1186/s13012-014-0157-1 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 101. Aarons GA. Mental health provider attitudes toward adoption of evidence-based practice: the Evidence-Based Practice Attitude Scale (EBPAS). Ment Health Serv Res 2004;6:61–74. 10.1023/B:MHSR.0000024351.12294.65 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 102. Bass BM, Avolio BJ, Improving organizational effectiveness through transformational leadership. Thousand Oaks, CA: Sage Publications, 1994. [Google Scholar]
- 103. NVivo qualitative data analysis software Version 12 2018.
- 104. Corbin J, Strauss A. Basics of qualitative research: techniques and procedures for developing grounded theory. 3rd ed Thousand Oaks, CA: Sage Publications, 2008. [Google Scholar]
- 105. Emerson RM, Fretz RI, Shaw LL. Writing ethnographic fieldnotes. Chicago, IL: University of Chicago Press, 1995. [Google Scholar]
- 106. Rogers J, Kelly UA. Feminist intersectionality: bringing social justice to health disparities research. Nurs Ethics 2011;18:397–407. 10.1177/0969733011398094 [DOI] [PubMed] [Google Scholar]
- 107. Aarons GA, Green AE, Willging CE, et al. Mixed-method study of a conceptual model of evidence-based intervention sustainment across multiple public-sector service settings. Implementation Sci 2014;9:183 10.1186/s13012-014-0183-z [DOI] [PMC free article] [PubMed] [Google Scholar]
- 108. Glaser BG, Strauss AL. The discovery of grounded theory: strategies for qualitative research. New York, NY: Aldine de Gruyter, 1967. [Google Scholar]
- 109. SPSS Statistics for Windows Version 19 2010.
- 110. Cantrill JA, Sibbald B, Buetow S. The Delphi and nominal group techniques in health services research. Int J Pharm Pract 1996;4:67–74. 10.1111/j.2042-7174.1996.tb00844.x [DOI] [Google Scholar]
- 111. Harvey N, Holmes CA. Nominal group technique: an effective method for obtaining group consensus. Int J Nurs Pract 2012;18:188–94. 10.1111/j.1440-172X.2012.02017.x [DOI] [PubMed] [Google Scholar]
- 112. Butterfoss FD. Coalitions and partnerships in community health. San Francisco, CA: John Wiley & Sons, Inc, 2007. [Google Scholar]
- 113. Alpert AB, CichoskiKelly EM, Fox AD. What lesbian, gay, bisexual, transgender, queer, and intersex patients say doctors should know and do: a qualitative study. J Homosex 2017;64:1368–89. 10.1080/00918369.2017.1321376 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 114. Willging CE, Israel T, Ley D, et al. Coaching mental health peer advocates for rural LGBTQ people. J Gay Lesbian Ment Health 2016;20:214–36. 10.1080/19359705.2016.1166469 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 115. Willging CE, Green AE, Ramos MM. Implementing school nursing strategies to reduce LGBTQ adolescent suicide: a randomized cluster trial study protocol. Implementation Sci 2016;11:145 10.1186/s13012-016-0507-2 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 116. Duda MA, Barrett S. Coaching for competence and impact brief 1: defining coaching. Washington, DC: OSEP Center on Positive Behavioral Interventions & Supports, 2013. [Google Scholar]
- 117. Edmunds JM, Kendall PC, Ringle VA, et al. An examination of behavioral rehearsal during consultation as a predictor of training outcomes. Adm Policy Ment Health 2013;40:456–66. 10.1007/s10488-013-0490-8 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 118. Fixsen DL, Blase KA, Naoom SF, et al. Core implementation components. Res Soc Work Pract 2009;19:531–40. 10.1177/1049731509335549 [DOI] [Google Scholar]
- 119. Nadeem E, Gleacher A, Beidas RS. Consultation as an implementation strategy for evidence-based practices across multiple contexts: unpacking the black box. Adm Policy Ment Health 2013;40:439–50. 10.1007/s10488-013-0502-8 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 120. Oxman AD, Fretheim A, Schünemann HJ, et al. Improving the use of research evidence in guideline development: introduction. Health Res Policy Sys 2006;4 10.1186/1478-4505-4-12 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 121. Weiner BJ, Lewis CC, Stanick C, et al. Psychometric assessment of three newly developed implementation outcome measures. Implementation Sci 2017;12:108 10.1186/s13012-017-0635-3 [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Data Citations
- Endnote X8 2018.
Supplementary Materials
bmjopen-2019-032787supp001.pdf (53.1KB, pdf)

