Abstract
The HIV epidemic remains a public health threat in the U.S., and the dissemination and implementation of evidence-based prevention and care programs are critical to addressing significant HIV health disparities. The provision of technical assistance (TA) to program providers and evaluators is key for uptake of these programs. The University of California San Francisco Prevention Research Center (UCSF PRC) model for TA delivery uses topics and strategies adapted to address HIV health disparities for a global audience. This model specifically matches TA requests to a TA provider who has expertise in that area upon receiving a request through various communication channels. Areas of expertise include research methods, community engagement strategies, interventions, and Implementation Sciences. Our evaluation of diverse TA services indicates that on-demand TA is effective for light-touch requests and well-suited for moderate to intensive requests. The model is a promising, broad-reaching, and responsive alternative for providing TA to a multitude of HIV workforce recipients.
Keywords: technical assistance, health disparities, HIV, capacity-building, evaluation
Highlights
What do We Already Know About This Topic?
Technical assistance (TA) is crucial for the uptake of HIV prevention and care programs to address health disparities, but there is no consensus on frameworks or models for delivering quality TA.
How Does Your Research Contribute to the Field?
The University of California San Francisco Prevention Research Center (UCSF PRC) presents a mixed-methods evaluation of an on-demand TA model for “light-, medium-, and heavy-touch” requests.
What Are Your Research’s Implications Towards Theory, Practice, or Policy?
The UCSF PRC model of TA provision based on a spectrum of intensity could serve as an alternative for public health academic centers conducting health equity research.
Introduction
The HIV epidemic remains a public health threat in the U.S. with more than 1 million people living with HIV and more than 37 000 becoming infected every year. 1 Given the progress in reducing new HIV infections, numerous federal agencies have included prevention in public health initiatives, including the recent federal multi-agency Ending the Epidemic, which aims to reduce new infections by 75% in the next 5 years and 90% in the next 10 years. 2 The dissemination and implementation of evidence-based prevention and care programs is critical to improving community health and eliminating significant HIV health disparities. Yet, ways to support program implementers are not well defined; notably, the best ways for providing technical assistance (TA) on the delivery of evidence-based interventions (EBIs) have not been articulated by TA providers or consumers of TA.
The University of California San Francisco Prevention Research Center (UCSF PRC) focuses on conducting, disseminating, and translating findings from innovative HIV research. The primary aims of the Centers for Disease Control and Prevention (CDC)-funded Prevention Research Centers are to: (1) establish, maintain, and operate multi-disciplinary academic-based centers that conduct high-quality applied health promotion and disease prevention research; (2) improve public health practice through applied prevention research; and (3) apply the knowledge and expertise of academic health centers to address practical public health problems. To achieve these aims, the UCSF PRC supports strong and sustainable long-term relationships with community stakeholders that are necessary to bridge the often siloed gap between researchers and communities. 3 A key component of this objective is the provision of TA.
Model of Technical Assistance Provision
A 2016 synthesis of the scientific literature on TA found no consensus on either frameworks for delivering quality TA or what should or does occur during the provision of TA. 4 TA activities can vary widely, from providing quick and easy access to resources to engaging in sustained capacity-building partnerships. Capacity-building organizations such as the UCSF PRC typically provide TA through consultations and relevant knowledge products and resources, including fact sheets, survey instruments, and research briefs. 5 TA can be generally defined as “the support and assistance that a prevention effort receives from someone or some organization that is not a part of a community team,” 6 or “an individualized and hands-on approach to capacity building in organizations and communities.” 4 This paper describes the evaluation of an innovative and flexible TA model that reaches broad audiences and accommodates a wide variety of TA needs.
Methods
Setting
While TA in public health is usually tied to EBIs, 7 the UCSF PRC model employs a broad definition of TA that extends beyond EBIs, because it always considers the role of HIV health disparities and health equity for a global audience by incorporating content from UCSF PRC constituents working in HIV prevention, care, and research. We employ an on-demand, responsive model for providing TA, which is available to the general public, free of charge, and with quick turn-around times for service. We also adapt our TA to the intensity of the request and incorporate a health equity lens in all responses. While the topical expertise is specific to HIV, the potential recipients are not limited by funding source, organization, or geographical region. In this case study, we present the results of our preliminary evaluation of this TA model: (1) a summary of the TA activities, intensity, and description of recipients, including communities, health care systems, government agencies, institutions, and universities and (2) feedback on how receipt of TA has contributed to health disparities reduction efforts by recipients.
Process for Responding to TA Requests
At the UCSF PRC, TA services are marketed through a public-facing website, social media, email, print materials, and word of mouth. The goal of marketing efforts is to reach academics and researchers, health departments and service providers, community organizations, and other individuals affected by HIV. We receive direct requests primarily through email, the website contact form, or in-person encounters. Upon receiving a TA request, the point-of-contact staff person triages the TA request to a subject matter expert within the UCSF PRC deemed the most appropriate to respond. Areas of expertise in the PRC Core include research methods, community engagement strategies, interventions, and Implementation sciences. We then confirm with the TA provider as to whether they were able to successfully fulfill the request. After x time, we assess satisfaction with TA provision by administering a brief satisfaction survey. The request and response are logged in an internal tracking and monitoring database. Upon confirmation of request fulfillment with the individual who provided the TA, we subsequently send a survey link to the TA recipient to assess their satisfaction and experience (Figure 1).
Figure 1.
University of California San Francisco, Prevention Research Center Model of Technical Assistance Delivery.
Multi-Method TA Evaluation Measures
The survey sent to TA recipients contains both closed-ended Likert-scale response items (see Appendix) as well as open-ended questions (e.g., “Tell us how the TA services resources or products you received affected your work.”). Survey responses are not linked to specific requests and are stored separately. For the analyses presented here, we use means and frequencies to understand the types of TA requests and time needed to respond, as well as the aggregate results of quantitative satisfaction measures. For open-ended responses, we employed an inductive thematic analysis approach. 8 Independently, two authors developed initial codes for the open-ended responses, piloted the codes to determine whether they needed additional detail or if new codes were needed, and met together to reach consensus and finalize the codes.
Results
Quantitative Findings
From January 2016 through May 2019, the UCSF PRC received 121 requests for TA. The time to complete a TA request ranged from 15 minutes to 8 hours with the most frequent time reported as 15 minutes (M = 1.06 hours, SD = 1.24) and 90% of requests taking 2 hours or less to complete. Table 1 lists the types of TA requested and whether the request assisted the requester in their work towards reducing health disparities related to ethnic, gender, or sexual minority groups (40.5% addressed at least one). TA involving evidence-based interventions or health disparities research that included populations that fell under the federal definition of any of these groups are noted.
Table 1.
Types of TA Requested and Intersection with Minority Status.
| Type of request | Frequency (% of total a ) | Ethnic (% of type a ) | Gender (% of type a ) | Sexual (% of type a ) |
|---|---|---|---|---|
| Referral to HIV resource (e.g., provide direction to resource or contact information of expert) | 22 (18.2) | 4 (18.2) | 7 (31.8) | 6 (27.3) |
| Linkage to expert (e.g., directly connect to an outside expert via hand-off) | 22 (18.2) | 7 (31.8) | 6 (27.3) | 5 (22.7) |
| Information transfer (e.g., provide information from research such as data collection protocol) | 10 (8.3) | 4 (40.0) | 4 (40.0) | 5 (50.0) |
| Consultation (e.g., provide real-time assistance and follow up) | 60 (49.6) | 19 (31.6) | 21 (35) | 25 (41.7) |
| Literature/Article search (e.g., provide specifically requested article or set of articles on requested topic) | 14 (11.6) | 5 (35.7) | 4 (28.6) | 4 (28.6) |
| Request for data (e.g., identify and provide access to requested datasets) | 3 (2.5) | 0 | 0 | 0 |
| Request for research instrument (e.g., locate and provide specific or relevant research instrument, survey, or scale) | 18 (14.9) | 2 (11.1) | 6 (33.3) | 5 (27.8) |
| Other (all others) | 8 (6.6) | 3 (37.5) | 1 (12.5) | 3 (37.5) |
aRequests can meet more than one category or minority issue, so no column or row will total 100%.
Of the 121 requests, 88 (72.7%) recipients completed TA evaluation surveys. Many were either from a UCSF employee (35.2%) or referral from a UCSF employee (14.8%). They were also received through online methods (Internet search 12.5%, UCSF PRC website 11.4%, and social media 3.0%). The most common service, product, or resource reported as being received was consultation (46.6%) followed by linkage to other resources (22.7%). Satisfaction with the TA provided was high across all items (see Appendix). All respondents said they would use the services again.
Qualitative Findings
Open-ended responses to the evaluation survey questions were generally positive, supporting the quantitative results. The primary codes that emerged in the data analysis were consistent with a conceptualization of technical assistance posited by Social Policy Research Associates, 9 with a spectrum of “light touch” to “heavy touch” based on the depth of the TA request and the amount of effort (i.e., the proportion of time and resources expended in any given timeframe).
Light-Touch
Light-touch requests are those that are very straightforward and easy to respond to; these requests typically required little time to address. These commonly included research assistance like data collection (e.g., identifying and locating appropriate survey instruments and implementing qualitative approaches) and background research (e.g., early stages of the research process and scientific literature review). In one case, a community-based organization asked UCSF to identify methods for conducting a community assessment. Other requests also pertained to immediate research dissemination.
The guide and sample for conducting a community assessment was very helpful and provided needed direction on how to approach assessment. [Data collection]
In another case, UCSF was able to provide access to academic literature that the requestor would not otherwise have had:
I’m using the journal articles to prepare a position paper on drug user health and safe consumption spaces. I would not have had access to them were it not for [UCSF PRC staff person]. [Background research]
Medium-Touch
Medium-touch requests are those that require moderate effort and tailoring, generally completed over the course of several days. These commonly included TA pertaining to writing, such as grant proposals or journal publications, and communication (e.g., marketing, social media, website management, and study recruitment), as illustrated by this request, regarding a research proposal:
The review came right before writing the final proposal draft and so comments were able to be incorporated. Terrific resource! We heard things that were surprising and that we hadn’t considered. [Grant proposal]
Other TA requests that required moderate effort were related to research-related dissemination and leveraging the power of community advisory boards:
I needed a perspective on how to describe activities related to research to community social media dissemination and got help immediately [Grant proposal]
My interview … provided me with valuable insight into how long-standing Community Advisory Boards (CABs) function and can be successful in their mission. [Consultation]
Heavy-Touch
Heavy-touch requests are work-intensive and long-term, generally requiring more effort or an extended period for ongoing support. These requests mainly pertained to research training and mentorship. Heavy-touch TA has included, for example, training a youth advisory board (YAB) for research activities as well as mentoring junior scientists and young professionals.
[UCSF PRC staff person] helped us in creating our Youth Advisory Panel (YAP). [They] guided us in creating by-laws and how to go about choosing candidates for the board. [Mentoring]
It was great to have [UCSF PRC staff person] present on the history of the UCSF PRC CAB and demonstrate some of the work they have been able to accomplish. The idea was to provide a model and inspire the newly found Youth Advisory Panel and their inspiration definitely delivered. [Training]
Discussion
TA has been shown to be an implementation strategy to help move prevention science into practice.10,11 Federal funding from the CDC has enabled UCSF and its sister PRCs across the U.S. to provide tangible support for reducing health disparities and increasing health equity; the provision of diverse TA services is an important component of our strategy toward that end. Our approach is an amalgam of the best TA practices by matching TA requests to a TA provider who has expertise in that area, and referral to subject matter experts is appreciated by recipients. In this multi-method evaluation of UCSF PRC’s model of TA delivery, not only were on-demand, intensity-adapted services reported as beneficial for light-touch requests, but feedback also indicates they can be well-suited for moderate to more intensive requests. All requests were grounded in the UCSF PRC’s mission of reducing health disparities. Despite not adhering to any particular TA framework in the public health evidence base,4,5 our findings imply that our generalized approach to providing TA may be promising. Overall, feedback indicated that our TA services met a wide breadth of needs across a spectrum of intensity and level of touch through either short-term (time-sensitive) or long-term (time-intensive), serving multiple topics and audiences and helping recipients address HIV-related health disparities in their communities. This supports the consideration for “TA as a continuum” focused on content vs relationships.9,12
Our strengths included providing concrete, practical help (e.g., sharing journal articles, study instruments, and data), social media promotion to increase the reach of programs, facilitating academic-community collaborations (e.g., CABs and YAPs), and predominantly supporting the research process (e.g., publication/proposal feedback, mentoring, and linkage). Areas of improvement that we identified from respondents include the following: increasing budget to hire more staff and increase TA efforts, wider advertising of our TA resources to community nonprofits, providing didactic courses, consultation in developing research-to-community education materials, and ongoing research project support for building dissemination services and products.
Our TA model breaks the mold in the public health context by expanding our definition of TA beyond EBIs, funder requirements, or geographical boundaries to provide a wide breadth of services to support: (1) research (support throughout different phases of the research process), (2) communication (input for designing promotional materials and leveraging communication channels), (3) dissemination (sharing research findings to wider community audiences), and (4) training and mentoring (guiding and equipping individuals with necessary knowledge and skills). Academic centers supporting public health and health disparities research could consider implementing and evaluating models of TA that reflect their capacity and audience’s needs, while remaining creative in how they conceptualize and deliver TA services.
Appendix A.
Likert-Scale and Open-Ended Survey Items from University of California San Francisco, Prevention Research Center’s Technical Assistance Services Satisfaction Survey, 2016–2019
Closed-Ended and Likert Items
| Responses to Satisfaction Survey Closed-Ended Items | |||
|---|---|---|---|
| Item | Response options | M (SD) | |
| 1. How satisfied are you with the | |||
| A. Timeliness of the delivery of TA services/products? | 1 “very dissatisfied” to 5 “very satisfied” | 4.82 (.69) | |
| B. Accessibility of the staff person? | 4.77 (.74) | ||
| C. Professionalism of the staff person? | 4.86 (.65) | ||
| D. Cultural sensitivity of the staff person? | 4.72 (.79) | ||
| e. Staff person’s flexibility to adjust to your unique needs? | 4.80 (.71) | ||
| f. Staff person’s knowledge of the topic area? | 4.75 (.77) | ||
| 2. How well did the TA services resources or products you received meet your needs? | 1 “not well at all” to 4 “extremely well” | 3.73 (.57) | |
| 3. Overall, how useful were the TA services/products provided to you? | 1 “not at all useful” to 5 “very useful” | 4.56 (.76) | |
| 5. Would you use our TA services/products again if the need arises? | 0 “no” or 1 “yes” | 1.00 | |
Open-Ended Items
3. [If response to #3 was 1 or 2] How could we have better met your needs?
4. Tell us how the TA services resources or products you received affected your work?
6. How can we improve our technical assistance/products?
7. What other TA services/products would you be interested in?
Footnotes
Declaration of Conflicting Interests: The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding: The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This article is a product of a Prevention Research Center and was supported by funding through Cooperative Agreement Number U48DP004998 to University of California San Francisco from the Centers for Disease Control and Prevention. The findings and conclusions in this article are those of the author(s) and do not necessarily represent the official position of the Centers for Disease Control and Prevention.
ORCID iDs
Daryl Mangosing, MPH https://orcid.org/0000-0002-6974-6148
Beth Bourdeau PhD https://orcid.org/0000-0001-7537-6350
References
- 1.Centers for Disease Control and Prevention . HIV in the United States and Dependent Areas; 2019. https://www.cdc.gov/hiv/statistics/overview/ataglance.html Accessed February 04, 20. [Google Scholar]
- 2.Department of Health and Human Services . What Is ‘ending the HIV Epidemic: A Plan for America. February, 21; 2020. https://www.hiv.gov/federal-response/ending-the-hiv-epidemic/overview Accessed February 22, 2020. [Google Scholar]
- 3.Green-Ajufo B, Mangosing D, Rebchook G, Lightfoot M. Walk the Talk: An Approach for Equitable Inclusion of Community Voices in Defining HIV Prevention Research in the San Francisco Bay Area. : National HIV Prevention Conference, Centers for Disease Control and Prevention; 2019. [Google Scholar]
- 4.Katz J, Wandersman A. Technical assistance to enhance prevention capacity: a research synthesis of the evidence base. Prev Sci. 2016;17(4):417-428. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Bergeron K, Abdi S, DeCorby K, MEnsah G, Rempel B, Manson H. Theories, models and frameworks used in capacity building interventions relevant to public health: a systematic review. BMC Publ Health. 2017;17:914. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Chilenski SM, Perkins DF, Olson J, et al. The power of a collaborative relationship between technical assistance providers and community prevention teams: A correlational and longitudinal study. Eval Progr Plann. 2016;54:19-29. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.West GR, Clapp SP, Averill EMD, Cates W. Defining and assessing evidence for the effectiveness of technical assistance in furthering global health. Global Publ Health. 2012;7(9):915-930. doi: 10.1080/17441692.2012.682075. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Guest G, MacQueen KM, Namey EE. Applied Thematic Anlaysis. : Sage Publications; 2012. [Google Scholar]
- 9.Lewis-Charp H, Estrella R, Henderson-Frakes J, Inouye TE, Yu HC, Sinicrope C. Building Capacity through a Technical Assistance Strategy: Promising Approaches Community, Leadership Project (CLP) 2011 Evaluation Report; 2012. [Google Scholar]
- 10.Darnell D, Dorsey CN, Melvin A, Chi J, Lyon AR, Lewis CC. A content analysis of dissemination and implementation science resource initiatives: what types of resources do they offer to advance the field? Implement Sci. 2017;12(1):137. doi: 10.1186/s13012-017-0673-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Leeman J, Calancie L, Hartman MA, et al. What strategies are used to build practitioners' capacity to implement community-based interventions and are they effective?: a systematic review. Implement Sci. 2015;10(1):80. doi: 10.1186/s13012-015-0272-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Le LT, Anthony BJ, Bronheim SM, Holland CM, Perry DF. A technical assistance model for guiding service and systems change. J Behav Health Serv Res. 2016;43(3):380-395. doi: 10.1007/s11414-014-9439-2. [DOI] [PubMed] [Google Scholar]