Abstract
We used the Practical, Robust Implementation and Sustainability Model to evaluate implementation of South Africa’s Central Chronic Medicine Dispensing and Distribution (CCMDD) program, a differentiated service delivery program which allows clinically stable HIV-positive patients to receive antiretroviral therapy refills at clinic- or community-based pick-up points. Across ten clinics, we conducted 109 semi-structured interviews with stakeholders (pick-up point staff, CCMDD service providers and administrators) and 16 focus groups with 138 patients. Participants had highly favorable attitudes and said CCMDD decreased stigma concerns. Patient-level barriers included inadequate education about CCMDD and inability to get refills on designated dates. Organizational-level barriers included challenges with communication and transportation, errors in medication packaging and tracking, rigid CCMDD rules, and inadequate infrastructure. Recommendations included: (1) provide patient education and improve communication around refills (at the patient level); (2) provide dedicated space and staff, and ongoing training (at the organizational/clinic level); and (3) allow for prescription renewal at pick-up points and less frequent refills, and provide feedback to clinics (at the CCMDD program level).
Keywords: Antiretroviral therapy, Differentiated service delivery, HIV/AIDS, Implementation science, Qualitative, South Africa
Introduction
In South Africa, 7.5 million people are living with HIV, of whom 5.2 million people take antiretroviral therapy (ART) [1, 2]. Since South Africa’s adoption of the World Health Organization’s Universal Test and Treat policy in 2016, increased demands have been placed on its health infrastructure to meet UNAIDS targets (95% aware of their serostatus, 95% on ART, and 95% virally suppressed by 2030) [3].
Community-based, decentralized ART delivery programs, a type of differentiated services delivery model, are a potential strategy for ART scale-up in sub-Saharan Africa by allowing clients to receive medication refills in community settings [4]. Quantitative evaluations of ART delivery programs, especially those that include clinical monitoring and/or adherence support, suggest that they contribute to lower mortality rates, higher retention, increased viral suppression, and reduced drug resistance for patients, alongside decreased burden on healthcare facilities [4–8]. Qualitative research has suggested that community-based ART delivery that includes clinical monitoring and adherence support improves HIV outcomes through the mechanisms of flexibility in refill pick-up times and places, integration of all HIV services, sufficient time for staff to address patient questions, and efficiency through reduced transit time for patients [9]. Implementation barriers have included insufficient staff, training, and infrastructure; increased workload and stigma; patient reluctance to leave clinics; and low retention [10, 11].
One such decentralized ART delivery program is the Central Chronic Medicine Dispensing and Distribution (CCMDD) program, initiated by the South African Department of Health in 2014 [12]. CCMDD allows clinically stable clients to obtain ART, as well as medication for chronic non-communicable diseases, at either external community pick-up points or clinic-based pick-up points (in designated queues), and to visit clinic providers less frequently (1–2 times annually vs. monthly). Pick-up points are selected by the Department of Health and include for example, retail pharmacies and churches. CCMDD differs from other more intensive differentiated service delivery models in that it does not include clinical monitoring or adherence support.
Limited prior research, much of it outside of KwaZulu-Natal, has examined the implementation of CCMDD and related programs in South Africa for ART dispensing [13–19]. Qualitative process evaluations with clients and clinic implementers have found positive attitudes, with perceptions of more efficient ART dispensing, reduced waiting times, more convenient refill pick-up hours, improved clinic working conditions, and reduced stigma (because non-HIV medications are dispensed with ART). Identified healthcare organizational barriers include low perceived accountability and ownership regarding the program among providers, increased workload (e.g., to track clients across pick-up points), insufficient training and staffing, communication gaps with pick-up points, limited storage for non-collected medications, and limited transportation to distribute medications to community pick-up points. CCMDD system- and policy-related challenges have included delayed refill reminder texts, unavailable medications, and inflexible pick-up dates and times.
Overall, there are limited recent data specifically on CCMDD implementation for ART (versus other differentiated delivery models), particularly recent evaluations to understand whether challenges identified in prior research have been addressed over time. Thus, we conducted a qualitative exploration of barriers and facilitators regarding CCMDD implementation for ART in ten healthcare facilities and their related pick-up points, in order to understand recent challenges to CCMDD implementation, several years after CCMDD was introduced. Our study extended prior evaluations of CCMDD and similar programs by gathering data on multiple levels of stakeholders associated with CCMDD at each healthcare facility and pick-up point.
Methods
Conceptual Framework
We used the Practical, Robust Implementation and Sustainability Model (PRISM) to evaluate the multi-level contextual factors that can affect CCMDD implementation: intervention characteristics (i.e., intervention-related barriers to CCMDD implementation); implementation and sustainability infrastructure (i.e., organizational barriers to implementation); external environment; and recommendations. We incorporated perspectives of multiple levels of stakeholders: clinic and external pick-up point providers, medication distribution administrators, and patients who were eligible for ART pickup through CCMDD. PRISM has been applied to study a range of healthcare topics and conditions across countries, including HIV [20–27].
Procedures
From November 1, 2018 to February 22, 2021, the team conducted 109 semi-structured stakeholder interviews and 16 focus groups with 138 patients living with HIV at clinics, external pick-up points, and stakeholder offices across ten study sites (see Table 1). The sample size was selected according to guidelines for qualitative research, which suggest that about 12 participants be included from each meaningful subgroup (i.e., drawn from a homogeneous population) to reach data saturation, with more participants (e.g., 20–40) required to discover cross-cutting meta-themes (across meaningful subgroups) [28–30]. In this study, meaningful subgroups were defined by study (clinic) site, and by providers and patients within each site. Thus, we aimed to collect data from approximately 10 stakeholders and 10 patients per site. Our sample size allowed us to explore themes within and across sites and participant subgroups, and meta-themes across participant subgroups.
Table 1.
Participants | Sample size | Number of clinics/pick-up points represented | Description |
---|---|---|---|
Interviews | |||
Public sector clinic providers and support staff | 79 | 10 clinics |
∙ Nurses and doctors who enroll and renew CCMDD patients) ∙ Managers, administrative staff (e.g., filing clerks, data capturers) ∙ Support staff (e.g., lay counsellors) |
Pick-up point staff | 23 | 10 clinics/24 pick-up points |
∙ Providers (who refer CCMDD patients to clinics for follow-up as necessary) ∙ Medication distribution staff, managers, stock clerks, pharmacy assistants ∙ Caregiver volunteers (clinic staff who provide home-based basic care and other support) |
Local and national CCMDD administrators | 7 | N/A | ∙ Includes program technicians and coordinators |
Interview total | 109 | 10 clinics/24 pick-up points |
∙ 77.1% female ∙ 98.2% Black African ∙ 1.8% Indian |
Focus groups | |||
Patients who received ART through CCMDD | 138 patients across 16 focus groups (6–12 patients each) | 10 clinics/4 pick-up points |
∙ 75.4% female ∙ 100% Black African ∙ 60.1% no high school diploma ∙ 25.4% employed ∙ HIV diagnosis Md = 5-years prior (range = 1–23) |
All study sites were in Umlazi, KwaZulu-Natal, South Africa, a high-density township in eThekwini municipality with one of the highest HIV prevalences in South Africa, where CCMDD was initiated in 2016. Interviewers who were fluent in isiZulu and English were trained in qualitative methods by a female senior behavioral/social scientist and social psychologist PhD, with extensive training and experience in using qualitative methods (LMB), who provided ongoing feedback. Data collection was paused from March 27, 2020 to October 1, 2020 during the COVID-19 pandemic because of a national lockdown; eight interviews (seven staff at one clinic, one CCMDD administrator) and three patient focus groups (in two clinics) were conducted after October 2020. Clinics were purposively sampled to capture a mix of performance levels, evaluated through chart audits (e.g., for patient tracking and wait-time, medical chart completeness) and client satisfaction surveys.
To recruit interview participants, a research assistant (who identified their organizational affiliation) attended clinic staff meetings to present the study overview and contacted external pick-up point and medication distribution staff to offer participation, based on a list supplied by each clinic. For focus groups, a convenience sample was used, in which a research assistant asked patients enrolling into CCMDD in person (at the clinic or pick-up point) if they were willing to participate in a study about CCMDD experiences.
Stakeholder (staff) interviews included questions on: overall knowledge, understanding, and attitudes about CCMDD; interactions with CCMDD stakeholders; organizational readiness of implementing organizations; patient factors that affect enrollment; and factors that limit and facilitate implementation. Patients were asked why they enrolled in CCMDD, any concerns about CCMDD, and suggestions for improving the program. Questions were not formally pilot tested, but questions and probes were refined if initial interviews indicated issues with comprehension.
Written informed consent was obtained. Participants received 150 South African Rand (~ USD $10.50) as compensation for the 1-h sessions. Interviews and focus groups were digitally recorded and translated and transcribed verbatim from isiZulu into English. Study interviewers also took field notes during each interview and focus group.
Eligibility
Inclusion criteria for stakeholder interviews required direct work with the CCMDD program. Inclusion criteria for patient focus groups were: 18 years-old or older; speak English or isiZulu; eligible for ART pickup through CCMDD; and willing to allow access to their 12-month lab data (HIV viral load and CD4 count from clinic files and the National Health Laboratory Service database). Eligibility for CCMDD participation for people living with HIV include: ≥ 15 years old, not pregnant, taking ART, and virologically suppressed (< 50 copies of HIV/ml). Prior to March 2020, patients also must have been taking ART at least 1 year; as of March 2020, this criterion was changed to at least 6 months, in order to fast-track patients into CCMDD during the pandemic.
Qualitative Analysis
Using a directed content analysis approach [31], three cisgender female researchers with qualitative training and experience (LMB: social psychologist, PhD; SM: public health researcher, ScD; and AMG: public policy graduate student, MPP) independently reviewed all transcripts and drafted a summary of preliminary themes within and across sites. The summary was reviewed by the rest of the U.S. and South African team for input on the salience of identified preliminary themes and to come to a shared understanding of the data. Three researchers (LMB, SM, AMG) developed a codebook based on the preliminary themes and interview guide. Three team members (SM, AMG, and NW, a research assistant/rising medical student, BA) jointly coded 24 transcripts, resulting in 223 excerpts (8.8%) using Dedoose qualitative software [32] and made modifications to the codebook as needed, refining definitions. Good inter-rater reliability was established using 45 excerpts (Cohen’s Kappa = 0.79). The remaining 51 transcripts were individually coded; SM, AMG, and NW met weekly to discuss any questions that arose.
Excerpts for each code were reviewed and used to update the preliminary summary noting potential differences based on participant type, pick-up location, and clinic performance (based on an overall performance rating that included, for example, patient chart completeness, patient wait-times, and patient satisfaction ratings). (The team did not identify thematic differences by clinic performance; thus, it will not be discussed further.) In addition, excerpts pre- and during the pandemic were compared. Themes across stakeholder interviews were categorized by PRISM domains. Tables 2, 3, and 4 include representative quotes.
Table 2.
Patient perspective | Provider/administrator perspective |
---|---|
Overall patient and provider enthusiasm for CCMDD | |
“[NAME] stores close after six. So, it is easy to collect your treatment and continue with your work routine." (Patient, Clinic) | “…number one, it’s a dream of every [patient], especially the ones who are working that when they get to the clinic, they just take and go…Number two, it really decreases the workload from the facilities. As much as there are some challenges but we really do appreciate CCMDD because they just come and take. Number three…Most people default because they don’t want long queues. And then when you go to that person, the person would tell you in front of your face that they didn’t come because they don’t want those long queues, they are busy. …so I think it is sustainable.” (CCMDD Eligibility Screener, Clinic Pick-up Point) |
Communication issues and errors | |
Communication with the CCMDD program | |
“… we are treated well at the pickup points but the problem is that I only received a [text] message once… I once raised it and they said it’s not a problem, even if I haven’t received it I must come and collect my pills. In fact the messages no longer come.” (Patient, Clinic Pick-up point) | “You get the patients who are like, ‘I've never got the [text] and then you get patients that say, ‘sometimes I receive [it], sometimes I don't… but I always tell them, whether you get a [text] or don't and you know it’s your date, come and check for your parcel… we only keep the parcel for 7 days, and after 7 days it goes back and you have to go back to your clinic.” (Pharmacy Assistant, External Pick-up Point) |
“My problem is, I usually do not get all my pills. I find my box opened and with a sticker written 'incomplete parcel'. When I ask why it is opened this month, I do not get a straight answer because I do not see the same nurse all the time. When there are missing pills inside, they say I should not be taking those pills all the time. Then I get confused because the doctor from [clinic] said I should take these pills.” (Patient, Clinic Pick-up Point) | “Sometimes a person takes two different pills…I don't know what happens, you find that they get the fixed dose combination. So, they come back saying, ‘I got the single pill’. So, people know, they are educated about their pills. They come back to the clinic and say, ‘this is not mine.’” (Nurse, Clinic Pick-up Point) |
N/A (no parallel patient quote about CCMDD distributor communication issues) | “We have a problem of not having a direct person that we contact if we have a problem with them. We only communicate with the consultants from the call centre, and there’s never a follow-up on the problem that you inquired about.” (Nurse, Clinic Pick-up Point) |
Poor communication among providers | |
N/A (no parallel patient quote about communication between clinic and pick-up point) | “We don't communicate with [the pick-up point] at all. If we have to relay a message, it's relayed through the patient. And if something [goes] wrong, it's with [the distributor]…It’s not one specific clinic where…you can communicate with them. There's like, I'd say, five, six clinics that we're dealing with or hospitals, now…If there was one, then yeah, maybe we'll phone and be like, 'Okay, this is the story.' So they know there's a relationship…but six different hospitals and clinics is like, that's chaos.” (Pharmacy Assistant, External Pick-up Point) |
“What is worrying me about the CCMDD is that you sometimes get there and they say your parcel did not arrive. Now you must start again and go to the clinic while the work that I do, I only get off when it’s weekends only. I cannot go to the clinic in the middle of the week because I work in someone’s house, so they don’t allow me to go during the week because I take care of children.” (Patient, Clinic Pick-up Point) | “[The NGO tracking patients] will give you dates for when you should collect the treatment and write them down on the client’s card and when they reach the pick-up points, they are told that the dates are wrong. And you end up not knowing what is happening because they do turn patients away.” (Nurse, Clinic Pick-up Point) |
Lack of patient education about CCMDD | |
“…a lot of people are initiated on treatment without being educated about it… without any knowledge as to why they are being initiated.” (Patient, Clinic) | “The manager recently asked me to deliver educational talks about CCMDD every morning because we are seeing a rise in the number of people being deregistered on CCMMD because they did not understand CCMDD even from the beginning…” (Provider, Clinic Pick-up Point) |
“The nurses… are harsh, they don’t have respect. Imagine a nurse that is the same age as your child speaking to you… you are wondering what you have done for them to speak to you that way. And then you end up making more mistakes because you are now confused what the problem is but instead of explaining to you, they shout at you. I’m not saying it’s all of them, but most of them.” (Patient, External Pick-up Point) | “There are those that are beyond control, you see that of patients who do not come on time… some of them I punish them on purpose, I say, ‘I am no longer taking you back even the next month because you did not honor your date…I told you that you must honor your date’…So there are ways you try to trick them as well using your own discretion.” (Nurse, Clinic Pick-up Point) |
Rigid CCMDD policies | |
“Collecting from here is okay because it is close by but if it happens that you forget to come and collect, you will find that [the medications] have been returned at the back. When you get to the back, even if you can come to the clinic in the morning, they put you aside and say, ‘oh, it’s this one who has defaulted, he/she will be served last’. So you will sit there until sunset while you see other people being served in front of you. It doesn’t matter how early you get there. It once happened to me. They said I had defaulted, even though I had just forgotten.” (Patient, External Pick-up Point) | “Some patients are not really loyal to the dates, they will come like a week or two after and they will tell you that ‘I was away.’ But then you have to take into consideration that the person has a life, they work … it's not easy for them to come on that particular date.” (Shop Assistant, External Pick-up Point) |
“I am a person who works in a job that requires me to travel to far places on and off, you see. Maybe sometimes I will go and work in [another location], and I will be away for 6 months.” (Patient, Both External and Clinic Pick-up Points) | “Another challenge may sometimes be the people who are in the trucking business…you find that they would go for 3 months to other countries. So, it comes to a point where they should stop for a short while from CCMDD, because CCMDD does not cater for 3 months.” (Nurse, Clinic Pick-up Point) |
N/A (no parallel patient quote about rejected prescriptions) | “Sometimes they don’t even tell you why [the prescription] is rejected…you have to find out yourself what did I do wrong and go through those scripts again.” (Nurse, Clinic Pick-up Point) |
External environment | |
HIV stigma | |
“Nobody knows what pills you have come to collect. It’s just a parcel. It could be for blood pressure or anything. It’s only you who knows what you are collecting.” (Patient, Clinic Pick-up Point) | “I think there was a message that was circulated to make patients understand that it’s not just ARVs…It’s chronic medicine… so the stigma is no longer there” (Pharmacy Manager, External Pick-up Point) |
“…if it’s a young person and [staff] are carrying such a big [clinic chart cover for patients in CCMDD], a person can see what this one has come to do…We don’t have blood pressure that we can be carrying such a big [clinic chart cover]…for some it’s obvious [that a young patient with chronic conditions is HIV-positive, rather than has another chronic condition like hypertension]. Once you say you are going to collect pills, they already know.” (Patient, Both External and Clinic Pick-up Points) | “Some of them don’t want us to give them this box. They are like, 'I’ll even buy a plastic if I have to.' I think there’s still a stigma. I think in terms of people still don’t understand that there is still so many chronic diseases and it’s just not confined to one thing.” (Pharmacy Manager, External Pick-up Point) |
COVID-19 | |
Even during the lockdown, they called me here at the clinic because we have to come collect the dates here, they called me and said they have revised my dates, I should continue collect up from [the external pick-up point]. I saw that way as easy. (Patient, Clinic) | We had an issue with the extended scripts… we got so excited and extended… It’s part of the contributing factor in defaulters because we will say they defaulted, yet they have received a communication at home that their scripts have been extended. So those we managed to get their files, we call them and tell them to just ignore the message and come to the facility. (CCMDD Administrator) |
Ever since the lockdown started, I do not get the (text reminder) message. (Patient, Clinic) | “It is their date to come inside and you find that they do not come and then they get here late. When you ask why, they say they were waiting to receive a message with a password because normally during the dates in between, people get messages. Then even if it’s the date for them to come to the clinic, they do not come because they say they are waiting for an SMS.” (Nurse/CCMDD Eligibility Screener/Champion, Clinic Pick-up point) |
N/A (no parallel patient quote for provider absence) | Covid started then we had an issue with the external pickup point…People were not available. I was one of them too, I was sick, I was in quarantine… That was our main challenge. (CCMDD Administrator) |
N/A indicates that no relevant patient excerpts were coded that parallel the provider quote in the same row, most likely because patients were not aware of all of the organizational barriers faced by providers
Table 3.
Inadequate space |
“To be honest, that filing room is getting smaller and smaller…because every day there are people being initiated…so we thought that it would be great if we would get a bigger filling to accommodate all the files because it seems like it’s going to be a lot of files for 2019 yet we are still halfway through the year.” (Data Clerk, Clinic Pick-up Point) |
“Space is needed too for CCMDD to have its own space in this clinic… the space is small. We squeeze with other people… Privacy is also needed sometimes. For example, maybe someone has some to review but you find that they are sick on that day and they need privacy to tell you, one-to-one.” (Nurse, Clinic Pick-up Point) |
“Our biggest challenge is storage…we do not even have adequate shelving space where we can store medication so that it can be sorted properly.” (Nurse, Clinic Pick-up Point) |
Insufficient staffing |
“Others got trained. I learned along the way because…I have never received any training.” (Nurse Assistant, Clinic Pick-up Point) |
“Let’s say a person comes, let’s say like [NGO] and they say they are here to assist us you in the sites, only to find that they tell you they are now closing down…Now I am left all alone.” (Supervisor/Adherence Club Leader, Clinic Pick-up Point) |
“…other clinics they had ART clubs but they didn't continue with them because there were no human resources to continue….” (Clinic Manager, Clinic) |
Lack of resources: transportation |
“The only problem that we experience at time is that fact that we share a vehicle with the clinic. You find that sometimes when we need to go out to pick up points, the vehicle is booked for something else. Sometimes when they have to fetch us you will find that they are still busy with something else.” (Lay Counsellor, Clinic Pick-up Point) |
“The clinic has discontinued delivering medication to pick-up points because we were experiencing high levels of vehicle theft. This has resulted in trying to have pickup points closer to the facility where we are able to walk to…” (Adherence Club Facilitator, Clinic Pick-up Point) |
“The reason [adherence clubs stopped] was the transport, because when the treatment was ready, the staff that was hired from [NGO] was delivering it. Then that staff from [NGO] would deliver it but the transport was not available here at the clinic, and [NGO] also didn’t have it. Patients would wait in those places until they decide to go back home. We saw that this will make the community to hate us, it's better to stop it because it wasn’t properly organized.” (Nurse, Clinic) |
Lack of resources: computing |
“When the [Wi-Fi data] ran out then we had to script them manually, so that makes work to be a lot whereas when there is a computer its easy. When you are hand-writing it takes time…” (Support/CCMDD Eligibility Screener, Clinic) |
“But now the system is frozen, then you have to write the IDs down on a page. God knows sometime that page gets lost, then it’s not synced and it’s your fault… But it’s challenging, if the systems are not working, what can you do?” (Pharmacy Assistant, External Pick-up Point) |
Table 4.
Patient-level recommendations |
Provide patient education |
“It would be better if they include it in their health education while we are still there in the front, that there is a program like this so that we can try to inform people…I'll make an example. My brother just started the treatment, but he does not know anything about the CCMDD program. I even asked him to take the transfer and come to my clinic so that I help him because he was not doing well. So, they don't advertise it enough.” (Patient, Clinic Pick-up Point) |
“If they could teach them properly, from the basics, then they won't be confused and they won't have lots of questions and they won't get angry and upset. If they are taught, you only come on that day for your medication and they told those things, we won't have problems at the pharmacy. And if you tell them it's only available for so many days, so they can [tell] themselves, ‘Okay, I don't have to take off on that day. I have 7 days I can collect in the weekend. I don't have to miss a day of work.’ ” (Pharmacy Assistant, External Pick-up Point) |
Foster clear communication with patients |
“…They should give the person the pills [anyway, even if the text reminder is not sent] because sometimes the message… comes late… I once had a problem because on my phone, I am used to getting a message. On this unfortunate day the message didn’t come through… and the date that is written on my card passed. I then came to the clinic, when I got to the clinic the nurses said ‘no, did the message come in?’ I said, ‘the message didn’t come but it is my date.’ They said, ‘no sisi, you have to go back and wait for the message.’ I was very disappointed because I did a fruitless trip, yet it was my date. They said you mustn’t come if you haven’t received a message, you must only come if you have received a message because it means that your pills are not yet ready. They haven’t arrived here for you to get them as you haven’t received a message. I said, ‘but it’s my date so what is wrong?’ They said, ‘if the message hasn’t come it means your pills are not ready and they haven’t arrived here from where they come from; so, it means they are not ready so please go home.’ I only came back after I had received a message. I had even defaulted as I didn’t have pills and I didn’t know where to get them.” (Patient, Clinic) |
“For the program to run more smoothly there must be better communication channels so patients get clear messages when [texts] are sent, for instance if the pick-up date is the next day then it must clearly state this in the message because people need information that is easy to understand.” (Lay Counsellor, Clinic Pick-up Point) |
Organizational-level recommendations |
Improve program infrastructure |
“An addition of pick-up points would have a good effect. …Inside the clinic, sisters get a lot of people and in this time that we are into right now, the time of Covid, I wish they could see fewer people. The clinic should be empty. Another thing, it makes life easy for us, the patients… we can go and pick up from the office. It’s near, I don’t take a taxi… I wish other places can be added if possible because in other places we struggle for money to go to [pick-up points].” (Patient, Clinic Pick-up Point) |
“…If there could be more [smart lockers], so that a person comes to collect their treatment and go home. Because sometimes it irritates people to stay in the facility for a long time.” (Data Capturer, Clinic Pick-up Point) |
Hire dedicated CCMDD staff |
“If there can be someone who deals specifically with people who come to collect, it would be fine because you would just come and take your parcel and leave. You’ll be able to get to work on time.” (Patient, Clinic Pick-up Point) |
“If they wanted this program to work and maximize the money they have put into it, they should have brought it’s own staff…You know that you have a nurse for the CCMDD, you have a clerk and a doctor; ok not necessarily a doctor but you have some [team].” (Pharmacist, External Pick-up Point) |
Increase training opportunities |
“Maybe if they can get someone who understands CCMDD and HIV because I don’t know whether [staff] here at the clinic don’t understand or what. I just don’t understand them.” (Patient, Clinic Pick-up point) |
“[There should be] a training to encourage to put people on CCMDD so that there will not be lines, complaints and all that” (Provider, Clinic Pick-up point) |
CCMDD program-level recommendations |
Change CCMDD renewal and refill policies |
“They can improve by increasing the pills to be taken every 4–6 months, not 2 months, as long as we are keeping our viral load and CD4 count low.” (Patient, Clinic) |
“I think it would help if patients can renew their scripts at the pick-up points because that would save patients times and lower the number of patients who end up defaulting…” (Lay Counsellor, Clinic Pick-up point) |
Add family planning |
“[CCMDD should have family planning at pick-up points] because family planning also has its own date, so if you are working, you have to be absent from work twice a month.” (Patient, Clinic Pick-up point) |
Monitor and provide feedback |
“What troubles me is the fact that we do not receive feedback on the performance of CCMDD in the facility. We need to know whether or not we meet targets on CCMDD enrolments. If we are having challenges on the program, they want us to send email; with whose data? Right now we are not even sure on who to direct or grievances if we have any.” (Nurse/CCMDD Eligibility Screener/Champion, Clinic Pick-up point) |
Results
Overall Patient and Provider Enthusiasm for CCMDD
Participants were highly enthusiastic about CCMDD. Patients valued the convenience and ease of collecting medications through CCMDD, with short queues, accessible locations, more flexible hours, and less frequent refills. Employed patients liked that they did not have to miss work to pick up refills; some patients walked to external pick-up points in their own community or near their worksites, contributing to reduced transport costs and time investment. Consistent with patient accounts, clinic staff said CCMDD led to clinic decongestion, including shorter queues and improved workflow. Some nurses referenced a reduced workload resulting from decongestion, allowing them to dedicate more time to sicker patients.
Some patients said that they had transport challenges when external pick-up points were not conveniently located in their community or if they could not afford transportation. Providers also mentioned that a minority of patients, especially those who were older, still preferred to be seen by a provider.
Intervention-Related Barriers to CCMDD Implementation
Communication Issues and Errors
Communication Issues with CCMDD Program Administration
The current CCMDD medication distributor was perceived by many staff across clinics and pick-up points as being inaccessible (e.g., for information about rejected prescriptions, incorrect medications, or incorrect medication quantities, which occasionally occurred). Several staff mentioned that emails or phone calls to the CCMDD administrator were never returned or that they did not have direct contact information. However, other staff described consistent and helpful communication with the distributor. Providers and staff also discussed how there were minimal feedback loops with the CCMDD program, specifically noting a lack of data for patient tracking or statistics about how the program was working (e.g., number of patients who did not pick up their medications per month and general enrollment numbers).
Patients also had communication issues with the CCMDD distributor regarding text reminders for refills. When such reminders were correct, patients found them to be helpful. However, patients said that text reminders for refills sometimes were not delivered, specified an incorrect date, or suggested that the refill was available at the pick-up point before it was actually ready.
Poor Communication Among Providers
Poor communication between clinics and pick-up point providers sometimes led to confusion around whether patients had defaulted or collected their medication at the clinic rather than the external pick-up point, and around understanding why a patient’s medication was not available at the external pick-up point (which meant that the patient had to return to the clinic to get their refill).
Lack of Patient Education About CCMDD
Patients discussed not having clear communication with providers about CCMDD, which led to misunderstandings about the program and eligibility. Although all patients were aware of CCMDD, many said they were not told explicitly about why they were switching to CCMDD, and some mentioned not being given a choice regarding CCMDD enrollment or their pick-up point. Some patients felt that providers scolded them for missed refills rather than explaining CCMDD’s rules. Providers said some patients with uncontrolled comorbid conditions who were virally suppressed were frustrated when they were disenrolled from CCMDD, because they did not understand that CCMDD requires all chronic conditions to be controlled, not only HIV.
Rigid CCMDD Policies
Across stakeholders, CCMDD policies were described as inflexible regarding medication pick-up dates, prescription writing, and enrollment. Patients had a 7-day grace period to pick up medications. Mobility (e.g., long-distance travel for work or holiday) limited patients’ ability to pick up medications on their refill date, and some employers did not allow patients sufficient time off to pick up medications. Although in most clinics patients were allowed to ask others to pick up their refills for them, such patients were unable to obtain blood draws to show continued CCMDD eligibility.
Many participants said that prescriptions were rejected due to administrative errors by clinic prescribers, even if the necessary information was included on the form. For example, some providers included multiple prescriptions on one form (e.g., for more than one chronic condition), which was not allowed. In addition, handwritten prescriptions were frequently rejected, but it was necessary for providers to handwrite prescriptions when laptops were not available at the clinic (a recurring issue, as noted below).
Organizational Barriers Related to CCMDD Implementation
Inadequate Space
Space constraints were common across most settings and included the absence of a separate CCMDD workspace (e.g., to sort medications and files) and CCMDD patient queue (sometimes presenting privacy issues), and inadequate storage for patient files and medications. Lack of a dedicated CCMDD filing space increased provider burden and workload; paper files needed to be pulled for patients coming in for refills every day, and if patients did not get their refill, their records needed to be refiled. Some external pick-up points did not have enough space for the program to operate, including for medication storage (since many external pick-up points were not designed to be medical spaces). Other external pick-up points, such as those at retail pharmacies, reported having enough space for storing and distributing medications. Additionally, there was sometimes limited waiting area space at clinics or external pick-up points, and some patients had to wait for their medications outside, in heat and rain.
Insufficient Staffing
Many participants spoke about insufficient staffing, highlighting the need for more staff to be trained on and dedicated to CCMDD (e.g., not enough nurses to enroll patients at the clinic and staff external pick-up points; too few data capturers to retrieve files and track patients). Although all staff were aware of CCMDD, many said they did not receive formal training; some learned from others or on the job. Inadequately trained staff led to errors on prescriptions, in filing patient records, and in tracking patients, and in enrolling ineligible patients or not enrolling eligible patients.
Clinic staff raised problems around high staff turnover, often due to the loss of supporting non-governmental organizations, which led to the need to redistribute tasks and train new and existing staff. Having even one person absent could mean that no one was available to distribute medications. Another staffing issue mentioned at some external pick-up points was a lack of resources to pay community (volunteer) caregivers, especially in sites providing child and elder care, and adherence clubs (in which groups of patients meet to obtain refills and discuss adherence).
Lack of Resources (Transportation and Computing)
Vehicles were seen as essential for implementation, facilitating transport of staff and medications to external pick-up points. However, some clinics only had one vehicle that was not consistently available, and others did not have any (e.g., one clinic’s car was stolen). Moreover, many clinics and pickup points had poor computing infrastructure, including poor connectivity/bandwidth, laptop unavailability, or no phone data, which led to difficulties in sending prescriptions and tracking patients. Some clinics only had one laptop, or no laptop (e.g., due to theft), limiting the ability of clinic staff to use CCMDD software for program administration (e.g., patient tracking), and necessitating hand-written prescriptions (which were frequently rejected, as noted above). Clinic and external pick-up point staff also discussed unreliable documentation in the CCMDD electronic patient record as leading to a lack of medication delivery (e.g., prescriptions marked as “ordered,” even if the medication was missing).
External Environment
HIV Stigma
HIV stigma was discussed as an external factor that continues to affect patients’ willingness to obtain HIV care and thus may affect CCMDD implementation. Across stakeholders, several aspects of CCMDD were consistently highlighted for mitigating stigma, including that CCMDD patients queue to pick up medications for all chronic conditions, rather than only ART, and that all medications are provided in generic packages. However, some patients remained self-conscious about picking up medications. For example, staff said some patients asked for plastic bags as they did not want to be perceived as carrying any medication—and possibly being identified as HIV-positive. Some younger patients also believed that their HIV-positive serostatus would be revealed, and that they would be stigmatized, if others noticed that they were in the CCMDD program, because other chronic conditions (e.g., hypertension) tended to be associated with older age.
COVID-19
Several changes to CCMDD implementation were made during the pandemic, to minimize interpersonal contact: Prescriptions were extended to every 3 months (from 2), and clinic visits were reduced to annually (from every 6 months). Moreover, CCMDD staff marked packages as “collected” in the patient record when medications were obtained, rather than having patients sign for medications. In addition, a limited number of patients were allowed inside the pick-up point, with queues starting outside the pick-up point. Key organizational barriers, including communication issues, did not generally change during the pandemic—and pandemic-related changes to refill and clinic visit frequency were not always communicated to clinic staff, leading to difficulties in tracking pick-ups. In addition, there were some issues with staff absences (e.g., due to illness).
Recommendations
Patient-Level
Patient-level suggestions for improving CCMDD centered on providing more patient education, including educating patients and communities on the purpose of CCMDD and how CCMDD works, and on eligibility and ineligibility criteria (so they understood when they were disenrolled or not offered the program). Suggestions included presenting about CCMDD to patients in waiting rooms and using radio announcements, as well as improving patient communication with clearer, more detailed texts about refill dates.
Organizational Level
Suggestions for improvement at the organizational level included dedicating or revising the CCMDD clinic infrastructure and staffing, and providing ongoing staff training opportunities. Providers, external pick-up point staff, and patients discussed the need to increase the number of pick-up points: some pick-up points were crowded, and some communities did not have nearby pick-up points. This led to challenges when patients did not have time to wait on queues (e.g., because of competing needs for work) or if they could not afford transportation. To increase the convenience of medication refill pick-up, a medication distribution machine was implemented in one clinic, whereby patients picked up their medications from smart lockers that were opened using a one-time PIN sent to patients’ cellphones. Clinic staff reported that the machine facilitated faster and more convenient pick-up by having longer daily hours than clinics and pick-up points. However, clinic staff referenced problems with the machine related to texts not being delivered, difficulties in access for patients without cellphones, and patient files not being updated after they accessed the machine.
Providers suggested having dedicated clinic staff for CCMDD-specific tasks (e.g., providing refills, pulling files, and tracking patients who missed refill pick-ups). Providers and external pick-up point staff asked for ongoing formal trainings and written materials, in addition to the standard operating procedures manual provided by the Department of Health.
CCMDD Program-Level
Program-level suggestions centered on changing CCMDD policies and providing regular feedback. A provider and some patients suggested allowing patients to renew prescriptions at external pick-up points rather than returning to the clinic. Patients requested increasing the number of months between refills. Patients and external pick-up point staff additionally suggested providing other chronic medications and contraceptives through CCMDD in general, and resources such as vital signs monitoring devices at external pick-up points. Providers and staff requested consistent access to regular statistics, assessments, and feedback about CCMDD in their clinic, so they could better monitor and track patients.
Discussion
In this comprehensive, multi-level assessment of CCMDD implementation, we found positive attitudes and high acceptability of the program across multiple levels of stakeholders, including patients, providers, and administrators, as well as ongoing logistical and implementation challenges. Consistent with prior research on CCMDD and other decentralized medication distribution programs [13, 16, 19], patients valued the convenience, accessibility, and shorter wait-times associated with refill pick-up points, and providers appreciated the clinic decongestion. Participants universally felt CCMDD was an improvement over prior clinic-based refill protocols.
Nevertheless, implementation challenges and contextual factors related to clinic infrastructure hindered achievement of CCMDD’s full potential. These challenges included medication errors, text reminders with incorrect refill dates, refill date inflexibility, and poor communication between clinics and pick-up points regarding missed refills, as well as failure to explain CCMDD to patients and to train providers. Such challenges led to confusion among patients about how and when to receive refills, and frustration among providers about how to track patients accurately. Contextual issues related to the clinic and pick-up point infrastructure—especially regarding space, computing, and staffing—threatened program feasibility and sustainability. Such issues perhaps left the most vulnerable patients behind—those who could not return for refills on designated dates due to greater barriers to care (e.g., competing work responsibilities, mobility, poverty, transportation barriers, lack of cell phone access) or difficulties navigating unfamiliar CCMDD protocols. A 2017 study on the implementation of decentralized medication delivery in South Africa identified similar challenges, such as inconsistent ART availability on refill dates, poor communication between pick-up points and clinics, and provider training needs [19]. We found these challenges several years later in some, but not all clinics, suggesting that, over time, implementation challenges are starting to be addressed. However, clinics may need additional technical assistance and resources, which may free up provider time to educate patients and answers their questions and concerns—a key feature that was thought to lead to greater effectiveness of differentiated service delivery in the DO ART study, in which ART delivery was combined with clinical monitoring and adherence support [9].
Our analysis did not find substantial differences in CCMDD program implementation prior to, versus during, the COVID-19 pandemic. Some previously identified tracking issues were amplified, such as challenges with documenting when patients obtained refills, because more patients were using CCMDD [33]. However, some changes to CCMDD during the pandemic may have strengthened the program [34], which not only continued uninterrupted throughout the pandemic, but also increased the number of people eligible, reduced refill frequency, extended the period in between clinic visits from 6 months to 1 year, and increased the number of community pick-up points (including in smart lockers that did not require staff to be present) [35]. Similar policy changes throughout sub-Saharan Africa during the pandemic resulted in scale-up of differentiated service delivery models [35, 36]. If such changes endure post-pandemic, they may contribute to CCMDD’s long-term success.
HIV stigma remains an ever-present issue affecting progress toward the 95-95-95 targets in sub-Saharan Africa [37, 38]. Similar to prior research on decentralized ART delivery [39], our study indicates that CCMDD may help to decrease HIV stigma concerns, because CCMDD integrates medication refills across chronic conditions. In contrast, other research has uncovered fears that community pick-up points could increase stigma, through unintended serostatus disclosure if people living with HIV are known to obtain medications at a particular community location [10, 40, 41]. Although the concealment of ART in plain packaging may reduce the likelihood of anticipated and experienced stigma (because patients’ serostatus is not apparent to others), it does not directly address or reduce societal stigma. However, as CCMDD pick-up points become widespread in communities, it is possible that HIV stigma may decrease as HIV becomes associated with less stigmatized chronic conditions served by the program.
Our study suggests several avenues for future research. Although case studies of differentiated ART delivery models in South Africa suggest effects on improved health outcomes [42], there is a need to test the comparative effectiveness of CCMDD versus standard clinic ART provision on increased viral load suppression and retention in care, overall and by pick-up point type (e.g., in-clinic, external, smart lockers). Research is also needed to document numbers and types of patients for whom CCMDD is infeasible (e.g., rural patients who do not live near pick-up points or without transportation access). Furthermore, quality improvement research is needed to evaluate ways to overcome the challenges identified in this and other research on CCMDD with revised implementation models, including how to improve staff training and patient education, and how to decrease errors (e.g., around medication packaging and text reminders). Moreover, CCMDD will not be able to reach its full potential unless funding is made available to improve clinic infrastructure and resources for the program.
Study limitations relate to sample characteristics and methods. Participants were drawn from one urban township in South Africa, and experiences with CCMDD may differ in other area of the country, particularly in rural areas. Moreover, the majority of patient and healthcare provider participants were female, reflecting HIV patient caseloads and healthcare occupational distributions in South Africa [43, 44]. In addition, patients who agreed to participate were present at the clinic or pick-up point, and thus they may have differed in their perceptions of CCMDD from patients who were previously in CCMDD, but who discontinued HIV care. Another limitation is the long qualitative data collection and analysis period, which could affect the relevance of our conclusions for current practice. Finally, although the research continued during the COVID-19 pandemic, relatively fewer interviews and focus groups were conducted during the pandemic, and data collection was completed in the middle of the pandemic; thus, our data cannot inform the full effects of COVID-19 on CCMDD.
Conclusion
CCMDD holds promise as a scalable means of managing HIV and other chronic conditions in resource-limited settings, if implementation challenges can be addressed. CCMDD was supported across stakeholder levels and was seen as a feasible program that has addressable logistical and staffing barriers. CCMDD can be strengthened through greater attention to healthcare worker training and patient education, and additional resources and infrastructure. Further research is needed to evaluate the effects of the program on patient outcomes, as well as CCMDD’s contributions to healthcare system resilience during the COVID-19 pandemic.
Author Contributions
Conceptualization (IVB, LMB); Data curation (LMB, SM, AM-G, NJW, ZS); Formal analysis (LMB, SM, AM-G, NJW); Funding acquisition (IVB); Investigation (IVB, ZS, ND, SG); Methodology (LMB, IVB); Project administration (IVB, ZS, DZ, SG); Supervision (IVB, LMB, ZS, ND); Roles/Writing—original draft (LMB, SM, AM-G, NJW); Writing—review & editing (ZS, DZ, ND, SG, IVB).
Funding
This study was funded by National Institute of Mental Health—Grant No. R01MH114997 (Bassett, PI), with additional support from P30MH058107 and P30AI060354.
Data Availability
All authors ensure that the data support the published claims and comply with field standards.
Declarations
Conflict of interest
The authors have no relevant financial or non-financial interests to disclose; the authors have no conflicts of interest to declare that are relevant to the content of this article; all authors certify that they have no affiliations with or involvement in any organization or entity with any financial interest or non-financial interest in the subject matter or materials discussed in this manuscript; and the authors have no financial or proprietary interests in any material discussed in this article.
Ethical Approval
This study was performed in accordance with the ethical standards as laid down in the 1964 Declaration of Helsinki and its later amendments. All study procedures were approved by the Biomedical Research Ethics Council of the University of KwaZulu-Natal and by the Partners Healthcare Institutional Review Board (protocol # 2017P001690).
Informed Consent
Informed consent was obtained from all individual study participants.
Footnotes
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
References
- 1.AIDSinfo|UNAIDS. 2021. Available from: http://aidsinfo.unaids.org/.
- 2.Avert. HIV and AIDS in South Africa 2020. Updated April 15, 2020. Available from: https://www.avert.org/professionals/hiv-around-world/sub-saharan-africa/south-africa.
- 3.UNAIDS Joint United Nations Programme on HIV/AIDS. Understanding fast-track-accelerating action to end the AIDS epidemic by 2030 Geneva, Switzerland. Updated June 2015; cited 2021. Available from: https://www.unaids.org/sites/default/files/media_asset/201506_JC2743_Understanding_FastTrack_en.pdf.
- 4.Nachega JB, Adetokunboh O, Uthman OA, Knowlton AW, Altice FL, Schechter M, et al. Community-based interventions to improve and sustain antiretroviral therapy adherence, retention in HIV care and clinical outcomes in low-and middle-income countries for achieving the UNAIDS 90-90-90 targets. Curr HIV/AIDS Rep. 2016;13(5):241–255. doi: 10.1007/s11904-016-0325-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Okoboi S, Ding E, Persuad S, Wangisi J, Birungi J, Shurgold S, et al. Community-based ART distribution system can effectively facilitate long-term program retention and low-rates of death and virologic failure in rural Uganda. AIDS Res Ther. 2015;12(1):1–9. doi: 10.1186/s12981-015-0077-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Chimukangara B, Manasa J, Mitchell R, Nyabadza G, Katzenstein D, Masimirembwa C. Community based antiretroviral treatment in rural Zimbabwe. AIDS Res Hum Retroviruses. 2017;33(12):1185–1191. doi: 10.1089/aid.2017.0029. [DOI] [PubMed] [Google Scholar]
- 7.Grimsrud A, Lesosky M, Kalombo C, Bekker L-G, Myer L. Implementation and operational research: community-based adherence clubs for the management of stable antiretroviral therapy patients in Cape Town, South Africa: a cohort study. J Acquir Immune Defic Syndr. 2016;71(1):e16–e23. doi: 10.1097/QAI.0000000000000863. [DOI] [PubMed] [Google Scholar]
- 8.Barnabas RV, Szpiro AA, van Rooyen H, Asiimwe S, Pillay D, Ware NC, et al. Community-based antiretroviral therapy versus standard clinic-based services for HIV in South Africa and Uganda (DO ART): a randomised trial. Lancet Glob Health. 2020;8(10):e1305–e1315. doi: 10.1016/S2214-109X(20)30313-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Gilbert HN, Wyatt MA, Pisarski EE, Asiimwe S, Van Rooyen H, Seeley J, et al. How community ART delivery may improve HIV treatment outcomes: qualitative inquiry into mechanisms of effect in a randomized trial of community-based ART initiation, monitoring and re-supply (DO ART) in South Africa and Uganda. J Int AIDS Soc. 2021;24(10):e25821. doi: 10.1002/jia2.25821. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Zakumumpa H, Rujumba J, Kwiringira J, Katureebe C, Spicer N. Understanding implementation barriers in the national scale-up of differentiated ART delivery in Uganda. BMC Health Serv Res. 2020;20(1):1–16. doi: 10.1186/s12913-020-5069-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Nkhata MJ, Muzambi M, Ford D, Chan AK, Abongomera G, Namata H, et al. Shifting human resources for health in the context of ART provision: qualitative and quantitative findings from the Lablite baseline study. BMC Health Serv Res. 2016;16(1):1–10. doi: 10.1186/s12913-016-1891-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Gcwabe L. CCMDD: “The right medicine, for the right patient, in the right parcel, in the right place, at the right time.”. Health-E News. 2021 May 3, 2021.
- 13.Magadzire BP, Marchal B, Mathys T, Laing RO, Ward K. Analyzing implementation dynamics using theory-driven evaluation principles: lessons learnt from a South African centralized chronic dispensing model. BMC Health Serv Res. 2017;17(2):15–23. doi: 10.1186/s12913-017-2640-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Magadzire BP, Marchal B, Ward K. Improving access to medicines through centralised dispensing in the public sector: a case study of the Chronic Dispensing Unit in the Western Cape Province, South Africa. BMC Health Serv Res. 2015;15(1):1–8. doi: 10.1186/s12913-015-1164-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Magadzire BP, Mathole T, Ward K. Reasons for missed appointments linked to a public-sector intervention targeting patients with stable chronic conditions in South Africa: results from in-depth interviews and a retrospective review of medical records. BMC Fam Pract. 2017;18(1):1–10. doi: 10.1186/s12875-017-0655-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Dorward J, Msimango L, Gibbs A, Shozi H, Tonkin-Crine S, Hayward G, et al. Understanding how community antiretroviral delivery influences engagement in HIV care: a qualitative assessment of the centralised chronic medication dispensing and distribution programme in South Africa. BMJ Open. 2020;10(5):e035412. doi: 10.1136/bmjopen-2019-035412. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Pillay B. The RK Khan Hospital pharmacy decongestion project: an innovative partnership in service delivery. 2017 International Awards. 2017;53(1):31. [PubMed]
- 18.Govender N, Suleman F. Comparison of patient satisfaction with pharmaceutical services of postal pharmacy and community pharmacy. Health SA Gesondheid. 2019 doi: 10.4102/hsag.v24i0.1105. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Pascoe SJ, Scott NA, Fong RM, Murphy J, Huber AN, Moolla A, et al. “Patients are not the same, so we cannot treat them the same”—a qualitative content analysis of provider, patient and implementer perspectives on differentiated service delivery models for HIV treatment in South Africa. J Int AIDS Soc. 2020;23(6):e25544. doi: 10.1002/jia2.25544. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Kanazawa JT, Saberi P, Sauceda JA, Dubé K. The LAIs are coming! Implementation science considerations for long-acting injectable antiretroviral therapy in the United States: a scoping review. AIDS Res Hum Retroviruses. 2021;37(2):75–88. doi: 10.1089/aid.2020.0126. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Ekawati FM, Emilia O, Gunn J, Licqurish S, Lau P. The elephant in the room: an exploratory study of hypertensive disorders of pregnancy (HDP) management in Indonesian primary care settings. BMC Fam Pract. 2020;21(1):1–11. doi: 10.1186/s12875-020-01303-w. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.McKay MM, Bahar OS, Ssewamala FM. Implementation science in global health settings: collaborating with governmental & community partners in Uganda. Psychiatry Res. 2020;283:112585. doi: 10.1016/j.psychres.2019.112585. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Leonard C, Lawrence E, McCreight M, Lippmann B, Kelley L, Mayberry A, et al. Implementation and dissemination of a transition of care program for rural veterans: a controlled before and after study. Implement Sci. 2017;12(1):1–8. doi: 10.1186/s13012-017-0653-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Trinkley KE, Kahn MG, Bennett TD, Glasgow RE, Haugen H, Kao DP, et al. Integrating the practical robust implementation and sustainability model with best practices in clinical decision support design: implementation science approach. J Med Internet Res. 2020;22(10):e19676. doi: 10.2196/19676. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Magee MF, Baker KM, Bardsley JK, Wesley D, Smith KM. Diabetes to go-inpatient: pragmatic lessons learned from implementation of technology-enabled diabetes survival skills education within nursing unit workflow in an urban, tertiary care hospital. Jt Comm J Qual Patient Saf. 2021;47(2):107–119. doi: 10.1016/j.jcjq.2020.10.007. [DOI] [PubMed] [Google Scholar]
- 26.McCreight MS, Rabin BA, Glasgow RE, Ayele RA, Leonard CA, Gilmartin HM, et al. Using the Practical, Robust Implementation and Sustainability Model (PRISM) to qualitatively assess multilevel contextual factors to help plan, implement, evaluate, and disseminate health services programs. Transla Behav Med. 2019;9(6):1002–1011. doi: 10.1093/tbm/ibz085. [DOI] [PubMed] [Google Scholar]
- 27.Liles EG, Schneider JL, Feldstein AC, Mosen DM, Perrin N, Rosales AG, et al. Implementation challenges and successes of a population-based colorectal cancer screening program: a qualitative study of stakeholder perspectives. Implement Sci. 2015;10(1):1–16. doi: 10.1186/s13012-015-0227-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Guest G, Bunce A, Johnson L. How many interviews are enough? An experiment with data saturation and variability. Field Methods. 2006;18(1):59–82. doi: 10.1177/1525822X05279903. [DOI] [Google Scholar]
- 29.Hagaman AK, Wutich A. How many interviews are enough to identify metathemes in multisited and cross-cultural research? Another perspective on Guest, Bunce, and Johnson’s (2006) landmark study. Field Methods. 2017;29(1):23–41. doi: 10.1177/1525822X16640447. [DOI] [Google Scholar]
- 30.Vasileiou K, Barnett J, Thorpe S, Young T. Characterising and justifying sample size sufficiency in interview-based studies: systematic analysis of qualitative health research over a 15-year period. BMC Med Res Methodol. 2018;18(1):1–18. doi: 10.1186/s12874-018-0594-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Hsieh H-F, Shannon SE. Three approaches to qualitative content analysis. Qual Health Res. 2005;15(9):1277–1288. doi: 10.1177/1049732305276687. [DOI] [PubMed] [Google Scholar]
- 32.Dedoose Version 8.0.35, web application for managing, analyzing, and presenting qualitative and mixed method research data. Los Angeles: SocioCultural Research Consultants, LLC; 2018.
- 33.Health Systems Trust. Annual report 2019/20. South Africa: Health Systems Trust; 2020.
- 34.Grimsrud A, Ehrenkranz P, Sikazwe I. Silver linings: how COVID-19 expedited differentiated service delivery for HIV. J Int AIDS Soc. 2021 doi: 10.1002/jia2.25807. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.Grimsrud A, Wilkinson L. Acceleration of differentiated service delivery for HIV treatment in sub-Saharan Africa during COVID-19. J Int AIDS Soc. 2021;24(6):e25704. doi: 10.1002/jia2.25704. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.Bailey LE, Siberry GK, Agaba P, Douglas M, Clinkscales JR, Godfrey C. The impact of COVID-19 on multi-month dispensing (MMD) policies for antiretroviral therapy (ART) and MMD uptake in 21 PEPFAR-supported countries: a multi-country analysis. J Int AIDS Soc. 2021;24:e25794. doi: 10.1002/jia2.25794. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37.Hargreaves JR, Pliakas T, Hoddinott G, Mainga T, Mubekapi-Musadaidzwa C, Donnell D, et al. HIV stigma and viral suppression among people living with HIV in the context of universal test and treat: analysis of data from the HPTN 071 (PopART) trial in Zambia and South Africa. J Acquir Immune Defic Syndr. 2020;85(5):561. doi: 10.1097/QAI.0000000000002504. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38.Kalichman S, Mathews C, El-Krab R, Banas E, Kalichman M. Forgoing antiretroviral therapy to evade stigma among people living with HIV, Cape Town, South Africa. J Behav Med. 2021 doi: 10.1007/s10865-021-00212-2. [DOI] [PubMed] [Google Scholar]
- 39.Sharer M, Davis N, Makina N, Duffy M, Eagan S. Differentiated antiretroviral therapy delivery: implementation barriers and enablers in South Africa. J Assoc Nurses AIDS Care. 2019;30(5):511. doi: 10.1097/JNC.0000000000000062. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 40.Adjetey V, Obiri-Yeboah D, Dornoo B. Differentiated service delivery: a qualitative study of people living with HIV and accessing care in a tertiary facility in Ghana. BMC Health Serv Res. 2019;19(1):1–7. doi: 10.1186/s12913-019-3878-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 41.Mudavanhu M, West NS, Schwartz SR, Mutunga L, Keyser V, Bassett J, et al. Perceptions of community and clinic-based adherence clubs for patients stable on antiretroviral treatment: a mixed methods study. AIDS Behav. 2020;24(4):1197–1206. doi: 10.1007/s10461-019-02681-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 42.Le Roux KW, Davis EC, Gaunt CB, Young C, Koussa M, Harris C, et al. A case study of an effective and sustainable antiretroviral therapy program in rural South Africa. AIDS Patient Care STDS. 2019;33(11):466–472. doi: 10.1089/apc.2019.0055. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 43.Mayosi BM, Benatar SR. Health and health care in South Africa—20 years after Mandela. N Engl J Med. 2014;371(14):1344–1353. doi: 10.1056/NEJMsr1405012. [DOI] [PubMed] [Google Scholar]
- 44.UNAIDS. Blind spot: reaching out to men and boys—addressing a blind spot in response to HIV 2017. Available from: http://www.unaids.org/sites/default/files/media_asset/blind_spot_en.pdf.
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Data Availability Statement
All authors ensure that the data support the published claims and comply with field standards.