Developing HIV indicators for clinical care quality assessment: relevance and practicality evaluated by healthcare practitioners in South Africa and Democratic Republic of Congo

Abstract

Background

The use of quality indicators in healthcare systems is one of the factors that improve the quality of health services. However, indicators for assessing HIV clinical care in the context of low- and middle-income countries are not fully explored. Some existing indicators were established within the context of developed countries and are primarily defined for community care, with limited emphasis on the hospital setting. Additionally, these indicators often do not account for the local practicality and relevance of quality measures in the context of low- and middle-income countries.

Aim

To assess the clinical significance and practical applicability of these indicators from the perspective of healthcare professionals specialising in HIV care in South Africa and Democratic Republic of Congo.

Methods

After performing a systematic review of quality indicators employed in the evaluation of clinical care for individuals with HIV, we conducted an observational, cross-sectional study. In this research, 30 physicians filled out two questionnaires with a core set of indicators, to establish the most pertinent and practically indicators for evaluation of the HIV clinical care. A Likert scale was used to rank the indicators. Kendall’s tau-B rank correlation analysis was also performed.

Results

From the initial list of 88 quality indicators, 43 were identified as the most relevant and practical in HIV clinical care. Healthcare professionals deemed the monitoring and therapy domain to be the most pertinent and useful indicators out of the seven different clinical domains, followed by the functional organisational structure domain.

Conclusion

This instrument may be a tool for healthcare professionals and hospital administrators to improve the quality of HIV clinical care.

WHAT IS ALREADY KNOWN ON THIS TOPIC

• Previous research has established that the use of quality indicators in healthcare systems is one of the main concerns regarding the quality of health services.

• Most clinicians working in health facilities are being faced with the need to measure the quality of care.

• For quality-of-care assessments, the core set of indicators is mainly well-defined for community care, with less emphasis on the hospital setting.

WHAT THIS STUDY ADDS

• Our study defined quality indicators for assessing HIV clinical care, considering local practicality and pertinence of quality indicators.

• We developed an appropriate tool providing the list of quality indicators for assessing HIV clinical care in the context of low- and middle-income countries.

HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY

• The findings of our study may contribute valuable insights into the refinement and optimisation of assessment tools and methodologies used in evaluating, monitoring and improving the quality of clinical care for HIV patients, thereby improving patient outcomes.

Introduction

HIV persists as a substantial global health concern, exerting profound impacts on millions of individuals across the globe.¹ The scholarly discourse has increasingly concentrated on quality indicators in response to practice disparities, substandard quality, rising healthcare expenditures and inefficiencies within healthcare markets.² As a result, most healthcare practitioners (HCP) employed in health establishments are confronted with the imperative to assess the quality of care. Assessing the quality of clinical care provided to HIV patients is critical for ensuring the best outcomes.^{3 4} Healthcare professionals play a central role in the provision of care and in negotiating the complexities associated with the management of HIV.^{5 6} Healthcare professionals’ perspectives, including those of physicians, nurses and other care providers, are invaluable in selecting and implementing appropriate indicators.^{7 8} The firsthand involvement of healthcare professionals in the management of patients with HIV and AIDS (PLWHA). equips them with the ability to discern critical facets of care that exert substantial influence on patient outcomes. Through their perspectives, we can elucidate the relevance and practicality of HIV indicators in assessing the quality of clinical care.

Several studies have concentrated on the identification and enhancement of quality indicators to assess the performance of clinical practices and enhance the delivery of care for people living with HIV.^{9 10} Therefore, most of these indicators have been established in developed countries. Few studies to determine indicators for the quality of HIV care in developing countries are primarily defined for community care, with less emphasis on the hospital setting.^{11 12} Although some indicators for assessing clinical care exist, they fail to consider local practicality and pertinence of quality indicators in the context of settings with low- and middle-income countries. Donabedian emphasises the significance of considering local factors when improving healthcare systems and enhancing patient health outcomes.¹³ Therefore, the insights and perspectives of local healthcare professionals are invaluable in guiding the selection and adoption of suitable quality-of-care indicators of HIV.¹⁴ The context of HIV in low- and middle-income countries is uniform across various regions. Most of the health systems in these countries suffer from inadequate infrastructure capacity, poor health information systems, inadequate financial and human resources, lack of collaboration and transparency, and a need for management capacity building.¹⁵ However, all healthcare systems and healthcare providers in low- and middle-income countries are facing unique challenges in assessing and improving the quality of HIV care.

Developed countries, while generally having more robust healthcare systems compared with developing countries, also face their own set of challenges in measuring and ensuring the quality of care. These challenges, however, are typically different in nature and scope. Despite these challenges, developed countries often have the resources and infrastructure to address and mitigate these issues through ongoing reforms, investments in technology and infrastructure, and continuous quality improvement initiatives. They also benefit from stronger regulatory frameworks and more robust health information systems that can support comprehensive care quality assessment and improvement efforts.¹⁶

In the process of indicator development, a critical phase for developers following an exhaustive literature review to identify existing indicators is the assembly of a diverse consensus group and the implementation of structured development protocols.¹⁷ In a previously documented investigation, a systematic review was conducted to identify and select quality indicators that have been published to assess the provision of hospital care in the context of HIV.¹⁸

From 20 studies selected, this systematic review identified existing quality indicators used for monitoring and evaluating HIV clinical care and proposed a fundamental set of 88 quality indicators. Each indicator was identified by assessing their endorsement by the WHO¹⁹ and South African guidelines.²⁰ To establish a comprehensive set of HIV and AIDS clinical care indicators, the researchers included only those concurrently used in at least two studies and endorsed by both guidelines and those endorsed by both guidelines only but most relevant.

The lack of a standardised and appropriate set of quality indicators for HIV/AIDS care has posed a significant challenge to quality assessment and improvement efforts. This study aimed to assess the clinical pertinence and the practicality of HIV and AIDS quality indicators for clinical care by healthcare practitioners (HCP). In addition, to develop an appropriate tool providing the list of quality indicators for assessing HIV clinical care in the context of low- and middle-income countries.

Materials and methods

Study design

This observational and cross-sectional study assessed the pertinence and practicality of HIV indicators in evaluating the quality of clinical care. Perspectives were collected from healthcare professionals with expertise in HIV clinical practice. The list of quality indicators for the assessment of HIV clinical care that were validated as pertinent and clinically practical by HCP was selected and considered relevant to be used as quality indicators to assess HIV clinical care in the context of low- and middle-income countries.

Participant selection

Conveniently, a selection was made among healthcare professionals, specifically physicians who are engaged in the Department of Internal Medicine, working in the antiretroviral (ARV) clinic at King Edward VIII Hospital and Panzi Hospital. The King Edward VIII Hospital is situated in Durban, KwaZulu Natal province in South Africa. Whereas Panzi Hospital is located in Bukavu, South Kivu province in the Democratic Republic of Congo. The objective and purpose of the study were elucidated to all healthcare professionals in attendance during a departmental meeting, and unanimous verbal agreement to participate in the study was obtained.

The Department of Internal Medicine at King Edward VIII Hospital, and Panzi Hospital house an ARV clinic that provides daily PLWHA care. The ARV clinic at King Edward VIII Hospital serves as a primary point of contact for HIV patients, accommodating an average of 4000 visits annually and attending to the needs of 2300 patients during follow-up. Among these patients, 1301 are currently receiving antiretroviral therapy (ART). Furthermore, the clinic welcomes an average of 10 treatment-naive HIV and AIDS patients each month. The ARV clinic at Panzi Hospital is the largest in South Kivu and serves as a referral centre for many healthcare facilities in the east of the DRC. This clinic provides daily care for PLWHA, serves as a primary point of contact for approximately 3000 PLWHA, and accommodates an average of 4500 visits annually. Among these patients, 2428 are currently receiving ART from the clinic. Furthermore, an average of 12 patients are initiated on ART at the clinic each month.

Inclusion and exclusion criteria

The inclusion criteria encompassed physicians actively providing care to HIV patients within the ARV clinic of the Department of Internal Medicine of Panzi and King Edward VIII Hospital, having devoted more than 2 years to attending to individuals with HIV infection, and willingly consenting to participate in the study. The exclusion criteria include healthcare professionals who did not complete the questionnaire. Physicians with less than 2 years of experience in HIV care were excluded. Healthcare practitioners who are not physicians (eg, nurses, clinical officers, community health workers). Residents in training were excluded. All 30 questionnaires distributed to HCPs (15 for each clinic) were satisfactorily completed, accounting for a 100% response rate.

Data collection instrument

Two questionnaires were used, each employing a similar indicator structure across different domains. The domains were established on the basis of the intended function of the indicators or the specific aspects they assessed. For instance, indicators related to initial evaluation and diagnosis were grouped under the domain ‘initial evaluation and diagnosis’. At the same time, those screening tests were gathered within the ‘screening’ domain, and so forth. Domains were named under the clinical care functions used in medical terminology. As a result, seven domains were defined (table 1). The boundaries between these domains were not precisely defined because some indicators could apply to multiple domains. Therefore, this model was employed to conveniently facilitate the comprehensive management of all indicators.

Table 1. Different domains and types of indicators included.

	Domain	Type of indicators included
1	Functional organisation structure	This domain encompassed indicators that evaluated the healthcare setting itself, including facilities, equipment, healthcare professionals-patient communication and the organisation of healthcare professionals
2	Initial evaluation and diagnosis	This domain focused on indicators related to patients’ baseline tests and screening specific conditions, such as opportunistic diseases indicative of advanced HIV infection
3	Screening	This domain involved indicators assessed various conditions and comorbidities
4	Prevention	This domain incorporated indicators to measure interventions to reduce the risk of HIV transmission. The fifth domain
5	Immunisation	This domain included indicators concerning the vaccination protocols recommended for HIV patients to minimise the likelihood of complications
6	Monitoring	This domain consisted of indicators for tracking specific conditions, detecting any changes and identifying new symptoms during follow-up
7	Therapy	This domain incorporated care indicators used to evaluate the effects of therapy and assess the interventions carried out

Data collection

Two questionnaires were developed on the basis of a previous systematic literature review, encompassing 88 indicator topics.¹⁴ One of the questionnaires aimed to evaluate the clinical pertinence of these indicators from the perspective of healthcare professionals, while the other sought to assess their practicality in daily care for PLWHA within a hospital setting. Clinical pertinence was defined as the degree to which these quality indicators provided valuable information for understanding the quality of care delivered to HIV patients. Practicality refers to the ease of using these indicators to evaluate and potentially enhance the care provided to HIV patients in a hospital setting. Data collection was conducted in two phases: phase 1 was conducted at King Edward VIII Hospital in Durban, and while phase 2 was at Panzi Hospital in Bukavu. The research team presented the study’s objectives to HCP during a meeting, after which self-administered questionnaires combined in a single brochure were distributed. Six days later, the completed questionnaires were collected.

A Likert scale was used in both questionnaires to determine the most relevant and valuable indicators for evaluating clinical care. The response options were as follows: 0=no opinion; 1=not pertinent/not practically; 2=less pertinent/less practically; 3=pertinent/practically; 4=more pertinent/more practically. Using a Likert scale of 0–4, a median score of 4 (excellent) indicates that at least half of the respondents rated the item at the highest possible score. A median score of 3 (good) shows that at least half of the respondents rated the item positively. A median score of 2 (average) indicates a moderate agreement or neutrality among respondents. A median score of 1 (below average) suggests a general trend toward negative evaluations. And 0 (poor) shows that at least half of the respondents rated the item at the lowest possible score.

In addition, a mean score of 3.5–4.0 (very good) indicates strong positive evaluations from the respondents. A mean score of 3.0–3.4 (good) reflects generally positive feedback. A mean score of 2.0–2.9 (average) shows moderate or neutral evaluations. A mean score of 1.0–1.9 (below average) suggests more negative than positive evaluations. And a mean score of 0–0.9 (poor) indicates strongly negative evaluations.

Indicators with a median score of 4 points (maximum agreement) and a mean score of 3.5–4.00 in both questionnaires were chosen as the most clinically pertinent and practical for evaluating the quality of HIV clinical care.

The 88 quality indicators presented in table 2 pertain to the following seven distinct clinical domains: functional organisational structure (9), initial evaluation and diagnosis (14), screening for opportunistic diseases (17), prevention (7), immunisation (5), HIV monitoring (20) and therapy (16).

Table 2. HIV and AIDS quality Indicators according to name, definition and periodicity.

No	Name	Definition (formula)	Periodicity
	A. Functional organisational structure
1	Attention by specialised healthcare professionals	Healthcare professionals affiliated with the HIV unit who fulfil the specified criteria×100/total count of healthcare professionals affiliated with the unit	3 years
2	Specific nursing consultation	Availability of specific nursing consultation	Quotidian
3	Availability of specialised pharmaceutical care for drug dispensing	Availability of an external pharmacy	Quotidian
4	Availability of diagnostic resources	The availability and accessibility of fundamental diagnostic laboratory resources	Quotidian
5	Availability of voluntary medical male circumcision	Availability of voluntary medical male circumcision	Quotidian
6	Experience of HIV-related discrimination in healthcare settings	Number of inquiries that meet discrimination conditions×100/total number of inquiries	Monthly
7	Conditions of privacy and structural confidentiality	Number of inquiries that meet confidentiality conditions×100/total number of inquiries	Monthly
8	Sexual and reproductive health services, including contraception, must be integrated with HIV services	Availability of sexual, contraceptive and reproductive health services	Quotidian
9	Diabetes and hypertension must be integrated into HIV services	Availability of diabetes and hypertension services	Quotidian
	B. Initial evaluation and diagnosis
10	Relevant anamnesis contents in the initial assessment	Number of patients with the basic examination in the initial assessment×100/number of patients with initial assessment	Monthly
11	Delay in referral to specialised care	Quantity of patients in primary care (PC) exhibiting an interval exceeding 30 days between the serological diagnosis and their initial visit×100/the count of patients diagnosed serologically within primary care (PC) settings and subsequently referred to primary care	Monthly
12	Late diagnosis of HIV infection in specialised care	The quantity of patients who received a belated diagnosis in specialised care (AE) settings×100/total number of patients undergoing their initial assessment in specialised care (AE) settings	As indicated
13	Complementary tests in the initial assessment	The total count of patients who underwent all the essential supplementary tests as part of their initial assessment×100/total number of patients who underwent the initial assessment	As indicated
14	Plasma HIV viral load (In the initial and follow-up)	The total count of patients who underwent measurement of HIV plasma viral load during the initial evaluation×100/number of patients with initial evaluation	At the initial and 3–6 months
15	Determination of CD4 cell count in the initial assessment	The total number of patients who underwent the determination of lymphocyte subpopulations during the initial evaluation×100/number of patients with the initial assessment	At the initial and 3–6 months
16	Renal basic assessment (blood urea nitrogen and creatinine)	Total number of HIV patients who underwent an annual basic renal study×100/number of HIV patients in regular follow-up	Annually
17	Bilirubin assessment	The total number of HIV patients on antiretroviral therapy (ART) whose bilirubin levels were assessed and documented x100/number of HIV patient hospital admitted to the hospital	3–6 months
18	Calculation of the body mass index (BMI)	The total count of HIV patients whose weight was assessed on admission and recorded in their medical charts×100/number of HIV patients hospital admitted to the hospital	Annually
19	Monitor the patient’s weight	Number of HIV patients with weight documented×100/number of HIV patients admitted at the hospital	3–6 months
20	Chest X-ray was performed on the patient at the initial assessment	The total number of HIV patients on antiretroviral therapy (ART) whose chest X-ray was evaluated and documented×100/number of HIV patient hospital admitted to the hospital	3–6 months
21	ECG performed in patients and cardiovascular risk assessment	The total number of HIV patients on antiretroviral therapy (ART) who underwent an ECG, and their cardiovascular risk was evaluated and documented as×100/number of HIV patients hospital admitted to the hospital	Annually
22	Health education at the initial assessment	The total number of patients who received health education during the initial assessment×100/number of patients with initial evaluation	3–6 months
23	Nutritional assessment and counselling at baseline	The total number of patients who received counselling and underwent nutritional assessment at the baseline×100/number of patients with initial assessment	3–6 months
	C. Screening
24	Assessment of resistance in cases of virological failure	The total number of patients undergoing antiretroviral therapy (ART) and diagnosed with virological failure who have undergone a genotypic resistance test×100/the total count of patients on ART who have experienced virological failure	3–6 months
25	Detection and treatment of latent tuberculosis infection (LTBI)	Number of patients in whom detection of LTBI has been performed×100/number of patients undergoing regular follow-up	3 months
26	Cervical cancer screening	Number of HIV patients who were screened for cervical cancer in the last year×100/number of HIV patients seen last year	Annually
27	Hepatitis C screening	Number of HIV patients who were screened for hepatitis C in the last year×100/number of HIV patients seen last year	Annually
28	Hepatitis B screening	Number of HIV patients who were screened for hepatitis B in the last year×100/number of HIV patients seen last year	Annually
29	Syphilis screening	The number of HIV patients who were screened for syphilis in the last year×100/number of HIV patients seen last year	Annually
30	Assessment and management of depression and other mental health disorders	Number of patients who have undergone the ‘anxiety/depression’ screening×100/number of patients in regular follow-up	Annually
31	Injection drug use screening	Number of HIV patients who were screened for injection drug use×100/number of HIV patients in regular follow-up	3–6 months
32	Screening for cryptococcal infection and fungal prophylaxis when appropriate	Number of HIV patients who were screened for cryptococcal infection and under fungal prophylaxis×100/number of HIV patients in regular follow-up	As indicated
33	High-risk sexual behaviour screening	The number of individuals engaging in high-risk sexual behaviour who were screened×100/total number of people living with HIV	As indicated
34	Sexually transmitted disease (STD) screening	Number of HIV patients who were screened for sexually transmitted disease×100/number of HIV patients in regular follow-up	Annually
35	Assessment of primary non-communicable chronic disease	Number of HIV patients who were assessed for primary non-communicable chronic disease×100/number of HIV patients in regular follow-up	Annually
36	Screening for anal cancer	Number of HIV patients who were screened for anal cancer in the last year×100/number of HIV patients seen last year	Annually
37	Detection of metabolic syndrome in the HIV population	Number of individuals with metabolic syndrome in the HIV population×100/total number of individuals in the HIV population	As indicated
38	Evaluation of cardiac risk	Quantification of patients partaking in biennial follow-up sessions for cardiovascular risk assessment was undertaken×100/number of patients undergoing follow-up	2 years
39	Alcohol intake assessment	The number of patients whose alcohol intake has been documented within the previous year has been established×100/number of patients under follow-up	Annually
40	Test for HIV, syphilis and HBsAg in all pregnant women	Number of pregnant woman in whom test for HIV, syphilis and HBsAg were performed×100/number of pregnant women in follow-up	3–6 months
	D. Prevention
41	Primary prophylaxis against Pneumocystis jiroveci and toxoplasmosis in patients with <200 CD4 lymphocytes	The count of patients exhibiting CD4 lymphocyte levels below 200 /mL and receiving prophylaxis against Pneumocystis jirovecii and toxoplasmosis has been determined×100/number of patients undergoing regular follow-up with CD4 lymphocytes <200/mL	Annually
42	Treatment and prevention of smoking	The number of patients with active smoking habit and annual intervention×100/number of patients with an active smoking habit	Monthly
43	Promote voluntary medical male circumcision as an additional effective HIV prevention method	Number of male patients to whom voluntary medical circumcision has been made×100/number of male patients in follow-up	As indicated
44	Promote the dapivirine vaginal ring as an additional prevention option for women	Number of female patients at risk of HIV infection who receive dapivirine vaginal prophylaxis×100/number of female patients at risk of HIV infection	As indicated
45	Oral pre-exposure prophylaxis (PrEP) containing TDF as an additional prevention strategy for people at substantial risk of HIV infection	Quantity of individuals at risk of HIV who are administered oral pre-exposure prophylaxis (PrEP) containing tenofovir disoproxil fumarate (TDF)×100/number of patients at risk of HIV infection	As indicated
46	Regular physical activity for all adults on ART	The number of adult HIV patients with regular physical activity×100/number of adult HIV patients in follow-up	As indicated
47	Co-trimoxazole prophylaxis for adults (including pregnant women) with severe HIV clinical disease (WHO stage 3 or 4) and/or CD4 cell count ≤350 cells/mm3	The number of HIV patients with CD4 counts below 350 cells/µL, who are concurrently receiving cotrimoxazole prophylaxis, and exhibit severe HIV clinical disease×100/the count of patients undergoing routine follow-up with CD4 counts below 350 cells/µL	Annually
	E. Immunisation
48	Vaccination against hepatitis A	The number of patients who have received a prompt indication for vaccines against hepatitis A virus (HAV) has been recorded×100/number of patients undergoing regular follow-up with more than 200 CD4/mL	Annually or at any time when indicated
49	Vaccination against hepatitis B virus	Number of patients in whom express indication of vaccines against HBV has been performed×100/number of patients undergoing regular follow-up with more than 200 CD4/mL	Annually or at any time when indicated
50	Vaccination against pneumococcal infection	Number of patients in whom an express indication of pneumococcal vaccination has been performed×100/number of patients undergoing regular follow-up with more than 200 CD4/mL	Annually or at any time when indicated
51	Vaccination against papillomavirus	Number of patients in whom an indication of vaccines against papillomavirus has been performed×100/number of patients undergoing regular follow-up with more than 200 CD4/mL	Annually or at any time when indicated
52	Vaccination against COVID-19	The number of patients in whom express indication of vaccines against COVID-19 has been performed×100/number of patients undergoing regular follow-up	As indicated
	F. HIV monitoring
53	Evaluation of frailty in patients older than 60 years	The number of HIV patients older than 60 years who have been evaluated for frailty×100/number of HIV patients older than 60 years in follow-up	Annually
54	Therapeutic conciliation in the polymedicated patient older than 60 years	The number of poly-medicated HIV patients older than 60 years to whom the process of therapeutic conciliation has been done×100/number of poly-medicated HIV patients older than 60 years in follow-up	Annually
55	Monitoring of adverse pregnancy outcomes related to ARV exposure	Number of adverse pregnancy outcomes×100/total number of pregnancies exposed to ARVs
56	Monitoring adverse drug reactions	Number of ADRs observed×100/total number of patients exposed to ARVs	3–6 months
57	Monitoring the toxicity of ARVs
58	Ultrasound control of cirrhosis	Number of patients with cirrhosis under regular follow-up who underwent ultrasound control every 6 months×100/number of patients with cirrhosis under regular follow-up	6 months
59	Patients with regular follow-up	Number of patients with appropriate follow-up who regularly attend consultations×100/number of patients undergoing follow-upRegular follow-up: when all the following requirements are fulfilled in the period analysed: attending at least one review every 6 months with documented records of the blood test: CD4+, PVL and recorded analyses	6 months
60	Loss to follow-up	Number of HIV-infected patients lost to follow-up in the last 12 months×100/number of HIV-infected patients in the programme	Annually
61	Recovery from failed appointments	Number of failed citations, with new citations×100/number of failed citations	6 months
62	Incidence of vertical transmission	Number of children born to HIV-infected mothers×100/number of children born to HIV-infected mothers	5 years
63	The overall AIDS-related mortality rate in patients in follow-up	Number of HIV patients in outpatient follow-up and deaths/1000 person-years in outpatient follow-up	Annually
64	Follow-up in the outpatient clinic after hospital discharge	Number of HIV patients discharged from the hospital with a scheduled outpatient visit within 3 months×100/number of HIV patients discharged from the hospital	3 months
65	Mortality rate due to AIDS-related causes in the hospital	Number of HIV of who died while hospitalised×100/number of HIV attended last year who were hospitalised	Annually
66	Continuing education	Number of professionals on staff who have obtained training credits in the last 24 months×100/number of professionals on staff	2 years
67	Evidence of previous HIV serology in men who have sex with men	The number of men who have sex with men with evidence documented of HIV serology×100/number of all male patients in follow-up	As indicated
68	Incidence of admissions because of AIDS-defining illnesses	The number of HIV patients who are hospitalised with severe clinical illness×100/number of patients in outpatient follow-up	Annually
69	Patients with discharge reports after hospitalisation	Number of HIV-infected patients discharged from the hospital with discharge report×100/number of patients discharged after hospitalisation	As indicated
70	Diagnosis of HIV with a previous negative serology	The number of patients with HIV who have previously tested negative for HIV but are suspected to have been exposed to the virus×100/number of patients diagnosed with HIV	Annually
71	Incidence of admissions in patients undergoing follow-up	Number of HIV patients in outpatient follow-up and hospitalised×100/1000 person-years in the outpatient follow-up	Annually
72	Discharge reports in patients who died in the hospital	The number of HIV of who died during hospitalisation with a discharge report×100/number of HIV of who died during hospitalisation	As indicated
	G. Therapy
73	Surveillance of the safety of new ARV drugs in adults	Number of adult patients on new ARV to whom the safety for the drug has been assessed×100/number of adults patients starting new ARV	3–6 months
74	Manage pain and symptoms when appropriate	Number of HIV patients for whom pain and symptoms have been managed×100/number of HIV patients in follow-up	As indicated
75	Patients on ART have clinical visits every 3–6 months	Number of HIV patients on ART who attend clinical visits every 3–6 months×100/number of HIV patients on ART in follow-up	3–6 months
76	People on ART have refills of ART lasting 3–6 months	Number of HIV patients who refills ART lasting 3–6 months×100/number of HIV patients on ART in follow-up	3–6 months
77	Tenofovir prophylaxis to prevent mother-to-child transmission of HBV	Number of HIV-positive pregnant women who received tenofovir to reduce the risk of mother-to-child-transmission×100/number of HIV-positive pregnant women attended	Annually
78	Adaptation of initial ART to national antiretroviral treatment guidelines	Number of patients who start ART with any of regimens of choice of national antiretroviral treatment guidelines×100/number of patients starting ART	As indicated
79	First visit after starting ART	Number of patients who start ART and attend the first visit after start×100/number of patients who start ART
80	Treatment changes during the first year	Number of patients who changed any of the ART drugs in the 12 months following treatment initiation×100/number of patients who started treatment	Annually
81	Specific treatment of chronic HCV hepatitis	Number of HIV patients in regular follow-up with positive HCV serology, evaluated and received hepatitis C treatment×100/number of HIV patients in regular follow-up with positive HCV serology	3 years
82	Undetectable viral load (<50 copies/mL) at week 48 of treatment	Number of patients starting ART who achieved a PVC <50 copies/mL at week 48 of treatment×100/number of patients starting ART	48 weeks after ART
83	Assessment of treatment adherence	Number of patients on ART with a diagnosis of virological failure who have a genotypic resistance test×100/number of patients on ART who have had a virological failure	Twenty 24 weeks after ART
84	ART in pregnant women with HIV infection	The total count of pregnant patients with HIV who are on antiretroviral therapy (ART) starting from week 14 of their pregnancy×100/number of pregnant patients with HIV	14 weeks
85	Treatment with abacavir (ABC) without previous HLA-B * 5701	Quantity of patients who initiated treatment with ABC (abacavir) without undergoing prior analysis of the HLA-B* allele 5701×100/total count of patients who began treatment with ABC (abacavir)	As indicated
86	Initiating ART 8 weeks after the initiation of corticosteroid treatment for patients living with HIV co-infected with TB meningitis	Quantity of HIV patients co-infected with TB meningitis who initiated ART 8 weeks after the commencement of corticosteroid treatment×100/total count of HIV patients co-infected with TB meningitis	8 weeks
87	Co-management of tuberculosis with HIV treatment	The total count of HIV patients who undergoing co-management of both tuberculosis and HIV treatment×100/number of HIV/TB patients receiving care	Annually
88	Initiating ART 4–6 weeks after the initiation of antifungal treatment for patients living with HIV co-infected with cryptococcal meningitis	The quantity of HIV patients co-infected with cryptococcal meningitis who started ART between 4 and 6 weeks after the initiation of antifungal treatment×100/total count of HIV patients who are co-infected with cryptococcal meningitis	4–6 weeks

Statistical analysis

A database using the SPSS V.29.0 was established, incorporating data gathered from questionnaires. Median and mean values were employed to identify the most significant and efficient quality indicators for evaluating HIV clinical care. Specifically, indicators with a median score of 4 and a mean score of 3.5–4.00 in both categories (clinical pertinence and practice usefulness) and not labelled as ‘not pertinent’ or ‘not practically’ by any healthcare professional were considered as consensus quality indicators for the assessment of clinical care.

Kendall’s tau-B rank correlation coefficients were computed to assess the strength of the relationship between the results of clinical pertinence and the inquiries regarding clinical practicality for each quality indicator.

Results

Sociodemographic and qualifications characteristics of physicians in South Africa and the Democratic Republic of Congo

100% of healthcare professionals (physicians) responded to the clinical pertinence and practicality questionnaire. Among these respondents, more than half (n=20; 66.7%) were 18–39 age group, while 10 (33.3%) were 40 years or older. Furthermore, the gender distribution showed that 18 (60%) were male and 12 (40%) were female. Regarding marital status, 9 (30%) were unmarried, 20 (66.7%) were married and 1 (3.3%) was divorced. In terms of professional qualifications, 9 (30%) were general practitioners, 13 (43.3%) held a master’s degree and 8 (26.7%) had completed a post-graduate fellowship. Experience levels varied among the respondents, with 17 (56.7%) having 5–10 years of experience, 12 (40%) having 11–20 years and 1 (3.3%) having 21 or more years of experience.

Clinical pertinence and practicality

Although a thorough analysis was conducted on the individual indicators, the outcomes depicted in this study have been categorised into distinct domains. Indicators of the functional organisational structure domain were evaluated for clinical relevance. A median was obtained (minimum and maximum value) of 4.0 (2.0–4.0), the initial evaluation and diagnosis domain was 3.0 (1.0–4.0), the screening for opportunistic diseases domain was 3.0 (2.0–4.0), the prevention domain was 4.0 (1.0–4.0), the immunisation domain was 3.5 (2.0–4.0), the HIV monitoring domain was 3.0 (1.0–4.0) and the therapy domain was 4.0 (2.0–4.0).

For practicality, the results for the indicators were 4.0 (2.0–4.0), 3.5 (1.0–4.0), 2.0 (1.0–4.0), 4.0 (2.0–4.0), 3.0 (2.0–4.0), 3.5 (2.0–4.0), 4.0 (2.0–4.0), respectively (table 3).

Table 3. Summary of the measures of quality indicators for HIV clinical care on clinical pertinence and practically.

Clinical pertinence	Number of indicators	Median	Minimum	Maximum
Functional organisational structure	9	4	2	4
Initial evaluation and diagnosis	14	3	1	4
Screening for opportunistic diseases	17	3	2	4
Prevention	7	4	1	4
Immunisation	5	3.5	2	4
HIV monitoring	20	3	1	4
Therapy	16	4	2	4
Practicality
Functional organisational structure	9	4	2	4
Initial evaluation and diagnosis	14	3.5	1	4
Screening for opportunistic diseases	17	2	1	4
Prevention	7	4	2	4
Immunisation	5	3	2	4
HIV monitoring	20	3.5	2	4
Therapy	16	4	2	4

HCPs deemed the functional organisational structure and the therapy domain to be the most pertinent and useful indicators out of the seven different clinical domains, followed by the prevention domain. Screening for opportunistic diseases domain was the least pertinent and useful (table 3).

Interdomain correlations

The application of Kendall’s tau-B rank correlation analysis (table 4) revealed noteworthy associations in terms of clinical pertinence. Specifically, significant correlations were observed between the ‘functional organisational structure’ domain and other domains such as prevention (0.74), immunisation (0.54) and therapy (0.55). Moreover, correlations were found between ‘prevention’ and the immunisation domain (0.53), and functional organisational structure (0.74). For practicality, significant correlations were observed between the domain ‘functional organisational structure’ and prevention (0.65), and the therapy domain (0.73). Moreover, correlations between the ‘prevention’ domain, and functional organisational structure (0.65) were found. Additionally, correlations were observed between the ‘therapy’ domain, and functional organisational structure (0.73).

Table 4. Correlation (Kendall’s tau-B rank correlation) between each domain for clinical pertinence and practicality.

	Functional organisational structure	Initial evaluation and diagnosis	Screening for opportunistic diseases	Prevention	Immunisation	HIV monitoring	Therapy
Clinical pertinence
Functional organisational structure	1.00	0.04	−0.27	0.74*	0.54*	0.30	0.55*
Initial evaluation and diagnosis	0.04	1.00	−0.11	0.33	−0.13	0.26	0.21
Screening for opportunistic diseases	0.27	−0.11	1.00	−0.32	0.13	0.18	0.21
Prevention	0.74*	0.33	0.33	1.00	0.53*	−0.25
Immunisation	0.54*	−0.13	0.13	0.53*	1.00	0.00
HIV monitoring	0.30	−0.26	0.17	0.24	0.00	1.00	−0.08
Therapy	0.55*	0.20	0.20			−0.08	1.00
Practicality
Functional organisational structure	1.00	0.31	−0.19	0.65*	−0.17	−0.18	0.73*
Initial evaluation and diagnosis	0.31	1.00	0.30	0.46	−0.28	0.08	−0.05
Screening for opportunistic diseases	0.19	0.30	1.00	0.35	−0.14	0.05	0.38
Prevention	0.65*	0.46	0.35	1.00	−0.25	−0.32
Immunisation	0.17	−0.28	−0.14	0.25	1.00
HIV monitoring	−0.18	0.08	0.05	−0.32		1.00	−0.28
Therapy	0.73*	−0.05	0.38			−0.28	1.00

^*Correlation is significant at the 0.05 level.

Pertinent and practical HIV quality indicators for evaluating clinic care, according to healthcare professionals

Indicators with a median score of four points (maximum agreement) and a mean score of 3.5–4.00 in both questionnaires were chosen as the most clinically pertinent and practical for evaluating the quality of HIV clinical care (table 5). As a result, out of the initial list of 88 quality indicators, 43 were selected and considered to be most pertinent and practical for assessing HIV clinic care.

Table 5. HIV quality indicators considered most pertinent and practical for evaluating clinic care, according to healthcare professionals.

No	Indicators	Mean	Median	WHO guidelines on HIV	South African guidelines for the management of HIV	DRC guidelines for the management of HIV
	A.Functionalorganisationalstructure
1	Attention by specialised healthcare professionals	3.85	4.00	+	–	+
3	Availability of specialised pharmaceutical care for drug dispensing	3.83	4.00	–	+	–
4	Availability of diagnostic resources	3.93	4.00	+	+	+
6	Experience of HIV-related discrimination in healthcare settings	3.95	4.00	+	+	+
7	Conditions of privacy and structural confidentiality	4.00	4.00	+	+	+
8	Sexual and reproductive health services, including contraception, must be integrated with HIV services	3.96	4.00	+	+	+
	B. Initial evaluation and diagnosis
11	Delay in referral to specialised care	3.71	4.00	+	+	+
14	Plasma HIV viral load (in the initial and follow-up)	3.83	4.00	+	+	+
15	Determination of CD4 cell count in the initial assessment	3.50	4.00	+	+	+
16	Renal basic assessment (blood urea nitrogen and creatinine)	3.98	4.00	+	+	+
22	Health education at the initial assessment	3.56	4.00	+	+	+
	C. Screening
24	Assessment of resistance in cases of virological failure	3.56	4.00	+	+	+
25	Detection and treatment of latent tuberculosis infection (LTBI)	3.92	4.00	+	+	+
30	Assessment and management of depression and other mental health disorders	3.58	4.00	+	+	+
40	Test for HIV, syphilis and HBsAg in all pregnant women	3.71	4.00	+	+	+
	D. Prevention
41	Primary prophylaxis against Pneumocystis jiroveci and toxoplasmosis in patients with <200 CD4 lymphocytes	3.98	4.00	+	+	+
44	Promote the dapivirine vaginal ring as an additional prevention option for women	3.52	4.00	+	+	+
45	Oral pre-exposure prophylaxis (PrEP) containing TDF as an additional prevention strategy for people at substantial risk of HIV infection	3.85	4.00	+	+	+
46	Regular physical activity for all adults on ART	3.75	4.00	+	+	+
47	Co-trimoxazole prophylaxis for adults (including pregnant women) with severe HIV clinical disease (WHO stage 3 or 4) and/or CD4 cell count ≤350 cells/mm3	3.67	4.00	+	+	+
	E. Immunisation
50	Vaccination against pneumococcal infection	3.90	4.00	+	–	+
	F. HIV monitoring
53	Evaluation of frailty in patients older than 60 years	3.53	4.00	+	–	+
54	Therapeutic conciliation in the polymedicated patient older than 60 years	3.58	4.00	+	–	+
56	Monitoring adverse drug reactions	3.60	4.00	+	+	+
59	Patients with regular follow-up	3.93	4.00	+	+	+
60	Loss to follow-up	3.86	4.00	–	–	–
61	Recovery from failed appointments	3.95	4.00	–	+	–
62	Incidence of vertical transmission	3.81	4.00	+	+	+
65	Mortality rate due to AIDS-related causes in the hospital	3.56	4.00	–	–	–
	G. Therapy
73	Surveillance of the safety of new ARV drugs in adults	3.96	4.00	–	+	–
74	Manage pain and symptoms when appropriate	3.96	4.00	+	+	+
75	Patients on ART have clinical visits every 3–6 months	3.85	4.00	+	+	+
76	People on ART have refills of ART lasting 3–6 months	3.86	4.00	+	–	+
77	Tenofovir prophylaxis to prevent mother-to-child transmission of HBV	3.91	4.00	+	+	+
78	Adaptation of initial ART to national antiretroviral treatment guidelines	3.76	4.00	+	+	+
79	First visit after starting ART	3.81	4.00	+	+	+
80	Treatment changes during the first year	3.77	4.00	–	+	–
82	Undetectable viral load (<50 copies/mL) at week 48 of treatment	3.85	4.00	+	+	+
83	Assessment of treatment adherence	3.81	4.00	+	+	+
84	ART in pregnant women with HIV infection	3.78	4.00	+	+	+
86	Initiating ART 8 weeks after the initiation of corticosteroid treatment for patients living with HIV co-infected with TB meningitis	3.96	4.00	+	+	+
87	Co-management of tuberculosis with HIV treatment	3.73	4.00	+	+	+
88	Initiating ART 4–6 weeks after the initiation of antifungal treatment for patients living with HIV co-infected with cryptococcal meningitis	3.70	4.00	+	–	+

Discussion

Our study examined the clinical pertinence and practicality of HIV quality indicators for evaluating clinical care. We sought the perspectives of local HIV experts who regularly generate indicator data and employ these measures to evaluate the quality of hospital care for patients with HIV. This paper presents an initial phase in formalising the development and validation process for a comprehensive set of quality indicators specifically designed for assessing the quality of hospital care provided to individuals with HIV in low- and middle-income countries.

Most healthcare professionals prioritise indicators related to HIV monitoring and the therapy domain as the most critical for assessing the quality of care provided to individuals with HIV and AIDS. These indicators are considered the key measures of effective care. Conversely, indicators associated with immunisation and the screening for opportunistic diseases are viewed as less critical by healthcare professionals, suggesting they are not seen as primary determinants of care quality in this study.

Despite the comparatively lower importance assigned to indicators for the screening for opportunistic disease and immunisation domains by healthcare professionals, it is imperative not to overlook the significance of these domains when evaluating the quality of care for HIV patients. It is essential to acknowledge that all individuals living with HIV are vulnerable to various types of opportunistic infections.^{21 22} These infections continue to substantially threaten the well-being and survival of HIV patients in low- and middle-income countries, leading to significant morbidity and mortality rates.^{23 24}

Considering the opinions of healthcare professionals on the correlation of indicators between the domains, it was found that there are notable correlations between the domain of ‘functional organisational structure’ and the areas of prevention, immunisation and therapy. These findings imply that a functional organisational structures’ clinical pertinence and practicality are associated with the three domains mentioned. A functional organisational structure is a setup in which tasks and responsibilities are divided on basis of specialised functions or departments.²⁵

Overall, the observed correlations provide empirical evidence for the practicality of a functional organisational structure in relation to prevention and therapy. Organisations that adopt such a structure may experience benefits in terms of improved planning, coordination, effectiveness and outcomes in these domains.²⁶

Additionally, a strong correlation was observed between the immunisation domain, functional organisational structure and the prevention domain.

The presence of correlations between the immunisation domain and functional organisational structure and prevention highlights the interconnections between these areas. This review shows that organisations with a functional structure emphasising prevention measures are more likely to have effective immunisation practices in place. This further supports the notion that a well-defined organisational structure can positively impact multiple healthcare-related domains.²⁷

Most quality indicators selected by healthcare professionals in this study are aligned with the guidelines endorsed by the WHO²⁶ on HIV and the South African guidelines for managing HIV.²⁷ These indicators are considered essential for assessing the clinical care provided for individuals with HIV. Quality in this context is defined as adherence to established standards; however, it is also crucial to comprehensively understand the patient’s overall health status.

In addition to the WHO and South African guideline-endorsed indicators, some indicators that were not explicitly endorsed by these guidelines were deemed necessary to be included as quality indicators for evaluating clinical care for HIV. These additional indicators are as follows: indicator number 1 (attention by specialised healthcare professionals), 3 (availability of specialised pharmaceutical care for dispensing drugs), 53 (evaluation of frailty in patients older than 60 years), 54 (therapeutic conciliation in the poly-medicated patient older than 60 years), 60 (loss to follow-up), 61 (recovery from failed appointments), 65 (the mortality rate due to AIDS-related causes in the hospital), 73 (surveillance of the safety of new ARV drugs for adults), 76 (ensuring people on ART have refills of ART lasting 3–6 months), 80 (treatment changes during the first year) and 88 (initiating ART after 4–6 weeks from the initiation of antifungal treatment for patients living with HIV coinfected with cryptococcal meningitis). These indicators were considered relevant and valuable for assessing the quality of care provided in the context of HIV.

Based on the study findings, it is suggested to include the following indicators, which are currently not endorsed by the South African guidelines: indicators 1 (attention by specialised healthcare professionals), 50 (vaccination against pneumococcal infection), 53 (evaluation of frailty in patients older than 60 years), 54 (therapeutic conciliation in the poly-medicated patient older than 60 years), 60 (loss to follow-up), 65 (the mortality rate due to AIDS-related causes in the hospital), 76 (ensuring people on antiretroviral therapy have refills of ART lasting 3–6 months) and 88 (initiating ART after 4–6 weeks from the initiation of antifungal treatment for patients living with HIV coinfected with cryptococcal meningitis).

It is also suggested to include the following indicators, which are currently not endorsed by the WHO and DRC guidelines: indicators 3 (availability of specialised pharmaceutical care for dispensing drugs), 60 (loss to follow-up), 61 (recovery from failed appointments), 65 (the mortality rate due to AIDS-related causes in the hospital), 73 (surveillance of the safety of new ARV drugs for adults) and 80 (treatment changes during the first year).

These findings give rise to several considerations that warrant attention in future research. First, it is crucial to incorporate the perspectives of patients with HIV and their assessments of the quality of clinical care. While acknowledging the limited scope of this study, it serves as a valuable starting point for developing specific quality indicators to evaluate clinical care for individuals with HIV in low- and middle-income countries. Second, it is important to recognise that the selected indicators may not encompass all possible metrics for assessing clinical care among HIV patients. This limitation does not stem from any shortcomings in the systematic review methodology employed but rather from the fact that healthcare professionals were solely requested to evaluate their clinical pertinence and practicality without evaluating their appropriateness.

Only a few expert physicians in DRC and SA were included in this study. Other expert physicians from other resource-limited countries may have divergent views. The consensus opinions reached on the list of quality indicators to assess HIV clinical care, therefore, are difficult to generalise for the entire resource-limited countries’ expert physicians.

Only physicians were selected in this study. The decision to select only physicians for evaluating HIV indicators for clinical care quality assessment has been based on several reasons: physicians typically have comprehensive training and expertise in diagnosing and managing HIV. Their insights are crucial for understanding clinical care quality; physicians are primarily responsible for making key clinical decisions regarding patient management and treatment protocols, making their perspective particularly relevant for assessing care quality; by selecting a specific group of healthcare providers, the authors could maintain consistency in responses, which is essential for comparative analysis. Nonetheless, while this study’s primary focus was on physicians, we agree that broadening the scope to include other healthcare providers, such as nurses, pharmacists and social workers, could offer a more holistic view of patient care in these settings. These elements constitute important directions for future research. Additionally, incorporating measurable indicators could enhance our findings’ robustness and plan to address this as a consideration for future work.

In addition, our investigation was single-site clinic-based in each selected country, so the results may need to be more generalisable to the overall population.

Despite these limitations, the results presented here draw attention to HIV-infected adult and highlight the feasibility of treatment success even in resource constraint settings.

Conclusion

In summary, comprehensive measurement of clinical care quality necessitates the availability of indicators spanning various domains of care. Among the initial pool of 88 indicators, healthcare professionals discerned a subset of 43 indicators deemed highly pertinent and valuable for evaluating the clinical care provided to individuals with HIV.

Data availability statement

All data relevant to the study are included in the article or uploaded as supplementary information.