Skip to main content
Bulletin of the World Health Organization logoLink to Bulletin of the World Health Organization
. 2014 Oct 30;93(1):29–41. doi: 10.2471/BLT.14.138149

Association between antiretroviral therapy adherence and employment status: systematic review and meta-analysis

Lien entre le suivi de la thérapie antirétrovirale et le statut professionnel: revue systématique et méta-analyse

La asociación entre la adherencia a la terapia antirretroviral y la situación laboral: examen sistemático y metaanálisis

الارتباط بين الالتزام بالعلاج بمضادات الفيروسات القهقرية وحالة التوظيف: استعراض منهجي وتحليل وصفي

抗逆转录病毒疗法的坚持和就业状况之间的关联:系统回顾和元分析

Связь между соблюдением режима антиретровирусной терапии и статусом занятости: систематический обзор и мета-анализ

Jean B Nachega a,, Olalekan A Uthman b, Karl Peltzer c, Lindsey A Richardson d, Edward J Mills e, Kofi Amekudzi f, Alice Ouédraogo f
PMCID: PMC4271680  PMID: 25558105

Abstract

Objective

To assess the association between the employment status of human immunodeficiency virus (HIV)-infected individuals and adherence to antiretroviral therapy (ART).

Methods

We searched the Medline, Embase and Cochrane Central Register of Controlled Trials databases for studies reporting ART adherence and employment status published between January 1980 and September 2014. Information from a wide range of other sources, including the grey literature, was also analysed. Two independent reviewers extracted data on treatment adherence and study characteristics. Study data on the association between being employed and adhering to ART were pooled using a random-effects model. Between-study heterogeneity and sources of bias were evaluated.

Findings

The meta-analysis included 28 studies published between 1996 and 2014 that together involved 8743 HIV-infected individuals from 14 countries. The overall pooled odds ratio (OR) for the association between being employed and adhering to ART was 1.27 (95% confidence interval, CI: 1.04–1.55). The association was significant for studies from low-income countries (OR: 1.85, 95% CI: 1.58–2.18) and high-income countries (OR: 1.33, 95% CI: 1.02–1.74) but not middle-income countries (OR: 0.94, 95% CI: 0.62–1.42). In addition, studies published after 2011 and larger studies showed less association between employment and adherence than earlier and small studies, respectively.

Conclusion

Employed HIV-infected individuals, particularly those in low- and high-income countries, were more likely to adhere to ART than unemployed individuals. Further research is needed on the mechanisms by which employment and ART adherence affect each other and on whether employment-creation interventions can positively influence ART adherence, HIV disease progression and quality of life.

Introduction

Recent data from the World Health Organization (WHO) and the Joint United Nations Programme on HIV/AIDS (UNAIDS) indicate that access to human immunodeficiency virus (HIV) treatment and care services in low- and middle-income countries has expanded dramatically.1 In 2012, over 9.7 million people living with HIV in these countries were receiving antiretroviral therapy (ART).1

Despite this increase, ensuring adherence to HIV treatment remains challenging in all countries. A meta-analysis of patients in North America (n = 17 573) and Africa (n = 12 116) estimated that only 55% and 77% in these areas, respectively, achieved over  80% adherence2 and a global meta-analysis, which included 33 199 patients on ART, reported that only 62% achieved over 90% adherence.3 Common barriers to adherence include medication side-effects, pill burden, the need to disclose HIV serostatus, a perception of feeling well, treatment fatigue and structural and psychosocial factors.4,5 For individuals, nonadherence can result in virological treatment failure, the development of drug resistance, disease progression and death.69 At the community level, nonadherence makes HIV transmission more likely,10,11 can substantially increase health-care costs, particularly for hospitalization to treat opportunistic infections,12 and can decrease productivity.13 One factor that has not been explored in depth is whether an individual’s employment status influences adherence to ART.14,15

Adherence to ART may be influenced by factors associated with the disease and its treatment, with the relationship between the patient and the health-care provider and with patients themselves, such as socioeconomic status which is often based on employment or occupational status in addition to educational level and income.14,16 Moreover, differences in adherence between people employed in the informal and the formal economy have been linked to gender roles and inequalities in employment status.17,18 A previous meta-analysis by our collaborative research group found that employed HIV-infected patients from low-, middle- and high-income countries were 39% more likely to adhere to ART than unemployed patients. However, the study included very few participants from middle- and low-income countries and therefore did not have sufficient statistical power to determine whether a country’s income level had a significant effect on the association between employment status and ART adherence.19

The aim of this systematic review and meta-analysis was to investigate the relationship between the employment status of HIV-infected individuals and ART adherence using updated data and a larger patient sample, which included more information from middle- and low-income countries.

Methods

This meta-analysis was reported in accordance with the Preferred reporting items for systematic reviews and meta-Analyses: the PRISMA statement.20 Studies of any design were included if they satisfied the following criteria: (i) the study involved people living with HIV; (ii) participants were receiving highly active antiretroviral therapy; (iii) treatment adherence was assessed using objective or self-reported measures; and (iv) employment was considered a possible factor influencing adherence.

We searched the Medline, Embase and Cochrane Central Register of Controlled Trials databases for the period January 1980 to September 2014 (Box 1). In addition, we used a narrative literature review approach to analyse and summarize information on HIV treatment, particularly on adherence, from a range of sources including UNAIDS secretariat reports, scientific conference abstracts and other grey literature. We contacted individual researchers for details of unpublished studies.

Box 1. Search terms used for studies on the association between employment status and adherence to antiretroviral therapy.

1. hiv infections/

2. HIV.ti.

3. human immunodeficiency virus.ti,ab.

4. HIV Infections/pc

5. HIV/ or HIV-1/

6. Acquired Immunodeficiency Syndrome/pc [Prevention & Control]

7. exp hiv/

8. exp hiv-1/

9. exp hiv-2/

10. Human immunodeficiency virus.mp.

11. hiv.mp.

12. or/ No. 1–11

13. blue collar.mp.

14. blue collar.ti,ab.

15. white collar.mp.

16. exp Social Class/

17. exp Adult/ or Occupations/

18. Agriculture/ec, ed, ma [Economics, Education, Manpower]

19. exp Employment/

20. job.mp.

21. exp work/

22. exp income/

23. manpower.mp.

24. socioeconomic.mp.

25. socio-economic.mp.

26. office.mp.

27. or/No. 13–26

28. exp Medication Adherence/

29. Adherence.mp.

30. Nonadherence.mp.

31. Compliance.mp.

32. or/ No. 28–31

33. No. 12 and 27 and 32

Two reviewers evaluated the eligibility of the studies identified and a third reviewer provided arbitration if there was a discrepancy. One reviewer extracted data, which were checked by others. The quality of the studies included was assessed using the Risk of Bias Assessment tool for Non-randomized Studies (RoBANS; details available from the corresponding author on request).21 The risk of bias in a study was graded as low, high or unclear on the basis of study features including the selection of participants (selection bias), consideration of confounding variables (selection bias), outcome measurement (detection bias), incomplete outcome data (attrition bias) and selective outcome reporting (reporting bias).

Data extraction

For each study included, we recorded: the first author’s last name; the year of publication; the country where the study was performed; details of the study design; the years when data were collected; the type of controls in case–control studies; the duration of follow-up in cohort studies; sample size; details of exposure measures, such as indicators of occupation or employment; age; sex; the odds that an employed patient versus an unemployed patient would adhere to ART; and the variables controlled for. Study countries were classified by geographical area and categorized as low-, middle- or high-income, as defined by the World Bank for 2014.22 Study participants were defined as being of working age either by the study investigators or using the International Labour Organization’s standard definition.23 In addition, the International Labour Organization’s definition of employment was applied: Persons in employment comprise all persons above a specified age who during a specified brief period, either one week or one day, were in the following categories: paid employment and self-employment.24

Consequently, studies were considered eligible if their authors defined employed people as those who, during a specified brief period such as one week or one day: (i) performed some work for wage or salary in cash or in kind; (ii) had a formal attachment to a job but were temporarily not at work during the reference period; (iii) performed some work for profit or family gain in cash or in kind; or (iv) were with an enterprise such as a business, farm or service but who were temporarily not at work during the reference period for any specific reason.24

Data synthesis

The meta-analysis was performed using the DerSimonian and Laird random-effects model to obtain a pooled estimate for the odds ratio (OR) and the associated 95% confidence interval (CI).25 The model was chosen since it takes into account both within- and between-study variability, as between-study heterogeneity was anticipated. Heterogeneity among studies was assessed by inspecting forest plots of the odds ratios from each study and by using the χ2 test for heterogeneity, with a 10% level of statistical significance, and the I2 statistic, with which a value of 50% represented moderate heterogeneity. We used leave-one-study-out sensitivity analysis to evaluate the stability of the results and to test whether any one study had an excessive influence on the meta-analysis.26 In addition, we performed subgroup analyses to assess the influence of study design (i.e. cross-sectional versus prospective cohort), the country’s income group (i.e. low, middle or high), the adherence threshold and adherence measures. We used the χ2 test to subgroup differences and reported the P-value for interaction between pooled OR and study-level characteristics. We used a variance-weighted, least-squares regression approach to estimate the effect of the year of publication (i.e. trend analysis) and the study sample size on the association between employment status and ART adherence. For all tests, a probability less than 0.05 was considered significant. All statistical tests were two-sided. Analyses were performed using Stata version 12 (StataCorp. LP, College Station, United States of America).

Results

A flow diagram of study selection is shown in Fig. 1. Twenty-eight studies, involving a total of 8743 patients from 14 countries, met criteria for inclusion in the systematic review (Table 1).4,2753 The studies were carried out between 1996 and 2012 and publication took place between 1996 and 2014. Overall, 24 studies, involving 7484 of 8743 patients (86%), were cross-sectional, whereas the other four were prospective cohort studies. Eight studies, involving 1775 patients (20%), were carried out in the United States, three each were carried out in Ethiopia, India and South Africa and two were carried out in Uganda.

Fig. 1.

Fig. 1

Flowchart showing the selection of studies for the meta-analysis of the association between employment status and adherence to antiretroviral therapy

Table 1. Studies in the meta-analysis of the association between employment status and adherence to antiretroviral therapy, 14 countries, 1996–2014.

First author of study Year of publication Study period Study design Study country Country's income groupa Participants receiving ART at enrolment Lower threshold for adherence to ART, % Adherence measure Sample size, n Male participants, % Age of participants,b years Unemployment, % % ART adherence Measure of associationc
Singh27 1996 Not reported Prospective cohort United States High Yes 80 Pharmacy refill data 46 ND 23–68 41.0 63.0 Unadjusted
Singh28 1999 March 1996 to December 1997 Cross-sectional United States High Yes Not reported Pharmacy refill data 123 ND 24–71 52.8 82.1 Unadjusted
Duong29 2001 Not reported Cross-sectional France High Yes Not reported Blood drug concentration 149 71.8 21–79 45.6 89.0 Unadjusted
Ickovics30 2002 Not reported Cross-sectional from RCT United States High Yes 95 Self-report questionnaire 93 88.0 19–61 23.9 63.0 Adjusted
Nachega31 2004 Not reported Cross-sectional South Africa Middle Yes 95 Self-report questionnaire 66 28.8 36.1 (10.1) 40.9 88.0 Unadjusted
Beyene32 2009 August to October 2009 Cross-sectional Ethiopia Low Yes 95 Self-report questionnaire 422 43.6 32.2 (7.2) 59.0 93.1 Adjusted
Duggan33 2009 August to November 2006 Cross-sectional United States High Yes Not reported Combination of methods 132 66.7 18–51 55.3 74.2 Unadjusted
Nakimuli-Mpungu34 2009 Not reported Cross-sectional Uganda Low Yes 90 Self-report questionnaire 122 21.3 36.0 (8.2) 34.4 82.8 Adjusted
Campos35 2010 May 2001 to May 2002 Prospective cohort Brazil Middle Yes 95 Self-report questionnaire 293 65.9 ND 35.1 62.8 Adjusted
Giday36 2010 August to September 2008 Cross-sectional Ethiopia Low Yes 95 Self-report questionnaire 510 38.6 15–63 39.6 88.2 Adjusted
Kunutsor37 2010 Not reported Prospective cohort Uganda Low Yes 95 Pharmacy refill data 392 35.2 32–45 55.4 93.1 Adjusted
Lal38 2010 2005 Cross-sectional India Middle Yes 95 Self-report questionnaire 300 72.0 30–45 31.8 75.7 Unadjusted
Li39 2010 Not reported Cross-sectional from RCT Thailand Middle Yes 100 Self-report questionnaire 386 32.7 38.0 (6.4) 15.5 68.6 Adjusted
Peltzer40 2010 October 2007 to February 2008 Cross-sectional South Africa Middle No 95 Self-report questionnaire 735 29.8 ND 59.6 82.9 Unadjusted
Sherr41 2010 2005 to 2006 Cross-sectional United Kingdom High Yes 100 Self-report questionnaire 449 78.9 ND 42.8 42.8 Unadjusted
Venkatesh42 2010 January to April 2008 Cross-sectional India Middle Yes 95 Self-report questionnaire 198 68.5 ND 21.9 49.0 Adjusted
Harris43 2011 June 2004 to December 2005 Cross-sectional Dominican Republic Middle Yes 95 Self-report questionnaire 300 45.0 ND 53.0 76.0 Unadjusted
Juday4 2011 April to May 2007 Cross-sectional United States High Yes 100 Self-report questionnaire 461 76.1 44.4 (9.3) 56.2 54.0 Adjusted
Kyser44 2011 March 2004 to June 2006 Prospective cohort United States High Yes 100 Self-report questionnaire 528 78.0 20–66 41.0 84.0 Adjusted
Wakibi45 2011 November 2008 to April 2009 Cross-sectional Kenya Low Yes 95 Self-report questionnaire 403 35.0 18–64 34.0 82.0 Unadjusted
King46 2012 February 2007 to December 2009 Cross-sectional from RCT United States High Yes 100 Self-report questionnaire 326 72.1 45.9 (7.6) 79.0 60.4 Adjusted
Kitshoff47 2012 Not reported Cross-sectional South Africa Middle Yes 95 Pill count 146 27.4 31–42 65.0 68.0 Adjusted
Berhe48 2013 August 2012 to October 2012 Cross-sectional Ethiopia Low Yes 95 Self-report questionnaire 174 46.0 38.5 (8.4) 20.1 40.8 Adjusted
Okoronkwo49 2013 Not reported Cross-sectional Nigeria Middle Yes 100 Self-report questionnaire 221 ND ND 21.8 14.9 Unadjusted
Vissman50 2013 November 2008 to April 2009 Cross-sectional United States High Yes 100 Self-report questionnaire 66 74.0 38.0 (10.3) 52.0 71.0 Unadjusted
Tran51 2013 2012 Cross-sectional Viet Nam Middle Yes 95 Self-report questionnaire 1016 63.8 35.4 (7.0) 17.8 74.1 Adjusted
Saha52 2014 2011 Cross-sectional India Middle Yes 100 Self-report questionnaire 370 58.4 33.5 (8.5) 33.0 87.6 Adjusted
Shigdel53 2014 2012 Cross-sectional Nepal Low Yes 95 Self-report questionnaire 316 64.6 ND 22.5 86.7 Adjusted

ART: antiretroviral therapy; ND: not determined; RCT: randomized controlled trial.

a Countries were categorized as low-, middle- or high-income, as defined by the World Bank for 2014.22

b Participants’ ages are given as a range or as a mean (standard deviation).

c Whether or not the measure of the association between employment status and adherence to antiretroviral treatment was adjusted for major confounding variables.

Twenty-seven studies, involving 8008 patients (92%), included participants who were already receiving ART. Twenty-three studies, involving 8171 patients (94%), defined adherence as receiving 95% or more of prescribed doses in a given period. In addition, 22 studies, involving 7755 patients (89%), used self-report questionnaires to assess adherence, whereas three used pharmacy refill data, one used the pill count, one used the blood drug concentration and one used a combination of methods. The median sample size was 300 participants (range: 46–1016). When reported, the percentage of males ranged from 21.3% to 88.0% between studies. The median percentage of unemployed participants was 41% (range: 16–79%) and the percentage of participants who adhered to ART ranged from 14.9% to 93.1%.

Risk of bias

The results of our assessment of the risk of bias in all studies included in the meta-analysis are shown in Fig. 2. The risk of bias in the selection of participants was low in all studies. However, the risk of selection bias due to inadequate confirmation or consideration of confounding variables was low in 16 studies but high in 12: 16 studies adjusted for major confounding variables during the analysis phase, whereas the remaining 12 reported unadjusted associations between employment status and adherence to ART. The risk of detection bias due to inadequate outcome assessment was low in the six studies that assessed adherence using objective measures and high in the remaining 22, which used self-report questionnaires. The risk of attrition bias due to inadequate outcome data handling was low in 24 studies, unclear in two and high in the two studies in which more than 20% of patients were lost to follow-up. The risk of selective reporting bias was low in all studies.

Fig. 2.

Fig. 2

Risk of bias in studies in the meta-analysis of the association between employment status and adherence to antiretroviral therapy, 14 countries, 1996–2014

Association between adherence and employment

The strength of the association between being employed and adhering to ART reported in each study in low-, middle- and high-incomes countries is shown in Fig. 3, Fig. 4 and Fig. 5, respectively. The figures also give pooled estimates for the association in each income group. The pooled estimate for the OR in all countries was 1.27 (95% confidence interval, CI: 1.04–1.55). There was evidence of substantial statistical heterogeneity between the results of all studies (I2: 77%). The leave-one-study-out sensitivity analysis showed that no single study had an undue influence on the pooled estimate.

Fig. 3.

Association between being employed and adhering to antiretroviral therapy in studies from low-income countries, 2009–2014

CI: confidence interval; OR: odds ratio.

Fig. 3

Fig. 4.

Association between being employed and adhering to antiretroviral therapy in studies from middle-income countries, 2004–2014

CI: confidence interval; OR: odds ratio.

Fig. 4

Fig. 5.

Association between being employed and adhering to antiretroviral therapy in studies from high-income countries, 1996–2013

CI: confidence interval; OR: odds ratio.

Fig. 5

Associations in subgroups

The magnitude and strength of the association between being employed and adhering to ART varied by country income group: it was highest in the seven studies from low-income countries, for which the pooled estimate for the OR was 1.85 (95% CI: 1.58–2.18). In the 11 studies from middle-income countries, the association was not significant as the pooled estimate for the OR was 0.94 (95% CI: 0.62–1.42). However, the association was significant in the 10 studies from high-income countries: the pooled estimate for the OR was 1.33 (95% CI: 1.02–1.74). In addition, the strength of the association was significantly higher in prospective cohort studies than in cross-sectional studies: the pooled OR was 2.05 (95% CI: 1.50–2.81) and 1.17 (95% CI: 0.95–1.44) for the two study types, respectively (P-value for interaction = 0.003). The association was significant in studies that used an adherence threshold less than 100% (OR: 1.59, 95% CI: 1.35–1.87) but not in those that used a threshold of 100% (OR: 0.90, 95% CI: 0.64–1.26; P-value for interaction = 0.003). We also found that the type of adherence measure used did not significantly influence the association between being employed and adhering to ART: the pooled OR was 1.21 (95% CI: 0.98–1.51) for studies that used self-report questionnaires compared with 1.67 (95% CI: 1.09–2.56) for those that used other measures (P-value for interaction = 0.19; Table 2).

Table 2. Association between being employed and adhering to antiretroviral therapy, by subgroup, 14 countries, 1996–2014.

Subgroup No. of studies Pooled association
Subgroup heterogeneity,a P
OR (95% CI) I2, %b
Income groupc 0.003
Low 7 1.85 (1.58–2.18) 0 NA
Middle 11 0.94 (0.62–1.42) 75 NA
High 10 1.33 (1.02–1.74) 56 NA
Study design 0.003
Cross-sectional 24 1.17 (0.95–1.44) 77 NA
Prospective cohort 4 2.05 (1.50–2.81) 0 NA
Adherence threshold 0.003
< 100% 20 1.59 (1.35–1.87) 17 NA
100% 8 0.90 (0.64–1.26) 78 NA
Adherence measure 0.19
Self-report questionnaire 22 1.21 (0.98–1.51) 81 NA
Other 6 1.67 (1.09–2.56) 0 NA

ART: antiretroviral therapy; CI: confidence interval; NA: not applicable; OR: odds ratio.

a Subgroups were compared using the χ2 test.

b I2 indicates the heterogeneity between the results of the studies in the subgroup.

c Countries were categorized as low-, middle- or high-income, as defined by the World Bank for 2014.22

Period and study size effects

The association between being employed and adhering to ART was observed to change over time, i.e. there was a period effect. The magnitude of the association declined with the publication year – the pooled OR for studies published between 2011 and 2013 was lower than that for those reported before 2011 (Fig. 6). Similarly, we found that study size also significantly influenced the association, which was more pronounced in those with small samples (Fig. 7). For each additional 100 study participants, the magnitude of the association decreased by 9% (P = 0.001).

Fig. 6.

Association between being employed and adhering to antiretroviral therapy, by study publication year, 14 countries, 1996–2012

a The odds ratio represents the odds that an employed patient would adhere to treatment compared with an unemployed patient.

b The area of each circle is in inverse proportion to the precision of the odds ratio.

Fig. 6

Fig. 7.

Association between being employed and adhering to antiretroviral therapy, by study sample size, 14 countries, 1996–2012

a The odds ratio represents the odds that an employed patient would adhere to treatment compared with an unemployed patient.

b The area of each circle is in inverse proportion to the precision of the odds ratio.

Fig. 7

Discussion

Overall, we found that patients with HIV infections who were employed were 27% more likely to adhere to ART than those who were unemployed. This is in agreement with the results of our previous meta-analysis19 and of others5 who reported that one of the barriers to ART adherence in both developed and developing countries was financial constraints, which may be considered a proxy for unemployment. One difference from our previous report is that the magnitude and strength of the association between being employed and adhering to ART were highest in studies from low-income countries. Hence, unemployment may have a greater effect on ART adherence in these settings.

It is possible that employment facilitates adherence to HIV treatment because it is associated with, for example, increased social support, better structuring of time and improved psychosocial well-being – these associations have all been documented in the general population.54,55 A review of 16 longitudinal studies showed evidence that unemployment has a negative effect on mental health56 and other studies have found that depression and impaired psychosocial well-being are associated with poor ART adherence.57 Employment may also promote increased material well-being, for example, by improving food security and housing quality and by reducing poverty – all three are known to be associated with adherence to HIV treatment.5,58,59 In addition, employment may promote adherence to ART by improving access to medical services through employer-sponsored health programmes that specifically encourage HIV treatment and by having a positive effect on physical and mental health-related quality of life.5,60 Conversely, the association may, in part, be due to the positive effect of ART on an individual’s ability to find and retain employment, i.e. reverse causation. For example, it was found that HIV-infected workers who were not receiving ART were almost twice as likely to report being unable to work in the previous week than those who recently began and were adherent to ART.61 Another study on more than 2000 HIV-infected adults enrolled in an ART programme estimated that four years after ART initiation, employment had returned to about 90% of the rates observed in the same patients three to five years before ART initiation.62 Further prospective, longitudinal, cohort studies of employment status, ART adherence and other factors could help tease out the relative effects of ART adherence and employment on each other.

Despite mutual reinforcement between employment and ART adherence, it can be difficult for individuals to maintain adherence. For example, it was found that the main reasons for the discontinuation of antiretroviral therapy in the workplace in South Africa were: individuals being uncertain about their own HIV status and about the value of ART; poor relationships between patients and health-care providers; and discrimination in the workplace.63 Furthermore, workplace participants also felt that the follow-up visits required by ART clinics created problems for them with their employers. The most frequently cited reason for treatment discontinuation among these individuals was harassment and discrimination by line managers who refused to grant time off from work for clinic attendance. For public sector employees, the main reasons included relocation away from HIV care providers and having insufficient money for transportation to clinical facilities.63 These findings suggest there is a considerable need to promote awareness of the importance of ART adherence and to develop employment arrangements among both employers and employees that encourage adherence.

Adherence, employment and gender

Unexpectedly, we found no evidence that gender had a significant influence on the association between employment status and ART adherence. Several factors could account for this finding. There is a gender gap in access to antiretroviral therapy. In most areas of the world, and especially in settings with a high burden of HIV infection, women are more likely than men to access both ART and supportive HIV services, such as targeted counselling and programmes for the prevention of mother-to-child HIV transmission offered by antenatal services.64 Conversely, men may have better access to employment than women and employment could facilitate increased adherence among men.65,66

Period effect

Our finding that the association between being employed and adhering to ART declined over time may reflect changes in ART regimens, such as the trend towards once-daily dosing, reductions in the pill burden and side-effects,67,68 better access to treatment for both employed and unemployed individuals or a drop in the cost of HIV medications.1 Further research is needed to determine whether this finding is specific to the studies we included or whether it signifies a true decrease in the association over time.

Limitations

The findings of this meta-analysis should be interpreted with caution. The observational nature of the data – 85% of the studies included had a cross-sectional design – limited our ability to draw causal inferences and our findings may be affected by reverse causation bias or by other unknown confounding factors that were not adjusted for. Additionally, we found significant heterogeneity across the studies, which suggests that a substantial percentage of the variability in effect estimates was due to heterogeneity rather than to sampling errors, i.e. to chance. Much of the heterogeneity observed may be explained by differences in the adherence threshold, study sample size and study design. Nevertheless, even when there is substantial heterogeneity, meta-analysis is regarded as preferable for data synthesis to qualitative or narrative interpretation since these approaches can lead to misleading conclusions that should not be generalized beyond the scope of the analysis.69 The ability to draw conclusions from quantitative data is an important feature of meta-analyses and its absence is one reason for avoiding narrative interpretations without a data synthesis. It is worth noting that the heterogeneity observed in the current study appears to be the norm rather than the exception in meta-analyses of ART adherence.14,15 An additional potential limitation is that none of the studies included compared treatment adherence across different types of occupation. Further, we found evidence for a small study effect and it is possible that the observed magnitude of the association between being employed and adhering to ART could have been inflated by studies with small sample sizes.

Despite these limitations, our study had important strengths. We conducted comprehensive searches of databases to ensure that all relevant, published studies were identified. We also carried out meta-regression analyses to determine whether any particular study-level factor explained the results or could account for the observed variations between studies. In performing this comprehensive and robust review of the existing literature, we identified gaps in the current literature on determinants of ART adherence.

Areas for future research

Our review was unable to address all pertinent questions on the association between employment status and optimal adherence to HIV treatment. Our study findings indicate a need for a range of further research: (i) to investigate mechanisms by which employment and ART adherence affect each other; (ii) to determine how different types of employment can differentially influence ART adherence; (iii) to study the association and interaction between employment and adherence to medications for chronic comorbid diseases, e.g. hypertension, diabetes and asthma in HIV-infected or uninfected individuals; (iv) to conduct interventional studies of how employment-creation programmes can positively influence HIV treatment adherence, disease progression and quality of life; and (v) to carry out cost-benefit and cost-effectiveness analyses of selected employment-creation interventions for people living with HIV.

Acknowledgements

We thank Meg Doherty, Eyerusalem Kebede Negussie and Caroline Connor.

Funding:

Funding was provided by the HIV/AIDS and the World of Work Branch (ILOAIDS) of the International Labour Organization.

Competing interests:

None declared.

References

  • 1.Global update on HIV treatment 2013: results, impact and opportunities. Geneva; World Health Organization; 2013. Available from: http://www.unaids.org/en/media/unaids/contentassets/documents/unaidspublication/2013/20130630_treatment_report_summary_en.pdf [cited 2014 Oct 17]. [Google Scholar]
  • 2.Mills EJ, Nachega JB, Buchan I, Orbinski J, Attaran A, Singh S, et al. Adherence to antiretroviral therapy in sub-Saharan Africa and North America: a meta-analysis. JAMA. 2006;296(6):679–90. 10.1001/jama.296.6.679 [DOI] [PubMed] [Google Scholar]
  • 3.Ortego C, Huedo-Medina TB, Llorca J, Sevilla L, Santos P, Rodríguez E, et al. Adherence to highly active antiretroviral therapy (HAART): a meta-analysis. AIDS Behav. 2011;15(7):1381–96. 10.1007/s10461-011-9942-x [DOI] [PubMed] [Google Scholar]
  • 4.Juday T, Gupta S, Grimm K, Wagner S, Kim E. Factors associated with complete adherence to HIV combination antiretroviral therapy. HIV Clin Trials. 2011;12(2):71–8. 10.1310/hct1202-71 [DOI] [PubMed] [Google Scholar]
  • 5.Mills EJ, Nachega JB, Bangsberg DR, Singh S, Rachlis B, Wu P, et al. Adherence to HAART: a systematic review of developed and developing nation patient-reported barriers and facilitators. PLoS Med. 2006;3(11):e438. 10.1371/journal.pmed.0030438 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6.Bangsberg DR, Perry S, Charlebois ED, Clark RA, Roberston M, Zolopa AR, et al. Non-adherence to highly active antiretroviral therapy predicts progression to AIDS. AIDS. 2001;15(9):1181–3. 10.1097/00002030-200106150-00015 [DOI] [PubMed] [Google Scholar]
  • 7.Clavel F, Hance AJ. HIV drug resistance. N Engl J Med. 2004;350(10):1023–35. 10.1056/NEJMra025195 [DOI] [PubMed] [Google Scholar]
  • 8.Nachega JB, Hislop M, Dowdy DW, Chaisson RE, Regensberg L, Maartens G. Adherence to nonnucleoside reverse transcriptase inhibitor-based HIV therapy and virologic outcomes. Ann Intern Med. 2007;146(8):564–73. 10.7326/0003-4819-146-8-200704170-00007 [DOI] [PubMed] [Google Scholar]
  • 9.Nachega JB, Hislop M, Dowdy DW, Lo M, Omer SB, Regensberg L, et al. Adherence to highly active antiretroviral therapy assessed by pharmacy claims predicts survival in HIV-infected South African adults. J Acquir Immune Defic Syndr. 2006;43(1):78–84. 10.1097/01.qai.0000225015.43266.46 [DOI] [PubMed] [Google Scholar]
  • 10.Baggaley RF, White RG, Hollingsworth TD, Boily MC. Heterosexual HIV-1 infectiousness and antiretroviral use: systematic review of prospective studies of discordant couples. Epidemiology. 2013;24(1):110–21. 10.1097/EDE.0b013e318276cad7 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 11.Cohen MS, Chen YQ, McCauley M, Gamble T, Hosseinipour MC, Kumarasamy N, et al. ; HPTN 052 Study Team. Prevention of HIV-1 infection with early antiretroviral therapy. N Engl J Med. 2011;365(6):493–505. 10.1056/NEJMoa1105243 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12.Nachega JB, Leisegang R, Bishai D, Nguyen H, Hislop M, Cleary S, et al. Association of antiretroviral therapy adherence and health care costs. Ann Intern Med. 2010;152(1):18–25. 10.7326/0003-4819-152-1-201001050-00006 [DOI] [PubMed] [Google Scholar]
  • 13.Liu GG, Guo JJ, Smith SR. Economic costs to business of the HIV/AIDS epidemic. Pharmacoeconomics. 2004;22(18):1181–94. 10.2165/00019053-200422180-00003 [DOI] [PubMed] [Google Scholar]
  • 14.Falagas ME, Zarkadoulia EA, Pliatsika PA, Panos G. Socioeconomic status (SES) as a determinant of adherence to treatment in HIV infected patients: a systematic review of the literature. Retrovirology. 2008;5(1):13. 10.1186/1742-4690-5-13 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 15.Peltzer K, Pengpid S. Socioeconomic factors in adherence to HIV therapy in low- and middle-income countries. J Health Popul Nutr. 2013;31(2):150–70. 10.3329/jhpn.v31i2.16379 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 16.Atkinson MJ, Petrozzino JJ. An evidence-based review of treatment-related determinants of patients’ nonadherence to HIV medications. AIDS Patient Care STDS. 2009;23(11):903–14. 10.1089/apc.2009.0024 [DOI] [PubMed] [Google Scholar]
  • 17.Arrivillaga M, Ross M, Useche B, Alzate ML, Correa D. Social position, gender role, and treatment adherence among Colombian women living with HIV/AIDS: social determinants of health approach. Rev Panam Salud Publica. 2009;26(6):502–10. 10.1590/S1020-49892009001200005 [DOI] [PubMed] [Google Scholar]
  • 18.Berg KM, Demas PA, Howard AA, Schoenbaum EE, Gourevitch MN, Arnsten JH. Gender differences in factors associated with adherence to antiretroviral therapy. J Gen Intern Med. 2004;19(11):1111–7. 10.1111/j.1525-1497.2004.30445.x [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19.Nachega JB, Uthman OA, Mills EJ, Peltzer K, Amekudzi K, Ouedraogo A. The impact of employment on HIV treatment adherence. Geneva: International Labour Organization; 2013. Available from: http://www.ilo.org/wcmsp5/groups/public/---ed_protect/---protrav/---ilo_aids/documents/publication/wcms_230625.pdf [cited 2014 Sep 21]. [Google Scholar]
  • 20.Moher D, Liberati A, Tetzlaff J, Altman DG; PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med. 2009;6(7):e1000097. 10.1371/journal.pmed.1000097 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21.Kim SY, Park JE, Lee YJ, Seo HJ, Sheen SS, Hahn S, et al. Testing a tool for assessing the risk of bias for nonrandomized studies showed moderate reliability and promising validity. J Clin Epidemiol. 2013;66(4):408–14. 10.1016/j.jclinepi.2012.09.016 [DOI] [PubMed] [Google Scholar]
  • 22.Country and Lending Groups [Internet]. Washington: World Bank; 2014. Available from: http://data.worldbank.org/about/country-and-lending-groups [cited 2014 Sep 20].
  • 23.Current international recommendations on labour statistics. Geneva: International Labour Office; 2000. [Google Scholar]
  • 24.Galobardes B, Shaw M, Lawlor DA, Lynch JW, Davey Smith G. Indicators of socioeconomic position (part 1). J Epidemiol Community Health. 2006;60(1):7–12. 10.1136/jech.2004.023531 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 25.DerSimonian R, Laird N. Meta-analysis in clinical trials. Control Clin Trials. 1986;7(3):177–88. 10.1016/0197-2456(86)90046-2 [DOI] [PubMed] [Google Scholar]
  • 26.Normand SL. Meta-analysis: formulating, evaluating, combining, and reporting. Stat Med. 1999;18(3):321–59. [DOI] [PubMed] [Google Scholar]
  • 27.Singh N, Squier C, Sivek C, Wagener M, Nguyen MH, Yu VL. Determinants of compliance with antiretroviral therapy in patients with human immunodeficiency virus: prospective assessment with implications for enhancing compliance. AIDS Care. 1996;8(3):261–9. 10.1080/09540129650125696 [DOI] [PubMed] [Google Scholar]
  • 28.Singh N, Berman SM, Swindells S, Justis JC, Mohr JA, Squier C, et al. Adherence of human immunodeficiency virus-infected patients to antiretroviral therapy. Clin Infect Dis. 1999;29(4):824–30. 10.1086/520443 [DOI] [PubMed] [Google Scholar]
  • 29.Duong M, Piroth L, Grappin M, Forte F, Peytavin G, Buisson M, et al. Evaluation of the Patient Medication Adherence Questionnaire as a tool for self-reported adherence assessment in HIV-infected patients on antiretroviral regimens. HIV Clin Trials. 2001;2(2):128–35. 10.1310/M3JR-G390-LXCM-F62G [DOI] [PubMed] [Google Scholar]
  • 30.Ickovics JR, Cameron A, Zackin R, Bassett R, Chesney M, Johnson VA, et al. ; Adult AIDS Clinical Trials Group 370 Protocol Team. Consequences and determinants of adherence to antiretroviral medication: results from Adult AIDS Clinical Trials Group protocol 370. Antivir Ther. 2002;7(3):185–93. [DOI] [PubMed] [Google Scholar]
  • 31.Nachega JB, Stein DM, Lehman DA, Hlatshwayo D, Mothopeng R, Chaisson RE, et al. Adherence to antiretroviral therapy in HIV-infected adults in Soweto, South Africa. AIDS Res Hum Retroviruses. 2004;20(10):1053–6. 10.1089/aid.2004.20.1053 [DOI] [PubMed] [Google Scholar]
  • 32.Beyene KA, Gedif T, Gebre-Mariam T, Engidawork E. Highly active antiretroviral therapy adherence and its determinants in selected hospitals from south and central Ethiopia. Pharmacoepidemiol Drug Saf. 2009;18(11):1007–15. 10.1002/pds.1814 [DOI] [PubMed] [Google Scholar]
  • 33.Duggan JM, Locher A, Fink B, Okonta C, Chakraborty J. Adherence to antiretroviral therapy: a survey of factors associated with medication usage. AIDS Care. 2009;21(9):1141–7. 10.1080/09540120902730039 [DOI] [PubMed] [Google Scholar]
  • 34.Nakimuli-Mpungu E, Mutamba B, Othengo M, Musisi S. Psychological distress and adherence to highly active anti-retroviral therapy (HAART) in Uganda: a pilot study. Afr Health Sci. 2009;9Suppl 1:S2–7. [PMC free article] [PubMed] [Google Scholar]
  • 35.Campos LN, Guimarães MD, Remien RH. Anxiety and depression symptoms as risk factors for non-adherence to antiretroviral therapy in Brazil. AIDS Behav. 2010;14(2):289–99. 10.1007/s10461-008-9435-8 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 36.Giday A, Shiferaw W. Factors affecting adherence of antiretroviral treatment among AIDS patients in an Ethiopian tertiary university teaching hospital. Ethiop Med J. 2010;48(3):187–94. [PubMed] [Google Scholar]
  • 37.Kunutsor S, Walley J, Katabira E, Muchuro S, Balidawa H, Namagala E, et al. Clinic attendance for medication refills and medication adherence amongst an antiretroviral treatment cohort in Uganda: a prospective study. Aids Res Treat. 2010;2010:872396. 10.1155/2010/872396 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 38.Lal V, Kant S, Dewan R, Rai SK, Biswas A. A two-site hospital-based study on factors associated with nonadherence to highly active antiretroviral therapy. Indian J Public Health. 2010;54(4):179–83. 10.4103/0019-557X.77256 [DOI] [PubMed] [Google Scholar]
  • 39.Li L, Lee SJ, Wen Y, Lin C, Wan D, Jiraphongsa C. Antiretroviral therapy adherence among patients living with HIV/AIDS in Thailand. Nurs Health Sci. 2010;12(2):212–20. 10.1111/j.1442-2018.2010.00521.x [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 40.Peltzer K, Friend-du Preez N, Ramlagan S, Anderson J. Antiretroviral treatment adherence among HIV patients in KwaZulu-Natal, South Africa. BMC Public Health. 2010;10(1):111. 10.1186/1471-2458-10-111 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 41.Sherr L, Lampe FC, Clucas C, Johnson M, Fisher M, Leake Date H, et al. Self-reported non-adherence to ART and virological outcome in a multiclinic UK study. AIDS Care. 2010;22(8):939–45. 10.1080/09540121.2010.482126 [DOI] [PubMed] [Google Scholar]
  • 42.Venkatesh KK, Srikrishnan AK, Mayer KH, Kumarasamy N, Raminani S, Thamburaj E, et al. Predictors of nonadherence to highly active antiretroviral therapy among HIV-infected South Indians in clinical care: implications for developing adherence interventions in resource-limited settings. AIDS Patient Care STDS. 2010;24(12):795–803. 10.1089/apc.2010.0153 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 43.Harris J, Pillinger M, Fromstein D, Gomez B, Garris I, Kanetsky PA, et al. Risk factors for medication non-adherence in an HIV infected population in the Dominican Republic. AIDS Behav. 2011;15(7):1410–5. 10.1007/s10461-010-9781-1 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 44.Kyser M, Buchacz K, Bush TJ, Conley LJ, Hammer J, Henry K, et al. Factors associated with non-adherence to antiretroviral therapy in the SUN study. AIDS Care. 2011;23(5):601–11. 10.1080/09540121.2010.525603 [DOI] [PubMed] [Google Scholar]
  • 45.Wakibi SN, Ng’ang’a ZW, Mbugua GG. Factors associated with non-adherence to highly active antiretroviral therapy in Nairobi, Kenya. AIDS Res Ther. 2011;8(1):43. 10.1186/1742-6405-8-43 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 46.King RM, Vidrine DJ, Danysh HE, Fletcher FE, McCurdy S, Arduino RC, et al. Factors associated with nonadherence to antiretroviral therapy in HIV-positive smokers. AIDS Patient Care STDS. 2012;26(8):479–85. 10.1089/apc.2012.0070 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 47.Kitshoff C, Campbell L, Naidoo SS. The association between depression and adherence to antiretroviral therapy in HIV-positive patients, KwaZulu-Natal, South Africa. S Afr Fam Pract. 2012;54(2):145–50 10.1080/20786204.2012.10874194 [DOI] [Google Scholar]
  • 48.Berhe N, Tegabu D, Alemayehu M. Effect of nutritional factors on adherence to antiretroviral therapy among HIV-infected adults: a case control study in Northern Ethiopia. BMC Infect Dis. 2013;13(1):233. 10.1186/1471-2334-13-233 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 49.Okoronkwo I, Okeke U, Chinweuba A, Iheanacho P. Nonadherence factors and sociodemographic characteristics of HIV-infected adults receiving antiretroviral therapy in Nnamdi Azikiwe University Teaching Hospital, Nnewi, Nigeria. ISRN AIDS. 2013;2013:843794. 10.1155/2013/843794 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 50.Vissman AT, Young AM, Wilkin AM, Rhodes SD. Correlates of HAART adherence among immigrant Latinos in the Southeastern United States. AIDS Care. 2013;25(3):356–63. [DOI] [PubMed] [Google Scholar]
  • 51.Tran BX, Nguyen LT, Nguyen NH, Hoang QV, Hwang J. Determinants of antiretroviral treatment adherence among HIV/AIDS patients: a multisite study. Glob Health Action. 2013;6(0):19570. 10.3402/gha.v6i0.19570 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 52.Saha R, Saha I, Sarkar AP, Das DK, Misra R, Bhattacharya K, et al. Adherence to highly active antiretroviral therapy in a tertiary care hospital in West Bengal, India. Singapore Med J. 2014;55(2):92–8. 10.11622/smedj.2014021 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 53.Shigdel R, Klouman E, Bhandari A, Ahmed LA. Factors associated with adherence to antiretroviral therapy in HIV-infected patients in Kathmandu District, Nepal. HIV AIDS (Auckl). 2014;6:109–16. 10.2147/HIV.S55816 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 54.Johada M. Work, employment, and unemployment: values, theories, and approaches in social research. Am Psychol. 1981;36(2):184–91 10.1037/0003-066X.36.2.184 [DOI] [Google Scholar]
  • 55.Sousa-Ribeiro M, Sverke M, Coimbra L. Perceived quality of the psychosocial environment and well-being in employed and unemployed older adults: the importance of latent benefits and environmental vitamins. Econ Ind Democracy. Epub 2014 Jul 26. 10.1177/0143831X13491840 [DOI] [Google Scholar]
  • 56.Murphy GC, Athanasou JA. The effect of unemployment on mental health. J Occup Organ Psychol. 1999;72(1):83–99 10.1348/096317999166518 [DOI] [Google Scholar]
  • 57.Gonzalez JS, Batchelder AW, Psaros C, Safren SA. Depression and HIV/AIDS treatment nonadherence: a review and meta-analysis. J Acquir Immune Defic Syndr. 2011;58(2):181–7. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 58.Milloy MJ, Kerr T, Bangsberg DR, Buxton J, Parashar S, Guillemi S, et al. Homelessness as a structural barrier to effective antiretroviral therapy among HIV-seropositive illicit drug users in a Canadian setting. AIDS Patient Care STDS. 2012;26(1):60–7. 10.1089/apc.2011.0169 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 59.Young S, Wheeler AC, McCoy SI, Weiser SD. A review of the role of food insecurity in adherence to care and treatment among adult and pediatric populations living with HIV and AIDS. AIDS Behav. 2014;18(S5) Suppl 5:505–15. 10.1007/s10461-013-0547-4 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 60.Rueda S, Raboud J, Mustard C, Bayoumi A, Lavis JN, Rourke SB. Employment status is associated with both physical and mental health quality of life in people living with HIV. AIDS Care. 2011;23(4):435–43. 10.1080/09540121.2010.507952 [DOI] [PubMed] [Google Scholar]
  • 61.Rosen S, Ketlhapile M, Sanne I, Desilva MB. Differences in normal activities, job performance and symptom prevalence between patients not yet on antiretroviral therapy and patients initiating therapy in South Africa. AIDS. 2008;22Suppl 1:S131–9. 10.1097/01.aids.0000327634.92844.91 [DOI] [PubMed] [Google Scholar]
  • 62.Bor J, Tanser F, Newell ML, Bärnighausen T. In a study of a population cohort in South Africa, HIV patients on antiretrovirals had nearly full recovery of employment. Health Aff (Millwood). 2012;31(7):1459–69. 10.1377/hlthaff.2012.0407 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 63.Dahab M, Kielmann K, Charalambous S, Karstaedt AS, Hamilton R, La Grange L, et al. Contrasting reasons for discontinuation of antiretroviral therapy in workplace and public-sector HIV programs in South Africa. AIDS Patient Care STDS. 2011;25(1):53–9. 10.1089/apc.2010.0140 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 64.Braitstein P, Boulle A, Nash D, Brinkhof MW, Dabis F, Laurent C, et al. ; Antiretroviral Therapy in Lower Income Countries (ART-LINC) study group. Gender and the use of antiretroviral treatment in resource-constrained settings: findings from a multicenter collaboration. J Womens Health (Larchmt). 2008;17(1):47–55. 10.1089/jwh.2007.0353 [DOI] [PubMed] [Google Scholar]
  • 65.Anema A, Vogenthaler N, Frongillo EA, Kadiyala S, Weiser SD. Food insecurity and HIV/AIDS: current knowledge, gaps, and research priorities. Curr HIV/AIDS Rep. 2009;6(4):224–31. 10.1007/s11904-009-0030-z [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 66.Puskas CM, Forrest JI, Parashar S, Salters KA, Cescon AM, Kaida A, et al. Women and vulnerability to HAART non-adherence: a literature review of treatment adherence by gender from 2000 to 2011. Curr HIV/AIDS Rep. 2011;8(4):277–87. 10.1007/s11904-011-0098-0 [DOI] [PubMed] [Google Scholar]
  • 67.Nachega JB, Mugavero MJ, Zeier M, Vitória M, Gallant JE. Treatment simplification in HIV-infected adults as a strategy to prevent toxicity, improve adherence, quality of life and decrease healthcare costs. Patient Prefer Adherence. 2011;5:357–67. 10.2147/PPA.S22771 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 68.Nachega JB, Parienti JJ, Uthman OA, Gross R, Dowdy DW, Sax PE, et al. Lower pill burden and once-daily antiretroviral treatment regimens for HIV infection: a meta-analysis of randomized controlled trials. Clin Infect Dis. 2014;58(9):1297–307. 10.1093/cid/ciu046 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 69.Ioannidis JP, Patsopoulos NA, Rothstein HR. Reasons or excuses for avoiding meta-analysis in forest plots. BMJ. 2008;336(7658):1413–5. 10.1136/bmj.a117 [DOI] [PMC free article] [PubMed] [Google Scholar]

Articles from Bulletin of the World Health Organization are provided here courtesy of World Health Organization

RESOURCES