Abstract
Objective:
In 2015, the Lebanese Ministry of Public Health launched the 2015–2020 National Mental Health Strategy in an effort to integrate Mental Health into Primary Healthcare centers. One of the key objectives of the strategy was to increase the detection, assessment, and management of mental disorders including depression. In addition to diagnosis and patient education, a successful management of depression requires that patients systematically follow-up with their healthcare provider to ensure that they are on the optimal path to recovery. This study evaluates the impact of a visual self-assessment card to increase patients’ attendance to a follow-up appointment with their healthcare provider.
Methods:
A clustered randomized controlled trial was implemented at a Primary Healthcare center located in South Lebanon. The final sample consisted of 405 patients (209 treated and 196 untreated patients) who were diagnosed with mild-to-severe depression using the Patient Health Questionnaire (PHQ-9 > 4). Patients were asked to attend a follow-up appointment with their health provider 2–3 weeks following the initial assessment. Treated patients were provided with a visual self-assessment card that tracked their daily mood changes and reminded them of their follow-up appointment.
Results:
Overall, the results provide evidence supporting the use of the self-assessment card to remind mental health patients of their follow-up appointments. Patients who received the card were 9 percentage points more likely to attend their follow-up appointment (p = 0.05), with new patients (vs existing patients) exhibiting a 15 percentage points increase (p < 0.05).
Conclusion:
Efforts focused on increasing compliance with follow-up appointments should be at the cornerstone of mental health integration approaches. Not only do our findings suggest a great potential for cost-effective interventions but also highlight new avenues to optimize on the use of reminders to promote compliance.
Keywords: mental health, primary healthcare, follow-up, attendance, reminder
Introduction
Mental health disorder is one of the most common health conditions, and the largest single source of economic burden globally, with depression constituting the leading cause of the disease burden.1–3 Over the past years, there have been widespread commitments to enhance the provision of mental health services.2,4–6 In Lebanon, efforts to integrate mental health into primary healthcare were initiated via the 2015–2020 National Mental Health Strategy developed by the Ministry of Public Health (MoPH). One of the main objectives of the strategy is to improve the detection, assessment, and management of depression in primary care settings. The strategy offered a cost-effective, evidence-based, and multidisciplinary approach to managing depression through a number of steps including systematic and timely diagnosis, training of primary care staff, patient education and participation in therapy, adoption of evidence-based medical treatments, and an effective collaboration between primary care providers and mental health specialists.1
Accordingly, nurses, social workers, general practitioners, and family doctors in 75 out of 220 Primary Healthcare (PHC) centers in Lebanon were trained by the MoPH using WHO’s Mental Health Gap Action Program (mhGAP) guidelines to strengthen the capacities of the existing healthcare system. The training addressed the gaps in primary care staff’s ability to assess and manage major mental disorders4 and involved training workshops on how to screen patients for depression using the Patient Health Questionnaire (PHQ-9), a commonly used screening tool for the detection and monitoring of depression.
Globally, despite efforts to improve the management of depression, low compliance with follow-up appointments remains a major challenge. Non-attendance contributes to a vicious circle of inefficiencies characterized by staff’s wasted resources and time on outreach, longer waiting time, non-attendance, and poorer patient care. Such disruptions are particularly worrying in the mental health field where close supervision is required until patients’ outcomes are considerably improved. Increased waiting time may also prevent many patients from receiving the proper treatment. The need to ensure that patients attend their follow-up appointments is further emphasized by findings that those who failed to attend were more psychologically unwell than those who attended, and needed more intensive and emergent care services in their next visit.7,8 Nelson et al (2000) found the chances of re-hospitalization for psychiatric patients to be one in 10 for those who responded to a follow-up appointment, with a much higher chance (one in 4) for those who did not keep or were not offered an appointment.8
A meta-analysis of behavioral interventions designed to increase patients’ attendance to health appointments shows that the use of behavior change techniques such as reminders, planning prompts, and feedback about the cost incurred due to non-attendance can increase attendance rates, with phone-based messaging being at the cornerstone of such nudge-type interventions.9–16 A study by Sims et al.15 implemented at four community health clinics in London found a 25%–28% decrease in missed psychiatric appointments as a result of text-message reminders, where non-attendance to psychiatric appointments was formerly judged to be two to three times the rate of other medical specialties. Studying different variations of reminders, Hallsworth et al.14 find that messages which highlight the specific costs incurred by the health facility due to non-attendance were more effective than calling patients to remind them of the date and time of their appointment, or using social norm messages that highlight the number of people who normally attend their appointments. Planning prompts in the form of tear-off slips—enclosed within invitation letters—that invite patients to record the date and time of their health checkups significantly encouraged the uptake of health checks.16 The prompt invites self-formulated plans including implementation intentions and self-motivated action plans, consequently acting as a relatively longer-lasting reminder to attend the health check.
The adoption of reminders and memory aids in the mental health field is supported by findings that highlight forgetfulness as one of the most reported reason for non-attendance.17–19 While phone-based reminders have generally been effective in the healthcare field, they remain less feasible than other prompts for several limitations including low response rates when patients fail to answer the calls, as well as undelivered messages due to phone-related issues (phone broken, turned off, etc.), not to mention limitations to the healthcare center in terms of investing in the appropriate software, as well as assigning dedicated individuals to ensure the timely delivery of reminders. Approaches that employ phone-based reminders are also less convenient for the underprivileged sections of the society with low access to mobile phones and/or low literacy rates. Such reminders are conceived to be relatively intrusive and may raise concerns over patient privacy when calls or messages are not read by the relevant patient.
Taking into consideration all these factors, this study implemented a randomized controlled trial to test the impact of a novel behavioral intervention aimed at addressing the low compliance rate with follow-up appointments at a PHC center located in South Lebanon. Parting from phone-based reminders, we opted for a pocket-sized visual self-assessment card that tracks patients’ daily mood changes for a period of 2–3 weeks prior to their scheduled appointment. Through engaging patients in the treatment and symptom monitoring processes, the self-assessment card was hypothesized to encourage patients’ attendance to their follow-up meeting and help improve their mental health conditions. As such, this intervention complements recent efforts by the Ministry of Public Health to systemize the follow-up process and enhance the management of depression in primary care settings, objectives set to be successfully achieved through 2015–2020 National Mental Health Strategy.
Methods
Setting
The intervention was conducted at a PHC center located in South Lebanon, one of the most underprivileged zones in Lebanon. The corresponding center was chosen, in agreement with the MoPH, for the following reasons. The center operated under the supervision of the MoPH, comprised of motivated, collaborative, and well-trained staff, as well as a trained general practitioner, and was judged by MoPH to be highly compliant with mental health requirements and procedures. In fact, later during the period of the intervention, the center was designated as a referral center for mental health by the MoPH.
Recruitment
The intervention took place between August 2018 and August 2019 and targeted patients aged 18+ years who were diagnosed with mild-to severe depression using the Patient Health Questionnaire.20 Patients who met the age and diagnosis criteria were randomly assigned to a control group or a treatment group. Screening and assignment to treatment were undertaken by one social worker who was familiar with the context and the details surrounding the design setting of the study. The intervention also involved a pilot study for a period of 2 weeks prior to the start date to ensure a smooth implementation process and to equip the social worker with the required skills for the delivery of the various procedures. Blinding was not performed given the nature of the intervention which involved providing patients with a tangible self-assessment card.
PHQ
Screening using the PHQ occurred in two stages. Patients were initially screened using the PHQ-2, a tool which examines if patients have experienced little interest or pleasure in doing things, and/or have felt down, depressed, or hopeless in the past 2 weeks. Patients with positive responses to any of the two questions proceeded to complete the PHQ-9, a 9-item screening tool widely used in primary care settings. The validity of the PHQ-9 as a tool to detect major depressive disorders has been confirmed across a number of studies indicating high sensitivity and specificity properties of around 88% for scores above 10.20,21
Procedure
Both existing patients and new patients visiting for the first time during the intervention period completed the PHQ screening. The screening was administered by the social worker who filled out the questionnaire on behalf of the patients.
Patients who were diagnosed with mild-to-severe depression (PHQ-9 > 4) were asked to attend a follow-up appointment 2–3 weeks after their initial assessment. In addition, patients visiting on a treatment day were provided with a self-assessment card with the date and time of the follow-up appointment enclosed herewith, such that the card would be returned during their follow-up appointment. Instructions on how to fill out the self-assessment card were provided by the social worker.
While it is customary to target patients with depression (PHQ-9 ≥ 10), we opted for a lower threshold of 4+ (i.e., patients with depressive symptoms) for the following reasons: (1) to assess any behavioral differences in response to the treatment (likelihood to attend the follow-up appointment) between patients with mild, moderate, and severe depression; and (2) to achieve the minimum required sample size suggested by the power analysis (N = 500) within the approved intervention period (12 months). Restricting the sample to patients with PHQ ≥ 10 would have resulted in a final sample of 281 patients, significantly lower than the required sample size to detect a 10% difference between the control and treatment groups.
Upon their visit, patients’ profiles were checked by the social worker to confirm if they were already screened for depression during the intervention period. The social worker then noted the date of the visit for patients who have already been screened and retrieved the self-assessment card from treated patients. This process ensured that no patient was offered the self-assessment card twice, and alleviated risks of the same patient being observed in both the control and treatment groups.
Sample size
The study was designed to run for a period of 1 year, targeting a sample of 500 patients allocated equally between control and treatment days to detect an impact of at least 10 percentage points increase in the likelihood of a patient attending their follow-up appointment. This was determined based on a power analysis that assumes 80% power and a type I error probability of 5%, using historical number of patients’ visits, depression screening rates and follow-up adherence rates. However, only 405 patients were eligible to be included in the study by the end of the trial period.
Randomization
A clustered randomization on the visiting day level was employed such that the treatment status of patients would be determined based on the day they visited the center. Randomization assignments were generated for the days of the year between August 2018 and August 2019. Randomization ensured that all days of the week are balanced among the two groups to account for any systematic differences across days. This randomization method was chosen to reduce the risk of contamination following discussions with the social worker who took charge of screening patients, and assigning them to control and treatment groups. The trial consisted of 327 intervention days, with 163 control days and 164 treatment days. A pre-specified randomization schedule was provided to the social worker in advance. Treatment days were marked green and control days were marked red to reduce confusion between respective assignments. In order to track any errors in the randomization process, the social worker was also provided with stickers to be pasted next to each patient’s name, depending on the group they were assigned to (green for treatment and red for control).
Treatment
Treated patients were provided with a visual self-assessment card that tracks their daily mood change for a period of 2–3 weeks leading up to their scheduled appointment date (see Figure 1). The card was delivered during the patient’s first reported visit over the period of the intervention and was returned to the social worker during their follow-up appointment. As it engages patients in this self-monitoring process, the card was conceptualized as a reminder for patients to attend their follow-up appointment provided that the exact date and time of the scheduled appointment are enclosed within.
Figure 1.
Self-assessment card provided to patients in the treatment group.
In addition, through reflecting on their mood changes, patients can also learn to recognize and differentiate between various emotions as well as identify the contextual factors leading up to the emotion.22 This can initiate a desirable change in behavior by setting the stage for patients to change their perceptions, attitudes, and coping strategies. As a result, the self-assessment card can help patients better understand their mental health conditions and may well improve their depressive symptoms. This hypothesis is related to the strand of literature examining the extent to which emotional self-awareness mediates the relationship between self-monitoring techniques and depressive symptoms.22,23 The self-assessment card, furthermore, involves patients in their path toward recovery through tangible and concrete doings and serves as concrete feedback for their behaviors, emotions, and attitudes throughout the day.
Building on this, the study focuses on testing two main hypotheses relating to the self-assessment card: (1) whether treated patients are more likely to come back for their follow-up screening and (2) whether treated patients show an improvement in their mental health conditions as measured by their PHQ-9 score.
Outcome measures
The primary outcome measure is the likelihood of a patient attending their follow-up appointment. Attendance was tracked by the social worker who took charge of the screening process and overseeing procedures. For our second hypothesis, we measure the effect on patient’s psychological well-being through the change in PHQ-9 score recognized between the first visit and the follow-up visit. For additional exploratory analysis to examine the relationship between the PHQ-9 screening tool and mood changes, we employ the PHQ-9 post score as the outcome measure.
Data collection
Data were collected by the social worker using a standardized data collection sheet to ensure reliability of the intervention processes and a proper tracking of patients over the study period. The data collection sheet encompassed all 327 days of the 1-year period such that intervention days were marked by green and control days were marked by red. Constituent entries included the patient’s ID number (to preserve confidentiality), the date of the first appointment, PHQ-9 score at first appointment, treatment status (whether or not they received the self-assessment card), the date of the scheduled follow-up appointment, whether or not they attended their follow-up appointment, date of attendance to follow-up appointment, and PHQ-9 scores reported during the follow-up visit. Collected data also included sociodemographic indicators like sex, age, nationality, education, and area of residence. The patient’s PHQ-9 scores at first and second visits were later retrieved from the PHC’s excel sheet through their patient de-identified IDs to ensure anonymity. Data on daily mood changes were extracted from the self-record cards returned at the follow-up session.
Statistical analysis
Prior to the analysis pertaining to our main outcomes of interest, randomization checks were performed using test of means for continuous variables (age and PHQ-9 score), Pearson chi-square test for binary variables (gender, nationality, area of residence, and type of patient) and Kruskal–Wallis test for categorical variables (education).
The impact of the intervention on follow-up attendance (coded as a dummy variable for whether or not the patient attended their follow-up appointment) and improvement in PHQ-9 score (coded as the difference between the initial and final PHQ-9 score) was assessed using a multivariate linear probability model (LPM). The use of the LPM model was based upon recent evidence that shows the benefits of using linear probability models for binary outcomes over binary logistic models with both yielding similar results.24 Nevertheless, average marginal effects from a logistic regression were estimated and compared to the LPM model to ensure the robustness of results.
The analytical models regressed each outcome variable on a treatment dummy while controlling for a number of covariates. Covariates included initial PHQ-9 score, age, nationality (coded as a dummy variable for Lebanese vs other subgroups), basic education (coded as a dummy for patients with basic education vs other subgroups), and type of patient (coded as a dummy for new patients vs existing patients). The choice of the educational dummy was made based on randomization checks which revealed significant imbalances solely at the level of basic education. Finally, month of visit dummies were added to capture any month-related influences on attendance rates. Control variables were added gradually to test the robustness of the treatment coefficient of interest. In addition to the primary analysis, a heterogeneity analysis at the level of patient type was conducted. For this purpose, we ran the same multivariate model with an added interaction between treatment variable on one hand and patient type dummy on the other hand. All regressions were estimated using robust standard errors, namely Huber–White standard errors, to correct for potential heteroscedasticity. Furthermore, given the clustered randomized feature of the study, we computed design effect adjusted standard errors, following Lehtonen and Pahkinen25 to take account of the intracluster correlation. Statistical analysis was conducted using Stata software.
In light of the limitations of the study, the approach we took to choosing the level of significance borrows its intuition from Fisher.26 Since the study was underpowered, we report results that are in essence suggestive (p < 0.1) but not necessarily conventionally statistically significant (p < 0.05). While directional results reported in this study warrant further testing, they are worth reporting and can offer new directions for further studies looking to evaluate reminder-based interventions in mental health settings. In general, our approach has been to report all statistical information on coefficients including standard error, p-value, sample size, and confidence intervals in addition to reporting changes in the coefficients of interest with the addition of control variables.
Ethical considerations
The randomized controlled trial was approved by the Institutional Review Board at the University Saint-Joseph of Beirut (USJ).
A verbal informed consent to participate in the study was obtained by the social worker from all eligible participants before being screened for depression. Only patients who provided consent to be screened for mental health and participate in the trial were included in the study. We opted for a verbal consent because most of the participants were either illiterate or had low literacy skills, and therefore would have struggled to read and sign a written consent.
The decision to acquire verbal consent was also motivated by the sensitivity of the topic and stigma around mental health in Lebanon. It was feared that having participants sign any documents could result in an adverse reaction, thereby impacting their trusting relationship with the social worker. The approach to acquire consent from participants was approved by the Institutional Review Board.
Anonymity and confidentiality of patients were preserved by way of non-identifiable data which are already routinely recorded as part of the PHC’s mental health program.
The trial was pre-registered with the American Economic Registration (AEA).
Results
Characteristics of sample
The final sample consisted of a total of 405 patients who visited the center over 206 days (control days = 101; treatment days = 105). The difference between the initial number of intervention days and the actual number of intervention days was generally due to lack of patients diagnosed with depression. The distribution of patients into control and treatment is 196 and 209 patients, respectively. The follow-up attendance rate for the control group was documented at 29%. While this is well below rates documented elsewhere, it is not surprising given that the study took place at an underprivileged community with suggestive evidence that people with lower socio-economic status are less likely to attend.27 The majority of the sample were women (85%) with almost one-third aged between 18 and 29 years (31%) or 30 and 39 years (36%) (see Table 1). Most were non-Lebanese (77%) with 88% residing in Sour, a main district in South Lebanon where the PHC is located. In terms of depression severity, 31% were diagnosed with mild depression, 31% with moderate depression, 25% with moderately severe depression, and 13% with severe depression. Almost two-thirds of visiting patients were existing patients (65%; 57% of which without healthcare) who have attended the PHC at least once before the intervention period.
Table 1.
Characteristics of the sample.
| Characteristic | N | % |
|---|---|---|
| Sex | ||
| Female | 343 | 84.69 |
| Male | 62 | 15.31 |
| Age | ||
| 18–29 | 123 | 30.67 |
| 30–39 | 143 | 35.66 |
| 40–49 | 75 | 18.70 |
| 50–59 | 45 | 11.22 |
| >60 | 15 | 3.74 |
| Education | ||
| Illiterate | 57 | 14.11 |
| Basic | 279 | 69.06 |
| Secondary | 44 | 10.90 |
| University | 24 | 5.94 |
| Nationality | ||
| Lebanese | 94 | 23.21 |
| Non-Lebanese | 311 | 76.79 |
| Patient type | ||
| New patient | 143 | 35.31 |
| Existing with healthcare | 31 | 7.65 |
| Existing without healthcare | 231 | 57.04 |
| Depression severity (according to PHQ-9) | ||
| Mild (5–9) | 124 | 30.62 |
| Moderate (10–15) | 127 | 31.36 |
| Moderately severe (15–19) | 103 | 25.43 |
| Severe (20–27) | 51 | 12.59 |
| Governorate | ||
| Beirut | 3 | 0.81 |
| Beqaa | 2 | 0.54 |
| Mount Lebanon | 12 | 3.23 |
| Nabatiyeh | 4 | 1.08 |
| North | 20 | 5.38 |
| South | 331 | 88.98 |
PHQ: Patient Health Questionnaire.
Randomization checks
There were no significant differences between the two groups on symptom severity diagnosis as well as sociodemographic characteristics relating to age, nationality, gender, and place of residence—refer to Table 2. However, comparisons on the patient type and education levels revealed some imbalances between the control and treatment groups. The difference with respect to type of patient was statistically significant at the 10% level. With respect to education, the difference was significant at the 5% level. In specific, patients in the treated group were significantly more educated than those in the control group, with the former group comprising a larger portion of patients with basic education and a smaller portion of illiterate patients. Patients with tertiary education, however, were better represented among the control group. Taken together, the results offer a narrow window for threats over unbalanced randomization. To correct for these differences, dummies for patient type and educational levels were included in all regression specifications. Such control variables were added gradually and changes in coefficients were traced to check for the sensitivity of results to varying model specifications.
Table 2.
Randomization checks.
| Characteristic | Treated group (N = 209) | Control group (N = 196) | Difference | |||
|---|---|---|---|---|---|---|
| M | SD | M | SD | t | p-value | |
| Age | 36.88 | 13.45 | 36.16 | 11.40 | −0.57 | 0.56 |
| Depression severity (PHQ-9) | 13.13 | 5.22 | 13.00 | 5.19 | −0.25 | 0.80 |
| N | % | N | % | χ2 | p | |
| Female | 178 | 85.16 | 165 | 84.18 | 0.07 | 0.78 |
| Lebanese | 46 | 22.00 | 48 | 24.48 | 0.35 | 0.55 |
| Place of residence (South) | 158 | 87.30 | 173 | 90.57 | 1.02 | 0.31 |
| New patient | 65 | 31.10 | 78 | 39.79 | 3.35 | 0.067 |
| Education | ||||||
| Illiterate | 23 | 10.58 | 34 | 17.53 | 4.03 | 0.045 |
| Basic | 161 | 77.40 | 118 | 60.82 | ||
| Secondary | 20 | 9.62 | 24 | 11.86 | ||
| University | 5 | 2.40 | 19 | 9.79 | ||
PHQ: Patient Health Questionnaire.
Kruskal–Wallis test was used for education and chi-squared statistic with ties was reported. It is worthy to mention that while the p-value associated with X2 (with ties) was <0.05, the p-value associated with X2 (without ties) was 0.23.
Indicators of attendance
PHQ-9 initial score, age, nationality, and patient type were significantly associated with follow-up attendance (see Table 3). The multivariate model shows that younger patients and patients diagnosed with higher levels of depression at their first visit are more likely to attend the follow-up appointment compared to older patients and patients with less severe depression. Lebanese patients are considerably less likely to follow-up with their health provider relative to non-Lebanese. Consistent with previous findings, patients visiting the PHC for their first time (new patients) were considerably less likely to follow-up with their health provider relative to existing patients.19,27 We also account for the time lag between the first visit and the scheduled follow-up appointment to test the hypothesis that patients are more likely to miss the follow-up appointment the further it was from their first appointment, but it was found to be very small in magnitude and statistically insignificant. It was not possible to explore the impact of place of residence or distance away from PHC—some of the most commonly reported reasons for non-attendance19,27 due to the low variability in the data with the majority residing in South Lebanon near the corresponding PHC. As for seasonality effects, patients were more likely to attend follow-up appointments scheduled in the fall season relative to each of winter, spring, and summer.
Table 3.
Follow-up assessment for mental health patients in a randomized controlled trial.
| Dependent variable: Follow-up status | |||||||
|---|---|---|---|---|---|---|---|
| (1) | (2) | (3) | (4) | (5) | (6) | (7) | |
| Main independent variable | |||||||
| Treatment | 0.092* | 0.091* | 0.086* | 0.082* | 0.089* | 0.072 | 0.045 |
| [0.050] (0.01, 0.17) |
[0.052] (0.01, 0.17) |
[0.066] (0.007, 0.16) |
[0.078] (0.003, 0.16) |
[0.060] (0.008, 0.17) |
[0.119] (–0.006, 0.15) |
[0.327] (–0.03, 0.12) |
|
| Other associated factors | |||||||
| PHQ-9 pre score | 0.008* | 0.0078* | 0.0079* | 0.0077* | 0.008* | 0.005 | |
| (0.004) | (0.004) | (0.005) | (0.004) | (0.004) | (0.004) | ||
| Age | −0.005*** | −0.004** | −0.003** | −0.0027 | −0.003* | ||
| (0.0016) | (0.0017) | (0.0017) | (0.0017) | (0.0017) | |||
| Lebanese | −0.110** | −0.113** | −0.148*** | −0.158*** | |||
| (0.054) | (0.054) | (0.054) | (0.055) | ||||
| Basic education | −0.05 | −0.07 | −0.07 | ||||
| (0.05) | (0.05) | (0.05) | |||||
| New patient | −0.23*** | −0.17*** | |||||
| (0.046) | (0.049) | ||||||
| Month of visit dummies | Yes | ||||||
| Observations | 405 | 405 | 401 | 401 | 400 | 400 | 400 |
PHQ: Patient Health Questionnaire; CI: confidence interval.
Coefficients from the linear probability model (LPM) were reported. Similar results were obtained from maximum likelihood estimation. The table shows seven different specifications with the gradual addition of control variables (referred to as other associated factors). The last specification includes month of visit dummies representing dummies for 11 out of 12 months of the year with the remaining month left as a reference category. Significance represented through stars corresponds to unadjusted p-values. For the main independent variable, we report both the unadjusted p-values (in brackets) and the design effects adjusted confidence intervals (90% CI) (in parentheses). For the other associated factors, we report the estimates along with the standard errors (in parentheses).
p < 0.1. **p < 0.05. ***p < 0.01.
Follow-up assessment
Table 3 presents results of the analysis conducted on the primary outcome of interest, specifically the likelihood to attend the follow-up appointment. Patients who were provided with the self-assessment card were 9 percentage points more likely to attend the follow-up appointment, significant at the 10% level (p-value = 0.05). This effect varies between 9 and 7 percentage points with the gradual inclusion of control variables such as PHQ-9 pre score, age, nationality, education, and patient type. The effect is statistically significant at the 10% level in five out of seven different specifications and represents an increase of 4.5 percentage points with the inclusion of month of visit fixed effects (p-value = 0.32). The inclusion of monthly dummies might have led to a major deterioration in power (type II error) required for the measurement of the impact. The results are similarly significant in 5 out of 7 specifications with the use of design effect adjustments due to clustered randomization at the visiting day level.
Table 4 presents differences in response based upon patient’s visit history. Notably, there were significant differences between new and existing patients. The impact of the self-assessment card is mostly captured for new patients who show a 14.8 percentage point increase in the likelihood of attending the follow-up appointment with minimal impact among existing patients. The impact on new patients is highly robust and significant (p < 0.05).
Table 4.
Follow-up assessment by type of patient.
| Type of patient | Dependent variable: Follow-up status | |||||
|---|---|---|---|---|---|---|
| Estimate | SE | p-value | 95% CI | Adjusted 95% CI | ||
| New patient | (1) Excluding control variables (n = 405) | 0.177 | 0.07 | 0.009 | (0.044, 0.309) | (0.036, 0.318) |
| (2) Including control variables (n = 400) | 0.148 | 0.07 | 0.035 | (0.011, 0.285) | (0.007, 0.289) | |
| Existing patient | (1) Excluding control variables (n = 405) | 0.017 | 0.06 | 0.784 | (–0.104, 0.138) | (–0.106, 0.138) |
| (2) Including control variables (n = 400) | −0.015 | 0.06 | 0.798 | (–0.135, 0.104) | (–0.136, 0.106) | |
CI: confidence interval.
Estimates are obtained from the linear probability model where patient type dummy is interacted with treatment status dummy. Adjusted 95% CI refer to design effects adjusted confidence intervals. The table presents the results of two different regression specifications: (1) without control variables and (2) with the addition of control variables.
In addition, we examined the differences in uptake between treated and non-treated patients based on the severity of their depression. While patients with mild or moderate symptoms were more likely to attend their follow-up appointment than those with severe depression, the differences were not statistically significant. Results are available in the Supplemental Appendix.
Improvement in mental health
The analysis to evaluate improvements in PHQ-9 score is evidently restricted to those who came for the follow-up appointment (n = 137). The mean of PHQ-9 post-score was 9.96 for the control group (SD = 7.5) and 9.28 for the treatment group (SD = 6.75). Considering the change in PHQ-9, the mean was −4.298 for the control group (SD = 6.72) and −4.246 for the treatment group (SD = 6.53). While the results of the linear multivariate model show no significant differences between control and treated patients on both PHQ-9 post score and ΔPHQ-9, it is worthy to note that the wide confidence intervals resulting from the small sample size restrict our ability to draw appropriate conclusions (see Table 5).
Table 5.
Additional results pertaining to secondary outcome measure and the relation between PHQ-9 and daily mood changes.
| Secondary outcome measure (N = 137) | Estimate | SE | p-value | 95% CI |
|---|---|---|---|---|
| PHQ-9 post score | −0.20 | 1.12 | 0.86 | (−2.41, 2.01) |
| ∆PHQ-9 score | 0.12 | 1.17 | 0.91 | (−2.18, 2.44) |
| PHQ-9 and daily mood changes (N = 55) | Estimate | SE | p-value | 95% CI |
| Happy vs Neutral | −0.17 | 0.22 | 0.43 | (−0.61, 0.26) |
| Sad vs Neutral | 0.83 | 0.23 | 0.001 | (0.38, 1.29) |
| Sad vs Happy | 1.04 | 0.2 | ⩽ 0.01 | (0.63, 1.45) |
PHQ: Patient Health Questionnaire; CI: confidence interval.
Regressions control for age and nationality. For PHQ-9 post score as the outcome variable, we also control for PHQ-9 pre score. A lower PHQ-9 post score points to a decrease in depressive symptoms.
Additional results
The self-assessment card serves as a self-monitoring tool allowing patients to evaluate their overall psychological well-being for the period leading up to their scheduled appointment. On their follow-up visit, patients are asked about their eating and sleeping patterns, fatigue, and nature of thoughts over the past two weeks. As such, one would expect that the self-reported mood changes be consistent, to some degree, with the final PHQ-9 screening outcome. This last investigation examines whether there is any association between self-reported mood changes (Happy, Neutral, and Sad) and mental health state assessed through the PHQ-9 rating score. One limitation, however, is the small sample size which is essentially restricted to the treated group who came back for the follow-up appointment but also returned their self-assessment card (n = 55). Results shown in Table 5 reveal a positive relation between the reported number of Sad days (vs Happy or Neutral days) and the PHQ-9 score. That is, keeping all other factors constant, an additional day reported as Sad (vs Happy or Neutral) is associated with about a 1-point increase in the final PHQ-9 score (p < 0.01).
Discussion
This randomized trial studied the impact of a novel behavioral intervention in the form of a self-assessment card to increase the attendance of patients diagnosed with depression to their follow-up appointments. Despite being underpowered, the trial provides reasonable evidence supporting the effectiveness of the self-assessment card, with new patients responding more strongly to the intervention than existing patients.
Overall, treated patients were around 9 percentage points more likely to attend their follow-up appointment compared to control patients. The effect was closely significant at the 5% level with a p-value in the range of [0.05, 0.078] in five out of seven different specifications. Notable was the impact on new patients (compared to existing patients) who experienced an increase of 15 percentage points in their likelihood of attending the follow-up appointment (p < 0.05). While the interpretation of this result is limited by the lack of stratified randomization on the type of patient level, the impact on new patients remained highly significant after controlling for basic demographics, symptom severity and month of visit fixed effects—factors that have previously been shown to influence attendance rates.19,27–29
The lack of information on existing patients’ visit history, specifically whether they had access to a similar service in their previous visits to the PHC center is another limitation. The absence of an effect for this sub-sample of patients who may have had prior experiences with health consultations could be due to loss of hope or tiredness from consulting a health specialist, or the belief that symptoms would resolve on their own. Research looking into the reasons for non-attendance highlight wariness about seeing a health specialist, patient’s poor psychological well-being, skepticism about the value of treatment, miscommunication with therapist, among other patient-related reasons like forgetting, oversleeping, or confusing the date of the appointment.19,27
In addition to the primary results, our evaluation of the relationship between patients’ PHQ-9 post score and mood changes reveals interesting insights. Only Sad versus Neutral mood changes—but not Happy versus Neutral—significantly explained variations in PHQ-9 post scores. In reference to prospect theory’s loss aversion,30 a possible explanation is that people react differently to their losses and gains. In our context, patients may have a tendency to recall feelings of sadness more strongly than to recall feelings of happiness, which might help explain the final effect on PHQ-9 captured during the course of the questionnaire.
Finally, informal feedback from treated patients suggests that the self-assessment card was engaging and may have contributed indirectly toward improving their mental health. Most treated patients who came back for their follow-up appointment expressed their satisfaction with the delivered card. Patients’ feedback also revealed that the card embodied a tangible step in their treatment process. The social worker also noted patients’ motivation and commitment to filling out the card and returning it during their follow-up visit. While more formal feedback is essential, the degree of engagement with the self-assessment card suggests a potential for a number of benefits for patients with mental health conditions, eventually supporting the objectives of the National Mental Health Strategy and embracing increased patient involvement in the management of depression.
Strengths and limitations
To the best of our knowledge, this is the first intervention that adopts a self-assessment card as a prompt to attend the follow-up appointment. Given its easy-to-use features, this self-monitoring tool could be introduced in the mental health field as a highly feasible and cost-effective treatment employed as an alternative to phone-based reminders that are relatively more costly. The findings provide new avenues for enhancing the use of reminders in the healthcare field, therefore contributing to closing the various gaps in efficiency.
The study was nonetheless associated with several limitations. The fact that the study was implemented in one primary healthcare center with specific characteristics—being located in an underprivileged area—limits the generalizability of our results. Originally designed as a 1-year intervention, the study also failed to meet its sample-size requirements due to the low number of patients with depressive symptoms attending over the period of the intervention (this could have been due to both false negative responses (due to stigma, for example) and true negatives indicating no depressive symptoms). Nevertheless, the study provides a promising benchmark for future evaluations looking to measure the impact of a self-assessment card.
Although the self-assessment card embodied a novel feature of this study, the study hinges on the assumption that patients would keep possession of the card. This could have led us to underestimate the impact of the intervention if a large number of patients lost or misplaced their cards. Potential follow-up interventions could try to examine the impact of phone-based tracking applications on the probability of attending the follow-up appointment in similar contexts.
The lack of stratified randomization by type of patient (existing or new), because of logistical limits, represented another limitation. This has restricted the interpretation of the observed impacts, despite the robustness of results to varying model specifications. Moreover, it would have been informative to know whether existing patients have accessed the same or different service before. In addition, information on what medical care or treatment plan mental health patients received at their first visit to the PHC center was lacking which limited the interpretability of our findings.
Finally, the PHQ-9 screener was not pretested with the target population group as it was already being used as a screening tool by all trained PHCs supervised by the MoPH, noting that the version used in the trial has been validated among the Lebanese population.31
Conclusion
The findings suggest a great potential for cost-effective interventions to promote the provision of mental health services. Providing patients with a self-assessment card which reminds them of the follow-up appointment and allows them to track their daily mood changes has been found to significantly increase attendance to follow-up meetings. Efforts focused on increasing compliance with follow-up appointments should be at the cornerstone of mental health integration approaches. Such efforts will require enhancing and optimizing behavior change interventions for the purpose of addressing the several negative consequences of non-attendance—manifested most notably in terms of poor patient care and relapse in patient’s health conditions.
Supplemental Material
Supplemental material, sj-docx-1-smo-10.1177_20503121221135990 for Promoting follow-up attendance among mental health patients at a primary healthcare center in Lebanon: A randomized controlled trial by Jana Kontar, Fadi Makki and Rabih El Chammay in SAGE Open Medicine
Acknowledgments
The authors express gratitude to all primary healthcare staff including supervisors, primary care staff, and general practitioners. We are especially grateful to Dr Randa Hamadeh and Imad Haddad in the primary care department and to the social worker for help in data collection and monitoring processes. We also thank Rasha Abi Hana and Bedros Kazazian from the Ministry of Public Health for helpful comments and suggestions and Ghia El Rifae from Nudge Lebanon for help with data collection.
Footnotes
Author contributions: F.M. and A.O. contributed to the study’s design and methodology. F.M., A.O., and R.E.C. supervised the implementation of the study and provided the necessary resources and materials to conduct the study. J.K. performed the analysis and took the lead in the interpretation of results with significant input from A.O. J.K. produced the draft of the manuscript that was reviewed by F.M., A.O., and R.E.C. All authors have read and approved the manuscript.
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Ethical approval: Ethical approval for this study was obtained from the Institutional Review Board at University Saint-Joseph of Beirut (Reference No: CEHDF 1178).
Funding: The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This project was funded by Carnegie Corporation of New York. The funders did not intervene in the design of the study, data collection, data analysis, and in the interpretation of results.
Trial registration: AEA RCT Registry, AEARCTR-0004627. Registered 4 November 2019, https://doi.org/10.1257/rct.4627-1.0.
ORCID iD: Jana Kontar
https://orcid.org/0000-0001-6484-3849
Supplemental material: Supplemental material for this article is available online.
References
- 1. Ministry of Public Health. Mental health and substance use prevention, promotion, and treatment—situation analysis and strategy for Lebanon 2015-2020. Beirut, Lebanon. Ministry of Public Health, 2015. [Google Scholar]
- 2. Charara R, Forouzanfar M, Naghavi M, et al. The burden of mental disorders in the Eastern Mediterranean Region, 1990-2013. PLoS ONE 2017; 12(1): e0169575. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3. Mental Health Foundation. Fundamental facts about mental health 2016. London: Mental Health Foundation, 2016. [Google Scholar]
- 4. World Health Organization. Mental Health gap action programme: scaling up care for mental, neurological and substance use disorders. Geneva: World Health Organization, 2008. [PubMed] [Google Scholar]
- 5. World Health Organisation. Strategy for mental health and substance abuse in the Eastern Mediterranean Region 2012–2016. Geneva: World Health Organisation, 2011. [Google Scholar]
- 6. Okasha A, Karam E, Okasha T. Mental health services in the Arab world. World Psychiatry 2012; 11(1): 52–54. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7. Killaspy H, Banerjee S, King M, et al. Prospective controlled study of psychiatric out-patient non-attendance: characteristics and outcome. Br J Psychiatry 2000; 176: 160–165. [DOI] [PubMed] [Google Scholar]
- 8. Nelson E, Maruish M, Axler J. Effects of discharge planning and compliance with outpatient appointments on readmission rates. Psychiatric Services 2000; 51: 885–889. [DOI] [PubMed] [Google Scholar]
- 9. Rusius C. Improving out-patient attendance using postal appointment reminders. Psychiatric Bulletin 1995; 19: 291–292. [Google Scholar]
- 10. Branson CE, Clemmey P, Mukherjee P. Text message reminders to improve outpatient therapy attendance among adolescents: a pilot study. Psychol Serv 2013; 10(3): 298–303. [DOI] [PubMed] [Google Scholar]
- 11. Linde D, Andersen M, Mwaiselage J, et al. Text messages to increase attendance to follow-up cervical cancer screening appointments among HPV-positive Tanzanian women (Connected2Care): study protocol for a randomised controlled trial. Trials 2017; 18: 555. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12. Blank MB, Chang MY, Fox JC, et al. Case manager follow-up to failed appointments and subsequent service utilization. Community Ment Health J 1996; 32(1): 23–31. [DOI] [PubMed] [Google Scholar]
- 13. Önür ST, Uysal MA, İliaz S, et al. Does short message service increase adherence to smoking cessation clinic appointments and quitting smoking? Balkan Med J 2016; 33(5): 525–531. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14. Hallsworth M, Berry D, Sanders M, et al. Stating appointment costs in SMS reminders reduces missed hospital appointments: findings from two randomized controlled trials. PLoS ONE 2015; 10(9): e0137306. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15. Sims H, Sanghara H, Hayes D, et al. Text message reminders of appointments: a pilot intervention at four community mental health clinics in London. Psychiatric Services 2012; 63(2): 161–168. [DOI] [PubMed] [Google Scholar]
- 16. Perry C, Chhatralia K, Damesick D, et al. Behavioural insights in health care. London: The Health Foundation, 2015. [Google Scholar]
- 17. Akhter K, Dockray S, Simmons D. Exploring factors influencing non-attendance at the diabetes clinic and service improvement strategies from patients’ perspectives. Practical Diabetes 2012; 29(3): 113–116. [Google Scholar]
- 18. Sparr LF, Moffitt MC, Ward MF. Missed psychiatric appointments: who returns and who stays away. Am J Psychiatry 1993; 150(5): 801–805. [DOI] [PubMed] [Google Scholar]
- 19. Binnie J, Boden Z. Non-attendance at psychological therapy appointments. Mental Health Review 2016; 21: 231–248. [Google Scholar]
- 20. Kroenke K, Spitzer RL, Williams JB. The PHQ-9: validity of a brief depression severity measure. J Gen Intern Med 2001; 16(9): 606–613. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21. Gilbody S, Richards D, Brealey S, et al. Screening for depression in medical settings with the Patient Health Questionnaire (PHQ): a diagnostic meta-analysis. J Gen Intern Med 2007; 22(11): 1596–1602. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22. Kauer S, Reid S, Crooke A, et al. Self-monitoring using mobile phones in the early stages of adolescent depression: randomized controlled trial. J Med Internet Res 2012; 14(3): e67. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23. Vago DR, Silbersweig DA. Self-awareness, self-regulation, and self-transcendence (S-ART): a framework for understanding the neurobiological mechanisms of mindfulness. Front Hum Neurosci 2012; 6: 296. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24. Gomila R. Logistics or linear? Estimating causal effects of experimental treatments on binary outcomes using regression analysis. J Exp Psychol Gen 2021; 150(4): 700–709. [DOI] [PubMed] [Google Scholar]
- 25. Lehtonen R, Pahkinen E. Practical methods for design and analysis of complex surveys. Chichester: John Wiley, 2004. [Google Scholar]
- 26. Fisher RA. Statistical methods for research workers. In: Kotz S, Johnson NL.(eds) Breakthroughs in statistics. New York: Springer, 1992, pp. 66–70. [Google Scholar]
- 27. Mitchell A, Selmes T. Why don’t patients attend their appointments? Maintaining engagement with psychiatric services. Adv Psychiatr Treat 2007; 13: 423–434. [Google Scholar]
- 28. Johnson E, Mellor D, Bran P. Differences in dropout between diagnoses in child and adolescent mental health services. Clin Child Psychol Psychiatry 2008; 13: 515–530. [DOI] [PubMed] [Google Scholar]
- 29. Baruch G, Vrouva I, Fearon P. A follow-up study of characteristics of young people that dropout and continue psychotherapy: service implications for a clinic in the community. Child Adolesc Ment Health 2009; 14: 69–75. [Google Scholar]
- 30. Kahneman D, Tversky A. Prospect theory: an analysis of decision under risk. Econometrica 1979; 47(2): 263–292. [Google Scholar]
- 31. Sawaya H, Atoui M, Hamadeh A, et al. Adaptation and initial validation of the Patient Health Questionnaire–9 (PHQ-9) and the Generalized Anxiety Disorder–7 Questionnaire (GAD-7) in an Arabic speaking Lebanese psychiatric outpatient sample. Psychiatry Research 2016; 239: 245–252. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Supplemental material, sj-docx-1-smo-10.1177_20503121221135990 for Promoting follow-up attendance among mental health patients at a primary healthcare center in Lebanon: A randomized controlled trial by Jana Kontar, Fadi Makki and Rabih El Chammay in SAGE Open Medicine