Is Informal Practice Associated with Outcomes in Loving-Kindness and Compassion Training? Evidence from Pre-Post and Daily Diary Assessments

Abstract

We investigated whether informal meditation practice (i.e., self-reported application of meditative techniques outside a period of formal meditation) was associated with outcomes in smartphone-based loving-kindness and compassion training. Meditation-naive participants (n = 351) with clinically elevated symptoms completed measures of psychological distress, loneliness, empathy, and prosociality at baseline and following a two-week intervention. Informal practice, psychological distress, and loneliness were also assessed daily. Steeper increases in informal practice had small associations with pre-post improvements in distress (r = −.18, p = .008) and loneliness (r = −.19, p = .009) but not empathy or prosociality. Using a currently recommended approach for establishing cross-lagged effects in longitudinal data (latent curve model with structured residuals), higher current-day informal practice was associated with decreased next-day distress with a very small effect size (βs = −.06 to −.04, p = .018) but not decreased next-day loneliness. No cross-lagged associations emerged from distress or loneliness to informal practice. Findings suggest that further investigation into a potential causal role of informal practice is warranted. Future studies experimentally manipulating informal practice are needed.

Introduction

Meditation can be conceptualized as a family of practices involving training in attention and emotion regulation (Lutz et al., 2008). Loving-kindness and compassion (LKC) meditation aims to promote kind and compassionate feelings toward self and others (Galante et al., 2014). Evidence suggests LKC meditation may improve psychological symptoms, positive emotions, and prosociality (Galante et al., 2014). With advances in digital technology, meditation-based interventions (MBIs) delivered by smartphone apps are widely used (Wasil et al., 2020). Recent randomized controlled trials (RCTs) have found that app-delivered LKC interventions improve mental health (e.g., Goldberg, Imhoff-Smith, et al., 2020). Although the potential benefits of LKC interventions are increasingly being reported, the mechanisms of action underlying these effects are largely unknown. Clarifying key mechanisms in LKC interventions can help guide efforts to optimize the effectiveness of these approaches (Kazdin, 2007).

MBIs, including LKC interventions, typically encourage meditators to engage in two forms of meditation practices: formal and informal practice (Kabat-Zinn, 2013). Formal meditation practice requires meditators to engage in specific mental training techniques, such as sitting still while non-judgmentally attending to bodily sensations. Traditional MBIs often provide audio recordings of guided meditation practices to support meditators’ formal practice, especially in the early stages of training (Kabat-Zinn, 2013). In contrast, informal mediation practice occurs in the midst of daily activities and without guidance (Fredrickson et al., 2019), for example, focusing on present-moment sensations while eating or walking from one place to another. Most MBIs consider both formal and informal practice to be core treatment ingredients, although formal practice involves a particularly substantial investment for participants. For example, the manuals of Mindfulness-Based Stress Reduction (MBSR) and Mindfulness-Based Cognitive Therapy (MBCT) encourage participants to engage in formal practice at home for 45 min per day throughout the eight-week intervention (Kabat-Zinn, 2013; Segal et al., 2013). Finding time for formal practice has long been reported as a perceived barrier to engaging in MBIs (Engbretson et al., 2020; Toivonen et al., 2020). Informal practice may be more accessible, given it does not require meditators to set aside time. Consistent with this possibility, there is evidence from qualitative studies that shorter practices and mindful activities (e.g., mindful walking to work) facilitate engagement with MBIs (Banerjee et al., 2017). In addition to reducing barriers to practicing meditation, informal practice may help practitioners implement meditative strategies more readily in daily life to address real-world situations (e.g., applying attention and emotion regulation strategies in response to actual stressors). For these reasons, informal practice may be critical in increasing the accessibility and effectiveness of MBIs.

There is both correlational and longitudinal evidence linking the amount of formal practice time with benefits in MBIs. A meta-analysis of 28 MBSR and MBCT studies showed a small-to-moderate association (r = .26) between cumulative formal practice time and post-intervention improvements (Parsons et al., 2017). Recent studies have also shown associations between daily formal mindfulness practice time and better daily outcomes measured on the same day (Goldberg, Hanley, et al., 2020; Levi et al., 2021). However, studies investigating associations between formal mindfulness practice and outcomes have not found significant cross-lagged effect of daily formal practice on next-day outcomes. In fact, Goldberg, Hanley, et al. (2020) found evidence suggesting that current-day formal practice time did not predict next-day mindfulness and negative affect; instead, higher current-day mindfulness and lower current-day negative affect predicted greater next-day formal practice time. Although not capable of demonstrating causality in the way that experimental manipulation can, cross-lagged effects demonstrating temporal precedence can be a valuable step toward establishing potential causal associations in observational data (Kazdin, 2007). For LKC interventions, Carson et al. (2005) found that current-day formal loving-kindness practice time predicted lower next-day anger in patients with chronic pain; however, current-day anger was not controlled in the analyses, making it difficult to interpret this result.

The effect of informal practice on outcomes within MBIs is less well understood and potentially less consistent across studies than associations with formal practice. Fredrickson et al. (2019) randomly assigned midlife adults to six weeks of either a mindfulness or a loving-kindness meditation workshop. They found significant within- and between-person associations between the amount of daily informal practice and daily outcomes. Specifically, more daily informal mindfulness or loving-kindness practice was linked with higher same-day perceived social integration and positive emotions. Manigault et al. (2021) found that daily informal mindfulness practice attenuated increases in stress and decreases in successful coping from pre- to post-intervention. In contrast, Canby et al. (2021) found that cumulative daily informal mindfulness practice was unrelated to changes in mindfulness and psychological distress following eight-week mindfulness training. Among these three studies, Canby et al. (2021) and Manigault et al. (2021) measured the frequency of applying mindfulness in daily life (e.g., “In the past 24 h, I practiced during these times [pick as many as apply],” Manigault et al., 2021, p. 2490); whereas Fredrickson et al. (2019) assessed the frequency of informal practice on a 4-point Likert scale (1 = no, not at all, 4 = yes, on many occasions).

Several noteworthy limitations exist in previous studies investigating the effects of informal practice on outcomes within MBIs. First, studies have not clarified the temporal relationship between informal practice and outcomes. Although temporal precedence alone is insufficient to establish causality, a lack of temporal precedence would suggest a lack of causality (Kazdin, 2007). Second, most studies on informal practice have focused on informal mindfulness practice, with only one study, to our knowledge, investigating the role of informal LKC practice (Fredrickson et al., 2019). While both informal mindfulness and informal LKC practice are forms of meditation integrated into daily life, they are conceptually distinct techniques, and their effects may well also be distinct. In contrast to informal mindfulness practice, which involves bringing nonjudgmental awareness to daily activities, informal LKC practice is characterized by cultivating feelings of empathy, compassion, and appreciation for others and oneself in everyday life. Compared with informal mindfulness practice, informal LKC practice may be especially beneficial for increasing other-focused concern (e.g., empathy, compassion for others) in people’s daily lives (Boellinghaus et al., 2014). Moreover, beyond the MBI context, research suggests that engaging in daily kindness and appreciation activities such as writing letters of gratitude may increase happiness and life satisfaction (Toepfer et al., 2012). Therefore, it is worthwhile specifically investigating the effect of informal LKC practice within MBIs, especially on social outcomes (e.g., loneliness, empathy, prosociality).

Current Study

The current study was conducted to clarify whether informal practice is a potential mechanism within LKC interventions while addressing many of the limitations described above. We investigated within-person changes in self-reported daily informal practice throughout a two-week app-based LKC intervention, i.e., Healthy Minds Program (HMP) - Connection module (Dahl et al., 2020; Goldberg, Imhoff-Smith, et al., 2020) in a sample of undergraduates with clinically elevated symptoms of anxiety and/or depression. In contrast to the approaches taken by Canby et al. (2021), Manigault et al. (2021), and Fredrickson et al. (2019), which assessed the frequency of informal practice, our study employed a visual analogue scale to measure participants’ self-reported application of LKC techniques in daily life, ranging from not at all (0) to all day long (100). Psychological distress, loneliness, and empathy were assessed using self-report measures, and prosociality was measured using a face rating task. The study occurred in the context of the COVID-19 pandemic, which has had a known impact on undergraduates’ mental health due to isolation and disruption in schooling (Li et al., 2021).

We used multilevel models and structural equation modeling (SEM) to examine: 1) whether trajectories of change in daily informal practice during the intervention were associated with pre-post changes in psychological distress and social outcomes (i.e., loneliness, empathy, prosociality), and 2) whether daily informal practice was associated with daily psychological distress and loneliness in same-day and cross-lagged models. Cross-lagged models in particular can help clarify the temporal associations between informal LKC practice and outcomes, which is an important criterion for establishing causal mechanisms (Kazdin, 2007). At once, it should be acknowledged that the cross-lagged models applied to observational data are inherently correlational rather than experimental, precluding the establishment of causality. While our focus was on exploring whether informal practice was a potential mechanism within LKC interventions, the observational nature of the data limits our ability to conclusively determine causal relationships. As few studies have examined these associations, the current study was exploratory and analyses were not preregistered. Therefore, a priori hypotheses were not proposed.

Transparency and Openness

The RCT from which these data were drawn was preregistered at ClinicalTrials.gov (NCT04741529) and through the Open Science Framework (https://osf.io/fmvw4;https://osf.io/rvhsb). Data and Mplus output are available (https://osf.io/rwj6x/). This study involved an analysis of existing data rather than new data collection. The study was approved by the Institutional Review Board at UW – Madison (2020-0197).

Method

The current study involved secondary data analysis of an RCT comparing the effect of a two-week LKC intervention delivered through the HMP app Connection module (Riordan et al., in press). Participants were randomly assigned to practice once a day for 20 minutes (massed practice condition) or twice a day for 10 minutes (spaced practice condition). As reported in the primary outcomes paper from this RCT (Riordan et al., in press), the groups did not differ in prepost or daily outcomes including informal practice, so groups were collapsed for the current analysis.

Participants

Participants were undergraduate students (≥ 18 years old) at UW – Madison who had access to a smartphone or device that enabled them to download and use the HMP app. To be eligible, participants needed to have clinically elevated symptoms indicated by T-scores ≥ 55 on the Patient-Reported Outcomes Measurement Information System (PROMIS) Anxiety or Depression scales (Cella et al., 2014; Pilkonis et al., 2011). Exclusion criteria included: 1) regular meditation practice (i.e., weekly meditation practice for at least one year or daily practice over the last six months), 2) previous meditation retreat experience, 3) previous meditation practice under the instruction of meditation teachers in non-introductory courses, and 4) severe depression indicated by a score > 70 on the PROMIS Depression Scale (Kroenke et al., 2020).

Procedure

Participants were recruited by email and flyers. A total of 772 potential participants were screened based on the inclusion and exclusion criteria. Among them, 351 eligible participants completed pre-test measures and attended an introductory meeting via video conferencing where they learned about the study and were randomized into the two groups. Participants completed daily diary measures each evening during the intervention period, for a total of 15 assessment time points. Post-test was administered two weeks after the start of the intervention. Participants received $25 for completing post-test measures and were given a $15 bonus for completing 80% of the assigned daily practices within the app and a $15 bonus for completing 80% of the daily diary measures. Of those randomized, 316 participants completed post-test measures (90.0% retention; see Supplemental Materials Figure 1 for CONSORT diagram). All assessments were delivered using REDCap.

Intervention

The intervention included LKC training delivered by the HMP Connection Module. HMP is an app that provides mental training to cultivate four core dimensions of psychological wellbeing: awareness, connection, insight, and purpose (Dahl et al., 2020; Goldberg, Imhoff-Smith, et al., 2020). HMP is freely available on both the Google Play Store and Apple App Store and includes the content that comprised the intervention in the current study. However, the specific two-week version tested in the current study (i.e., an abbreviated version of the HMP Connection Module) is not publicly available. The Connection module of HMP was designed to support the cultivation of healthy relationships with oneself and others. The module is comprised of audio recordings that include didactic materials discussing the scientific bases of these practices along with guided meditation practices (i.e., formal practice). Meditation practices in this module include giving appreciation, kindness, and compassion to oneself and extending these qualities to others, including loved ones, “neutral people,” and people one finds challenging (Dahl et al., 2020). The guided meditation practices in the app encourage participants to engage in informal practice (i.e., bring the attitudes cultivated during formal practice into daily interactions).

Measures

Pre-post Outcome Measures

Psychological distress was measured by the four-item versions of PROMIS Depression and Anxiety scales (Pilkonis et al., 2011). Participants rated the severity of anxiety (e.g., “I felt fearful”) and depressive symptoms (e.g., “I felt worthless”) in the past week using a 5-point Likert scale (1 = never, 5 = always). Consistent with the preregistration for the RCT, a distress composite score was computed by averaging across the two total scores (T-scores, with a population mean of 50 and a standard deviation of 10), with a higher composite score corresponding to higher psychological distress. Internal consistency was adequate for both measures (αs = .81 and .88, for anxiety and depression, respectively) as well as for the distress composite (α = .75).

Loneliness was measured by the Loneliness Scale of the NIH Toolbox Adult Social Relationship Scales (Cyranowski et al., 2013). The Loneliness scale contains five items asking about participants’ feelings of being isolated from others (e.g., “I feel alone and apart from others”). Participants rate how often each item applied to them on a 5-point Likert scale (1 = never, 5 = always). A higher total score of the five items indicates higher feelings of loneliness. Internal consistency was high (α = .89).

Empathy was measured by Toronto Empathy Questionnaire (TEQ; Spreng et al., 2009). The TEQ measures the emotional process of empathy through 16 items (e.g., “It upsets me to see someone being treated disrespectfully”). Participants reported how often they felt or acted as described in the items on a 5-point Likert scale (0 = never, 4 = always). A higher total score represents higher empathy. The TEQ has previously shown high internal consistency and concurrent validity in university students (Spreng et al., 2009). Internal consistency was high (α = .87).

Prosociality was measured by a face rating task. This task involved viewing 32 images from the Chicago Face Data Database (Ma et al., 2015) of human faces with neutral, closed-mouth expressions. Images were evenly distributed by binary gender (male, female) and race (non-Latinx White, Latinx, African American/Black, Asian/Asian American) and balanced across a wide array of ratings (e.g., age, attractiveness, facial symmetry, eye spacing). Participants were asked, “How much do you like this person?” and responded on a 7-point Likert scale (1 = dislike a great deal, 7 = like a great deal). An average likability score was computed reflecting prosociality, with a higher likability score indicating greater prosociality. Internal consistency was high (α = .90).

Daily Diary Measures

Daily psychological distress was measured by the highest-loading items of the PROMIS Anxiety (“Today I found it hard to focus on anything other than my anxiety”) and PROMIS Depression (“Today I felt hopeless”) scales (Pilkonis et al., 2011). This practice is consistent with Moore et al. (2016). Participants rated these items on a 5-point Likert scale (1 = never, 5 = always). Responses to each item were z-scored and then averaged to create a distress composite. Internal consistency was adequate for the daily diary distress composite (overall α = .73, within-person α = .67, between-person α = .81).

Daily loneliness was measured by a single item (“How lonely did you feel today?”) adapted from the highest-loading item on the NIH Toolbox Loneliness Scale. Participants rated their daily loneliness on a 7-point Likert scale (1 = not at all lonely, 7 = extremely lonely). A higher score on this item indicates higher daily loneliness. Measuring daily loneliness through one single item has also been adopted by previous studies (e.g., Zilioli et al., 2017).

Daily informal practice (i.e., self-reported application of LKC techniques in daily life) was measured by a single item: “As you reflect on today, to what extent did you apply these practices? (For example, intentionally feeling more warmth or appreciation toward others).” Participants responded on a visual analogue scale (horizontal slider), where 0 = not at all and 100 = all day long. Using one item to measure daily meditation practice (including daily informal practice) has been adopted by previous studies (Fredrickson et al., 2019; Goldberg, Hanley, et al., 2020; Levi et al., 2021). Single-item measures of aspects of meditation practice (e.g., meditation practice quality) have shown acceptable psychometric properties in previous studies (Goldberg, Knoeppel, et al., 2020).

Data Analyses

Data analyses were conducted in R 4.1.1 (R Core Team, 2021) and Mplus Version 8.8 (Muthén & Muthén, 1998). All models characterized longitudinal changes in informal practice within a multilevel model framework, with daily repeated measures (Level 1) nested within participants (Level 2). We used SEM in Mplus to examine whether changes in informal practice were associated with changes in pre-post outcomes. Specifically, we modeled pre-post change in psychological distress, loneliness, empathy, and prosociality using latent change scores (McArdle, 2009) and we modeled longitudinal change in informal practice using latent growth curves (i.e., multilevel models). We tested the significance of the correlation between pre-post latent change scores and latent growth curve slopes and intercepts (see Figure 1). Fit acceptability was evaluated based on Brown (2015). Specifically, root mean square error of approximation (RMSEA) < .050 and comparative fit index (CFI) and Tucker-Lewis fix index (TLI) > .950 were considered indicative of acceptable fit.

Figure 1. Structural Equation Model Examining Associations between Latent Pre-post Change Scores and Latent Growth in Informal Practice.

Note. Change in pre-post outcomes (x_Pre and x_Post) modeled using a latent change score (Δx). y_i (where i = observation 1, 2, …, 15) are the values of informal practice from day 0 to day 14 throughout the intervention; α_y is the intercept factor of y; (β_y is the linear slope factor of y with factor loadings set to 0, 1, 2, 3, …, 14. The correlation between latent change in pre-post outcomes and latent growth in informal practice (i.e., latent intercepts and slopes) depicted with curved two-headed arrows.

To examine same-day linkages between daily informal practice and daily outcomes, we constructed models in which informal practice was the dependent variable and daily outcomes were the independent variable as well as models where daily outcomes were the dependent variable and informal practice was the dependent variable. We used the ‘lmer’ package in R (Bates et al., 2015) for same-day models. Given the possibility that both informal practice and daily outcomes changed over the course of the intervention, time was also included in these models.

We examined cross-lagged associations using SEM in Mplus. A key question when examining cross-lagged relations is to separate between-person variability (i.e., what is consistent for an individual across time for informal practice, psychological distress, and loneliness) from within-person variability (i.e., what is unique for an individual at specific time for informal practice, psychological distress, and loneliness). A latent curve model with structured residuals (LCM-SR; Curran et al., 2014), see Figure 2, can separate the between-and within-person variability. The LCM-SR model involves fitting latent intercepts and slopes for both daily informal practice and daily outcomes (a separate model was conducted to examine associations between daily informal practice with daily psychological distress and loneliness). These latent intercepts and slopes represent the between-person component of the model. This allows assessment of whether individuals who generally reported higher informal practice generally reported lower psychological distress or loneliness (i.e., correlation between latent intercepts) and whether individuals who showed increases in informal practice over time also showed decreases in psychological distress or loneliness over time (i.e., correlation between latent slopes).

Figure 2. Latent Curve Model with Structured Residuals (LCM-SR) Diagram.

Note. y_i and z_i (where i = observation 1, 2, …, 15) are the values of constructs y (i.e., informal practice) and z (psychological distress or loneliness) from day 0 to day 14 throughout the intervention. ε_yi and ε_zi are residuals of y and z at each timepoint i. ay and az are the intercept factors of y and z with all factor loadings set to 1.0; β_y and β_z are the linear slope factors of y and z with factor loadings set to 0, 1, 2, 3, …, 14. The curved two-headed arrows linking ε_yi and ε_zi represent within-time covariances. Figure adapted from Curran et al. (2014).

For the purposes of evaluating cross-lagged effects, the key parameters are at the within-person level. As seen in Figure 2, the LCM-SR includes auto-regressive (i.e., relationship over time for a single construct) and cross-lagged effects (i.e., relationship over time across constructs). These relationships are at the within-person level because we use person-level residuals—the portion of each observation left over after modeling the between-person intercept and slope. In other words, we examine the time-specific within-person effects (Curran et al., 2014), which is consistent with our theoretical interest in the cross-lagged influence of current-day informal practice or daily outcomes on subsequent-day levels of the opposite construct. We started with the most parsimonious LCM-SR which constrained the associations between residuals (cross-lags and auto-regressive parameters) to be constant across time.

Missing Data

Both the multilevel models and the SEM models used maximum likelihood estimation, which is robust to data missing at random (MAR missingness; Graham, 2009). Completion of both the daily diary assessments (~84% of assessments) and completion of the intervention (~85% of the days of practice) was generally high within the study. Following Goldberg, Hanley, et al. (2020), we examined the association between the number of non-missing observations for a specific variable (e.g., number of non-missing observations for loneliness) and the average of that variable across all non-missing observations (i.e., the average level of loneliness across all non-missing observations) to assess the potential bias brought by missing data. Participants’ number of non-missing daily diary assessments was positively associated with their average informal practice ratings (r = .11, p = .044) and negatively associated with ratings of distress (r = −.18, p = .001) and loneliness (r = −.17, p = .002). All of these variables were included in the models for use by the maximum likelihood estimator.

Results

Descriptive Statistics

Of the 351 randomized participants, 348 completed at least one daily diary and were included in analyses. Sample demographics were as follows: 20.17 years old on average (standard deviation [SD] = 1.58); 77.9% female, 20.4% male, 1.4% non-binary, 0.3% prefer not to respond; 83.0% non-Latinx White, 11.8% Asian American, 5.5% non-US Asian, 4.9% Latinx, 2.0% African American, 0.3% non-US African, 0.9% Native American, 1.1% other race, 1.4% prefer not to respond (note race/ethnicity percentages do not sum to 100% as participants could select more than one race/ethnicity). Participants completed a total of 4,387 daily diary assessments (84.0% of 5,220 potential assessments), on average completing 12.61 (SD = 3.01) out of 15 potential assessments.

Means and SDs of pre-post and daily diary measures are reported in Table 1 and visualizations of these variables are displayed in Supplemental Materials Figures 2, 3, and 4. Informal practice time and daily diary outcomes are reported at Level 1 (repeated measure level) and Level 2 (participant level). Intraclass correlations (ICCs) were calculated to estimate the proportion of variance in daily ratings occurring at the within-participant level. ICCs ranged from .41 to .50.

Table 1.

Sample Descriptive Statistics

	Pre-Post Measures
Variable	Timepoint	n	Mean	SD	Min	Max	Skew	Kurtosis
Distress	Pre	351	61.08	6.03	44.45	77.4	−0.15	−0.16
Distress	Post	316	57.76	6.70	40.65	74.75	−0.05	−0.23
Loneliness	Pre	351	2.88	0.89	1.00	5.00	−0.11	−0.54
Loneliness	Post	316	2.34	0.80	1.00	4.60	0.44	−0.41
Empathy	Pre	351	3.09	0.44	1.44	3.94	−0.63	0.49
Empathy	Post	316	3.15	0.45	1.56	4.00	−0.71	0.49
Prosociality	Pre	351	4.12	0.55	2.00	7.00	1.16	6.07
Prosociality	Post	316	4.17	0.65	2.47	7.00	1.25	3.66

	Daily Diary Measures
Variable	Level	n	Mean	SD	Min	Max	Skew	Kurtosis	ICC
Informal	1	4387	50.82	23.56	0.00	100.00	−0.33	−0.82	0.50
Distress	1	4387	−0.18	−0.32	−1.30	2.61	0.65	−0.22	0.45
Loneliness	1	4387	2.73	1.59	1.00	7.00	0.77	−0.24	0.41
Informal	2	348	50.37	17.42	0.00	92.60	−0.53	0.01	-
Distress	2	348	−0.16	0.64	−1.30	2.12	0.41	−0.23	-
Loneliness	2	348	2.78	1.10	1.00	6.43	0.67	0.04	-

Note. Pre-post measures included 4-item Patient-Reported Outcomes Measurement Information System [PROMIS] Anxiety and Depression scales (averaged to compute Distress composite), National Institutes of Health [NIH] Loneliness Scale (Loneliness), Toronto Empathy Questionnaire (Empathy), and a face rating task (Prosociality). Daily diary measures included single items assessing informal practice (Informal), highest loading anxiety and depression items from PROMIS Anxiety and PROMIS Depression scale (Distress), and item adapted from NIH Toolbox Loneliness (Loneliness). ICC = intraclass correlation coefficient; Level = level of nesting with Level 1 being the daily repeated measures (i.e., time) and Level 2 being participant. Note that daily diary distress scores are negative on average due to this variable being the average of two z-scored variables that decreased over time.

As reported previously (Riordan et al., in press), psychological distress (t[315] = −11.83, p < .001, d = −0.52) and loneliness (t[315] = −12.32, p < .001, d = −0.63) decreased from pre- to post-test. Empathy (t[315] = 3.96, p < .001, d = 0.14) and prosociality (t[315] = 2.26, p = .025, d = 0.09) significantly increased from pre- to post-test, albeit with small effect sizes. Daily informal practice significantly increased over time (b = 0.82, 95% confidence interval [CI] [0.71, 0.94], p < .001), whereas daily psychological distress (b = −0.010, CI [−0.016, −0.004], p < .001) and loneliness (b = − 0.024, CI [−0.033, −0.016], p < .001) significantly decreased over time.

Associations between Trajectories of Change in Informal Practice with Pre-post Measures

Latent change score SEM models all showed acceptable fit (RMSEA < .050, CFI and TLI > .95; see Supplemental Materials Table 1). Latent pre-post change in psychological distress was negatively correlated with longitudinal change (i.e., latent slope) in informal practice (r = −.18, CI [−.31, −.05], p = .008). The direction of this effect indicated that steeper improvement in informal practice was associated with larger pre-post reductions in psychological distress. Latent pre-post change in psychological distress was not correlated with the latent intercept for informal practice (r = −.08, CI [−.20, .03], p = .140). Latent pre-post change in loneliness was also negatively correlated with longitudinal change in informal practice (r = −.19, CI [−.32, −.05], p = .009), with steeper improvement in informal practice being associated with larger pre-post reductions in loneliness. Latent pre-post change in loneliness was not correlated with the latent intercept for informal practice (r = −.03, CI [−.14, .09], p = .662). Latent pre-post change in empathy was not correlated with longitudinal change in informal practice (r = .10, CI [−.06, .27], p = .225) nor the latent intercept for informal practice (r = .03, CI [−.09, .15], p = .584). Latent pre-post change in prosociality was also not associated with longitudinal change in informal practice (r = .15, CI [−.02, .31], p = .092) nor the latent intercept for informal practice (r = .01, CI [−.10, .12], p = .852).

Same-day Effects

When using daily informal practice as the predictor variable, it was related to lower same-day psychological distress (β = −.24, CI [−.27, −.21], p < .001) and loneliness (β = −.27, CI [−.30, −.24], p < .001). Significance tests were unchanged when daily informal practice was entered as the dependent variable and psychological distress (β = −.21, CI [−.24, −.19], p < .001) or loneliness (β = −.23, CI [−.26, −.21], p < .001) as the predictor.

Cross-lagged Effects

The LCM-SR model examining cross-lagged associations between psychological distress and informal practice showed acceptable fit (RMSEA = .033, CFI = .959, TLI = .960).¹ Significant auto-regressive paths were detected from current-day to next-day informal practice (b = 0.13, CI [0.08, 0.19], p < .001, βs = .12 to .17²) and from current-day to next-day distress (b = 0.22, CI [0.16, 0.28], p < .001, βs = .20 to .25). A significant cross-lagged path was detected from current-day informal practice to next-day distress (b = −0.050, CI [−0.091, −0.008], p = .018, βs = −.06 to −.04) but not from current-day distress to next-day informal practice (b = −0.009, CI [−0.053, 0.036], p = .702, βs = −.01). At the between-person level, the informal practice intercept was not associated with the distress intercept (covariance = 0.013, CI [−0.056, 0.082], p = .717, r = .03) nor was the informal practice slope associated with the distress slope (covariance = 0.000, CI [−0.000,0.001], p = .259, r = .18).

The LCM-SR model examining cross-lagged associations between loneliness and informal practice also showed acceptable fit (RMSEA = 0.028, CFI = .968, TLI = .969). Significant auto-regressive paths were detected from current-day to next-day informal practice (b = 0.14, CI [0.09, 0.19], p < .001, βs = .13 to .18) and from current-day to next-day loneliness (b = 0.17, CI [0.11, 0.23], p < .001, βs = .15 to .19). However, neither the cross-lagged path from current-day informal practice to next-day loneliness (b = −0.030, CI [−0.077, 0.017], p = .205, βs = −.03) nor the cross-lagged path from current-day loneliness to next-day informal practice (b = 0.016, CI [−0.018, 0.049], p = .356, βs = .02) was significant. At the between-person level, the informal practice intercept was not associated with the loneliness intercept (covariance = −0.048, CI [−0.12, 0.026], p = .206, r = −.11) nor was the informal practice slope associated with the loneliness slope (covariance = 0.000, CI [−0.001, 0.000], p = .316, r = −.26).

Discussion

The present study examined the relationship between daily informal LKC practice (i.e., self-reported application of LKC techniques in daily life) with daily and pre-post changes in psychological symptoms and social outcomes (i.e., loneliness, empathy, prosociality) among meditation-naïve participants with clinically elevated symptoms. Steeper increases in daily informal practice over the course of a two-week, app-based LKC intervention had small magnitude associations with pre-post improvement in psychological distress and loneliness. Specifically, those who increased more in their application of meditative techniques in daily life (i.e., informal practice) benefitted more from the intervention from pre- to post-test. These results support the possibility that psychological distress and loneliness may be modifiable through informal LKC practice. This may be important, particularly given high levels of psychological distress and loneliness experienced by many during the COVID-19 pandemic when there were limited opportunities for in-person social interaction (McGinty et al., 2020). However, longitudinal changes in daily informal practice were not associated with pre-post changes in empathy or prosociality. Thus, the linkage between changes in informal practice and pre-post changes in outcomes may be restricted to more internal experiences (psychological distress and loneliness) and manifest less in explicitly interpersonal experiences (empathy and prosociality).

When examining same-day associations, we found that greater daily informal practice was associated with lower same-day psychological distress and loneliness with small effect sizes. In addition to mirroring what we found in the longitudinal analyses, these results are in line with Fredrickson et al. (2019) who found associations between daily informal loving-kindness practice and same-day positive emotions and perceived social integration. Our results suggest that informal LKC practice may be associated with not only increases in positive outcomes (i.e., positive emotions and perceived social integration) but also decreases in negative outcomes (i.e., psychological distress and loneliness).

When examining cross-lagged associations using the LCM-SR (Curran et al., 2014), we found no evidence for cross-lagged relationships between current-day psychological distress or loneliness on next-day informal practice. However, we did find evidence that greater current-day informal practice was linked with lower next-day psychological distress (the preregistered primary outcome of the larger RCT) with a very small effect size, although this association did not hold for next-day loneliness. To our knowledge, this is the first study to use cross-lagged models to investigate the temporal ordering of linkages between informal practice and outcomes within an MBI. As noted, cross-lagged effects demonstrating temporal precedence can be a valuable (although far from a definitive) step toward establishing causal relationships (Kazdin, 2007). Therefore, the cross-lagged effect of informal practice on psychological distress supports the possibility that informal practice is a potential mechanism underlying the effect of LKC training on psychological distress.

Interestingly, this cross-lagged effect did not emerge for loneliness, despite the fact that steeper increases in daily informal practice throughout the two-week training were associated with pre-post improvement in loneliness. Loneliness may be more dependent than psychological distress on the influence of social interactions (Kuczynski et al., 2022), which during the period of this study (i.e., the COVID-19 pandemic) may have been limited. However, the specific daily outcomes that informal LKC practice can and cannot impact and the reasons for this will need to be examined closely in future work.

In examining the relationship between informal practice and pre-post outcomes in the latent change score SEM models, we found that participants’ mean level (intercept) of informal practice was not linked to pre-post changes in any of the four outcomes assessed (i.e., distress, loneliness, empathy, prosociality). This contrasts the association observed between longitudinal change in informal practice (slope) with pre-post changes in distress and loneliness. One potential explanation for this discrepancy may be related to the way that response set biases (e.g., social desirability bias, tendency to use the upper or lower end of rating scales) influence intercept and slope parameters differently. For example, intercepts may be very strongly impacted by respondents tending to use upper or lower portions of the rating scale. In contrast, slopes, characterizing changes over time, may be less impacted by whether respondents tend to use the upper or lower portions of the rating scale. (Of course, tending to use the upper or lower portions of the rating scale can still impact slopes, particularly in the case of floor and ceiling effects that restrict the range of change over time.) In this regard, slopes may be less influenced by some response set biases and therefore contain more valid signal for characterizing the development of informal practice. Interestingly, in the LCM-SR models which disaggregated effects into within-person (i.e., cross-lagged) and between-person components, neither the latent intercepts nor slopes were associated at the between-person level. This suggests that the day-to-day variation in participants’ informal practice may carry more signal than the latent intercept and slope values which aggregated across the two weeks.

The current findings may inform efforts to improve LKC interventions. In particular, the essentially correlational findings of the current study warrant future experimental manipulation of informal practice dosage. Several candidate designs could help clarify the role of informal practice in LKC interventions and MBIs more broadly. One design could include comparing an entirely informal practice-based LKC intervention with a waitlist control. Such a study could clarify whether informal practice alone produces benefits. An alternative design could compare an LKC intervention that includes instructions to engage with informal practice versus an LKC intervention that does not (i.e., augmented vs. non-augmented). Using such a design, changes in self-reported informal practice could be examined as a mediator of the effects of the augmented versus non-augmented LKC intervention. Similarly, investigators could manipulate the amount of informal practice recommended to participants within an LKC intervention that includes both formal and informal practice. A micro-randomized trial (Qian et al., 2022) could be used to randomly assign participants several times per day to mobile-based prompts that encourage informal practice and employ experience sampling to assess subsequent self-reported informal practice and momentary psychological outcomes. Data from such a study design could be used to investigate whether prompting informal practice in the moment facilitates proximal self-reported practice (e.g., in the next few hours) which in turn improves momentary psychological distress and other outcomes.

Prior intervention development including user experience testing and qualitative research may be necessary to inform the design of such experiments. Whether or not informal LKC practice can be delivered as a standalone intervention (i.e., without formal LKC practice) will be an important area to explore. Clarifying barriers to and facilitators of informal practice can also inform intervention development. Future LKC interventions may also benefit from the integration of just-in-time-adaptive interventions (JITAIs) that encourage informal practice (Nahum-Shani et al., 2018). JITAIs are interventions that leverage digital technology to tailor the delivery of in-the-moment support to the person’s internal state and context in daily life (Nahum-Shani et al., 2018). Future studies could investigate the conditions (e.g., level of psychological distress or loneliness, being alone or with others) in which individuals are more likely to benefit from in-the-moment recommendations to engage with informal LKC. Results from these studies can inform when and how to best intervene in daily life in order to effectively promote informal practice.

Limitations

Several limitations of the current study warrant discussion. For one, the current study did not manipulate the dosage of informal LKC practice. Therefore, we cannot directly infer causal relationships between informal practice and outcomes. Despite evidence suggesting temporal precedence for the association between informal practice and daily psychological distress, it will be necessary to establish causality in future experimental designs.

There are several important limitations related to measurement. First, bias could have been introduced through our repeated assessment of informal practice, psychological distress, and loneliness. Measurement reactivity of this kind is a common challenge in intensive longitudinal studies (Hamaker & Wichers, 2017). Informal practice in particular may have been influenced, given the daily prompts may have encouraged participants to practice informally. A future study could include a randomized condition that does not receive daily assessments of informal practice to establish whether completing these assessments impacts pre-post measures of informal practice. Second, although informal practice, psychological distress, and loneliness may fluctuate in the course of daily activities (Hjartarson et al., 2021; van Roekel et al., 2015), we only assessed them once per day. It is possible that linkages between informal practice with psychological distress and loneliness are different than reported here when examined on a finer-grain timescale. It would be valuable for future studies to examine the temporal associations between informal practice dosage and outcomes using ecological momentary assessment (EMA; Verhagen et al., 2016). In addition to providing high temporal resolution, EMA can increase ecological validity by assessing variables in the midst of daily life. Third, the use of a single-item measure for meditation practice in our study, while common in previous research (Fredrickson et al., 2019; Goldberg, Hanley, et al., 2020; Levi et al., 2021), may have reduced the reliability of our measurement of informal practice. Future studies will ideally employ multiple-item measures for assessing informal practice. At once, it is worth noting that the lower reliability associated with a single-item measure would theoretically introduce greater measurement error and ultimately decrease (rather than artifactually increase) the likelihood of observing associations between informal practice and outcomes. Fourth, our single-item measure of informal practice (i.e., “As you reflect on today, to what extent did you apply these practices?”) may have assessed participants’ perceived success applying the meditative techniques they learned in the HMP, rather than the duration to which they attempted to apply these techniques. Thus, this item, while intended to measure the amount of informal practice, may be confounded with participants’ proficiency in experiencing compassion and appreciation in daily life. To address this, future studies could enhance clarity by modifying the item to explicitly assess intentional engagement in informal practice (e.g., “As you reflect on today, to what extent did you try to apply these practices?”). Fifth, certain measures in our study, such as those related to informal practice and prosociality, lacked thorough validation in prior research. Notably, we employed the likability of faces as an indicator of prosociality, a measure that might not fully encompass the broader concept involving prosocial intentions and behaviors (e.g., offering comfort to others in times of hardship; Baumsteiger & Siegel, 2019). Subsequent studies validating this measure or employing previously validated measures to study prosociality would be worthwhile. The field has yet to determine a consensus definition for informal practice and a gold standard method for assessing this construct (for a recent review; Andersen, 2020). Validated measures do exist that purport to assess informal practice. For example, the Mindfulness Adherence Questionnaire (Hassed et al., 2021) assesses mindfulness in everyday life by having participants reflect on their past week. However, to our knowledge, this measure has not been shown to correlate with momentary assessments of mindfulness in everyday life (Hassed et al., 2021). It would be valuable for future studies to examine the degree to which various methods for assessing informal practice such retrospective assessments, daily diary assessment of informal practice used in the current study, and other methods (e.g., assessing when participants practiced informally after they complete a formal practice; Manigault et al., 2021) correspond to informal practice assessed via EMA. Given EMA is specifically designed to capture momentary and daily life experiences in situ (Shiffman et al., 2008), it may be an inherently more valid method for evaluating informal practice and can therefore serve as a criterion measure in validation studies. Sixth, we did not measure and control for potential confounding variables (e.g., conscientiousness, social desirability) that might influence the reporting of informal practice. Future studies could control for these variables while examining the relationship between informal practice and outcomes.

Other limitations are related to our sample. Our study included only psychologically distressed undergraduates and participants were predominantly female and non-Latinx White. It is unknown to what degree the current results generalize to non-college student populations, treatment-seeking populations, and more racially and gender-diverse samples. Furthermore, our study sample was derived from the United States, representing a Western, Educated, Industrialized, Rich, and Democratic (WEIRD) society. The extent to which these findings can be extrapolated to non-WEIRD populations remains unclear (Henrich et al., 2010).

Conclusion

In summary, our study supports the notion that informal practice plays a role in the beneficial effects of LKC on both daily and longer-term measures of psychological distress. An upward trajectory of change in informal practice over the course of training is also linked to benefits on loneliness but was not associated with changes in empathy or prosociality. These findings encourage continuing to emphasize informal practice within the context of LKC interventions and MBIs, although effects may be more limited on explicitly interpersonal experiences like empathy and prosociality. Future studies manipulating informal practice dosage can help clarify whether informal practice plays a causal role within LKC interventions. Examining ways to increase informal practice in daily life may ultimately help maximize the potential benefits of this aspect of LKC interventions, should informal practice be shown to be causally linked with outcomes. Such work may eventually produce lighter weight app-based LKC interventions that require less time commitment and integrate more seamlessly into participants’ daily life. Such interventions have the potential to increase the acceptability and accessibility of this approach.

Highlights.

• Increased informal practice was correlated with pre-post reductions in distress

• Increased informal practice was correlated with pre-post reductions in loneliness

• Higher current-day informal practice predicted lower next-day distress

• No cross-lagged effects of distress or loneliness on informal practice

• Further investigation into a causal role of informal practice is warranted