Abstract
Since more than 450 million people worldwide suffer from mental disorders, interventions that promote mental health have been called for. Mindfulness-based coping (MBC) is an intervention based on coping skills from cognitive behavioral therapy integrating mindfulness practices. The aim of this study was to examine the effectiveness of the MBC program for psychiatric outpatients. The study employed a mixed research method with a qualitative approach using semi-structured patient interviews and clinical assessments from patients’ therapists and a quantitative approach using instruments measuring mindful coping, mental ill health, and life satisfaction. The study sample included 38 psychiatric outpatients from a district psychiatric outpatient service in Norway. Results suggested that although use of the different skills varied, participants had a positive experience with the program and positive changes in psychological functioning were observed. Findings provide knowledge regarding the design of interventions integrating mindfulness to promote more adequate psychological coping.
Key words: mindfulness, coping, psychiatry, mixed model
Introduction
Mental health is considered the foundation of individual well-being and the effective functioning of a community.1 However, more than 450 million people worldwide suffer from mental disorders, whereas many more experience mental problems.1 Although Norway has fewer problems regarding mental health than other European countries, issues such as symptoms of anxiety and depression, sleep deprivation, and reduced coping ability are widespread in the Norwegian population.2 Based on studies from Norway and other Nordic countries, important measures in promoting the population’s mental health include support groups, self-help groups, and psycho-educative programs aiming to empower individuals.3 The discussion above points to a need for interventions that aid people in coping with emotional and/or stress-related problems so as to prevent and treat such problems. One such intervention is mindfulness-based coping (MBC).4 MBC is an intervention based on coping skills from cognitive behavioral therapy (CBT) that integrates mindfulness practices to help psychiatric outpatients deal with emotional and stress-related problems. The main objective of this effectiveness-focused study was to evaluate the MBC program by using a mixed model design.
Background
MBC was developed with the aim of providing a transdiagnostic group-based program for psychiatric outpatients.4 MBC aims to help participants with inter- and intrapersonal emotional and stress-related problems, and draws heavily upon two established and documented traditions within psychiatric health services: mindfulness and CBT. Mindfulness as a statelike quality refers to being in the present moment, intentionally and without judgment toward whatever the moment brings in the way of thoughts, feelings, or bodily sensations. Kabat-Zinn’s5 commonly used definition of mindfulness describes it as paying attention in a particular way: on purpose, in the present moment, and non-judgmentally, explaining mindfulness as a dynamic process of life that includes both intra- and interpersonal aspects.6 The use of mindfulness in integrative medicine7 was first presented through mindfulness-based stress reduction (MBSR) for patients with chronic pain disease8,9 and soon followed by dialectical behavior therapy (DBT) for women with a borderline personality disorder,10,11 acceptance and commitment therapy (ACT) for psychotherapy in general,12 and mindfulness-based cognitive therapy (MBCT) for preventing a relapse of depression.13 These four interventions are collectively referred to as third wave cognitive therapies,14 and much of the current psychological literature on mindfulness is based upon these therapies and the cognitive operationalization of mindfulness they have provided.
The last decades have seen increasing research into mindfulness and mindfulness-based psychological interventions.15-21 Numerous variants of these four mindfulness-based interventions (MBIs) have since been developed using modified content and targeting other populations.6 Such mindfulness-based approaches vary in their components, although they all include meditation practices, behavioral practices, cognitive strategies, and/or emphatic strategies,22 as does MBC. Cognitive behavioral coping skills in MBC are mainly derived from the skills training program of DBT, which is part of the more complex and holistic treatment program for the DBT target group. However, the dialectical approach, a main focus in the DBT program, was removed from MBC, as was the training in certain skills specific to the DBT target group; however, other exercises were added. The main reason for such adjustments was to design MBC as a transdiagnostic group program specifically for psychiatric outpatients with emotional and stress-related problems.
Following the above, the main themes of MBC are mindfulness exercises, skills for coping with distressing situations and negative emotions, and practical training in handling relations. One specific focus of MBC is awareness training, emphasizing non-judgmental acceptance of situations and emotional experiences connected to them. Such awareness is thought to bring forth more adaptive coping strategies. Research has demonstrated the importance of interpretation of stressful encounters compared to the situation experienced as stressful, and how the latter has less influence on emotional outcome of the situation than the former.23 Furthermore, some researchers have suggested that appraisal processes may affect outcomes directly,24 and that a focus on appraisal processes in preference to mere coping may have a positive influence on how one perceives stress.25 Although not differentiating as clearly between the appraisal process and coping, this is consistent with Lazarus’ perspective on the meaning of appraisal in coping with stress.26 In the short term, awareness-training can lead to decreased rumination and less impulsive action in that one does not allow negative thoughts, feelings, or impulses to guide one’s life or actions. More specifically, it may reduce potential self-blame and rumination, as well as general distress. In the long term, this may further lead to an accepting attitude toward difficult situations and emotions that arise in life. This acceptance is likely to enhance the appraisal of problematic thoughts, actions, and/or situations; in turn, this may allow for more adequate coping.
The use of coping skills for constructive self-distraction is another specific topic in the MBC program, referring to so-called healthy distractions,27 (i.e., mindful and short-term). This involves using distractions to better tolerate negative arousals, but without falling into long-term avoidance. In the short run, by creating necessary mental breaks from distressing situations, mindful distractions help one to avoid acting impulsively and perhaps inadequately, which may lead to self-criticism and negative thinking. This makes it possible to get through distressing situations when for some reason it is inappropriate to deal with the distress immediately. Thus, constructive self-distraction can allow participants to retain coping resources for long-term adequate coping. Regulating affect by learning coping skills that help prevent unnecessary negative emotions from arising is also a specific part of MBC. Such skills do not imply the avoidance of negative emotions, but mindfully building one’s coping resources for future reference through taking care of oneself. By learning emotion theory and how emotions work, participants become aware of what emotions can do for and to them. Furthermore, learning how to regulate emotional activation and to seek experiences that increase positive emotions can build coping resources for future coping. In other words, psycho-education on emotions teaches participants skills that help them moderate negative emotions in the short term, as well as skills to increase positive experiences, providing a long-term positive change.
The final specific topic is constructive self-assertion, which emphasizes awareness of one’s goals in a given situation, and adequate and mindful interpersonal communication. By learning to create a balance among one’s own needs, social relations, and self-respect, participants learn how to optimize the fulfilling of their own needs while maintaining social relations; they come to value their own meanings. Participants become aware of and reflect on these topics and learn skills to enhance interpersonal situations by improving their arsenal of relational skills (e.g., saying no to requests). Becoming more assertive and aware of what relations with others imply, both good and bad, may increase participants’ sense of mastery in relation to others and furthermore encourage participants to employ and receive health-promoting qualities, such as social support. In short, learning to adequately handle relations leads to positive changes in both the short and long term. Thus, the mindful coping aspects of the MBC program can be experienced as useful to participants in terms of both short-term and long-term coping. This can further contribute to positive changes via increased functioning and reduced symptoms of mental ill health.
Generally there has been limited attention in psychotherapy research into understanding therapy processes and patients’ perspectives, especially within cognitive behavioral interventions.28 In line with this, effectiveness-focused research seems to be appropriate for the current study. Effectiveness-focused research involves explaining what makes a treatment work.29 This implies an emphasis on consumer benefit,30 as well as a broader approach to clinical practice with respect to process and dynamics rather than mere outcome research. By examining how the target group experiences MBC, it may be possible to make the program more motivating and suitable for that target group. The goal of this effectiveness-focused study was therefore to evaluate different aspects of the MBC program and how participants experience these aspects by qualitatively exploring participants’ own comprehension of useful skills and potential experience of subjective change and by quantitatively investigating participants’ change in symptoms of mental ill health and subjective well-being. Research questions were: Do participants find skills taught useful and, if so, what skills do participants refer to as useful? Do participants make use of mindful coping skills in daily life and, if so, what skills do they use in daily life and in what situations? Finally, is participation in the MBC program associated with a reduction in symptoms of mental ill health and an increase in psychological functioning?
Materials and Methods
One objective for mixing methods is to achieve complementarity, that is, broader, deeper, and more comprehensive social understandings by using methods that tap into different facets or dimensions of the same complex phenomenon.31 In the current study, by using a concurrent embedded strategy within a mixed model design,31,32 treatment outcome as measured by different instruments (quan) was nested within a study of participants’ accounts of MBC (QUAL).
Intervention
Mindfulness-based coping
MBC is a manualized educative mindfulness-based coping skills training program developed for psychiatric outpatients4 at a district psychiatric outpatient service (DPS) located in a district in southwest Norway. DPSs in Norway have responsibility for establishing local mental health care programs to provide services for users in their local environment. MBC was offered as a transdiagnostic group program with the aim of teaching participants coping skills for general life stressors. MBC consisted of 27 weekly group meetings of six to eight participants based on four modules, that is, mindfulness, stress management, affect regulation, and handling of relations. Regarding the mindfulness tradition, MBC was inspired especially by MBSR and MBCT. Specifically, the 3-minute breathing space, cognitive therapy exercises (e.g., thoughts and feelings exercise), and deliberately bringing difficulties to mind in sitting meditation derive from MBCT. From MBSR, poems such as The Guesthouse by the Sufi poet Rumi, exercises using mindfulness in everyday life, exercises for experiencing the present moment mindfully, and the raisin exercise were incorporated into MBC. DBT skills training inspired MBC in terms of structuring the program, in part using mindfulness skills, selected skills for distress tolerance and emotion regulation, and use of homework sheets for skills taught. Additionally, inspiration for using stories, fairytales, and metaphors to highlight important subjects came from ACT, as did the emphasis on acceptance and avoidance. In short, with a main emphasis on skills from the DBT skills training program, parts of MBSR, MBCT, DBT, and ACT believed to be beneficial for the target group were adjusted with the aim of developing the MBC program. The use of short stories, cases, quotations, visualization techniques, comic strips, and role playing were emphasized within MBC. In addition, other coping skills and homework focusing on coping skills inspired MBC, such as investigating personal stressors using worksheets and building coping resources.33 Each meeting lasted 1 1/2 hours plus a 15 min break and began and ended with a short mindfulness meditation. Topics discussed were elaborated on through pedagogical methods such as storytelling, reading poems, role playing, discussing quotations, and solving cases related to psychosocial problems. Homework sheets were handed out at each meeting. Because the group members were not allowed to discuss particular private matters such as diagnosis, a therapist was appointed to each participant. Each participant had 10 to 12 meetings with his or her therapist throughout the duration of the program. The objective of participating in individual consultations during the program was to ensure that the participant was able to relate skills from the program to his or her problems; thus, the program was the main topic during these consultations. The educational background of the therapists with patients in the program was varied, including psychiatric nurses, psychiatrists, psychologists, and social workers. All had experience of working individually with patients attending interventions with elements of mindfulness and cognitive psychology, and, they had all worked within psychiatric institutions no fewer than 15 and no more than 30 years. Criteria for inclusion in the program were the participants’ own wish to participate, a therapist’s recommendation, a specific and explicit need for at least one of the components of the MBC program (e.g., skills for interpersonal communication), commitment to participate in all modules, a reasonable level of functioning in terms of being able to analyze one’s own thought processes, feelings, and actions, as well as status as an outpatient at the clinic at least for the duration of the program. Criteria for exclusion were psychotic patients or patients whose therapists did not believe would benefit from the program, patients with current drug or alcohol abuse, currently suicidal patients, or other patients for whom it was believed that the program in any way constituted a risk factor. Group leaders had all undertaken an MBC instructor training. Their educational background was varied, including nurses, psychologists, occupational therapists, and pedagogues. Two group leaders shared responsibility for each group, and, were present in each MBC-group.
Participants
Purposeful procedures for sampling were used. Participants referred to MBC experienced continual problems of an emotional and stress-related character, including a relational component, and skills from the program were thought to ease these problems. Patients with a serious depressive disorder were not referred to the program, but patients who, for instance, experienced emotional and stress-related distress with depressive symptoms in intercommunicative situations were considered suitable for the program. The main recruitment criteria were the participants’ potential for improving their ability to handle relations, regulate their affect, and manage their stress. All participants were referred from the municipal health services to the same district psychiatric outpatient service. Most of the patients were diagnosed with anxiety, depression, or stress-related disorders, such as bereavement, burnout, and/or relational problems. Group participants were informed about the current study by the research leader and their group leader. Participants in each group were enrolled concurrently, and between-groups enrollment was sequential. Fifty-four participants were included and allocated to seven MBC groups. Sixteen participants dropped out during the course of the program; two were hospitalized, seven experienced severe life changes that challenged program participation (e.g., new family member, change in job/school situation, relocation), two were known to have difficulties fulfilling their commitments, one abused medical drugs, one gave religious beliefs as the reason for dropping out, and three gave no reasons at all. In all, 38 participants finished the program (84.2% female and 15.8% male, Mage=40.4 years, age range 21-72 years).
Measures
Interviews
An interview guide was developed for a semi-structured interview, focusing on participants’ reflections on expectations of the program, what coping skills they found useful and in what ways, which skills they found less useful and/or difficult, as well as whether or not they experienced any changes in thoughts or actions as a result of learning coping skills.
Global assessment of functioning -split version
The Global assessment of functioning - split version (S-GAF) is based on the Global Assessment of Functioning (GAF) scale.34,35 The GAF scale aims to measure adaptive functioning by having therapists score their patients on psychiatric symptoms and social and occupational functioning levels on a scale of 0 to 100. The main purposes are to reflect the need for psychiatric treatment and to measure treatment outcome.36 The S-GAF was developed assuming that the scoring of functioning and symptoms refers to different clinical aspects of a psychiatric patient’s condition. This split version is divided into one symptom (GAF-S) and one function score (GAF-F), each on a scale from 0 to 100 with a 10-point range. According to Karterud,34 GAF-F scores ranging from 81 through 100 indicate that the patient functions extremely well regarding work, social activities, leisure, and interpersonal relations. Scores ranging from 61 through 80 indicate an adequate functioning level for patients not currently ill, including passing and situational conditioned functioning without necessarily alarming others. Scores ranging from 41 through 60 show increasing impairment of functioning that is conspicuous to and has consequences for others, including patients not yet recovered to a normal level of functioning after a psychiatric illness. On the GAF-S, scores ranging from 81 through 100 cover joy of life and wellness in relation to minor distressing experiences. Scores ranging from 61 through 80 indicate moderate reactions of distress to easier and limited symptoms. Scores ranging from 41 through 60 indicate evident psychiatric symptoms; the strength of the symptoms will increasingly be a serious nuisance and worry for the patient and people close to the patient experience him or her as in need of psychiatric treatment, although the patient is not considered psychotic. None of the participants in the current study scored below 40, and scores below 40 are therefore not elaborated on. Studies have shown that S-GAF scores from single independent raters hold acceptable reliability.36
Mindful coping scale
The Mindful Coping Scale (MCS)37 is a 23-item, self-report questionnaire. The scale was constructed with four subscales to assess four different aspects of mindful coping: awareness, distraction, preventing negative emotions, and constructive self-assertion. Each item was rated on a 5-point Likert scale (1=never/hardly ever, 5=always). Scale validity was acceptable (Chi-square=632.2, df=222) and reported reliability values in terms of alpha values ranged from 0.76 to 0.85.38 Cronbach’s alphas for the MCS subscales ranged from 0.73 to 0.82 in the current study.
Symptom checklist-90-revised
The Symptom Checklist-90-Revised (SCL-90-R)38 is a 90-item, self-report symptom inventory measuring current psychological symptom status. Each item is rated on a 5-point scale of distress (0=not at all, 4=extremely). The scale includes nine primary symptom dimensions (somatization, obsessive-compulsive, interpersonal sensitivity, depression, anxiety, hostility, phobic anxiety, paranoid ideation, psychoticism) and three global indexes of distress, of which the Global Severity Index (GSI) is the most important because it is the best single indicator of the current level or depth of disturbance.39 The GSI was used in this study. Higher GSI scores indicated higher overall distress. The SCL-90-R has proven to have high reliability in terms of alpha values. In the current study, the coefficient of reliability was 0.98.
The satisfaction with life scale
The Satisfaction with Life Scale (SWLS)40 is a 5-item measure of subjective well-being, assessing global satisfaction with life. The 7-point Likert scale ranges from 1 (strongly disagree) to 7 (strongly agree), and scores are calculated by summing item ratings to obtain a total. The SWLS has been found to have favourable psychometric properties. The coefficient of reliability was 0.91 in the current study.
Ethics
The current study was formally approved by the Regional Ethical Committee (Norwegian abbreviation: REK Vest) and has official permission from the Personal Data Registers Act § 9 (Norwegian abbreviation: NSD). Prior to study participation, oral and written information was provided and informed consent forms were distributed to participants. The information and consent form emphasized that participation was strictly voluntary and that participants at any point in time could withdraw their consent. That some participants chose to do so indicates that the participants experienced the study as voluntary. To ensure anonymity, all names mentioned in this study are fictional.
Procedure
Data collection was concurrent. For practical reasons, patient interviews were conducted by the individual participant’s therapist at the clinic. All therapists attended a meeting on how to conduct the interviews. If agreed by the participant, the interview was tape recorded in addition to the interviewer taking notes. Each interview required 30 to 45 min to carry out. All interviews were conducted within a month after the program ended. Interviews were coded and transcribed verbatim. S-GAF was scored after individual consultations by participants’ therapists pre- and post-intervention. All therapists were experienced in scoring S-GAFs on patients. Therapists enclosed the respective scores in envelopes, serving the purpose of keeping scores from the researcher and group leaders until post-intervention. Quantitative data were gathered pre-intervention, post-intervention, and at a three-month follow-up. The respective group leaders administered the survey in accordance with verbal and written instructions from the researcher. The questionnaires were completed after group meetings. The questionnaire required approximately 45 to 60 min to complete.
Data analysis
A parallel mixed analysis was conducted. With the assistance of NVivo 8 Software,41 qualitative data were analyzed by content analysis. Software tools for computer-based qualitative research provide measures of convenience and efficiency because packages such as NVivo increase the overall level of organization of projects involving qualitative research.42 To identify skills labelled as useful, how these skills were used in daily life, and the potential experience of change, participants’ qualitative data were quantified and references to either skills or module were counted into the predefined categories of awareness, constructive self-distraction, preventing negative emotions, and constructive self-assertion. In accounts where references were made to several modules or to skills from different modules, each reference was counted within the category to which it belonged. Researchers have suggested that counting qualitative data and narrative descriptions may provide more meaning.43 Furthermore, to understand contextual details, accounts of situations in which participants made use of the skills or modules were explored. Quantitative measures were analysed by comparing scores at different time points. Statistical analyses of quantitative data included descriptive statistics, reliability statistics (Cronbach’s alpha), and repeated measures analyses, all conducted using the SPSS 18 program.44 In addition, effect sizes were calculated by use of Cohen’s d estimation. Individual effect sizes were calculated using the formula for Cohen’s d, but with the individual change score as numerator.
Results
Qualitative results
Awareness-training
Results from qualitative analyses showed that the majority of participants referred to skills from awareness as useful to them, and, mindful breathing exercises and moment-to-moment experience were most frequently mentioned. Awareness-skills were also chiefly represented when describing use of skills in situations from everyday life. One participant stated that (…) it was at least a way to think ‘wow, you are alive, you experience this once in your life, use what you can to experience as much as possible. And I was fortunate enough to attend the movie festival in Haugesund [city in Norway] almost right away. (…) and if it wasn’t for me thinking mindfulness it wouldn’t have been as great. Instead I would’ve been a bit scared and lost in that kind of environment. But I was there with glowing eyes and had a wonderful weekend (Hulda). Awareness was also referenced to as contributing to change, as in the following example: I have experienced very large changes in my own thoughts and actions after the program. I manage to stop. [I then] consider what are emotions and what are facts (Lena). Another participant stated: Now I breathe before I act and before I have to deal with difficult questions. I can actually use my breath to get through difficult situations, and also by taking one moment at a time. I use that part of mindfulness a lot. It has kind of become a part of me after participating in the program, I think. But sure, it was the part we worked with continuously through the program so it just fell into place; there was room for it in my life (…). Right now I mostly use breathing exercises. (…) I have also been good at letting things go. Not clinging to them (Sofie).
Constructive self-distraction
Regarding useful skills, constructive self-distraction was also mainly referred to. Skills for coping with acute stress, such as distraction and doing something pleasant for oneself, and skills for coping with chronic stress, such as breathing exercises for acceptance, were emphasized. When describing use of skills in situations from daily life, constructive self-distraction was chiefly represented. One statement pointed to coping with acute distress: I deal with stressing situations by sitting down and breathing. When bad thoughts arise, I use distraction skills to get my thoughts on something else (Karen). An example of coping with chronic distress through acceptance was the following account: I have to mention [name of son]. The fact that he was leaving [for Afghanistan] was something that I had been dreading very much, just the fact that I had to say goodbye to him. But it turned out ok. I decided that this situation should not get the best of me, I am going to deal with this, I don’t want him to see that I get problems with my neck because of this, [and] I am going to cope with it. And I did, it was fine. How it turns out after he has left, we don’t know (Maria). Regarding experience of change in coping distraction for coping with acute stress and acceptance for coping with acute stress and acceptance for coping with chronic stress were emphasized Examples are: I manage to create a distance and see that it’s not necessary to use that much energy. [I] benefit from leaving the situation, both mentally and physically (Ada) and (…) on several occasions I manage to choose acceptance although I don’t like the situation (Linn).
Preventing negative emotions
Preventing negative emotions had fewer references than the other mindful coping strategies, with no specific skills pointing out. This strategy was somewhat referenced to as difficult. Few participants referred to situations in which they had used skills from preventing negative emotions. Still, one example was the following account: My husband and I had decided to have a quiet conversation, just the two of us. He came home from work and said that some friends had asked him to help them do some work on their building site. My husband said that he wished to work on the site and postpone our conversation. I was mindful of the feeling I experienced and named it. I calmly said: I don’t like this. We talked quietly together, and after a little while I thought that it would be ok for him to go to the site. He chose not to do so and called his friends and postponed the work. We had a great conversation later on that evening, the two of us (Lena). Skills from preventing negative emotions were the least emphasized as contributors to change, with no specific skill being pointed out as more important. An example is that the program has helped me in that I now become less angry when the children defy me (Lena). Another participant pointed to difficulties in giving examples of using skills from this module: I trust my own feelings more, I can be right even though others don’t feel the same. [It is] difficult to give a concrete example (Kate).
Constructive self-assertion
Also constructive self-assertion had somewhat fewer references compared to awareness and constructive self-distraction, and, was referenced to as being somewhat difficult. The skills referenced to as most useful were rejecting requests and being self-assertive, with an example being: [I] notice the change in my own ways, especially when it comes to saying no. [I] experience that people find it ok (Mira). One participant stated that skills for constructive self-assertion had helped in that I feel that I’m capable of expressing myself in a different manner, solving problems more easily, and that I’m much more confident in my own opinions (Edna).
Experience of change, difficult skills, and program expectations
Qualitative findings showed that almost all of the participants (N=32) expressed that they experienced changes in thoughts or actions as a consequence of participating in the program. Only two participants did not; one replied a plain no (Jane) and the other not really (Thea). Participants who did experience changes referred to those changes as consequences of practicing skills mainly from awareness, constructive self-assertion, and constructive self-distraction. Some participants experienced changes that could not have been foreseen. As one participant stated: I don’t have anxiety anymore. [I] am in a process with my doctor to reduce [my] medication. [The] challenges in everyday life don’t get as big anymore. [I] am better at taking care of myself. [I am] planning to remove my disability aid. [I] believe this is all thanks to the program (Randi). Half of the references to skills as difficult were restricted to mindful coping themes, not going into detail on concrete skills, as mentioned by one patient: Affect regulation was a bit more vague (Tom). Almost half of the participants stated that skills experienced as difficult were only difficult initially and that their understanding increased during the program’s course; for example, saying no was difficult, but [is] easier now (Mira). Participants’ expectations of the program were categorical in terms of either having expectations (N=24) or not (N=10). The majority of the participants had high expectations of the program, and almost all of these expectations were met (N=21); for example, I had huge expectations for stress management. It gave me some tools I use daily (Karen). Expectations centered on coping skills to manage specific problems in daily life, such as learning ways to deal with my problems and get an easier everyday life, especially regarding stress management and handling of relations (Kate). Most of the participants with no expectations did not go into further detail than, for example, not really. Started with a white sheet (Olga), no (Jane), and not big (Tom). One participant stated: I didn’t have any expectations. [I] just decided that ‘I’m going to try this program. [I] came to the first meeting with an open mind and from day one the course was fantastic. I have had much use of what I’ve learned. [It was a] great help for me in my situation. I’d say it was a turning point in my life (Lena). None of the participants expressed disappointment with their program participation. Results are shown in Table 1.
Table 1.
Qualitative results: number of participants’ references to the mindful coping themes regarding useful skills, difficult skills, skills used in daily life, skills experienced as contributing to change, and references in total (N=34). The number of participants giving references is given in brackets below the number average references made.
| Useful skills |
Difficult skills |
Skills used |
Contribution to change |
References in total |
|
|---|---|---|---|---|---|
| Awareness | 55 | 6 | 21 | 16 | 98 |
| (33) | (5) | (16) | (16) | (34) | |
| Constructive self-distraction | 66 | 6 | 26 | 14 | 112 |
| (29) | (6) | (19) | (12) | (32) | |
| Preventing negative emotions | 14 | 8 | 8 | 10 | 40 |
| (11) | (7) | (7) | (10) | (20) | |
| Constructive self-assertion | 21 | 8 | 26 | 14 | 69 |
| (17) | (8) | (18) | (13) | (27) |
In sum, regarding the research questions, participants referred mostly and almost equally to constructive self-distraction and awareness. Constructive self-assertion had half as many references, and preventing negative emotions half as many as constructive self-assertion. Neither constructive self-distraction nor awareness was referred to as frequently in examples of skills as usefulness. Nor were the two modules referred to frequently as contributors to change in comparison to usefulness. Regarding constructive self-assertion, more references were made in participants’ examples of skills used rather than to its usefulness and almost half as many references were made to this module as a contributor to change. In regard to preventing negative emotions, almost half as many references were made to examples of skills used and as a contributor to change than to usefulness. Difficult skills, as such, received few references.
Quantitative results
Results from quantitative analyses are shown in Table 2. Results indicate that participants increased their use of mindful coping skills during the course period and that the increment in coping skills was maintained during the follow-up period. Changes from pre-to post- and follow-up tests yielded Cohen’s d coefficients ranging from 0.59 through 1.01, indicating that increments in the use of mindful coping strategies were relatively large. Moreover, scores for life satisfaction suggest that participants improved their sense of satisfaction with life during the intervention period and that the sense of satisfaction was even more enhanced during the follow-up period. Changes from pre- to post- and follow-up tests yielded Cohen’s ds ranging from 0.50 through 0.77, indicating a relatively large increase in life satisfaction. Individual effect sizes for the SWLS for pre- post- changes showed medium effect sizes of 0.5 or higher among 42.9% of the patients, and large effect sizes of 0.8 or higher among 25.7% of the patients. Accordingly, scores for the GSI dropped significantly during the course period and even more during the follow-up period; changes from pre- to post-and follow-up tests yielded Cohen’s ds ranging from -.54 through -.88 changes in sample mean scores. For the GSI pre-post changes yielded medium individual effect size scores of 0.5 or higher among 57.1% of the patients, and large effect size scores of 0.8 or higher among 28.6%. Corresponding percentages for pre-follow-up-changes were 57.7% and 34.3%, respectively. Both GAF-S and GAF-F scores showed a significant increase from pre- to post-intervention (Table 3). The number of participants categorized as having adequate or good functioning increased from 15 in the pre-test to 28 in the post-test, whereas the number of participants categorized with evident symptoms decreased from 27 to 14. GAF-S changes of 4 points or more are considered a reliable change,36 whereas changes of 10 points or more indicate change of symptomatic or functional level. Analyses showed that 71.1% had a GAF-S improvement of 4 points or more, whereas 39.5% of the patients had a S-GAF improvement of 10 points or more. Corresponding percentages for GAF-F were 73.7% and 36.8%, respectively.
Table 2.
Quantitative results: mean scores and standard deviations for study variables at pre-, post-, and follow-up test, as well as results of analyses of changes from pre-test to post-test and pre-test to follow-up test.
| Pre-Test | Post-Test | Follow-Up | Pre-Post Changes | Pre-Follow-Up Changes | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| M | SD | M | SD | M | SD | N | F | p | d | N | F | p | d | |
| Mindful coping | ||||||||||||||
| Awareness | 2.93 | 0.64 | 3.48 | 0.45 | 3.40 | 0.46 | 36 | 21.31 | 0.000 | 1.01 | 32 | 18.10 | 0.000 | 0.85 |
| Distraction | 2.51 | 0.65 | 2.92 | 0.53 | 2.96 | 0.49 | 36 | 15.56 | 0.000 | 0.69 | 32 | 18.67 | 0.000 | 0.79 |
| Preventing negative emotions | 3.14 | 0.76 | 3.57 | 0.55 | 3.50 | 0.47 | 36 | 15.74 | 0.000 | 0.66 | 32 | 6.68 | 0.015 | 0.59 |
| Constructive self-assertion | 3.21 | 0.65 | 3.67 | 0.54 | 3.67 | 0.50 | 36 | 28.05 | 0.000 | 0.77 | 30 | 8.24 | 0.008 | 0.80 |
| Mental health indicators | ||||||||||||||
| Life satisfaction | 3.50 | 1.40 | 4.20 | 1.40 | 4.56 | 1.34 | 35 | 10.96 | 0.002 | 0.50 | 32 | 15.96 | 0.000 | 0.77 |
| Global Severity Index | 1.28 | 0.58 | 0.94 | 0.69 | 0.75 | 0.62 | 35 | 18.01 | 0.000 | -0.54 | 31 | 30.27 | 0.000 | -0.88 |
M, mean scores; SD, standard deviations; N, number; F, F-value; P, P-value; d, effect sizes
Table 3.
Clinical assessment: mean scores and standard deviations for study variables at pre- and post-test, as well as results of analyses of changes from pre- to post-test (N=38).
| Pre-Test | Post-Test | Pre-Post Changes | |||||
|---|---|---|---|---|---|---|---|
| M | SD | M | SD | F | p | d | |
| GAF-F | 60.39 | 9.85 | 67.82 | 9.91 | 17.84 | 0.000 | 0.75 |
| GAF-S | 57.26 | 7.70 | 65.55 | 9.29 | 38.46 | 0.000 | 0.98 |
GAF-F, level of functioning; GAF-S, level of symptoms; M, mean scores; SD, standard deviations; N, number; F, F-value;P, P-value; d, effect sizes
Discussion
The main objective of the current research was to evaluate the effectiveness of the mindfulness-based coping program using a mixed model design. Research questions related to the perceived usefulness of skills, utilization of skills, and changes in psychological well-being.
Useful mindful coping skills and skills used in daily life
Overall, the qualitative findings indicate that participants perceived skills related to awareness and constructive self-distraction as most useful. This finding probably reflects that the mindfulness aspect is the core of the program and hence were introduced initially and continuously practiced and discussed throughout the program. It is also possible that awareness and constructive self-distraction skills were perceived as the most novel and unique elements of the intervention, compared to other approaches to coping in which the patients may have participated.
Concerning the utilization of awareness and constructive self-distraction skills, findings were more mixed. The quantitative data suggest a significant increase in the use of awareness and self-distraction from pre- to post-intervention, an increase that was maintained in the follow-up assessment three months after the intervention ended. On the other hand, qualitative data contained markedly fewer references to the use of awareness in examples from daily life and constructive self-distraction skills than to the perceived usefulness of such skills. Regarding awareness, findings could reflect that awareness is a relatively abstract state of mind that can be challenging to implement in stressful situations. The discrepancy between reported usefulness and examples of use could also be due to the fact that the ability to be in the present moment, as well as to tolerate stressful feelings and bodily sensations, could be closely related to the individual’s personality or even biological features45,46 and, therefore, difficult to change. Similarly, complicated problems and resistance,47 highly avoidant patients,48 and other difficulties related to pathology28 in CBT interventions may hinder patients from tuning in to their own experiences. These arguments may also apply to constructive self-distraction for chronic distress. However, findings are a bit puzzling, since distraction skills for acute distress were designed to be concrete and easy to implement in daily life situations.
Findings indicate that participants see potential for the use of awareness and constructive self-distraction skills, although some modifications of the MBC program may be necessary to make these skills more applicable to participants. Mainly drawing from experiences with CBT, Lambert47 distilled the motivational patient factor as one main factor promoting nonresponse to treatment, especially in regard to homework assignments. This is supported elsewhere.48 Hence, an emphasis on educative aspects within MBC could be adequate to increase applicability of skills presented. Therefore, a potential fruitful perspective on the aspect of motivating for change is Roger’s49 learner-centered teaching, emphasizing among other factors a focus on background and experiences of the learner, making the learning process relevant to the learner, and facilitating open-mindedness for learning. Managing motivation and ambivalence in a group format has been called a therapeutic art and may be challenging.50 Suggestions for improving memory of coping skills learned include having clients take notes during sessions, writing reminders regarding homework, using written assignments between sessions, and engaging in behavioral experiments;48 in other words, one should focus on the program as a learning process rather than a therapeutic intervention. This can perhaps inspire MBC group leaders to focus even more on the learning process and to use pedagogic initiatives to make awareness and constructive self-distraction more transferable from the course setting and, hence, more applicable in the real world. The relational and social context in which patients make sense of difficulties conflicting with CBT models can make it problematic for participants to generalize what they have learned.28 One modification regarding MBC could be to ensure that group leaders are more cognizant that awareness and constructive self-distraction skills can be difficult to apply in stressful situations between sessions. The difficulty of changing long-established habits in a short time is another challenge learned from CBT.28 Within MBC, one should probably put stronger emphasis on establishing concrete and realistic goals for behavior change and encourage participants to practice between meetings, in both relaxed and stressful situations. A stronger emphasis on smaller goals between sessions (e.g., improve relational skills) as well as having a superior goal with participation (e.g., acceptance of stressor such as diagnosis) may help participants reach their goals of change step by step and increase their sense of mastery. Furthermore, emphasizing mindfulness practices between sessions may make awareness more applicable to participants.
Findings concerning the prevention of negative emotions and constructive self-assertion demonstrated a somewhat different pattern. As for awareness and self-distraction, the quantitative data suggest a significant increase in the use of these coping strategies from before to after the intervention, whereas the qualitative data indicate moderate perceived usefulness of coping strategies related to the prevention of negative emotions and constructive self-assertion compared to awareness and self-distraction. Moreover, for constructive self-assertion, references to examples of use were at the same level as for awareness and self-distraction, whereas references to examples of use of strategies to prevent negative emotions were few, with perceived usefulness more frequent than for awareness and self-distraction. In this way, the qualitative findings suggest that participants found these aspects of the MBC program less appealing than awareness and self-distraction, but less difficult to apply when they realized the relevance of these approaches to coping. Regarding preventing negative emotions, the low reference rate may be due to the more theoretical design of these subjects in the MBC program compared to other subjects. Another reason for the few references regarding usefulness may be that the more concrete skills for preventing negative emotions, such as working out and eating and sleeping right, are skills participants were familiar with prior to the program and, therefore, they may not have seen the use in relearning these skills. The same may apply to constructive self-assertion skills. The relatively high use of constructive self-assertion may be because communication skills are of a practical and concrete nature and participants can relatively easily put them to use in real-life situations. Results suggest that in contrast to the themes of awareness and self-distraction, the challenge related to the topics of preventing negative emotions and constructive self-assertion is making these topics more appealing. Stimulating positive emotions to a higher degree within preventing negative emotions as well as increasing the numbers of cases discussed within constructive self-assertion could be fruitful initiatives to accommodate this goal.
Regarding the contexts for using skills, the findings suggest that skills were used both for intra- and interpersonal problems, specifically in terms of how to express one’s meaning and reduce rumination. Participants reported mainly using skills from the program in everyday situations, with an effort to cope with difficulties regarding work situations, school-related problems, and/or problems with family and friends. The use of skills reported in participant accounts were for coping with both emotional distress, such as when dreading work for different reasons, and self-assertion in interaction with others, such as being more confident in one’s own opinions and/or feelings and expressing these opinions and feelings. Furthermore, several participants mentioned a mix of both skills, such as being able to stress less regarding amount of work and hence being able to structure and also share workloads with colleagues. Others mentioned being aware of one’s own needs in the family and not only doing chores for and helping others, but also being selfish in a positive manner (Mira). This may indicate that the MBC program provided participants with skills for general and everyday problems, helping them in both emotion-based and problem-focused coping. However, some participants’ accounts also pointed to skills helpful in more specific contexts, such as Maria’s working to accept her son leaving for Afghanistan, or skills practiced with the objective of coping through acceptance of diagnoses or other areas experienced as difficult in that participants avoided avoidance and faced their problems.
Changes in perceived life satisfaction and psychological symptoms
Both qualitative and quantitative results indicate that participants experienced better functioning, increased life satisfaction, and reduced psychological symptoms after the intervention, and quantitative data indicate that this improvement was maintained or even increased in the three-month follow-up period. It was not possible to include a control group in the present study and one should, therefore, be cautious in attributing this change to the MBC intervention. One cannot rule out that the positive change in psychological functioning is due to naturally occurring fluctuations or changes in participants’ life situation (e.g., history or maturation).51 Analyses of individual effects showed that almost half of the patients experienced evident enhancement in quality of life and more than half experienced a reduction in psychological symptoms. Almost three out of four patients were scored by their respective therapists as better in terms of functioning and symptoms, and, almost half of the patients were scored by their therapists as bettering level of functioning and symptoms on the S-GAF scale. This suggests that the majority of patients experienced improvement of their condition that could be considered clinical significant. In accordance with this, in Keng et al.’s15 review of empirical studies of mindfulness’ impact on psychological health, a clear convergence of findings from correlational studies, clinical intervention studies, and laboratory-based, experimental studies suggests a positive association between mindfulness and psychological health and that mindfulness training brings about positive psychological changes such as increased subjective well-being, reduced psychological symptoms and emotional reactivity, and improved regulation of behavior. Moreover, the fact that the qualitative data in the current study demonstrated that (at least) some participants attributed changes in psychological functioning to increased use of coping skills introduced by the MBC intervention lends support to the notion that participation in MBC may have induced the positive changes in psychological symptoms, functioning, and life satisfaction. Changes in GAF scores suggest a move toward a normal score for the majority of participants, potentially pointing to clinical significance. The overall goal of clinical significance refers to data from the research that can be utilized by clinicians, however, no simple formula exists for determining the necessary amount of change experienced by the individual in order to judge the change as clinically significant.52 Here it has been shown to a suggested individual effect size to account for this, as well as, a significant group effect size. Results from the follow-up further suggest that participants learned skills they use successfully on their own after completion of the program.
Conclusions
Findings provide knowledge regarding the design of interventions that integrate mindfulness so as to promote enhanced psychological coping. One important finding is that skills related to the ability to be in the present moment tolerating stressful feelings and bodily sensations (awareness), as well as coping skills in adequate self-distraction, may be quite difficult for participants to implement in real-life situations. Interventions to promote mindful coping should, therefore, emphasize the presentation of skills that have been adapted to the needs and prior experiences of participants. It is also likely to be crucial to provide training in such skills in realistic situations. Concerning strategies to prevent negative emotions and promote constructive self-assertion, the challenge seems to be connected to presenting the strategies in ways that appeal to participants and allowing participants to realize that these strategies can be effective in preventing or coping with stressful life situations.
Despite these potentials for improvement, findings suggest that participants had a positive experience with the MBC program. Participants seemed to have learned and to some degree adopted useful coping skills. These reported changes in coping skills could underlie the observed positive changes in psychological functioning. However, to draw conclusions about the effects of MBC on psychological functioning among people with emotional or other stress-related psychological problems, studies with randomized controlled designs are warranted. A longer follow-up period would also be important to gain information regarding long-term effects of the intervention.
Acknowledgements
The authors would thank Professor Ingvard Wilhelmsen, Institute of Medicine, University of Bergen, Norway, and the specialist in psychiatry Henk Otten, Psychiatric Division, Stavanger University Hospital, Norway.
References
- 1.World Health Organization. 2010; Fact sheet N220. Available from: http://www.who.int/mediacentre/factsheets/fs220/en/ [Google Scholar]
- 2.Bang RN, Clench-Aas J.Psykisk helse i Norge. Tilstandsrapport med internasjonale sammenligninger. Folkehelseinstituttet. 2011:2 Available from: http://www.fhi.no/dokumenter/960011f100.pdf [Google Scholar]
- 3.Major EF, Dalgard OS, Mathisen KS, et al. Bedre føre var - Psykisk helse: helsefremmende og forebyggende tiltak og anbefalinger. Folkehelseinstituttet. 2011:1 Available from: http://www.fhi.no/dokumenter/1b2e13863a.pdf [Google Scholar]
- 4.Tharaldsen KB, Otten H.Mestringsteknikker for livsvansker. Mindfulnessbased coping: MbC-manual. Hertervig Forlag; 2008. [Google Scholar]
- 5.Kabat-Zinn J.Wherever you go, there you are. Mindfulness Meditation for Everyday Life. New York: Piatkus; 1994. [Google Scholar]
- 6.Cullen M.Mindfulness-based interventions: an emerging phenomenon. Mindfulness; 2011;2:186-93. [Google Scholar]
- 7.Davidson RJ, Kabat-Zinn J, Schumacher J, et al. Alterations in brain and immune function produced by mindfulness meditation. Psychosom Med 2003;65:564-70. [DOI] [PubMed] [Google Scholar]
- 8.Kabat-Zinn J.An outpatient program in behavioural medicine for chronic pain patients based on the practice of mindfulness meditation: theoretical considerations and preliminary results. Gen Hosp Psychiatry 1982;4:33-47. [DOI] [PubMed] [Google Scholar]
- 9.Kabat-Zinn J.Full catastrophe living: using the wisdom of your body and mind to face stress, pain and illness. New York: Random House; 1990. [Google Scholar]
- 10.Linehan MM.Cognitive-behavioral treatment of borderline personality disorder. New York: Guilford Press; 1993. [Google Scholar]
- 11.Linehan MM.Skills training manual for treating borderline personality disorder. New York: Guilford Press; 1993. [Google Scholar]
- 12.Hayes SC, Strosahl K, Wilson KG.Acceptance and commitment therapy: an experiential approach to behavior change. New York: Guilford Press; 1999. [Google Scholar]
- 13.Segal Z, Williams JMG, Teasdale JD.Mindfulness-based cognitive therapy for depression. A new approach to preventing relapse. New York: Guilford Press; 2002. [Google Scholar]
- 14.Hayes S.C.Acceptance and commitment therapy, relational frame theory, and the third wave of behavioural and cognitive therapies. Behav Ther 2004;35:639-65. [DOI] [PubMed] [Google Scholar]
- 15.Keng SL, Smoski MJ, Robins CJ.Effects of mindfulness on psychological health: a review of empirical studies. Clin Psychol Rev 2011;31:1041-56. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Allen NB, Blashki G, Chambers R, et al. Mindfulness-based psychotherapy: a review of conceptual foundations, empirical evidence and practical considerations. Aust N Z J Psychiatry 2006;40:285-94. [DOI] [PubMed] [Google Scholar]
- 17.Baer RA.Mindfulness training as clinical intervention: a conceptual and empirical review. Clin Psychol Sci Pr 2003;10:125-43. [Google Scholar]
- 18.Dunn BR, Hartigan JA, Mikulas WL.Concentration and mindfulness meditations: unique forms of consciousness? Appl Psychophys Biof 1999;24:147-65. [DOI] [PubMed] [Google Scholar]
- 19.Bishop SR, Lau M, Shapiro S, et al. Mindfulness: a proposed operational definition. Clin Psychol Sci Pr 2004;11:230-41. [Google Scholar]
- 20.Brown KW, Ryan RM.Perils and promise in defining and measuring mindfulness: observations from experience. Clin Psychol Sci Pr 2004;11:242-8. [Google Scholar]
- 21.Chambers R, Gullone E, Allen NB.Mindful emotion regulation: an integrative review. Clin Psychol Rev 2009;29:560-72. [DOI] [PubMed] [Google Scholar]
- 22.Singh NN, Lancioni GE, Wahler RG, et al. Mindfulness approaches in cognitive behavior therapy. Behav Cogn Psychother 2008;36:659-66. [Google Scholar]
- 23.Siemer M, Mauss I, Gross JJ.Same situation-different emotions: how appraisals shape our emotions. Emotion 2007;7:592-600. [DOI] [PubMed] [Google Scholar]
- 24.Oliver J, Brough P.Cognitive appraisal, negative affectivity, and psychological well-being. N Z J Psychol 2002;31:2-7. [Google Scholar]
- 25.Lequerica AH, Forchheimer M, Albrught KJ, et al. Stress appraisal in women with spinal cord injury: supplementary findings through mixed methods. Int J Stress Manag 2010;17:259-75. [Google Scholar]
- 26.Lazarus RS.Stress and emotion: a new synthesis. London: Free Association Books; 1999. [Google Scholar]
- 27.Salovey P, Bedell BT, Detweiler JB, Mayer JD.Coping intelligently: emotional intelligence and the coping process. In: Snyder CR.Coping The psychology of what works. New York: Oxford University Press; 1999. pp 141-164. [Google Scholar]
- 28.McManus F, Peerbhoy D, Larkin M, Clark DM.Learning to change a way of being: an interpretative phenomenological perspective on cognitive therapy for social phobia. J Anxiety Disord 2010;24:581-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Persons JB, Silberschatz G.Are results of randomized controlled trials useful to psychotherapists? J Consult Clin Psychol 1998;66:126-35. [DOI] [PubMed] [Google Scholar]
- 30.Borkovec TD, Castonguay LG.What is scientific meaning of empirically supported therapy? J Consult Clin Psychol 1998;66:136-42. [DOI] [PubMed] [Google Scholar]
- 31.Creswell JW.Research design. Qualitative, quantitative, and mixed methods approaches. Publications Inc; 2009. [Google Scholar]
- 32.Johnson RB, Onwuegbuzie AJ.Mixed methods research: a research paradigm whose time has come. Educ Res 2004;33:14-26. [Google Scholar]
- 33.Tingnæs G.Mestringsbok ved posttraumatisk stresslidelse. Stiftelsen Psykiatrisk Opplysning; 2005. [Google Scholar]
- 34.Karterud S, Pedersen G, Loevdahl H, Friis S.Global assessment of functioning - split version (S-GAF): background and scoring manual. Oslo: Department of Psychiatry, Ullevaal University Hospital; 1998. [Google Scholar]
- 35.American Psychiatric Association. Diagnostic and statistical manual of mental disorders. Fourth Edition Text Revision (DSM-IV-TR). Arlington, VA: American Psychiatric Association; 2000. [Google Scholar]
- 36.Pedersen G, Hagtvet KA, Karterud S.Generalizability studies of the global assessment of functioning - split version. Compr Psychiatry 2007;48:88-94. [DOI] [PubMed] [Google Scholar]
- 37.Tharaldsen KB, Bru E.Validation of the mindful coping scale. Emotional and Behavioural Difficulties 2011;16:87-103. [Google Scholar]
- 38.Derogatis LR.SCL-90-R: Administration, scoring, and procedures manual-II. Towson, MD: Clinical Psychometric Research; 1983. [Google Scholar]
- 39.Ogles BM, Lambert MJ, Masters KS.Assessing Outcome in Clinical Practice. Massachusetts: Allyn & Bacon; 1996. [Google Scholar]
- 40.Diener E, Emmons RA, Larsen RJ, Griffin S.The satisfaction with life scale. J Pers Assess 1985;49:71-5. [DOI] [PubMed] [Google Scholar]
- 41.QSR. NVivo 8. Melbourne: QSR International; 2008. [Google Scholar]
- 42.Lu CJ, Shulman SW.Rigor and flexibility in computer-based qualitative research: introducing the coding analysis toolkit. International Journal of Multiple Research Approaches 2008;2:105-17. [Google Scholar]
- 43.Sandelowski M.Combining qualitative and quantitative sampling, data collection, and analysis techniques in mixed-method studies. Res Nurs Health 2001;23:246-55. [DOI] [PubMed] [Google Scholar]
- 44.Norusis MJ.SPSS 16.0 statistical procedures companion. US: Pearson Education; 2008. [Google Scholar]
- 45.Brown KW, Ryan RM.The benefits of being present: mindfulness and its role in psychological well-being. J Pers Soc Psychol 2003;84:822-48. [DOI] [PubMed] [Google Scholar]
- 46.Giluk TL.Mindfulness, big five personality, and affect: a meta-analysis. Pers Indiv Differ 2009;47:805-11. [Google Scholar]
- 47.Lambert MJ.What have we learned about treatment failure in empirically supported treatments? Some suggestions for practice. Cogn Behav Pract 2011;18:413-20. [Google Scholar]
- 48.Newman CF.When client’s morbid avoidance and chronic anger impede their response to cognitive-behavioral therapy for depression. Cogn Behav Pract 2011;18:350-61. [Google Scholar]
- 49.Rogers C.Client-centered therapy: its current practice, implications and theory. London: Constable; 1951. [Google Scholar]
- 50.Lowe MR, Bunnell DW, Neeren AM, et al. Evaluating the real-world effectiveness of cognitive-behavior therapy efficacy research on eating disorders: a case study from a community-based clinical setting. Int J Eat Disorder 2011;44:9-18. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 51.Cook TD, Campbell DT.Quasi-experimentation. Design and analysis issues for field settings. Boston: Houghton Mifflin Company; 1979. [Google Scholar]
- 52.Onwuegbuzie AJ, Leech NL.Enhancing the interpretation of “significant” findings: the role of mixed methods research. The Qualitative Report 2004. Available from: http://www.nova.edu/ssss/QR/QR9-4/abstract.html#onwuegbuzie [Google Scholar]
