Abstract
Background
This study examined the efficacy of the red flag screening procedure, an approach used in a largescale immersion program for the treatment of adolescent obesity (Wellspring Camps). The intention of the red flag screening was to screen out potentially highly disruptive applicants, provide an impetus for additional support when high-risk campers were admitted, and help parents develop appropriate expectations. When compared to non-flagged peers, red-flagged campers who attended camp were expected to be sent home early more frequently.
Methods
Wellspring's admissions coordinators ‘red-flagged’ potential campers whose enrollment forms showed evidence of a history of psychological distress, major medical challenges, and/or developmental delays; a licensed psychologist then determined whether or not to admit the child after talking with the identified applicant, parents, and health care providers. 554 overweight young people attended three Wellspring Camps in the summer of 2007 (445 females, 109 males, mean age = 15.8 years, mean BMI = 35.6 kg/m2, mean % overweight = 66.7%).
Results
8.7% of applicants were red-flagged (n = 48), and 8% of all of the campers (n = 44) were dismissed prematurely. As expected, a much higher percentage of red-flagged campers who were admitted to camp were dismissed prematurely (31,3% of the red-flagged campers) than were non-flagged campers (5,1%; p < 0.0005).
Conclusion
These findings support the efficacy of Wellspring's red flag screening procedure. While further improvements can be made, such a tool may be useful to other immersion programs.
Key Words: Weight loss, Obesity, Behavior problems, Immersion program
Introduction
Immersion programs remove obese teenagers from the negative influences of an obesogenic culture, surround them with adult staff members who model effective weight controlling behaviors, and provide them with appropriate education and cognitive-behavior therapy. These programs have demonstrated substantial promise [1–4]. For example, in three consecutive 6- to 12-month follow-ups, on average, participants in Wellspring Camps’ immersion program decreased the amount that they were overweight to a significant degree during the follow-up periods [1].
Managing these immersion programs, however, poses challenges which are rarely confronted by more traditional outpatient clinics. Camps function as closed units, like little towns where everyone lives together. As such, even a few especially disruptive campers can have an immediate and sustained adverse effect on many of the other participants. Disruptive campers may also contribute to a decrease in effectiveness of staff by causing sustained frustration and burnout during long and tiring days [5]. Thus, managers of immersion programs must overcome the classic dilemma faced by every school teacher: 5% of the participants can create 95% of the problems.
If this adage proves accurate, it behooves operators of immersion programs to identify as soon as possible those participants at greatest risk for becoming quite disruptive to the community. An effective screening procedure could help clinical staff identify potentially disruptive participants before they begin treatment. Depending on the nature of the problems identified during screening and the resources available in the programs, screening could result in deciding not to admit some potential participants or in creating a plan for treatment that could help manage the anticipated challenges. For example, even in therapeutic camps, accepting a camper with a recent history of violent behavior toward peers may create too much of a risk for other participants. Actively suicidal individuals might also require more intensive supervision than the kind typically available in a camp setting. On the other hand, campers who have struggled with depression, a relatively common problem sometimes caused by obesity, may benefit from the type of treatment provided in a therapeutic immersion program [1,4].
Research shows that screening techniques (e.g. brief questionnaires or behavioral rating forms completed by teachers or parents) can identify high-risk students who subsequently demonstrate substantial behavioral and academic difficulties over extended periods of time [6]. These brief screening techniques were developed primarily for schools. The screening procedures used in school settings generally have teachers complete rating scales that describe behaviors which accurately predict the development of significant problems over time (e.g. dropping out of school; problems in school). As such, these ratings measure constellations of behaviors like acting out / aggressiveness, moodiness/withdrawal, and learning difficulties [6]. The purpose of the present study was to examine the efficacy of Wellspring Camps’ screening procedure – red flag screening – within an immersion program for the treatment of adolescent obesity.
In the present case, we used prior research on factors associated with success/failure at weight control as one guideline for inclusion in red flag screening. We also relied on two theories about successful self-regulation to inform selection of items. Unlike the screening checklists used in schools which focus on a few constructs, we were looking for a broader array of factors that might predict ability to use the intervention successfully or disruptiveness to others in immersion programs.
From a strictly empirical perspective, the evidence indicates that several noteworthy factors contribute to failures in weight control efforts, including psychological distress, problems with maintenance of commitment and key self-regulatory behaviors, and poor parental support and modeling [7–10]. The theoretical approaches that we use help explain these correlations.
The healthy obsession model of effective weight control [1,7,11] posits that weight controllers must develop a very consistent and intensive way of focusing attention on the planning and executing of key target behaviors (i.e. minimizing intake of fat, maximizing activities); such a healthy obsession may be necessary to manage effectively the challenges faced by weight controllers from their own resistant biology and from the obesogenic culture in which they live. The second theory, ‘strength of self-regulation’, suggests that self-regulation can break down when demands from everyday life become overwhelming. Sjoberg's and Persson's [12] view of limitations in our self-regulatory ‘resources’ and Schmeichel's and Baumeister's [10] related notion of ‘strength of self-regulation’ provide the basis for this view of self-regulatory failure. Integrating the healthy obsession and strength of self-regulation models indicates that we would expect young people who have relatively poor psychological skills, greater problems in everyday life, and weak social support at home to have more difficulty developing the requisite healthy obsession – thereby failing to succeed in the long run at weight control.
This conceptualization and empirical evidence led us to identify a range of variables for inclusion in red flag screening. In particular, we selected items that indicated high levels of psychological distress (e.g. medications for severe psychological problems, history of hospitalization for suicide attempts, or other psychological problems), factors that might compromise self-regulatory strength during an immersion program (e.g. learning problems and challenging medical conditions), and signs of challenges at home (e.g. camper opposition to attending program).
After conducting Wellspring immersion programs for several years, it became clear that some campers – whether red-flagged or not – will significantly and repeatedly negatively impact the milieu at camp. These disruptive campers have to be sent home early for the good of the vast majority of their peers – sometimes well before the end of the summer. Therefore, the red flag screening's ability to predict the disruption potential of incoming campers is the critical test of its usefulness. If the red flag screening proves useful in predicting the disruption potential of incoming campers, then managers of immersion programs could use it to make decisions about admission and to modify existing clinical protocols to increase support for the high-risk cases who are actually admitted.
Wellspring's prior experience with the screening procedure showed that the vast majority of all of the red-flagged campers were admitted to camp after investigating the issues that caused them to be red-flagged. These individuals were viewed as high-risk campers, but they were also considered to be functioning well enough to benefit from the immersion program. If this view was accurate (meaning that the red flag procedure accurately identified high-risk campers), then we would expect more of the red-flagged campers to create disruptions at camp when compared to their non-flagged peers. Therefore, we hypothesized that red-flagged campers who were admitted to camp would be much more likely than their peers who were not red-flagged to be dismissed prematurely (i.e. sent home prior to scheduled departure dates).
Material and Methods
Participants
During the summer of 2007, 554 young people (445 female, 119 male, age = 15.8 ± 2.82 years (mean ± SD), range = 9.0–25.0 years) attended the three Wellspring Camps included in this study (Wellspring California; Wellspring New York; Wellspring Adventure Camp North Carolina). At the beginning of camp, participants averaged 66.6 ± 31% overweight (range = 20–220%) and a BMI of 35.57 ± 6.6 kg/m2 (range = 23.3–66.1 kg/m2) based on comparisons to normative data for BMIs [13]. 90% were Caucasian, and 98% were from upper socio-economic backgrounds. Wellspring's approach, which is described in greater detail elsewhere [1,7,11], includes: four sessions per week of cognitive-behavior therapy; education (e.g. culinary, nutrition); strong emphasis on consistency of self-monitoring; journaling and the development of healthy obsessions; a very low-fat diet with ability of campers to access ad lib ‘uncontrolled food’ at every meal (very low-calorie density and virtually non-fat foods like fruits, soups, salads, and yogurt); ample opportunities and encouragement to engage in a wide array of activities with recording of steps on a pedometer; educational workshops for parents, and an internet-based interactive aftercare system to promote sustained self-monitoring and to allow supportive interactions with friends from camp.
Procedures
Table 1 presents the red flag screening items. Note that this set of items represents a nominal scale. The campers either demonstrated the presence of an item or they did not. For example, they either were hospitalized for psychological problems or took the medications on the list or they did not.
Red Flag Screening
The following five steps were taken to implement red flag screening:
i) Campers and parents submitted several pages of biographical, open-ended questions as part of the application process as well as medical exam/approval forms.
ii) Wellspring clerical staff (admissions coordinators) reviewed the application materials and determined whether any of the 18 items in the red flag list (table 1) applied to each applicant. If even one item applied, that camper was red-flagged.
iii) A licensed doctoral-level psychologist, supervised by the first author (a licensed psychologist with over 30 years of experience), reviewed and investigated all red-flagged applications. If the reviewer agreed with the admission staff member's assignment of the red flag, she then talked to the parent and the potential camper; sometimes therapists or physicians were also contacted, with proper consent, if deemed helpful or necessary.
iv) The reviewer in consultation with the first author determined that three of the red-flagged applicants ought to be excluded from admission to camp. These campers were excluded because their issues were such that the reviewing and supervising psychologists both believed the campers would have great difficulties benefiting from immersion treatment and/or were highly likely to cause disruptions at camp. In all cases, referrals were made to the parents for appropriate alternative treatments.
v) Camp clinical directors and program directors at each site received documentation about the investigation and outcome of the red flag screening; however, most staff members (counselors, trainers, nutrition instructors) were unaware of the red flag status of the campers. Campers and parents were not provided with feedback about red flag screening.
Reliability of Red Flag Screening
During the summers of 2007 and 2008, 151 campers were red-flagged. The psychologists who reviewed the decisions of the admission coordinators concurred 100% of the time that the red-flagged campers met at least one of the red flag criteria. This suggests that false-positives rarely (if ever) occurred in red flag screening. To assess the probability of false-negatives, the second author and a research assistant randomly selected 20 applications. They independently agreed on red flag designations on 18/20 applications (90% reliability), 13 of which were not red-flagged. This suggests that red flag screening produced few false-negatives. Inter-item correlations were not computed because the items on the list were not expected to correlate; they reflect a diverse array of issues (from opposition to attending camp to learning difficulties to history of suicide attempts).
Documentation of which Campers Were Sent Home Early
During the course of the summer, camp directors and Wellspring administrators kept careful track of every participant who left camp early. Before a camper was allowed to leave early, camps employed every intervention feasible within a 4- to 8-week immersion program to maintain attendance and improve conditions, including changing therapy or activity groups, providing additional sessions of cognitive-behavior therapy, and establishing contingencies via behavioral contracting. Decisions to send campers home early were made by camp directors in consultation with the first author and the president of Wellspring, an experienced educator/administrator. These decisions were exclusively based on concerns about safety (e.g. suicide threats) or, much more commonly, about the apparent adverse impact that the disruptive campers were having on other participants (e.g. refusal to participate in required cognitive-behavioral treatment and activities; inappropriate behaviors like smoking or sexual acting out). Wellspring seeks to maintain the highest possible satisfaction of its clients (both parents and participants), a far more important priority for a service business than validation of a screening device. Parents rarely received pro-rated reimbursement when campers were sent home early (a key element in the contract parents signed prior to camp). Accordingly, many parents were quite distressed about their children having to go home early (i.e. not happy with Wellspring). Therefore, consideration of red flag status virtually never occurred among decision makers when determining which campers had to be sent home early.
Within 1 month after the close of each camp, camp directors provided data to the authors that identified which campers were sent home prematurely, regardless of the reason. Percentages were computed for red-flagged campers and non-flagged campers to determine proportions of those that were sent home prematurely.
Results
Of the 554 admitted campers attending the three camps studied here, 48 (8.7%) were red-flagged. A total of 44 (7.9%) campers were sent home early; of this 44, 15 (34.1%) were red-flagged before arriving at camp.
As shown in table 2, comparisons of pre-treatment characteristics showed no significant differences between campers who left early, stayed at camp, were red-flagged or non-red-flagged in age or gender (all p > 0.30). However, also as noted in table 2, a significant interaction was observed between red flag and sent home variables for % overweight. Tukey LSD comparisons showed that only among campers who were sent home, red-flagged campers were less overweight (63% overweight) initially than non-flagged campers (75% overweight; < 0.05). Regarding the primary hypothesis in this study, as shown in figure 1, 5.5 times as many campers who were red-flagged were sent home prematurely compared to campers who were not red-flagged (15/48 or 31.3% versus 29/506 or 5.7%; Z = 3.78, p = 0.0002). All three camps showed this pattern of greater percentage of red-flagged campers being sent home early compared to campers who were not red-flagged.
Four aspects of the red flag screening as a methodology were determined: sensitivity, specificity, positive predictive value, and negative predictive value.
– Sensitivity = proportion of red-flagged campers who were sent home early divided by campers who were sent home early (true-positives + false-negatives): 15/(15 + 29) = 34%.
– Specificity = proportion of non-red-flagged campers who stayed at camp (true-negatives) divided by all campers who stayed (true-negatives + false-positives): 477/(477 +33) = 94%.
– Positive predictive value = proportion of red-flagged campers who were sent home early divided by all campers who were red-flagged (true-positives + false-positives): 15/(15 + 33) = 31%.
– Negative predictive value = proportion of not red-flagged campers who stayed (true-negatives divided by all campers who were not red-flagged (true-negatives + false-negatives): 477/(29 + 477) = 94%.
Discussion
This study showed that red flag screening did, as hypothesized, identify campers who were far more likely to be sent home early than those who were not red-flagged. The data do not indicate which types of red flag criteria predicted this unfavorable outcome but point out that, in general, campers identified with various medical or psychological challenges may have greater difficulties sustaining effective behaviors within an immersion program for the treatment of adolescent obesity.
These results are consistent with prior findings in the obesity literature, showing that greater degrees of psychological distress and other signs of personal and environmental stress predict unfavorable outcomes [8,9,14]. From a theoretical perspective, this makes good sense. Sjoberg's and Persson's [12] view of self-regulatory resources and Schmeichel's and Baumeister's [10] similar notion of strength of self-regulation are consistent with these findings. When weight controllers have substantial personal barriers to manage, they will have, according to these empirically based principles, fewer resources (or less strength) available to focus on self-monitoring, cognitive behavior therapy, exerting effort in their choices of foods and activities, and devotion to success.
In addition to the kinds of personal barriers identified in red flag screening, all weight controllers within immersion programs face additional challenges. They must grapple with their own numerous biological barriers to weight loss [7,11] as well as manage the stress created by being away from home and such comforts as access to cell phones, computers, friends, family, and their usual foods. Campers who struggle with the kinds of problems identified in the red flag screening, in addition to the usual challenges for weight controllers in immersion programs, certainly risk overloading their self-regulatory systems. This can lead to problematic behaviors and apparently a relatively high incidence of expulsion from the immersion program.
Alternative explanations could account for these results. For example, bias in memory and attention has been demonstrated for more than 75 years [15]. People often selectively attend to and remember information that is consistent with their expectations and beliefs. Perhaps camp staff viewed the behavior of red-flagged campers selectively along these lines, resulting in overly negative perceptions of them and pessimistic views of their abilities to adapt to camp. Arguments against this explanation, however, include the fact that most of the staff at the camps were unaware of the red flag status of the campers and that administrators clearly and consistently did all that they could to maintain every camper for the duration of enrollment. The latter is based on the policy espoused and promoted in Wellspring, but it is, of course, anecdotal in nature. On the other hand, the fact is that part of Wellspring's substantial growth in just a few years is likely attributable to providing high-quality customer service, as evidenced by testimonials [16]. Sending young people home prematurely, especially to parents who have invested considerable effort and thousands of dollars toward camp, can produce very dissatisfied customers. Camp administrators who determine whether or not to expel campers quickly receive very negative reactions from some parents and from the campers themselves – another set of factors that would discourage a cavalier or biased attitude toward this important decision.
The limitations in this study include the diversity of factors listed in the red flag list, from bedwetting to blindness to developmental delay to suicide risk. More specific categorization with much larger samples (data collected over several years or with inclusion of more camps) might help specify the types of red flag indicators that require additional support at camp or that cause disruptions eventuating in early dismissal. In addition, more extensive assessment of campers would allow for analyses of which red flag factors interact with dispositional variables to predict negative outcomes. It would also be possible to determine which types of supports provided at camp would allow those with certain risk factors to benefit from the immersion experience. For example, perhaps more psychologically distressed campers would benefit from more frequent individual sessions with therapists during camp or from changes or additions to their medication regimens.
Future research could further refine the categorization of red flags, for example into psychological versus medical categories; obtain more objective evidence of the functioning of campers in immersion programs via variables such as consistency and quality of participation in the process of change, weight loss during camp, and ratings of adaptation during camp; and also study factors associated with successful adaptation to the immersion experience by the majority of red-flagged campers. It might be worthwhile to examine pre-camp differences in greater detail, including further analyses of the observed differences in initial % overweight among campers who were sent home early, as predictors of the efficacy of red flag screening. It would also be useful to examine characteristics of the red-flagged campers who were not admitted to camp versus those who were admitted.
Wellspring's red flag screening may be useful to others who provide this promising form of treatment to children, adolescents, and their families. It is the hope of the authors that this red flag screening procedure will shed light on the difficulties encountered by participants in immersion programs and that early identification of factors contributing to these difficulties will improve successful performance in such programs. By proving the hypothesis that more red-flagged campers were sent home early than non-red-flagged campers, we offer an efficient solution to identify potentially high-risk participants, with the hope that, once identified, other options may be suggested to children and adolescents who would not benefit from the immersion experience. Early identification may further save potentially high-risk campers and their families from the psychological and health consequences of a negative experience. At present, this screening procedure seems useful in advising parents about factors that can impact the likelihood of successful completion of immersion treatment, making decisions regarding admission and retention, and determining which campers may benefit from additional support during the immersion program.
Disclosure
Wellspring sponsored the research described in this article; the first and second authors are employees of Wellspring, while the third author is a part-time consultant to Wellspring.
Fig. 1.
Percentages of 2007 Wellspring campers who were sent home early based on whether or not they were red-flagged.
Table 1.
Red flag screening criteria
| Acting Out/History of Violence* |
| Autism, Asperger's disorder |
| Bedwetting |
| Biography in handwriting similar to parent's |
| Blindness or deafness |
| Bulimia |
| Cutting/self-harm |
| Developmental delay – as determined by falling behind in school by at least one year |
| Medical/physical challenges: seizures, type I diabetes, other serious medical issues; weight over 300 lbs (≈ 136 kg) and some evidence of mobility difficulties; weight over 300 lbs for an adventure camp; weight over 350 lbs for any camp |
| Medications: abilify, lithium, depakote, risperadol, geodon, lamictal, haldol |
| Opposed to attending Wellspring |
| Probation |
| Psychological problems, major: bipolar, panic attacks, psychosis |
| Serious learning disabilities |
| Significant trauma, posttraumatic stress disorder |
| Substance abuse: history of, current |
| Suicide attempts/ psychiatric hospitalizations |
| Truancy / flight risk |
Items in italics require especially careful investigation.
Table 2.
Pre-treatment gender, age and % overweight for four groups (red-flagged × sent home)
| Red-flagged / sent home (n = 15) | Red-flagged / stayed at camp (n = 33) | Non-red-flagged / sent home (n = 29) | Non-red-flagged / stayed at camp (n = 477) | |
|---|---|---|---|---|
| Gendera, % female | 84.6 | 80.6 | 73.5 | 80.6 |
| Ageb, years | 16.92 ± 4.15 | 16.17 ± 2.70 | 16.55 ± 2.96 | 15.75 ± 2.78 |
| % Overweightc | 63 ± 25 | 68 ± 33 | 75 ± 38 | 66 ± 30 |
Chi-square (gender × 4 red flag groups) (3) = 1.6, p > 0.25.
ANOVA (age) red flag × sent home, all p > 0.35.
ANOVA (% overweight) red flag × sent home, main effects = n.s.; red flag × sent home interaction F (1,537) = 4.4, p = 0.04.
Acknowledgments
Research conducted at Wellspring California, Reedley, CA; Wellspring New York, Paul Smiths, New York; Wellspring Adventure Camp-North Carolina, Asheville, NC, USA.
References
- 1.Kirschenbaum DS, Craig RD, Kelly KP, Germann JN. Immersion programs for treating pediatric obesity follow-up evaluations of Wellspring Camps and Academy of the Sierras a boarding school for overweight teens. Obes Manag. 2007;3:261–266. [Google Scholar]
- 2.Braet C, Van Winckel M. Long-term follow-up of a cognitive behavioral treatment program for obese children. Behav Ther. 2000;31:55–74. [Google Scholar]
- 3.Gately PJ, Cooke CB, Butterly RJ, Mackreth P, Carroll S. The effects of a children's summer camp programme on weight loss with a ten month follow-up. Int J Obes. 2000;24:1445–53. doi: 10.1038/sj.ijo.0801405. [DOI] [PubMed] [Google Scholar]
- 4.Braet C, Tanghe A, Decaluwe V, Moens E, Rosseel Y. Inpatient treatment for children with obesity weight loss, psychological well being and eating behavior. J Ped Psych. 2004;29:519–529. doi: 10.1093/jpepsy/jsh054. [DOI] [PubMed] [Google Scholar]
- 5.Paine WS. Beverly Hills: Sage Publications; 1982. Job Stress and Burnout. [Google Scholar]
- 6.Allen GJ, Chinsky JM, Larcen SW, Lochman JE, Selinger HV. Mahwah, New Jersey: Lawrence Erlbaum; 1976. Community Psychology in the Schools. [Google Scholar]
- 7.Kirschenbaum DS, Craig RD, Tjelmeland L. New York: Penguin Books; 2007. The Sierras Weight Loss Solution for Teens and Kids. [Google Scholar]
- 8.Goldfield GS, Raynor HA, Epstein LH, Treatment of pediatric obesity . Handbook of Obesity Treatment. In: Wadden TA, Stunkard AJ, editors. New York: Guilford; 2002. pp. pp 532–555. [Google Scholar]
- 9.Haddock CK, Shadish WR, Klesges RC, Stein RJ. Treatments for childhood and adolescent obesity. Ann Behav Med. 1994;16:235–244. [Google Scholar]
- 10.Schmeichel BJ, Baumeister RF, Self-regulatory strength . Handbook of Self-Regulation: Research, Theory, and Applications. In: Baumeister RF, Vohs KD, editors. New York: Guilford; 2004. pp. pp 84–98. [Google Scholar]
- 11.Kirschenbaum DS, The Healthy Obsession Program . Dallas: BenBella Books; 2005. Smart Weight Loss Instead of Low-Carb Lunacy. [Google Scholar]
- 12.Sjoberg L, Persson L. A study of attempts by obese patients to regulate eating. Addict Behav. 1979;4:349–359. doi: 10.1016/0306-4603(79)90006-6. [DOI] [PubMed] [Google Scholar]
- 13.National Center for Health Statistics Prevalence of overweight and obesity among adults. United States. 2000 www.cdc.gov/nchs/products/pubs/pubd/hestats/obese/obse99tab2.htm Accessed January 21, 2007. [Google Scholar]
- 14.Kramer FM, Jeffery RW, Forster JL, Snell MK. Long-term follow-up of behavioral treatment for obesity patterns of weight regain among men and women. Int J Obes. 1989;13:123–136. [PubMed] [Google Scholar]
- 15.Schacter DL. The seven sins of memory insights from psychology and cognitive neuroscience. Am Psychol. 1999;54:182–203. doi: 10.1037//0003-066x.54.3.182. [DOI] [PubMed] [Google Scholar]
- 16.‘Campers talk about their life-changing experiences’. www.wellspringcamps.com/campers.html