Abstract
Behavioral medicine training is due for an overhaul given the rapid evolution of the field, including a tight funding climate, changing job prospects, and new research and industry collaborations. The purpose of the present study was to collect responses from trainee and practicing members of a multidisciplinary professional society about their perceptions of behavioral medicine training and their suggestions for changes to training for future behavioral medicine scientists and practitioners. A total of 162 faculty and 110 students (total n=272) completed a web-based survey on strengths of their current training programs and ideas for changes. Using a mixed-methods approach, the survey findings are used to highlight seven key areas for improved preparation of the next generation of behavioral medicine scientists and practitioners, which are grant writing, interdisciplinary teamwork, advanced statistics and methods, evolving research program, publishable products from coursework, evolution and use of theory, and non-traditional career paths.
Keywords: behavioral medicine, academic training, experiential learning, interdisciplinary studies, psychology, public health
Introduction
Behavioral medicine is an interdisciplinary field of research and practice that focuses on the modifiable behavioral factors contributing to a range of health behaviors and conditions, including but not limited to diet, physical activity, diabetes, cancer, HIV/AIDS, and tobacco use (Fisher et al. 2011). Though the majority of students do not receive degrees explicitly in behavioral medicine (Society of Behavioral Medicine, 2016), students instead receive degrees in fields allied to the broad field of behavioral medicine including psychology, public health, nursing, and medicine. Behavioral medicine training is offered as a focus or a part of master’s and doctoral programs, clinical psychology pre-doctoral internships, medical schools, and postdoctoral fellowships (e.g., Boston University, 2016; Cornell University, 2016; San Diego State University, 2016; University of California San Diego School of Medicine, 2016; University of Indianapolis, 2016; University of Miami College of Arts & Sciences, 2016; University of South Carolina, 2016; University of Washington School of Public Health, 2016; The Warren Alpert Medical School of Brown University, 2016; Yeshiva University, 2016). Subgroups of students self-identify with behavioral medicine due to research, clinical, or policy-related interests (Society of Behavioral Medicine, 2016). Often training encompasses courses in research methods, statistics, clinical and community settings, specialty training, and applied research opportunities, but offerings depend on institution, training site, and sub-specialty (Tulane University, 2016; University of Maryland Baltimore County, 2016).
The Society of Behavioral Medicine (SBM) is a multidisciplinary organization of scientists, clinicians, and educators dedicated to advancing knowledge about medicine (SBM, 2016). In 2010, the SBM Education, Training, and Career Development Council conducted a survey of career trajectories of 737 members currently working in behavioral medicine. According to the survey, 21.5% did not start in behavioral medicine, thus requiring additional training upon changing careers (SBM, 2010). While over half the respondents were early career (53.0%) and 73.0% were psychologists who dedicated at least half of their time to research, participants spanned other fields including epidemiology and health services research (6.0%), nursing (5.0%), medicine (2.0%), and other fields (8.0%; SBM, 2010). Based on this research, it can be concluded that behavioral medicine represents a diverse field of professionals who often require additional resources for training and career development. Given that this survey was conducted among behavioral medicine faculty and professionals years ago, and it appears no previous work has investigated behavioral medicine training trainees’ perspective, a more comprehensive investigation is overdue.
Given behavioral medicine’s interdisciplinary foci, the opportunity for interdisciplinary collaboration exists naturally, and trainees may benefit from these collaborations. Cross-training, the training of an individual in relevant tasks in another field, could take place amongst differing fields of study (e.g., psychology, public health) in behavioral medicine to help ensure that students are exposed to different methodology and theoretical approaches. Encouraging interdisciplinary training and work experiences may result in more comprehensive research and interventions: it could prepare trainees for the shifting landscape of real-world behavioral medicine partnerships.
Continuity across training programs that are housed in different departments (e.g., psychology versus public health) is lacking (Onken et al., 2014). Within types of programs (e.g., across all American Psychological Association-accredited clinical psychology doctoral programs, or across all Council of Education for Public Health-accredited doctoral programs), accrediting bodies facilitate cohesion, yet much is left to the discretion of the program (American Psychological Association, 2006; American Psychological Association, 2016; Council on Education for Public Health, 2015). Flexibility is necessary for feasibility of execution, but further modernized guidelines could establish a standard background for professionals coming to behavioral medicine from different fields. Presently, the interdisciplinary field of behavioral medicine itself lacks a set of uniform expectations for training future scientists and practitioners, and students need a more integrative view of the field (Onken et al., 2014).
Behavioral medicine training appears to need a revolution if behavioral medicine scientists and practitioners currently in training are to be adequately prepared for the future of the field. Current faculty and trainees face a need for education reform that will allow both parties to diversify and hone the skills that will continue to move the field, as well as their individual careers, forward.
The purpose of the present study was to collect responses from a diverse sample of trainee and faculty SBM members about their perceptions of behavioral medicine training gaps and needs for additional education in preparing future behavioral medicine scientists and practitioners both in research and clinical settings. The results provided a snapshot of how to evolve training in behavioral medicine from both trainee and faculty perspectives. Using a mixed-methods approach, the survey findings were used to outline suggestions for future educational directions and implementation that focus on the preparation of trainees.
Methods
Survey Composition
Two web-based surveys were developed by the study authors to identify the clinical and research training opportunities necessary to advance behavioral medicine. The SBM Executive Committee reviewed the recruitment documents and survey before implementation. The first survey was targeted toward behavioral medicine faculty and professionals and comprised 16 items. The second survey was designed for graduate students and post-doctoral trainees (herein cumulatively referred to as “trainees”) and included 18 items. Both surveys contained closed-and opened-ended items and included demographic questions followed by questions pertaining to future behavioral medicine training needs. A free text response option was included in most questions to capture those responses not reflected in the given options; these results were categorized by the study authors following the survey implementation period. The areas of interest for behavioral medicine research and intervention were obtained from the SBM website (SBM, 2015). The web-based surveys were administered via Yale Qualtrics, a secured online survey engine.
Procedures
The web-based survey was circulated to current trainees (i.e., those in master’s, professional, or doctoral programs as well as post-doctoral trainees) and practicing professionals (i.e., academics, professors, or health professionals such as psychologists or physicians) who were current SBM members. The inclusion criteria comprised that the respondent was a member of SBM, 18 years of age or older, and had access to a computer, tablet, or smartphone with internet accessibility. Instructions and a hyperlink to access the surveys were circulated through SBM newsletters and Listservs from October 28, 2015 to November 16, 2015. Given the anonymous nature of the survey distribution, it is unknown how many individuals received the invitation to participate in the surveys. We sought to recruit 100 respondents (50 trainees and 50 faculty/professionals) to complete the surveys. Given that SBM membership was part of the recruitment strategy and trainees across programs with and without clinical components were being targeted, it was understood that this survey would likely be most reflective of individuals heavily involved in research and academic pursuits.
Human Subjects Considerations
The survey and recruitment materials were approved by the Yale University Human Subjects Committee and the SBM Executive Committee. Participation was voluntary. No identifiable information was sought and results were reported in the aggregate, thus, confidentiality was ensured. The consent form was posted on the first page of the online survey and respondents had to click “agree” before starting the survey.
Analyses
Data were checked for missing values, distribution, and normality. Those surveys that were not at least 50.0% complete were excluded from analysis (faculty survey: n=10; trainee survey: n=8). Respondents of all excluded surveys had clicked “agree” but did not respond to any of the items. Descriptive statistics and bivariate analyses were calculated for closed-ended questions. We used qualitative content analysis methods to analyze the open-ended responses. The content analysis approach entailed systematically coding and categorizing a set of data to generate findings (Vaismoradi et al., 2013). All quantitative analyses were performed using STATA version 14.0 (STATA®, College Station. TX), and findings at p<.05 were considered statistically significant.
Results
Participant Characteristics
A total of 162 faculty and 110 trainees (total n=272) completed the web-based survey over the course of 19 days. Participant characteristics are reported separately for faculty (Table 1) and trainee (Table 2) respondents. Overall, a larger proportion of women than men responded to the survey for both faculty (72.3%) and trainees (83.6%; p<.0001 for both). The majority of faculty respondents was affiliated with institutions located in the Midwest (28.9%), Mid-Atlantic (17.6%), and Northeast (15.7%), whereas the majority of trainees were affiliated with institutions in the South (27.3%), Midwest (26.4%), and West (18.2%). When compared with membership statistics collected by SBM (E. Linc, personal communication, January 4, 2016), the respondents to the present survey were similar to the SBM 2014 membership with few exceptions. In the present sample, there were fewer faculty from the South (compared to 20.9% in the 2014 membership) and more from the Midwest (19.1% in 2014), and there were fewer students in this sample from the Mid-Atlantic region (22.7% in 2014). The distribution of gender in this sample was similar to the gender distribution of the 2014 SBM membership. The vast majority of faculty respondents held a Ph.D. (93.0%), while almost two-thirds and one-fifth of trainee respondents were enrolled in a Ph.D. and post-doctoral program, respectively. The top five areas of interest in behavioral medicine were similar among faculty and trainees: physical activity, cancer, obesity, public health, and minority health. Similarly, half of faculty and trainee respondents had or were pursuing a clinical degree.
Table 1.
Characteristic | Total (n=162) n (%) |
|
---|---|---|
Woman | 118 (72.3) | |
Region* | ||
Northeast | 25 (15.7) | |
Mid-Atlantic | 28 (17.6) | |
South | 19 (12.0) | |
Midwest | 46 (28.9) | |
Southwest | 14 (8.8) | |
West | 17 (10.7) | |
Outside US | 10 (6.3) | |
Highest Academic Degree | ||
PhD | 151 (93.2) | |
MD/DO | 4 (2.5) | |
PsyD | 4 (2.5) | |
DrPH | 2 (1.2) | |
EdD | 1 (0.6) | |
Department/Academic School | ||
Medicine | 63 (38.9) | |
Psychology | 28 (17.3) | |
Public Health | 24 (14.8) | |
Other | 18 (11.1) | |
Nursing | 12 (7.4) | |
Kinesiology | 6 (3.7) | |
Nutrition/Exercise Science | 9 (5.6) | |
Human Development | 2 (1.2) | |
Licensed Clinician | 81 (50.0) | |
Post-terminal Degree Position | ||
Post-doc | 100 (61.7) | |
Junior Faculty | 39 (24.1) | |
Research Scientist | 12 (7.4) | |
Clinical Practice | 9 (5.6) | |
Nonacademic Position | 2 (1.2) | |
Tenured or Tenure-Track | 100 (62.5) | |
Academic Career Stage | ||
Assistant Professor | 72 (45.9) | |
Associate Professor | 34 (21.7) | |
Full Professor | 41 (26.1) | |
Non-Tenure Track | 10 (6.4) | |
Primary Academic Responsibilities | ||
Research | 140 (53.2) | |
Teaching | 79 (30.0) | |
Administration | 24 (9.1) | |
Clinical | 13 (4.9) | |
Service | 5 (1.9) | |
Consultant | 2 (0.8) | |
Currently Mentored Predoctoral Trainees, n | ||
0 | 63 (39.9) | |
1 | 38 (24.1) | |
2–4 | 38 (24.1) | |
5+ | 19 (12.0) | |
Position Trainees Pursue Post-degree | ||
Post-doc | 70 (49.0) | |
Junior Faculty | 28 (19.6) | |
Clinical Practice | 19 (13.3) | |
Nonacademic Position | 16 (11.2) | |
Research Scientist | 10 (7.0) |
US States by Census Region: Northeast (CT, MA, ME, NH, RI, VT); Mid-Atlantic (DE, MD, NJ, NY, PA, DC); South (AL, AR, FL, GA, KY, LA, MS, NC, SC, TN, VA, WA, WV); Midwest (IL, IN, IA, KS, MI, MN, MO, NE, ND, OH, SD, WI); Southwest (AZ, NM, OK, TX); West (AK, CO, CA, HI, ID, MT, NV, OR, UT, WA, WY)
Table 2.
Characteristic | Total (n=110) n (%) |
|
---|---|---|
Woman | 92 (83.6) | |
Age Range | ||
18–25 | 17 (15.5) | |
26–30 | 60 (54.6) | |
31–35 | 20 (18.2) | |
> 35 | 13 (11.8) | |
Region* | ||
Northeast | 10 (9.1) | |
Mid-Atlantic | 12 (10.9) | |
South | 30 (27.3) | |
Midwest | 29 (26.4) | |
Southwest | 7 (6.4) | |
West | 20 (18.2) | |
Outside US | 2 (1.8) | |
Current Degree Program | ||
PhD | 75 (68.2) | |
Post-doc | 24 (21.8) | |
PsyD | 5 (4.6) | |
MPH | 3 (2.7) | |
MS | 2 (1.8) | |
DrPH | 1 (0.9) | |
Department/Academic School | ||
Psychology | 57 (52.3) | |
Public Health | 17 (15.6) | |
Medicine | 12 (11.0) | |
Kinesiology | 8 (7.3) | |
Other | 6 (5.5) | |
Nursing | 4 (3.7) | |
Nutrition/Exercise Science | 3 (2.8) | |
Human Development | 2 (1.8) | |
≥ 50% Program Completed | 74 (67.3) | |
Pursuing Clinical Degree | 53 (48.2) | |
Training Fellowship (e.g., T32, F31, F32) | 23 (20.9) | |
Stipend Received for Training | 94 (85.5) | |
Intended Post-degree Position | ||
Post-doc | 54 (50.9) | |
Junior Faculty | 23 (21.7) | |
Research Scientist | 12 (11.3) | |
Clinical Practice | 3 (2.8) | |
Nonacademic | 14 (13.2) | |
Ideal Post-degree Academic Responsibility | ||
Research | 51 (50.0) | |
Teaching | 7 (6.9) | |
Clinical | 13 (12.8) | |
Not Applicable | 31 (30.4) | |
Nonacademic Position Career Track | ||
Government | 15 (33.3) | |
Consulting | 13 (28.9) | |
Industry | 7 (15.6) | |
Non-profit | 4 (8.9) | |
Nonacademic Clinician | 3 (6.7) | |
Think Tank | 3 (6.7) |
US States by Census Region: Northeast (CT, MA, ME, NH, RI, VT); Mid-Atlantic (DE, MD, NJ, NY, PA, DC); South (AL, AR, FL, GA, KY, LA, MS, NC, SC, TN, VA, WA, WV); Midwest (IL, IN, IA, KS, MI, MN, MO, NE, ND, OH, SD, WI); Southwest (AZ, NM, OK, TX); West (AK, CO, CA, HI, ID, MT, NV, OR, UT, WA, WY)
Post-degree Considerations
Almost two-thirds of faculty respondents completed a post-doctoral training program and felt that about half (49.5%) of trainees in their program would pursue this position post-degree. Similarly, 51.4% of trainees reported the intention to pursue a post-doctoral training program (p=0.700). There were also equal proportions of faculty (19.6%) and trainees (21.9%) who reported pursuing or planning to pursue a junior faculty position following graduation (p=0.655). Over half of faculty reported research as their primary responsibility and a similar response was noted among trainees interested in pursuing academia. Almost one in three faculty members reported teaching as their primary responsibility, whereas 6.9% of trainees sought teaching as a primary responsibility for an academic position. Faculty and trainees reported anticipating that 7.0% and 10.5% (p=0.330), and 11.2% and 13.3% (p=0.609) of trainees would pursue either a research scientist or nonacademic position post-degree, respectively. Trainees intending to pursue nonacademic positions were most interested in government (33.3%), consulting (28.9%), and industry (15.6%). Significant differences were observed among faculty and trainee perceptions of the percentage of trainees interested in pursuing a clinical position post-degree (13.3% faculty vs. 2.8% trainees, p<0.01).
Perceptions of the Future of Behavioral Medicine
The majority of both faculty (73.9%) and trainee (85.0%) respondents agreed or strongly agreed that the program they are affiliated with will adequately train trainees to become successful behavioral medicine researchers or clinicians (Table 3; p=0.318). Trainees felt that at present, training programs were best at preparing them in: advanced statistical and research methods, mentorship and advising, exposure to theory and theory development, grant writing, and scientific writing (26.1%, 16.4%, 9.4%, 9.4%, and 6.7% of responses, respectively).
Table 3.
Item | Faculty (n=162) n (%) |
Trainee (n=110) n (%) |
|
---|---|---|---|
How much do you agree with the
following statement: “The program I am affiliated with will adequately train my trainees to become successful behavioral medicine researchers or clinicians” |
|||
Strongly Disagree | 9 (6.2) | 3 (2.8) | |
Disagree | 5 (3.4) | 3 (2.8) | |
Neither Agree nor Disagree | 24 (16.4) | 10 (9.4) | |
Agree | 64 (43.8) | 52 (49.1) | |
Strongly Agree | 44 (30.1) | 38 (35.9) | |
Ways in which clinical training
needs to evolve to prepare students to be modern, successful behavioral medicine specialists |
|||
Interdisciplinary/multidisciplinary clinical teams | 50 (30.9) | 52 (32.1) | |
Integrating technology in clinical practice | 27 (21.4) | 29 (17.9) | |
Training in policy/advocacy | 15 (11.9) | 28 (17.3) | |
Communication with advisors | 12 (9.5) | 16 (9.9) | |
Seeking nontraditional funding for practice | 11 (8.7) | 20 (12.3) | |
Other | 6 (4.8) | 6 (3.7) | |
Applying to psychology internships | 4 (3.2) | 11 (6.8) |
The top capacities (defined as >50% of categories respondents’ selected) in which faculty felt training should evolve to prepare trainees to be modern, successful behavioral medicine researchers and practitioners (Table 4) included: instruction in grant writing, training in multidisciplinary team science, and training in advanced statistical and research methods. Trainees most highly endorsed instruction in grant writing and training in advanced statistical and research methods. There were significant differences between faculty and trainees with faculty having a greater emphasis on the importance of exposure to theory and theory development and building an evolving program of research (both p<0.05). Trainee perceptions significantly differed from those of faculty in that trainees endorsed the importance of developing publishable manuscripts from coursework and exploration of nontraditional careers in behavioral medicine (both p<0.05). Among programs that have a clinical training component, respondents felt the top domains in which clinical education needs to evolve include: training on interdisciplinary or multidisciplinary clinical teams, integrating technology into clinical practice, and training in policy or advocacy.
Table 4.
Faculty | Trainee | p-value | |||
---|---|---|---|---|---|
Item | n | % | n | % | |
Advanced statistical and research methods | 96 | 59.3% | 65 | 59.6% | 0.774 |
Exposure to theory and theory development | 75 | 46.3% | 30 | 27.5% | 0.002 |
Integrating technology in research | 38 | 23.5% | 35 | 32.1% | 0.172 |
Training in policy | 20 | 12.4% | 19 | 17.4% | 0.182 |
Grant writing | 107 | 66.1% | 73 | 67.0% | 0.919 |
Serve as a reviewer of a scientific journal | 15 | 9.3% | 19 | 17.4% | 0.050 |
Multidisciplinary team science | 81 | 50.0% | 52 | 47.7% | 0.659 |
Seeking nontraditional funding for
academic and research efforts |
27 | 16.7% | 20 | 18.3% | 0.746 |
Experience developing and delivering
lectures to students |
10 | 6.2% | 12 | 11.0% | 0.160 |
Training in mentoring undergraduate
and graduate students |
18 | 11.1% | 13 | 11.9% | 0.958 |
Training in supervision of research
project staff |
24 | 14.8% | 12 | 11.0% | 0.244 |
Time management | 29 | 17.9% | 18 | 16.5% | 0.597 |
Networking | 29 | 17.9% | 24 | 22.0% | 0.536 |
Media awareness and public engagement
(e.g., press releases, blogging, social media, speak and present effectively in public) |
22 | 13.6% | 19 | 17.4% | 0.525 |
Developing publishable manuscripts
from coursework |
26 | 16.1% | 44 | 40.4% | <0.001 |
Job searches (academic and nonacademic) | 19 | 11.7% | 19 | 17.4% | 0.136 |
Communication with advisors | 4 | 2.5% | 4 | 3.7% | 0.895 |
Nontraditional careers in behavioral medicine | 21 | 13.0% | 36 | 33.0% | <0.001 |
Building an evolving program of research | 75 | 46.3% | 32 | 29.4% | 0.003 |
No change is needed | 0 | 0.0% | 0 | 0.0% | 1.000 |
Other | 12 | 7.4% | 6 | 5.5% | 0.525 |
Several faculty and trainees had suggestions for ways to improve these areas (Appendix Table 1). For instance, on the topic of writing publishable manuscripts during a Ph.D. program, one trainee stated: “Publishing and creating reports that are of high caliber take significant time. Early support to move forward on this deliverable would be very helpful.” A highly endorsed faculty topic, building an evolving program of research, was expressed by one respondent: “…Building a program of research helps trainees to have a story and be more marketable. Suggestions to improve this is tough because it depends on the individual, but enhancing a mentorship culture and graduate training around this ideal would be helpful.” In other instances, both faculty and trainees provided different perspectives about how behavioral medicine can evolve. For instance, on the topic of nontraditional careers in behavioral medicine, a faculty respondent stated, “The ACA [Affordable Care Act] is opening many doors for well-trained biomedical psychologists to contribute to new models of population health. We need to have a firm seat at the table of ACOs [Accountable Care Organizations]…” Meanwhile, a trainee felt it would be useful to apply doctoral training competencies to other fields, stating “I would like to learn more about bridging the gap between academia and industry, especially the technology sector given the push for tech use in health care.”
Discussion
The results of the present study describe the current and desired training opportunities of a representative sample of members of an interdisciplinary scientific society for practitioners and researchers in behavioral medicine. The feedback received from faculty and trainees demonstrate that professionals at all levels of their careers feel that training can be improved.
Suggestions for training targets varied between trainees and faculty, but some were consistent. For both groups, grant writing was a top selection. The interest in grant writing training and support is to be expected given the decreasing availability of funds and increasing competition. Suggestions included providing more resources to support preparing small and foundation grant applications as well as nontraditional funding sources. Trainees noted that programs should offer flexible schedules to allow time to take a grant writing course. Programs could also offer students training in grant literacy or exposure to grant components (e.g., specific aims) through existing coursework (e.g., research methods). Such training can be evaluated through competency exams (e.g., writing a mock F31 or R21 grant).
Another highly ranked area in both clinical and research training was working with interdisciplinary and multidisciplinary teams. There has been a push to work in diverse teams, which has been necessitated by the need for collective expertise on diverse topics of study or interventions. For example, health technology demands that researchers and clinicians work cooperatively with nontraditional partners such as computer scientists and engineers (Kumar, 2013). There is an increasing opportunity for scientists to learn about team science through conferences (Science of Team Science, 2016) and workshops (Stanford Medicine Office of Faculty Development and Diversity, 2015) and there are growing opportunities for students to take courses in team science, such as the course at Northwestern University that seeks to train master’s students in effective techniques for scientific inquiry, including tackling complex questions through collaboration with an interdisciplinary team (Northwestern University NUCATS Clinical and Translational Science Institute, 2015).
A traditional area of interest for students and faculty was advanced statistical training. In an era of “big data” and new methods of collecting continuous monitoring from participants in behavioral research, strong empirical skills will likely remain of great importance (Kaplan, Riley, & Mabry, 2015). Programs should continue to require related courses, but conferences can serve as outlets for the dissemination of new techniques, including pre-conference seminars on emerging techniques for study design and analysis such as those often held at the SBM Annual Meeting (SBM, 2016). Trainees and professionals are able to access training and refresh their knowledge, such as the recent rise in popularity of non-traditional study designs (e.g., multiphase optimization strategy [MOST], sequential, multiple assignment, randomized trial [SMART]; Collins et al., 2007).
There were a few significant differences in the ratings of additional areas of interest, where faculty were interested in how to build an evolving research program and increased exposure to theory, and trainees expressed the importance of learning how to publish class papers, pursuing nontraditional careers, and serving as a reviewer of a scientific journal. Both groups’ concerns are well-founded. For example, in a recent paper that characterized how much theory was used in funded cancer screening and prevention R01 proposals to the National Cancer Institute, few proposals used theory to advance the science in this area (Kobrin et al., 2015). While most funded proposals mentioned a theory or construct by name, none fully described all the components of an established theory and many mentioned constructs that were not part of the conceptual model governing the study design (2015). Faculty members may be more mindful of utilizing conceptual models as it is a component of successful grant applications, but behavioral medicine appears to under-utilize complete theories to drive research; trainees may be less aware of this issue or the impact thereof.
Opportunities to publish coursework and other projects was an area of interest for trainees but was endorsed less frequently by faculty. One option is to encourage trainees to develop publishable products from coursework and other competency exercises required in graduate programs. If trainees are allowed freedom to tailor class project topics and papers to their interests, these papers can be used towards other important goals such as dissertation proposals and can be transformed into publishable manuscripts. The course instructor may also merit authorship on manuscripts resulting from coursework, providing the opportunity for collaboration, team science, and publication. Also, doctoral competency exams are an important part of ensuring only qualified trainees obtain degrees, but exams can be time-consuming. Though some programs offer students options (e.g., a comprehensive paper or an exam), and others try to make the exam more useful (e.g., the exam is turned into the introduction of the dissertation), these exams may limit time spent generating products that will prepare the candidate for the next career phase. Some programs allow trainees to design independent studies, and this can be an opportunity for students to write a publishable manuscript.
These discrepancies may reflect the new pressures and opportunities that trainees face that differ from their mentors’ experiences. As large shifts are occurring in healthcare, federal funding for research, and health technology, there are more paths available to trainees than before. Yet, many trainees are trained by mentors who took a traditional route through academia and have minimal experience with nontraditional job paths or how to advise trainees in the process to pursue them. The Chronicles of Higher Education recently published an advice column issuing a call to action for faculty to better prepare themselves to assist their trainees in the face of a changing job market (Lay, 2015). While most behavioral medicine trainees are adequately prepared for the careers of their mentors, the beauty of the versatility of a behavioral medicine degree is lost to under-preparation for working in non-traditional settings. Furthermore, many trainees may be interested in making professional contributions outside their graduate program, hence trainees’ increased interest in serving as reviewers of a scientific journal. It is also possible that given the increased pressure to publish during graduate school, trainees want to take advantage of this important service opportunity because it will further their own training goals related to scientific writing.
Trainees expressed interest in gaining experience integrating technology in research, an item that was slightly less endorsed by faculty respondents (though this difference was not statistically significant). Similar to the discrepancy in endorsement of the item about non-traditional careers, technology is an emerging focus of behavioral medicine and likely not a part of many training programs. However, there is a rising call for more of a focus on the use of technology in health care across wide platforms. A recent editorial in the Journal of the American Medical Association highlighted the need for development of this area, such as increased collaboration between health care professionals and non-traditional partners including designers and engineers to harness the power of technology for health (Thibault, 2015). In the era of wearables, such as Fitbits, that results in the “quantified self”, this is an area ripe for behavioral medicine training and subsequent research.
The call for revisiting training curricula has been raised through several recent editorials. A recent issue of the American Journal of Public Health was dedicated to “Innovations in Public Health Education,” calling for a push to move beyond traditional classroom-based education (Thibault, 2015). One editorial focused on students learning to interact effectively with a range of stakeholders in making public health decisions, including the “7 P’s of public health”: policymakers, purchases, penurious budget officials, providers, passionate advocates, the press, and the public (Koh, 2015). Another was on updating curricula on interprofessional education, new models for clinical education, and new educational models based on competency. Also, multiple primers have been published instructing students and early career faculty about the importance of service as a journal article reviewer (Lovejoy et al., 2011; Northridge et al., 2015).
Proposal to Revise Current Behavioral Medicine Curricula
Based on the findings of the present study, this writing group proposes the following seven changes to curricula of programs offering training to behavioral medicine graduate students and post-doctoral fellows. The suggestions are geared towards programs offering formal or informal concentrations in behavioral medicine and individual students aiming to work within behavioral medicine following graduation. These suggestions are intended to help programs prepare the next generation of behavioral medicine scientists and practitioners.
Grant Writing: Include a course to improve grant literacy and grant writing skills to traditional and non-traditional funding sources for conducting research, community outreach, and provision of clinical and advocacy services.
Interdisciplinary Team Work: Interdisciplinary team science experience should be mandatory for graduation in the form of research collaboration, participation in a treatment team, or another relevant project. Ideally, this topic should be addressed from a variety of perspectives in coursework and practica.
Advanced Statistics and Methods: Offer advanced quantitative and qualitative research and statistical methods semester-long or short courses. Alternatively, departments could subsidize students’ workshop attendance in other departments.
Evolving Program of Research: Advise students from the start about how to build an evolving program of research. This can be addressed by a trainee’s mentor(s) and/or a core research course.
Publishable Products from Coursework: Trainees need guidance on how to develop publishable products from coursework, and this should be a core aim of required and elective courses. For some students, the end goal should be publications in peer-reviewed journals. For others, this can include presentations to local, regional, or national groups (both lay and professional), newsletter columns, blog posts, and programs implemented in community, school, judicial, or healthcare organizations.
Evolution and Use of Theory: Exposure to theory and theory development should be interwoven throughout graduate education. Trainees should be encouraged to develop research projects grounded in behavioral theory, to learn about theory development and psychometrics, and to incorporate behavioral theory into work with clients.
Non-Traditional Career Paths: Trainees should be exposed to nontraditional careers in behavioral medicine through panel discussions consisting of speakers from outside their departments, guidance through an institution’s career services, and networking opportunities at professional conferences such as the SBM Annual Meeting.
Anticipated Challenges and Suggestions for Dissemination of Curriculum Revisions
There will be challenges to disseminating and implementing these suggestions. Through outlining challenges and potential solutions, the process of implementing these changes will hopefully be more feasible for most graduate training programs.
One concern for implementing these curriculum revisions is the hardship placed on department faculty; some programs could require overhaul and may require resources beyond a single department. Departments should seek opportunities to collaborate with other departments on campus and within the region to offer a wide breadth of coursework. Through programming offered by SBM and Special Interest Groups (SIGs) at the SBM Annual Meeting, materials circulated online, and the SBM website, networks of professionals can expose trainees to resources a single university could not contain. Through workshops at the annual meeting, mentoring programs, and Listservs, trainees can access professionals at other institutions for consultation, mentorship, and networking. Recently, the American Psychological Association’s Society for Health Psychology created an online graduate course in integrated primary care psychology for faculty members to present to graduate students (American Psychological Association, 2016); this high-quality resource is an innovative opportunity for graduate programs to spare resources (e.g., coursework preparation time) while offering trainees high-quality training from a trusted organization free of charge.
Other training programs are in remote areas, and they may lack proximity to resources, such as internship sites (e.g., technology companies or think tanks) or a large local network of behavioral medicine scientists and practitioners. Technology makes it possible for students to collaborate with professionals from a distance. Alternatively, some students could spend a semester or year collaborating with others off-campus or doing an internship in their ideal work setting. Otherwise, students could moonlight part-time for a company (with their department’s approval) or contribute from campus utilizing conference calls, Skype meetings, and other platforms that facilitate distance learning, communication, and collaboration.
A challenge to the push for increased grant writing opportunities is that most graduate students are not allowed to be the principal investigator (PI) of most grants. Graduate students should be encouraged to apply for larger grant mechanisms (e.g., F31 from NIH) with support from their departments, but departments should draw significant attention to developing grantsmanship skills by encouraging trainees to apply for small, local funding mechanisms (e.g., a grant offered by the department or graduate student association, professional organizations’ dissertation awards). These options can be less competitive and offer funds to execute a small-scale research study, such as a dissertation or pilot work for a larger grant application. Grantsmanship can be covered in a research methods course, and departments should consider a formal system that recognizes faculty mentoring of student grant writing as a mentoring or teaching activity, so as to offer protected time and incentive to faculty for this added responsibility. Entire courses focused on grant writing can incorporate students from various departments so that such courses attract adequate enrollment.
Many of the suggestions included in this paper will take time for programs to implement, and many of the suggestions themselves are time-intensive. These are most likely to be accomplished through gradually revising curricula. We highly recommend that departments collaborate with other departments or universities. National organizations, like SBM, should consider developing coursework that could be offered across a variety of programs as an alternative to the currently offered pre-conference workshops (which are not always convenient or financially feasible for all students; SBM, 2015). SBM could develop an online grant writing seminar focused on applying for NIH funding for behavioral medicine. Students or departments could pay for this course, where students could take the course on their own schedules. Other departments and centers, such as The Methodology Center of Pennsylvania State University, could consider adapting some of their existing workshops to be marketed to students (The Pennsylvania State University, 2015); flexible timelines (e.g., spread out over the semester) and payment structures (e.g., an entire department purchases a site license to give their students access to the course to supplement their own course offerings) could facilitate implementation.
While the bulk of this responsibility falls on the shoulders of master’s and doctoral programs, the changes in curricula proposed herein apply to clinical psychology pre-doctoral internships that offer behavioral medicine training experiences as well. Incoming applicants to internship programs may present with even more heterogeneous training than applicants to graduate schools. This presents challenges for creating training experiences that will be the appropriate level for all trainees, but the structure of pre-doctoral internships in clinical psychology tends to inherently offer unique benefits as well. Many internship sites make use of interdisciplinary integrated clinical teams; this hands-on experience is invaluable for trainees. Trainees should be taught about the behavioral theories that support their treatments so that they can explain them to another member of the scientific community as well as to clients (for example, Beck, 1979; Hayes, 2004; Kabat-Zinn & Hanh, 2009; Miller & Rollnick, 2012). Further, internship sites are highly encouraged to disseminate training across networks. Sites are encouraged to make this information accessible via the Internet at the trainees’ convenience, which is necessary given the implausibility of coordinating numerous trainees’ schedules across sites. For example, some of this training can be offered online through the Veterans Health Administration or a state or regional psychological association. This collaboration will be necessary at smaller sites; there is no need to reinvent the wheel when a coordinated, collaborative approach uniting internship sites can successfully meet trainees’ needs.
SBM’s Education, Training, and Career Development Council is a great resource for program and training directors and early career professionals. We recommend that interested parties meet at future annual meetings to discuss program development, methods for rolling out revised curricula, and to forge relationships with other programs in the service of resource-sharing. The Council maintains a repository of behavioral medicine syllabi and other training materials for SBM members; training directors can access these materials and share their materials with the Council. The Council should be involved in dissemination and implementation of the proposed changes with their added perspective on future training directions.
According to the Education, Training, and Career Development Council’s 2010 survey, SBM members reported that SBM could offer the following resources to address career development issues: Annual Meeting panel discussions (47%), an SBM-run web page/forum and Listserv (41%), Annual Meeting small groups (41%) and one-on-one discussions (31%), SIG-based mentoring programs (38%), and a small grant program that allows an SBM member to visit the work setting of a colleague/consultant/mentor (38%; SBM, 2010). We recommend providing these resources and training using empirically-supported learning methods. For example, a recent study of online learning of behavioral activation for community health providers demonstrated that a system including didactics, modeling, skills rehearsal, feedback on trainees’ performance (evidenced by trainees’ self-report of skills utilization, and three phone-based roleplays with a trainer [pre-training, immediately post-training, and six weeks post-training]), and discussion is associated with increased skills implementation (Puspitasari et al., 2013); it is imperative to make use of teaching methods that have produced meaningful gains in similar populations. We recommend training directors meet with the Council at annual meetings to discuss which educational targets to pursue first, methods for providing experiences, and ways to collaborate across programs in an empirically-based, efficient way. We recommend that SBM’s Student SIG and the Education, Training, and Career Development Council re-evaluate how programs are implementing these changes and what work remains to be done.
The present study must be interpreted with the following limitations. First, there may be limits to generalizability as the sample consisted of trainees and faculty who were all members of SBM. There are many trainees and professionals in the field of behavioral medicine who are not SBM members, particularly individuals working in industry or who are not involved in research; this likely explains why clinical issues were rarely mentioned in the qualitative responses. Future work should gather more information on clinical training issues by contacting directors of clinical training, medical schools, and Psy.D. programs, and collaborating with organizations that include more clinicians such as the American Psychological Association and the Association for Behavioral and Cognitive Therapies. Second, only two trainees and 10 faculty members from outside of the United States were represented in this sample. While this small contingent of international respondents relative to the total sample is similar to the SBM membership at large (E. Linc, personal communication, January 4, 2016), concerns of behavioral medicine scientists and practitioners outside of the United States were minimally represented. Future research should seek out participants in other countries to ensure that education recommendations meet the needs of the international behavioral medicine community. Moreover, once some of the changes outlined in the present study are implemented in programs of varying size and type, future research should examine barriers and solutions for putting these changes into practice.
Conclusion
The present study captured faculty and trainees’ perceptions about the evolving needs of behavioral medicine training. Most agreed or strongly agreed that the program they are affiliated with will adequately prepare trainees to become successful behavioral medicine researchers or clinicians. No respondents reported that reform is presently unnecessary. Participants highlighted that grant writing, interdisciplinary teamwork, advanced statistical and research methodology, turning coursework into manuscripts, and building a program of research are all areas for improvement. Given the changing landscape of behavioral medicine as a field, training targets need to be updated to adequately prepare trainees for careers in academia, industry, and policy.
The present study offers a snapshot of the current training needs as expressed by a sample of faculty and trainees. Future research should continue to evaluate how education is meeting the needs of trainees and the academic institutions and organizations that hire behavioral medicine scientists and practitioners. As public policy, such as the Affordable Care Act, evolves to naturally include behavioral medicine, as the role of technology in healthcare interventions expands, and as clinical interventions are created and refined to prevent disease and alleviate human suffering, the behavioral medicine workforce will be prepared to make a meaningful difference in creative ways.
Acknowledgments
The authors wish to thank the 2014–2016 executive boards of Society of Behavioral Medicine’s Student Special Interest Group, the Special Interest Group leaders who disseminated the study to their groups, and all the members that participated in the present study for their priceless time and support.
Funding:
This work was supported by NHLBI T32 HL076134-10, NIDDK T32 DK07718, and NCI R25 CA047888.
Appendix Table 1. Select Excerpts of Faculty and Trainee Suggestions of Ways to Improve Behavioral Medicine Training Capacities
Faculty Suggestions | Trainee Suggestions |
---|---|
Advanced Statistical And Research Methods | |
Workshops and mentored projects. | |
Exposure To Theory And Theory Development | |
Exposure to theory, theory
development, and how research is used for evidence-based practice is most important. |
I think that more training that
facilitates the connection of theory to the application would be helpful. |
Integrating Technology In Research | |
Coursework and technology should
be integrated into labs in which students work, and written into grants. |
Collaborative agreements and
YouTube videos/Webinars of how to integrate [technology in research]. |
Training In Policy | |
I believe we need to educate our
graduate students in the importance of engagement in professional organizations. I believe it is our professional organizations that have the greatest ability to advocate for and help protect our access to research funding and ability to practice within behavioral medicine. |
At a minimum translating clinical work
and research on helping an individual to policies that support and fund behavioral medicine is most important in my eyes. I would recommend that academic departments actively seek partnered research opportunities with entities represented in local communities, public health, and policymaking. It doesn’t have to be “big” or national to be important and actually help people. I believe the more people we help as a field, the larger influence we will have in the political landscape. |
Serve As A Reviewer Of A Scientific Journal | |
It would allow for us to be exposed to
a multitude of analyses and other ways of thinking, while serving as a form of networking (in the sense of realizing what else is being done) |
|
Multidisciplinary Team Science | |
Learning to work in true
multidisciplinary teams. Having students work on projects with other departments/fields, continuing projects over multiple semesters and years. Looking to obtain viable publications, presentations, field experiences, and community-based service opportunities. |
In order to be a successful
behavioral medicine researcher or practitioner, it is crucial to collaborate with others who have different areas of expertise. Networking can help improve this, but institutions (within and between) need to initiate more efforts focused on connecting people… |
Seeking Nontraditional Funding For Academic And Research Efforts | |
It is a changing world. There are too
many people being trained for research than there will be the standard research funds available - particularly at NIH. The earlier days of PhDs supported primarily by research dollars are over, for the near future. |
Not just non-traditional careers, but
also non- traditional funding sources. As the job market and the funding climate become more difficult, I feel students need to be trained in flexibility…not thinking that there is only “one” career path in behavioral medicine. |
Training In Supervision Of Research Project Staff | |
We tend to underappreciate the
challenges associated with managing people: staff, mentees, even our own supervisors (i.e., managing up). Being head of a team and being a supervisor are not skills taught at most graduate programs and yet they are critical to success as a researcher and even a practitioner. Including more opportunities to learn and try out these skills would be valuable (e.g., coaching, peer mentoring, management training) |
|
Time Management | |
Time management is of high
importance because it would be important to allocate time/effort towards the training and credentialing needed to meet [trainees] career goals. |
Time management - prioritizing all of
the key elements and balancing work with life. |
Media Awareness And Public Engagement | |
I believe behavioral medicine
continues to struggle for recognition amongst an atmosphere that continues to focus on biology and pharmacology. We need to improve our message and increase our message to the public and state/federal legislatures. |
I think that learning how to develop a
career goals and identify how to network/build skills toward those goals are invaluable. Experience with grant writing, public speaking, networking, and social media all play into that. |
Developing Publishable Manuscripts From Coursework | |
I think learning how to publish is the
most important. It’s a big hurdle for students to get over, especially the need to persevere once the rejections and requests for major revisions start rolling in. I think building in better training and support for the revision and resubmission process is really critical. |
Publishing and creating reports that
are of high caliber take significant time. Early support to move forward on this deliverable would be very helpful. |
Job Searches (Academic And Nonacademic) | |
I particularly wish that there was a
more guided structure for the job search. I have sought out advice from faculty and other resources, but my department does not have any formal instruction/training on the job search process. I think it would be important to discuss the job search both in a general sense (so it’s applicable to a wide audience) as well as more specifically depending on the career path (post-doc vs. junior faculty position, etc.). |
|
Nontraditional Careers In Behavioral Medicine | |
The ACA is opening many doors for
well- trained biomedical psychologists to contribute to new models of population health. We need to have a firm seat at the table of ACOs… |
I would like to learn more about
bridging the gap between academia and industry, especially the technology sector given the push for tech use in health care. |
Building An Evolving Program Of Research | |
By the 3rd or 4th year of grad school,
trainees should start to have an idea of their broad area of research and specific studies they want to conduct in that area. However, this doesn’t happen for most students because they are not mentored on how to build a program of research but how to finish their thesis and dissertation and move on to the next stage. Building a program of research helps trainees to have a story and be more marketable. Suggestions to improve this is tough because it depends on the individual, but enhancing a mentorship culture and graduate training around this ideal would be helpful. |
Teaching or modeling ways to advance
our research and interests over time in a way that is fundable and thus feasible as a career, but also takes into account the fact that interests change and research can take unexpected directions. |
Other | |
My bias is toward the
scientist-practitioner model, so I believe in promoting excellent research and clinical training opportunities as well as the integration of both (i.e., clinical research, evidence-based practice). |
Negotiating, defining and
re-evaluating your 2-year post-doctoral plan related to what you what to achieve, learn, and how to position yourself for the next job. |
Footnotes
Declaration of Conflicting Interests:
None declared.
References
- American Psychological Association. Guidelines and principles for accreditation of programs in professional psychology (G&P) 2006 Retrieved from http://www.apa.org/ed/accreditation/about/policies/guiding-principles.pdf.
- American Psychological Association. Office of Program Consultation and Accreditation. 2016 Retrieved from http://www.apa.org/ed/accreditation/index.aspx.
- American Psychological Association. Society for Health Psychology. 2016 Retrieved from http://www.health-psych.org/
- Beck AT, editor. Cognitive therapy of depression. Guilford Press; 1979. [Google Scholar]
- Boston University. MPH: Social & Behavioral Sciences concentration. 2016 Retrieved from http://www.bu.edu/academics/sph/programs/mph/social-behavioral-sciences/
- Collins LM, Murphy SA, Strecher V. The Multiphase Optimization Strategy (MOST) and the Sequential Multiple Assignment Randomized Trial (SMART): New methods for more potent eHealth interventions. American Journal of Preventive Medicine. 2007;32(5, Supplement):S112–S118. doi: 10.1016/j.amepre.2007.01.022. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Cornell University. Concentration in Applied Behavioral Economics and Individual Choice. 2016 Retrieved from http://behavioralecon.dyson.cornell.edu/about#areas.
- Council on Education for Public Health. Understanding accreditation. 2015 Retrieved from http://ceph.org/about/understanding-accreditation.
- Fisher EB, Fitzgibbon ML, Glasgow RE, Haire-Joshu D, Hayman LL, Kaplan RM, Ockene JK. Behavior matters. American Journal of Preventive Medicine. 2011;40(5):e15–e30. doi: 10.1016/j.amepre.2010.12.031. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hayes SC. Acceptance and commitment therapy, relational frame theory, and the third wave of behavioral and cognitive therapies. Behavior Therapy. 2004;35(4):639–665. doi: 10.1016/j.beth.2016.11.006. [DOI] [PubMed] [Google Scholar]
- Kabat-Zinn J, Hanh TN. Full catastrophe living: Using the wisdom of your body and mind to face stress, pain, and illness. Delta. 2009 [Google Scholar]
- Kaplan RM, Riley WT, Mabry PL. News from the NIH: Leveraging big data in the behavioral sciences. Translational Behavioral Medicine. 2014;4(3):229–231. doi: 10.1007/s13142-014-0267-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Kobrin S, Ferrer R, Meissner H, Tiro J, Hall K, Shmueli-Blumberg D, Rothman A. Use of health behavior theory in funded grant proposals: Cancer screening interventions as a case study. Annals of Behavioral Medicine. 2015;49(6):809–818. doi: 10.1007/s12160-015-9714-3. [DOI] [PubMed] [Google Scholar]
- Koh HK. Educating future public health leaders. American Journal of Public Health. 2015;(105 Suppl 1):S11–S13. doi: 10.2105/AJPH.2014.302385. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Kumar S, Nilsen WJ, Abernethy A, Atienza A, Patrick K, Pavel M, Swendeman D. Mobile health technology evaluation: The mHealth evidence workshop. American Journal of Preventive Medicine. 2013;45(2):228–236. doi: 10.1016/j.amepre.2013.03.017. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Lay J. Job-market advice — for faculty. 2015 Oct 12; Retrieved from http://chronicle.com/article/Job-Market-Advice-for/233724.
- Lovejoy TI, Revenson TA, France CR. Reviewing manuscripts for peer-review journals: A primer for novice and seasoned reviewers. Annals of Behavioral Medicine. 2011;42(1):1–13. doi: 10.1007/s12160-011-9269-x. [DOI] [PubMed] [Google Scholar]
- Miller WR, Rollnick S. Motivational interviewing: Helping people change. Guilford Press; 2012. [Google Scholar]
- Northridge ME, Holtzman D, Bergeron CD, Zambrana RE, MR G. Mentoring for publication in the American Journal of Public Health. American Journal of Public Health. 2015;(105 Suppl 1):S14–S16. doi: 10.2105/AJPH.2014.302543. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Onken LS, Carroll KM, Shoham V, Cuthbert BN, Riddle M. Reenvisioning clinical science unifying the discipline to improve the public health. Clinical Psychological Science. 2014;2(1):22–34. doi: 10.1177/2167702613497932. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Puspitasari A, Kanter JW, Murphy J, Crowe A, Koerner K. Developing an online, modular, active learning training program for behavioral activation. Psychotherapy. 2013;50(2):256–265. doi: 10.1037/a0030058. [DOI] [PubMed] [Google Scholar]
- San Diego State University. Behavioral Medicine/Health Psychology. 2016 Retrieved from http://clinpsyc.sdsu.edu/about-us/major-areas-of-study/behavioral-medicinehealth-psychology/
- Science of Team Science. SciTS and Team Science Resources. 2016 Retrieved from http://www.scienceofteamscience.org/scits-a-team-science-resources.
- Society of Behavioral Medicine. The SBM career trajectories survey: Findings and next steps. 2010 Retrieved from http://www.sbm.org/about/councils/education-training-and-career-development/career-trajectories.
- Society of Behavioral Medicine. Behavioral medicine. 2015 Retrieved from http://www.sbm.org/resources/education/behavioral-medicine.
- Society of Behavioral Medicine. About. 2016 Retrieved from http://www.sbm.org/about.
- Society of Behavioral Medicine. Retrieved from http://www.sbm.org/resources/education/behavioral-medicine.
- Society of Behavioral Medicine. FAQ. 2016 Retrieved from http://www.sbm.org/about/faq.
- Society of Behavioral Medicine. Pre-Conference Sessions. 2016 Retrieved from http://www.sbm.org/meetings/2016/pre-conference-sessions.
- Stanford Medicine Office of Faculty Development and Diversity. Team science training. 2015 Retrieved from https://med.stanford.edu/facultydiversity/faculty-development/workshops/team-science-training.html.
- The Pennsylvania State University. Training: The Methodology Center. 2015 Retrieved from https://methodology.psu.edu/training.
- Thibault GE. Public health education reform in the context of health professions education reform. American Journal of Public Health. 2015;(105 Suppl 1):S8–S10. doi: 10.2105/AJPH.2014.302407. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Tulane University. Pre-doctoral internship in clinical psychology. 2016 Retrieved from https://tulane.edu/som/departments/psychiatry/Internship/psych-internship.cfm.
- University of California San Diego School of Medicine. Division of Behavioral Medicine. 2016 Retrieved from http://behavioralmedicine.ucsd.edu/index.html.
- University of Indianapolis. School of Psychological Sciences Clinical Psychology Doctorate (PsyD) 2016 Retrieved from http://www.uindy.edu/psychologicalsciences/psyd/concentrations.
- University of Maryland Baltimore County. Department of Psychology Program Concentrations. 2016 Retrieved from http://psychology.umbc.edu/hsp/concentrations/
- University of Miami College of Arts & Sciences. Department of Psychology. 2016 Retrieved from http://www.psy.miami.edu/graduate/health-division/
- University of South Carolina. Health Promotion, Education, and Behavior. 2016 Retrieved from http://sph.sc.edu/hpeb/
- University of Washington School of Public Health. Social and Behavioral Sciences Concentration. 2016 Retrieved from http://depts.washington.edu/hservmph/sbs/
- Vaismoradi M, Turunen H, Bondas T. Content analysis and thematic analysis: Implications for conducting a qualitative descriptive study. Nursing & Health Sciences. 2013;15(3):398–405. doi: 10.1111/nhs.12048. [DOI] [PubMed] [Google Scholar]
- Wang C, Huang AT. Integrating technology into health care: What will it take? Journal of the American Medical Association. 2012;307(6):569–570. doi: 10.1001/jama.2012.102. [DOI] [PubMed] [Google Scholar]
- The Warren Alpert Medical School of Brown University. Brown Clinical Psychology Training Consortium. 2016 Retrieved from https://www.brown.edu/academics/medical/about/departments/psychiatry-and-human-behavior/training/clinical/clinical-psychology-internship/clinical-psychology-internship-1/clinical-psychology-internship-0.
- Yeshiva University. Clinical Psychology Health Emphasis Program, PhD. 2016 Retrieved from https://www.yu.edu/ferkauf/clinical-psychology-health-emphasis/about/