Postdoctoral Clinical-Research Training in Psychiatry

Abstract

The authors describe a model for teaching grant writing and other research survival skills to postdoctoral clinical-research fellows in psychiatry and for improving research mentoring. Over the past 4 years, the authors have developed a course on writing grant applications for postdoctoral clinical-research fellows, using peer-review processes modeled after a National Institutes of Health study section. At the same time, the authors have clarified expectations of mentors in ways designed to help fellows prepare “K” (Research Career Development) applications and to receive mentored practice in skills being taught in the course. Sixteen of 30 fellows have succeeded in receiving their first extramural support by the end of their two-year fellowship tenure or during the succeeding year. The authors conclude that by teaching grant-writing skills in a supportive peer environment, providing peer review of proposals, and sharpening expectations of mentors, it may be possible to reduce the time between the end of fellowship and the receipt of the first extramural grant.

Future accomplishments and changes in psychiatry, including improvements in patient care, depend upon the training of clinical researchers and the ability of such researchers to compete successfully for extramural support. The need to launch, maintain, and enhance careers in clinical research has given rise to increasing concern about the national shortage of clinical researchers in psychiatry (1). Concurrently, today's health care environment makes it much more difficult to train outstanding investigators. The shortage of clinical investigators in psychiatry is chronic, and this shortfall is worsening (2,3). Postresidency psychiatrists and clinical psychologists, who need time to develop as clinical researchers, face increasing pressure to generate their salaries and maintain high levels of clinical productivity. Thus, the availability of postdoctoral fellowship support can be, and often is, a pivotal step in the pathway to extramural funding and a successful academic career in psychiatry.

The two-year postdoctoral fellowship, while critically important, is often not enough in itself to ensure achievement of dedicated research status. A survey by Pincus et al. (1) found that the median time period between the end of research training and receipt of the first extramural grant is 3.0 years. Unless this time can be reduced, a significant hiatus in research training and experience is all but inevitable, since graduates of research training programs increasingly must engage in clinical service to generate salary support. Therefore, complementary to fellows' acquisition of core knowledge in research ethics, methodology, and statistics is the vital need for acquiring skills in project development and grant writing. Mentoring focused on complementary activities such as manuscript review, data analysis, and manuscript preparation may also better position fellows to develop their own independent projects and to practice the skills necessary for developing as an independent scientist.

Four years ago, the Department of Psychiatry at Western Psychiatric Institute and Clinic (WPIC), University of Pittsburgh Medical Center, Pittsburgh, surveyed its current and past postdoctoral fellows about how institutional training grants awarded to the department might better serve the needs of trainees. A consensus emerged on the need for “research survival skills” in an increasingly competitive and uncertain environment, particularly skills that would enable fellows to write competitive proposals for extramural funding. The faculty directing three separately funded institutional training grants (T32's) conjointly designed a grant-writing skills course. It has now evolved through four annual iterations with considerable input from the fellows themselves. The seminar was designed to focus primarily on grant-writing skills for several reasons. This focus is desired by the fellows who are aware of the critical importance of extramural funding for career advancement. The seminar provides a forum for peer review of grant proposals, improving the quality of these applications, and giving fellows experience in review activities. Writing high-quality grant proposals sharpens scientific thinking and communication skills, thereby benefiting the fellows' research skills in general. Finally, the emphasis on grant-writing skills reflects a pragmatic emphasis on the skills fellows will need to survive in academic psychiatry in the current fiscal environment.

Course Description

The WPIC research survival skills course is a weekly, problem-based learning seminar aimed primarily at helping fellows learn how to write a carefully planned, well-reasoned, and coherent grant application. The course content and organization is shown in Table 1. This course is offered in addition to other courses in research design, statistics, and ethics. The weekly course is attended by 12–15 postdoctoral fellows (adult psychiatrists, child psychiatrists, and psychologists: about 50% M.D.'s, 50% Ph.D.'s) from 3 separate National Institute of Mental Health (NIMH)-funded institutional training grants (T32's), together with the faculty directors of the “Clinical Research Training in Psychiatry” program (CFR and CM). Additional faculty mentors attend on an as-needed basis. The course consists of about 25 sessions each academic year, from September to May. The shared effort across several training programs has helped to provide the critical components necessary for lively discussion and to ensure more efficient use of resources. In addition, the conjoint meeting has been a way to provide trainees with wide access to the intellectual resources of a large department, across a broad range of disciplines. Fellows work in a broad range of research laboratories, from the psychosocial to the heavily biological, from health service delivery to basic neuroscience and genetics. Meetings have provided a forum for cross-fertilization of research ideas (basic to clinical and vice versa). The meetings have also provided social support through the establishment of a peer network.

Table 1. Course content and organization.

Primary focus: skills development, percentage of time devoted

Grant writing: 80%

Oral presentation: 5%

Manuscript writing: 5%

Manuscript and proposal review: 10%

Primary method: problem-based learning in a small group

Review successful and unsuccessful applications

Review National Institutes of Health (NIH) summary statements

Discuss grant-writing topics (e.g., specific items)

Develop/write own proposals

Model NIH study section review of fellows' proposals

Review available funding mechanisms

Present oral progress report annually

Critique manuscripts

Practice survival skills in mentor's laboratory

During the initial course sessions, examples of successful and unsuccessful grant applications authored by recent graduates of the postdoctoral fellowship program are used as learning resources. Recent graduates of the program, now serving as faculty members of the department, return to the seminar to discuss their proposals and their experiences with peer review. Different components of these National Institutes of Health (NIH) or private foundation grants (e.g., National Alliance for Research in Schizophrenia and Affective Disorders, Stanley Foundation, or Scottish Rite Foundation), written by previous and current fellows for extramural funding, are examined to promote discussion of different aspects of preparing scientific grant applications. The following components of a grant are covered in a problem-based learning format: 1) specific aims (i.e., asking clear questions that translate into clear hypotheses); 2) background and significance (i.e., how to review the literature succinctly, what elements actually make the proposed research significant); 3) preliminary studies; 4) experimental design and methods; 5) human subject issues and informed consent; 6) developing a budget and budget justification; 7) issues related to the review process; 8) the organization of the NIH and its component agencies; and 9) how to interpret and respond to an initial review group summary statement. In this context, the NIH summary statements from recent proposals are a useful resource that allows fellows to compare their own critiques with those of NIH study sections and to develop a deeper appreciation of the peer-review process.

Since there are several different grant mechanisms by which fellows may apply, discussion is also directed at determining the most appropriate mechanism. In most cases, fellows are encouraged to initially apply for a scientist development award (e.g., Research Career Development Award [K]: K08 or K01) as a way to ensure continued career enhancement activity, protection of base salary support for an extended period (5 years), and a modest level of start-up funds to support pilot research activity. Hence, in addition to the topics described earlier, considerable discussion is also devoted to career enhancement plans and activities, and their relation to scientific objectives.

As the year evolves, many sessions are modeled after study sections, in which participants take turns presenting critiques of proposals being written by fellows. Primary, secondary, and tertiary reviewers are asked to prepare comments based on the study section format, including a review of human subjects issues such as consent and ethical treatment of research subjects. By acting as referees of each other's work, fellows learn firsthand about the process of research review and how to focus and communicate their own scientific ideas in a more compelling fashion.

By the time fellows complete the course, they are knowledgeable about how to prepare an application. They understand what reviewers are looking for and how to effectively revise and resubmit a proposal. Since trainees come in at different levels of experience, this is an ongoing course for the full 2 years of the fellowship.

The culmination of the research survival skills seminar is an annual oral presentation to the joint training committees in May, attended by mentors, faculty serving on the training committee, and all postdoctoral fellows. The impetus and structure for the oral presentation grew out of the NIMH-mentored awardee meeting, in which K-awardees present to the Research Scientist Development Review Committee. Each fellow presents a 30-minute oral progress report of his/her research work, followed by a 30-minute general discussion with the training committee in which the mentor participates. The oral presentation day allows the training committee to assess each fellow's progress and the quality of the mentoring being received. Another important purpose of this exercise is to give fellows the experience of organizing and delivering an oral presentation to a critical and diverse audience of scientists. The mechanism for providing fellows feedback after the oral presentation is a private one-on-one meeting with the program director in which written feedback from participating faculty is presented and discussed. The quality of the fellow's scientific activity, as conveyed in the oral presentation, is rated by using NIH descriptors of merit (“outstanding,” “excellent,” etc.). The training program director also conveys these ratings to the fellow.

Clarification of Mentoring Expectations

The process of implementing the course has led to greater clarity in expectations of research mentors in the department. For example, mentors are expected to encourage peer-reviewed productivity during the first year of fellowship by allowing fellows access to extant data sets relevant to the fellow's scientific interest and to encourage the earlier development of review skills by providing opportunities for supervised review of articles submitted to scientific journals. Mentors assist fellows with formulating project ideas, using archival or other data sets; submitting proposals for seed-money support; and obtaining opportunities for co-authorship on extant projects. In the second year of the fellowships, mentors are expected to guide fellows toward extramural funding mechanisms and to assist fellows with grant preparation.

Evolution in the Seminar Brought About by Discussions with Fellows

The primary focus on preparing grant applications has been retained throughout all four iterations of the seminar. In more recent iterations, there has been greater emphasis and encouragement to pursue career development (K) awards in order for fellows to secure both long-term (5-year) salary support and start-up research funds. This change in content has taught fellows to conceptualize their work within a broader context, rather than simply in terms of one study, and to formulate longer term goals for research career enhancement necessary to attain specific research aims.

In addition, during the past year, more time was devoted to critiquing manuscripts prepared by the fellows before journal submission. These manuscripts are often theoretical or initial empirical reports related to their primary research interest. In this context, responding to comments from journal reviewers has also become an important focus of the course.

Finally, the course provides practice sessions and peer review for oral platform presentations at national and international meetings. The focus is on achieving a clear platform presentation, with concise statement of question or hypothesis, rationale, methods, results, and conclusion. Simplicity and clarity of slides and overheads receive emphasis, as do asking and responding to questions in succinct, direct fashion.

Thus, the specific content of meetings and the sequence of activities have come to be driven by the fellows themselves and the external deadlines in their own professional lives. This approach to organization provides a taste of scientific life, in which multiple and often competing priorities have to be balanced.

Descriptive Outcome Data

Sixteen of 30 postdoctoral fellows who have been course participants have received extramural research funding either during the fellowship or within a year of graduating. Surveying the entire history of grant applications by previous fellows appointed under “Clinical Research Training in Psychiatry,” about 75% eventually received funding but typically after longer intervals (2–4 years) than has been the experience of more recent fellows. Of the nine recent graduates in the “Clinical Research Training in Psychiatry” program, eight have academic positions and five have already secured extramural funding for their research (K awards, FIRST awards, and foundation grants). During the immediate past academic year, three of six current fellows have secured external support from NARSAD, Stanley, or Scottish Rite private foundations. With respect to postdoctoral fellows in the “Clinical Research Training for Psychologists” program, nine fellows have participated in the course during the past 4 years; six have since graduated, of whom all have academic positions and three have secured extramural funding. Of the four current fellows, one has outside funding. Of six current and immediate past fellows in the child psychiatry fellowship, two of two psychiatrists and one of three psychologists have received K awards; in addition one current fellow has received an Eli Lilly seed-money award.

Critiques of Seminars by Fellows

At the end of each academic year, fellows are asked to complete an anonymous, written evaluation of the fellowship generally, and of the grant-writing skills course in particular. Almost universally, the fellows have rated the seminar, with its emphasis on active learning, as an outstanding forum to get their own work reviewed and discussed. Fellows view the seminar as more beneficial than a didactic (and more passive learning) seminar in which they hear about other people's ideas. In addition, the availability of accomplished researchers as role models, time specifically dedicated to project development, learning about other funding sources, and support from peers have all been viewed as useful.

Three weaknesses of the course were commonly cited by fellows: 1) occasional loss of control and focus in the discussions born of the tendency of “first reviewers” to monopolize the time, 2) redundancy in some of the discussions of budget preparation, and 3) inadequate attention to job search strategies for the initial faculty appointment.

Discussion

Because of the absence of a control group and random assignment, we cannot make a claim for the efficacy of the course. The positive outcomes observed might be attributable to the seminar, its effects on mentoring, and/or to some other factor(s). In any case, it appears that participation in the seminar on grant-writing skills, together with clarification of mentoring expectations, can aid in the achievement of early success in securing the first grant, often before the postdoctoral fellowship is completed or during the first year afterward. This change from the historical pattern, in which fellows more often took 2–4 years after fellowship completion before receiving the first grant, may reflect the fact that the course has helped many fellows begin the process of writing a grant application earlier in their training. As a result, revisions and funding may have occurred somewhat earlier than with the traditional pathway. It is also possible that the training mix of M.D.'s and Ph.D.'s has been associated with greater research productivity than was the case before (because of exposure to “translational” research concepts and methods), in which M.D.'s and Ph.D.'s tended to work in relative isolation from one another. Greater clarity in the expectations of mentors has been achieved through their direct participation in the course itself and has probably been an important contributory factor in the program's apparent success. In addition, training directors continue to meet individually with fellows and conjointly with fellows and mentors, outside the seminar, to monitor progress in career development plans and activities.

Other characteristics of successful postdoctoral fellowship programs, well documented in the research training literature, include sufficient duration of postdoctoral training (at least 2 years [1,4]); personal contact with senior investigators, including exposure to ways in which senior and accomplished scientists came to make important career decisions (5,6); and location in a large academic institution with its concentration of senior researchers (1). It is our view that these “ingredients” of successful postdoctoral research training are enhanced by the existence of a longitudinal research survival-skills seminar, extending throughout the 2 years of fellowship, and directly involving many faculty from diverse fields. Research development and grant-writing abilities connect directly, since grant writing deals with clear, focused communication; the development of important, testable hypotheses; specification of rationale; and choice of appropriate samples, outcome measures, and analytic strategies. Thus, grant writing is a problem-based learning activity, in which the objective is to apply scientific method to solve a problem and defend the approach chosen. Finally, when grant-writing skills address the demands of a K-award application, fellows create a “platform” or plan for long-term research and career development.