OPERATIONAL LESSONS FROM THE PATHWAYS TO DESISTANCE PROJECT

Abstract

Implementing a large, longitudinal study of any sample is a major undertaking. The challenges are compounded when the study involves multiple sites and a high-risk sample. This article outlines the methodology for the Pathways to Desistance study, a multisite, longitudinal study of serious juvenile offenders, and discusses the key operational decisions with the greatest impact on the study design.

Over the past 2 decades, a great deal has been learned about the risk indicators associated with adolescent antisocial behavior and delinquency. Indeed, much has been clarified about how delinquent behavior starts, the general trajectory of this behavior during adolescence, and the relative predictive power of certain risk indicators (see, for example, Blumstein, Cohen, Roth, & Visher, 1986; Cicchetti & Cohen, 1995; Farrington, 1997; LeBlanc & Loeber, 1998; Loeber, Farrington, Stouthamer-Loeber, Moffitt, & Caspi, 2001; Moffitt, 1993; Sampson & Laub, 1990, 1997). One particularly important finding to emerge from this literature is that relatively few adolescent offenders go on to serious adult offending (Loeber & Farrington, 2000; Moffitt, 1993). Consequently, one of the most pressing current challenges for the field is to reliably distinguish between juvenile offenders who will continue problem behavior beyond adolescence and those who will not.

Successfully meeting this challenge requires a greater empirical understanding about how and why juveniles desist from committing crime. Several calls have been made for researchers to study desistance from criminal activity with the same vigor exerted toward issues surrounding the onset of criminal activity (e.g., Farrington, 1997), yet the literature in this domain remains scant. One comprehensive approach to understanding this process would be to examine desistance from criminal activity prospectively, using multiple sources of information beyond official reporting (Farrington, 1997). This approach would be guided by the literature on child and adolescent development, and sensitive to the potential implications of findings for designing interventions and developing rational justice policies.

The Pathways to Desistance Project highlighted in this special edition is an attempt to take up this challenge. It is a large-scale, two-site longitudinal examination of desistance from crime among adolescent serious offenders. The goal of the current study was to elucidate how developmental processes, social context, and intervention and sanctioning experiences affect the process of desistance from crime. The current study employed a prospective design with a broad measurement focus and multiple sources of information (self-report, collateral report, and official record) to provide a picture of intraindividual change over time. The goals of the current study are to provide a rich description of changes in functioning, psychological development, and social context among adolescent serious offenders during late adolescence, and to assess the effects of maturation, changes in social context, and sanctioning and intervention experiences on positive and negative changes in behavior, psychological functioning, and the transition into adult roles.

This article describes some of the key practical and logistical challenges we found most salient for maintaining the integrity of the original intent of the current study. Some issues that we faced are generic to multisite longitudinal research (e.g., enlisting funding partners, obtaining clearances from site research ethics review boards that do not always interpret guidelines in the same way) and will not be discussed here. Instead, here we address the subset of issues we confronted that, although not unique, had the greatest impact on the study design and the study’s eventual interpretability. The hope is that this information will be useful to researchers confronting the challenges associated with making valid, scientific inferences in a complex research venue. In the subsequent sections of this article, we present an operational overview of the Pathways study, discuss specific challenges to the implementation of the project (as well as the strategies used to meet these challenges), and summarize the implications and lessons that can be drawn from our experience.

Operational Overview of the Pathways Study

Recruitment

The Pathways study sought to recruit a sample of adolescent offenders with sufficiently serious charges and histories to be relevant for policy discussions, but with enough heterogeneity to examine the relative impact of interventions, sanctions, and life changes. With this goal in mind, we recruited 1,354 adjudicated adolescents who were between the ages of 14 and 17 years at the time of their committing offense from the juvenile and adult court systems in Philadelphia, Pennsylvania, and Phoenix, Arizona. The youth were selected for potential enrollment after a review of the court files in each locale revealed that they had been adjudicated delinquent or found guilty of a serious offense. Eligible crimes included all felony offenses with the exception of less serious property crimes, as well as misdemeanor weapons offenses and misdemeanor sexual assault.¹ Because drug law violations represent such a significant proportion of the offenses committed by this age group, and because boys account for the vast majority of those cases (Stahl, 2003), we were concerned about compromising the heterogeneity of the sample if we did not limit the number of study participants who were drug offenders. Therefore, we capped the proportion of male juveniles with drug offenses to 15% of the sample at each site. All female juveniles meeting the age and adjudicated crime requirements and all youths whose cases were being considered for trial in the adult system were eligible for enrollment, even if the charged crime was a drug offense. The enrollment of the sample began in late 2000, and the study is currently in the field.

Interview Schedule

After informed consent was obtained from the juveniles and their parents or guardians, youths who agreed to participate in the study completed a baseline interview. An adult collateral informant (a parent in 80% of cases) was also interviewed at baseline. For youths in the juvenile system, the baseline interview was conducted within 75 days of their adjudication hearing. For youths in the adult system, the baseline interview was conducted within 90 days of either (a) the decertification hearing in Philadelphia, a hearing at which it is determined if the case will remain in adult court or if it will be sent back to juvenile court or (b) the adult arraignment hearing in Phoenix, the point in the Arizona adult system at which charges have been formally presented and the defendant has the opportunity to enter a plea of guilty or not guilty to the charges. There is no waive-back provision to the juvenile system under Arizona law.

Participants complete two types of interviews after their baseline interview: “time-point” interviews and “release” interviews. The time-point interview includes a standard set of measures we administer at 6-month intervals, beginning 6 months after the baseline interview and continuing for the 3-year follow-up period. The date for each of the time-point interviews is calculated based on the date of the baseline interview, ensuring approximately equal measurement periods for all participants. These equal measurement periods simplify the statistical analyses required to assess developmental processes, environmental changes, and their relations to changes in behavior.

A window of opportunity to complete each follow-up interview opens at 6 weeks prior to the follow-up interview target date and closes at 8 weeks after the target date. To help interviewers stay on schedule, they receive weekly workload reports that are divided into three sections: the “search window,” the “do window,” and the “late window.” The search window includes cases whose target date for the next interview will occur within the next 6 weeks. For these cases, interviewers attempt to locate participants but do not conduct an interview unless there is a reasonable expectation that it cannot be completed at a later time (i.e., the participant is homeless and does not know where he or she will be in a few weeks). The do window, which lists cases that should be interviewed at that time, include cases that are within 8 weeks of the target date, (i.e., 4 weeks prior and 4 weeks after the target date). Finally, cases in the late window have passed their target date by 4 weeks and are of the highest priority for locating the participant and conducting an interview. If an interview is not completed within 8 weeks of the target date, that particular time-point interview is considered missed, and no further attempts are made to interview these individuals until the next time-point interview. Unless the participant explicitly withdrew from the study, we continue to attempt to contact a research participant for future interviews even after one or more of the previous time-point interviews was missed.

One year after the baseline interview, and at annual intervals after that, additional collateral information is obtained from peers nominated by participants as individuals who know the participant well. This shift from a parent collateral at the baseline to a peer informant for subsequent interviews was motivated by our desire to capture information about the ongoing behavior of the participant, rather than historical information. Previous work has found that peers are better able than a parent to report on deviant activity (Chassin, Pitts, & Delucia, 1999; Smith, McCarthy, & Goldman, 1995).

The second type of interview completed by research participants after their baseline is a “release interview.” A release interview is completed following any stay at a residential facility. These interviews obtain the adolescents’ reports of services received and their perceptions of the environment experienced in institutional care. The window of opportunity for release interviews opens 30 days before the research participant is scheduled to be released and closes 30 days after he or she has left the community from a facility.

Interviewers make a concerted effort to interview research participants in a location where the adolescent is comfortable. Most of these interviews are conducted in the adolescent’s home or, for those participants in institutional placement, in a private room within the facility, unless there are concerns about the interviewer’s safety. Of the interviews conducted to date, 53% (N = 5,500+) were conducted in the research participant’s home, 36% were conducted in a facility, and only a small percentage (11%) were conducted elsewhere.

Interview Content

The baseline and time-point interviews cover six domains: (a) background characteristics (e.g., demographics, academic achievement, psychiatric diagnoses, offense history, neurological functioning, psychopathy, personality), (b) indicators of individual functioning (e.g., work and school status and performance, substance abuse, mental disorder, antisocial behavior), (c) psychosocial development and attitudes (e.g., impulse control, susceptibility to peer influence, perceptions of opportunity, perceptions of procedural justice, moral disengagement), (d) family context (e.g., household composition, quality of family relationships), (e) personal relationships (e.g., quality of romantic relationships and friendships, peer delinquency, contacts with caring adults), and (f) community context (e.g., neighborhood conditions, personal capital, social ties, and community involvement). Because of the comprehensive nature and length of the baseline assessment, the interview was broken into two, 2-hour sessions. Follow-up interviews, which assesses changes during the previous 6 months in domains covered in the baseline interview, are conducted in one 2-hour session.

The release interview is more limited in scope. It attempts to document the treatment content and program dynamics for each residential intervention experienced by the research participant. The release interview contains measures of program operations, program dynamics (e.g., contact with caring adults in the facility, perceptions of fairness and equity connected with treatment by facility staff), and the adolescents’ assessments of the type and utility of services offered.² It was logistically impossible to reliably track the involvement of youth in community-based programs because involvement in these programs often is not recorded with sufficient detail in court records to determine the exact program, and because it was not possible to obtain accurate or timely information about the dates of participants’ release from community-based programs.

Important Issues to Be Addressed

The operational procedures outlined above were adopted after extensive discussions among the investigators and consultants working on this project. In looking back over the choices made regarding research design and procedural rules, we saw four key issues critical to the potential success of a project like this. These are discussed below.

Site Selection: Choosing and Setting the Stage

The processing of juvenile offenders varies dramatically from locale to locale, (Feld, 1991; Ghezzi & Kimball, 1986; Krisberg, Litsky, & Schwartz, 1984; Snyder & Sickmund, 1999). Failure to consider the implications of court practices at a given site can seriously limit the generalizability of one’s findings and the subsequent applicability of one’s findings to relevant policy issues. When two or more data collection sites are involved, it is important for investigators to understand the site procedures well enough to recognize the points at which the systems are parallel and to devise a sampling plan that capitalizes on those parallel points to maximize comparability (e.g., our enrollment procedures for youth in the adult system). A thorough knowledge of each local system is needed to make these determinations intelligently and to recognize the benefits and limitations of conducting research in that locale.

Preliminary detailed examination of the juvenile justice system in potential research sites allows the investigators to gain an understanding of the guiding legal mandates, operating principles, and data capacities (e.g., types, completeness, and format of information retained on youth in the juvenile and adult system and our ability to access specific portions of that information) of the potential site. This information provides a context for the later interpretation of findings. In the Pathways study, for example, it was critical to determine what the governing legislation was regarding transfer to adult court in each site, the frequency with which this procedure was invoked, the distribution of placements used by the court, and the availability of data about individual case processing in the juvenile and adult system. This information was needed to determine whether the sites would be sufficiently different to allow for relevant policy contrasts (e.g., whether similar types of adolescents have different outcomes when processed differently). In addition, having this type of information allowed us to address basic feasibility issues before launching the study (e.g., whether there would be enough adolescents placed in different sanctioning and intervention environments to allow for an examination of the effects of these placements). The availability of detailed processing information was also needed to make a valid assessment of the degree to which the final sample was representative of the overall population of offenders at each site.

There is also substantial political value to careful preliminary examination of any potential site. Meeting with officials to learn about local juvenile justice system operations and to discuss the procedures by which study results will be shared is an appropriate and respectful way to establish long-term cooperative relationships with those whose support is vital. In our case, well-placed and broad support was essential to obtain permission to conduct interviews in a variety of settings throughout the juvenile and adult justice systems. We needed the support and cooperation of many different stakeholders, including representatives from the court, probation, corrections, and service providers in the juvenile and adult systems in each locale. Without a solid basis of support among the key players in these systems, it would have been difficult to obtain a representative sample or to track participants effectively over the course of the project.

For the current study, investigators spent many hours conducting a systematic investigation of six potential sites before selecting the two locations for data collection.³ This investigation included an extensive review of the juvenile justice system reports from each of these six locations, a review of existing and pending legislation regarding juvenile processing, face-to-face interviews with key administrators and potential collaborators, and visits to detention and long-term secure facilities. In the end, Philadelphia and Phoenix were selected because they offered (a) high enough rates of serious crime committed by juveniles to ensure the enrollment of a large enough sample (based on statistical power analyses) within a reasonable time frame (determined by budgetary considerations); (b) a diverse racial/ethnic mix of potential participants (Philadelphia’s offender population is mainly African American, whereas Phoenix’s is predominantly Hispanic and White); (c) a sizable enough number of female serious offenders to examine sex differences in patterns of development and desistance from crime; (d) a contrast in the way the systems operate (Phoenix has a sparse treatment system, and Philadelphia has a more extensive one); (e) political support for the study and cooperation from the practitioners in the juvenile and criminal justice systems; and (f) the presence of experienced research collaborators to oversee data collection on-site.

Measurement Selection: Covering the Right Domains Well

Deciding what measures will elicit the desired information and choosing the formats for presenting those measures are important considerations for any empirical study. However, the current study had additional demands because of the nature of our sample and our defined goals. We wished to observe developmental and contextual change in a relatively understudied and difficult sample moving from adolescence through early adulthood.

Finding measures of psychological and social functioning that are appropriate for use with a juvenile offender population was difficult and time-consuming, for several reasons. First, we were tracking development across two different age periods—adolescence and young adulthood; many measures developed for use with one age group have not been validated in the other. Second, our sample contained a high proportion of individuals with limited literacy. Third, juvenile offenders come from social and cultural backgrounds often very different from those of research participants in studies of community samples of adolescents and young adults. Fourth, the life circumstances of juvenile offenders are more diverse than those of typical community samples of high school students or college undergraduates. Finally, the variability in age and ethnicity within the sample raised questions about measurement equivalence (see Knight, Little, Losoya, & Mulvey, 2004 [this issue]) for a more detailed consideration of this point). After an exhaustive search of the literature, we found little in the way of validated measures for the range of social contexts in which our study participants lived. As a result, considerable time was devoted to testing, revising, and retesting an array of measures with youths in detention centers in Pittsburgh and Philadelphia. This pilot work was a critical first step to constructing a comprehensive battery of measures that would work with this population.

A second major challenge was the development of a strategy for tracking change over time with sufficient resolution to draw meaningful conclusions. Juvenile offenders’ lives often are chaotic and unstable, with frequent changes in residence, education, employment, and interpersonal relationships. We wished to capture not only information that could characterize the period covered in each interview but also information about the nature, number, and timing of important changes in life circumstances. For example, although we were interested in whether an adolescent worked during the period covered by an interview, we also wished to know when and how long employment periods lasted, and whether those periods preceded or followed criminal activity. Previously developed methods for structuring life-event recall have been shown to provide reasonably accurate information about the temporal sequencing of events during the period covered by an interview. Such methods for constructing life-event calendars have been used successfully in studies of criminal offending, antisocial behavior, and mental health service use (Caspi et al., 1996; Horney, Osgood, & Marshall, 1995). We modeled the construction of our time-point interviews after the life calendars developed by previous investigators.

Choosing an Efficient and Effective Method for Data Collection

In the Pathways study, interviews are conducted on laptop computers, and interviewers and participants usually sit side by side with the computer screen visible to both. This technology was attractive for a number of reasons. Foremost, with the proper programming, the computer-assisted interviewing has the ability to tie prior reports (from either earlier time-point interviews or previous points within the current interview) to current responses. Cross-referencing of information during the interview ensures that responses in one section are consistent with those in other sections or other interviews and allows for identification of significant status changes (e.g., whether the person who is identified as having raised the adolescent in the current interview is the same person named in prior sections or interviews). Error messages can be built in to alert the interviewer to these inconsistencies. In addition, this technology permits a visual and interactive display of life calendar information that makes the interview more engaging for the research participant.

Secondary benefits related to computer-assisted interviewing include data that are immediately accessible and can be translated for use with a variety of computer software packages. Our design calls for the regular transfer of data (in an encrypted format) from interviewer laptops to a centralized database maintained at the study’s coordinating center at the University of Pittsburgh. This structure makes it possible to combine data from both sites quickly, allowing for ongoing monitoring of the psychometric properties of the measures, identification of inconsistencies or problems with the data, and detection of studywide and site-specific trends in the data as they emerge.

There are some downsides to using this technology as well. One clear one is that a major investment must be made in programming and software testing before data collection can begin. For the current study, development time was reduced significantly by using commercial software designed specifically for the development of interviews with complicated skip patterns (Statistics Netherlands, 2002); however, the up-front costs were still substantial. In addition, staff must be trained and monitored closely. Interviewers need to know how to navigate the interview (e.g., how to skip backward to a question when a participant wants to change a response) and how to manage their data (e.g., knowing when and how to back up their hard drives, understanding how to avoid overwriting data). Finally, there is the mundane, but nonetheless important, consideration of electrical power. Not all facilities and homes have an electrical outlet that is accessible from the location where the interview takes place. Therefore, interviewers must be prepared with adequately charged batteries.

Using computers with an offending population or in high-crime neighborhoods poses other concerns. Precautions must be taken to avoid theft of equipment and to safeguard the personal information stored on that equipment. In addition, the use of computers raises security concerns for many of the facilities in which our interviews occurred. In these situations, a special arrangement with the facility administrator is necessary before data collection can begin.

The use of technology can go beyond just computer-assisted interviewing. Our multisite design and ambitious sampling goals led us to use computer technology in the larger realm of study operations as well. We developed a secure Web site to house information on each juvenile meeting our age criteria and petitioned on an eligible charge at each site for the entire enrollment period of the study (N = 10,461). Background information (e.g., aliases, petition dates, petitioned and adjudicated charges, case disposition) from court records at each site was collected for these individuals. For individuals adjudicated on an eligible charge, we recorded additional information about the outcome of our efforts to recruit them to participate in the study. Finally, for those individuals who have agreed to participate in the study, we maintain real-time tracking data and information on the ongoing status of each case. This Web site information serves a dual function: We have a comprehensive database about case processing in each site and a valuable administrative tool to monitor the progression of enrolled participants through the course of interview waves. At any point during the enrollment process, for example, we were able to ascertain how our sample was developing with respect to diversity in offending, ethnicity, or gender (all of which were sampling concerns). This information was readily available to interviewing staff, site coordinators, and coordinating center staff at varying levels of security clearance.

Setting Up a System for Tracking and Retaining Research Participants

As in other longitudinal studies of high-risk populations, we face the difficult challenge of maintaining research participant involvement in repeated testing over an extended time. This challenge presents particular issues, however, when dealing with serious adolescent offenders. As with any adolescent, this group selects (and are selected into) a more diverse and often frequently changing set of novel social contexts (e.g., changes in residence, entrance into and out of correctional facilities, new school experiences, changing peer groups). Although these are the very factors that make the study of our participants’ lives scientifically interesting, they also create considerable problems for maintaining contact with research participants. The information used to locate a participant at one time is not necessarily valid 6 months later because the youth may have subsequently been incarcerated, transferred to a new facility, or moved away from his or her family. In addition, many of the research participants in this sample do not want to be found because of active bench warrants or involvement in illegal activities. Although we have obtained a certificate of confidentiality from the U.S. Department of Justice, it is reasonable for study participants, many of whom have been betrayed by adults in their life, to wonder about our trustworthiness. Our participants’ high degree of mobility and engagement in deviant activity make tracking and retention more demanding for this group than the norm.

Finding these adolescents for repeated interviews requires consideration of the individual characteristics of the participant being located as well as the use of multiple sources of information beyond the individual. For example, Menendez, White, and Tulsky (2001) found that in a sample of released inmates, English-speaking participants were most often successfully located in shelters or treatment programs, whereas Spanish-speaking participants were more successfully located through outreach efforts in their community gathering places. Similarly, tracking strategies for a 15-year-old might emphasize contacts with the family and/or school whereas strategies for a 19-year-old may rely more heavily on official record searches (e.g., through the motor vehicle bureau). Investment of time and effort in searching for participants in the most strategic fashion can often reduce the frustration felt by interviewers in doing tracking activities.

We have also found it necessary to use a range of data sources at each follow-up to locate the research participants, often relying heavily on locating and maintaining contact with each adolescent’s family. The Pathways study has used a multifaceted tracking protocol that includes strategies found to be successful in our prior work (Schubert, Mulvey, Lidz, Gardner, & Skeem, in press) as well as in other studies with high-risk populations (Cottler, Compton, Ben-Abdallah, Horne, & Claverie, 1996; Craig, 1979; Demi & Warren, 1995; Desland & Batey, 1991; Given, Keilman, Collins, & Given, 1990; Grant & DePew, 1999; Marmor et al., 1991; McKenzie, Tulsky, Long, Chesney, & Mos, 1999; Menendez et al., 2001; Senturia et al., 1998; Whelan, McBride, & Colby, 1993; Wright, Allen, & Devine, 1995). These methods include using a wide range of tactics to reach the participant (e.g., phone calls during odd hours and unscheduled visits to the participant’s home, neighborhood, and hangouts), enlisting support and obtaining information from every possible contact among family members and friends mentioned in previous interviews, and conducting address searches with credit databases, community agencies, and criminal justice facilities.

In the end, however, our experience is that much of the success of tracking comes down to the level of individual effort put forth by interviewing staff. Our tracking protocol includes a set of clearly defined guidelines for interviewers, including a requirement to maintain a log of all attempts made to reach a participant (so that observed patterns might help to locate this person at a subsequent interview), a locator information sheet updated at every contact with the participant, and a standardized sequence of contact attempts to be made at every time-point interview (e.g., telephone and written contacts, personal visits, and the use of computerized databases). As an additional resource for interviewing staff, we encouraged each data collection site to designate a “tracker.” The tracker can either be an interviewer with a reduced interviewing load or noninterviewing project staff. In either case, the tracker specializes in locating the most difficult-to-find cases through the use of community, computer, and official database resources. We encourage interviewers who had made repeated unsuccessful attempts to locate a participant to request the assistance of the staff tracker to not only provide a more focused effort to locate the participant but also alleviate the frustration associated with not reaching the participant.

We have also found it helpful to maintain contact with professionals who are responsible for monitoring the participant’s activities, such as case workers and probation officers. Sometimes these individuals know information about an adolescent’s location or activities that are not recorded in official records. Maintaining a positive relationship with probation officers or police officials, however, requires a mutual understanding about the limits of confidentiality. It is not uncommon, for example, that a probation officer who has been helpful will subsequently request information from an interviewer about how to find an adolescent, especially if there is an outstanding bench warrant in the case. The researcher, however, cannot provide that information if it was obtained as part of the regular interview because this information is protected by the confidentiality provided to the research participant. This unrequited information exchange is often frustrating to individuals working in the juvenile and adult justice systems and can lead to confrontation or organizational stonewalling if the confidentiality requirements are not made clear at the outset of the study.

Similar to the structure of probation and case management departments, we have followed a caseload model when assigning participants to interviewing staff, as noted earlier. Cases were assigned to interviewers randomly and with the idea that the participant and interviewer would remain paired throughout the course of the study, unless a safety concern or staff turnover dictated a change. There was no matching on gender or ethnicity because we have found, similar to others (McKenzie et al., 1999), that the ability to form a respectful relationship with a participant matters more than being of the same race or gender.

This approach has several advantages. Because cases are randomly assigned, interviewers all have a comparable mix of difficult and easy cases. Interviewers thus feel that they are all facing similar tracking challenges (i.e., no one is singled out to handle only difficult cases) and are individually accountable for retaining a group of participants for the duration of the study (i.e., this is their case). Maintaining consistency in the pairing of interviewers and participants also promotes rapport, providing continuity for the participant and hopefully increasing disclosure. Getting to know individuals better and getting along with them makes an interviewer’s job easier in the long run, especially given the sensitive nature of the interview topics (e.g., illegal activity, victimization). In addition, keeping this pairing allows the interviewer to accumulate a historical knowledge base that is often useful for validating information shared in the interview and for locating participants over the course of the study.

Our caseload model is augmented by meetings that provide support and review productivity. Weekly staff meetings are held at each site, where staff members share stories about unusual experiences or difficult cases. The group support received during these meetings counteracts the often-felt sense of working in a vacuum when tracking and interviewing hard-to-find adolescents. In addition, we hold a biweekly “late cases” telephone conference involving the coordinating center staff and the site coordinators. During this meeting, we review each of the cases that are reaching the end of the window of opportunity for completion of an interview. Interviewers supply their contact logs for review at these meetings, listing all the steps that have been taken to date to contact the research participant. The late cases meetings serve to provide accountability and a way to explore strategies that might not yet have been taken with a particular case. Similar to others (Craig, 1979; Menendez et al., 2001), we have found that this model of case assignment and frequent meetings introduces a healthy level of competition among the interviewing team members as they struggle to attain the highest retention rate with their individual caseloads. To promote this, we regularly provide feedback on case retention rates to each interviewer so that he or she was able to compare his or her performance to that of the group as a whole.

Although these management practices help to convey clear standards of performance to interviewing staff, in the end we believe that much of the success of tracking and retention in a sample such as this rests on the manner in which interviewers interact with participants. Over and above the tracking guidelines and support, we repeatedly emphasize to staff that high retention of participants has been a direct result of their persistence and the respectful interaction they maintain with these individuals. To us, these two basic elements rise above all other protocol features for maintaining participant involvement over time, a conclusion that others have reached as well (Grant & DePew, 1999; Marmor et al., 1991; McKenzie et al., 1999; Menendez et al., 2001). A motivated and conscientious staff that treats adolescents and their families with courtesy is the most important element in the array of management strategies used to track and retain research participants. To that end, treating interviewers with courtesy and respect helps provide a model for how they ought to be interacting with adolescents and their families.

Enrollment and Retention Results

In many respects, the operational success and eventual theoretical contribution of a longitudinal study can be judged by whether the appropriated research participants are identified, convinced to take part in the study, found repeatedly, and interviewed successfully. Without doing these activities well, even the richest data set is limited in its potential contribution. So far, we have achieved considerable success in finding and interviewing the group we set out to sample using the methods outlined above.

The results of our enrollment efforts are presented in Figure 1. During the enrollment period (November 2000 to January 2003), approximately 10,461 individuals meeting our age and petitioned charge criteria were processed in the court systems in Philadelphia and Phoenix (see Figure 1). Although some individuals (approximately 42% of the 10,461) came through the court system more than one time in the 27-month recruitment period, we will consider individuals once for the purposes of this analysis. This approach avoids a skewed view of case-processing practices because of the presence of repeat offenders. The petition selected to represent the individual is the first petition on which the youth met the study criteria during the enrollment period (if that individual was not enrolled in the study) or the petition on which a participant was enrolled.

Figure 1.

Pathways to Desistance Sample Enrollment

Although petitioned on an eligible serious charge, some adolescents did not qualify for enrollment because they were not adjudicated (found guilty) on an eligible charge. In a sizable number of the petitioned cases (n = 5,382), the charges were reduced below a felony-level offense at adjudication. In another 1,272 cases, the court data were not sufficiently clear during the enrollment period to determine eligibility status at adjudication.

Slightly more than one half of the youth determined to be adjudicated on an eligible charge were approached for enrollment. Those not approached (n = 1,799) were excluded because of operational and design constraints. We did not approach all eligible cases when the flow of these cases would have overloaded the available interviewers or when we were close to enrolling our predetermined cap of 15% drug offenders. In the end, we managed to enroll more than 1 of every 3 (36%) of the identifiable adjudicated felony offenders who came before the courts in these locales during the enrollment period. Our participation rate, defined as the number of participants enrolled divided by the number attempted for enrollment, was 67%. Our refusal rate, defined as the number of adolescents or parents who would not take part in the study divided by the number we approached, is 20%. These figures compare quite favorably with those from other studies of high-risk populations.

We examined how our case identification and enrollment process may have filtered out particular groups along the way. The total sample of petitioned youth (exclusive of the 1,272 cases for which the court records were incomplete; n = 9,189) was divided into three mutually exclusive groups: (a) those individuals petitioned on an eligible charge but then adjudicated on a lesser, noneligible charge (“petitioned, but not adjudicated” group; n = 5,392); (b) those petitioned and subsequently adjudicated on an eligible charge but not enrolled into the study (“adjudicated, but not enrolled” group; n = 2,443); and (c) those petitioned and adjudicated on an eligible charge and then enrolled into the study (the “enrolled” group, n = 1,354).

Two sets of comparisons using these three groups allowed us to obtain a picture of how our enrollment process influenced sample characteristics. In the first analyses, the petitioned group was compared to the combined adjudicated and enrolled group. This provided a view of the filtering connected with the adjudication process. Next, we compared the adjudicated, but not enrolled, group to the enrolled group, providing a perspective on potential biases connected with our enrollment criteria (i.e., the cap on drug charges at 15% of the sample) and our recruiting process.

The descriptive statistics of the petitioned, but not adjudicated, group and the combined adjudicated and enrolled groups are presented in Table 1. Many of the statistically significant differences seen in Table 1 are reasonable because the comparisons are between petitioned cases and adjudicated cases with a very large sample. The petitioned group and the adjudicated group differed in their average age, number of prior petitions, gender, and race. The adjudicated group is more likely to be male, slightly older, and with more prior petitions. This group is also less likely to be White (test of proportions z = 10.95, p < .001) and more likely to be Black (test of proportions z = 4.51, p < .001) and Hispanic (test of proportions z = 5.59, p < .001). Blacks and Hispanics in this sample are significantly more likely to be adjudicated on a serious charge meeting our criteria (p = .001) than to be found not guilty.

TABLE 1.

Descriptive Statistics of Petitioned Versus Combined Adjudicated Groups

	Petitioned (n = 5,392)		Combined Adjudicated (n = 3,797)
	M	SD	M	SD	Significance
Age at adjudication	15.8	1.2	16.0	1.2	t = 7.97, p < .001
Number of prior petitions	0.6	1.4	1.7	2.1	t = 30.33, p < .001
Age at first prior petition	14.0	1.5	14.1	1.7	t = 1.09, p < .001
Gender
Male	81%		90%		χ2 = 133.18, p < .001
Female	19%		10%
Race
White	32%		21%		χ2 = 118.74, p < .001
African American	43%		48%
Hispanic	23%		28%
Other	2%		3%

Table 2 presents the differences between the adjudicated, but not enrolled, group and the enrolled group. These groups differ in several ways. First, the enrolled group is younger at their adjudication hearing, has had more prior petitions, and appeared in the court for the first time at an earlier age. There are also a larger proportion of girls in the enrolled group. None of these results are surprising given that we purposefully sought to enroll more serious youth and every possible female offender to increase the size of this subgroup for later analyses. Finally, although our enrollment criteria did not include any restrictions on race, we did enroll proportionately more White offenders (test of proportions z = 3.27, p < .005) and fewer African Americans (test of proportions z = 3.09, p < .005). We know that this discrepancy was not related to differential rates of participant refusal across racial groups because African Americans were not significantly more likely to refuse. It is instead most likely that the imposition of a cap on the proportion of the sample adjudicated on drug charges probably affected this race proportionality because there is likely to be an association between adjudications for drug charges and race, especially among African Americans in Philadelphia. Indeed, African Americans were significantly more likely (p = .001) to be in the drug cap group than were other racial groups.

TABLE 2.

Descriptive Statistics of the Adjudicated Versus Enrolled Group

	Adjudicated (n = 2,443)		Enrolled (n = 1,354)
	M	SD	M	SD	Significance
Age at adjudication	16.1	1.2	15.9	1.4	t = −4.42, p < .001
Number of prior petitions	1.5	1.9	2.1	2.4	t = 8.78, p < .001
Age at first prior petition	14.2	1.7	13.9	1.7	t = −3.29, p = < .001
Gender
Male	91%		86%		χ2 = 22.81, p < .001
Female	9%		14%
Race
White	20%		25%		χ2 = 33.66, p < .001
African American	49%		44%
Hispanic	28%		29%
Other	3%		2%
Disposition at adjudication
Adjudicated with later case dismissal	2%		1%		χ2 = 118.51, p < .001
Fines/restitution	1%		1%
Probation	50%		41%
Nonincarcerated residential placement	15%		21%
Incarcerate/jail	10%		21%
Pending	22%		15%

As mentioned earlier, we are still in the midst of conducting follow-up interviews, so in many ways a report of retention figures is necessarily premature. Nevertheless, a discussion of this topic is pertinent as not only a way to demonstrate the current success of the tracking and retention strategies but also an illustrative example for alternative ways to report performance in this area. For the current study, retention can be considered in two ways.

The first method, time-point retention rate, measures the success in completing a particular interview wave. This calculation considers only the enrolled cases that have passed through the window of opportunity to complete a particular time-point interview. For example, the retention rate at the 24-month interview considers the number of 24-month interviews that were completed for those cases that had passed out of the window of opportunity for the completion of the 24-month interview. The time-point retention rates to date indicated that we had a 95% retention rate at the 6-month, 12-month, and 18-month time points and 93% at the 24-month time point.⁴

The second way to characterize retention, a cumulative retention rate, reflects the completeness of the data. This figure reflects the proportion of possible interviews we have completed for an individual across all time points. Similar to the time-point retention rates, the cumulative retention rates are calculated on only those cases that have had the opportunity to be interviewed at a particular time point. For example, cases considered for their completeness at the 12-month time point would be restricted to only those cases that have passed through the window of time in which they could have completed a 12-month follow-up interview. Using this method of examining cases, we then looked at how many cases had completed each successive interview up to that point. Considering retention in this way gives us an idea about the number of missing data points we will have at the individual participant level. Table 3 presents our cumulative retention figures. As the table indicates, we have been very successful in collecting complete data from a large proportion of our sample.

TABLE 3.

Cumulative Retention Rates Through September 2003

Time Point	Data Completion	Percentage of the Sample
12-month point	Full data (6 and 12 months)	92
	One interview	6
	No interviews	2
18-month point	Full data (6, 12, 18 months)	89
	Two interview	7
	One interviews	3
	No interviews	1
24-month point	Full data (6, 12, 18 and 24 months)	81
	Three interviews	13
	Two interviews	4
	One interview	1
	No interviews	1

Conclusion

This article attempted to accomplish several things at once, without doing full justice to the complexity of each of the tasks undertaken. It presented the outline of an ambitious longitudinal study, laid out some of the key operational issues critical to making such an ambitious project a reality, and provided initial findings about the sample obtained and research participant retention. One purpose of this piece is obviously archival. It provides documentation of the operations of the Pathways study to inform future scholars. At the same time, the larger goals here are clearly to convince the reader that this project has taken on an important question, achieved notable initial success, and learned some valuable operational lessons in the process.

It is worth noting that the operational lessons highlighted here do not encompass the full range of issues to consider in mounting a longitudinal study nor are they ones that we “discovered” on our own. The issues presented here are, nonetheless, ones that we see as key considerations when doing research with “deep end” offenders. As stressed above, we are convinced that the currently promising status of the study is attributable largely to operational decisions made along the way. Without careful consideration of site characteristics, we do not think that there would be the desired diversity within the sample. In addition, without the in-depth knowledge of the sites obtained initially, there would not be the extensive cooperation needed to carry out a study of this scope. Without careful review of instruments and testing of new measurement strategies, the available data would have questionable validity and weak spots. Without computerized methods for data collection and organization, the ability to administer and monitor widely distributed data collection efforts would be greatly compromised, and quick management responses to problems would not have been possible. Without a clear and comprehensive set of steps for tracking adolescents and a consciously constructed positive environment for supporting interviewers, it is unlikely that the data set would be so complete. Enrolling, interviewing, and keeping up with serious adolescent offenders is a daunting task. Without a clear plan regarding key issues, the task is much harder and less likely to succeed.

Biographies

Carol A. Schubert is a research program administrator for the Law and Psychiatry Program at Western Psychiatric Institute and Clinic at the University of Pittsburgh School of Medicine. She has managed several large-scale research projects (funded by the National Institute of Mental Health and the MacArthur Foundation) that followed individuals who were violent and mentally ill in the community. She is currently the director and a member of the working group for the Pathways to Desistance Project, a longitudinal study of adolescent serious offenders.

Edward P. Mulvey is a professor of psychiatry and director of the Law and Psychiatry Program at Western Psychiatric Institute and Clinic, University of Pittsburgh School of Medicine. His research has centered on issues related to the use of social service treatment as a method of social control, with an emphasis on the prediction of violence in individuals with mental illness and on treatment systems for serious adolescent offenders.

Laurence Steinberg is the Distinguished University Professor of Psychology at Temple University and the director of the MacArthur Foundation Research Network on Adolescent Development and Juvenile Justice. His research concerns normative and atypical development during adolescence and the influence of parents and peers on psychosocial development.

Elizabeth Cauffman is an assistant professor in the Psychiatry Department at Western Psychiatric Institute and Clinic, University of Pittsburgh. She is a developmental psychologist interested in the assessment of mental health and psychosocial maturity among juvenile offenders, the exploration of factors associated with female delinquency, juvenile psychopathy, and the study of maturity of judgment as it develops during the course of adolescence.

Sandra H. Losoya is a research assistant professor at Arizona State University. She received her Ph.D. in developmental psychology at the University of Oregon. Her interests include emotional development and individual differences in emotional responding, emotional coping, and other sources of resilience in high-risk children.

Thomas Hecker is a licensed clinical psychologist in Pennsylvania and currently assistant dean for Administration and Planning in the College of Liberal Arts at Temple University.

Laurie Chassin is a clinical psychologist, and professor of psychology at Arizona State University. Her research interests are in the area of substance use disorders, including their natural history over the life course, familial intergenerational transmission, and etiological models of risk and resilience

George P. Knight is a professor in the Department of Psychology at Arizona State University. He received his Ph.D. in social psychology, specializing in social development, from the University of California at Riverside. His research interests include the acculturation/enculturation of Mexican American children and families, cross-ethnic/race measurement equivalence, and prosocial development.