Maximizing Efficiency of Enrollment for School-Based Educational Research

Abstract

In this observational study, influences on research enrollment were examined in the context of an educational research study. Recruitment materials were sent to parents of kindergartners in one of 23 classes across 7 public schools, as an invitation to enroll in a longitudinal study. For the 7 classes from 2 of the schools, the school principal prepared a cover letter as an introduction to the educational study. Inclusion of this cover letter was associated with a significantly swifter pace of recruitment and, to a lesser degree, with an increase in the number of children enrolled. However, when potential confounding variables were controlled, no significant increase in final enrollment count was associated with including the cover letter. Thus, inclusion of a cover letter from the school principal did not appear to dramatically increase the number of parents who elected to enroll their child in the educational study, and it did significantly decrease (by 6 weeks) the number of weeks required to achieve final enrollment counts.

The recent movement toward school reform is one of several factors potentially associated with an increase in school-based educational research. Federal funding for education research, broadly defined, increased by 48% between 1980 and 1998, with $17.1 billion allocated to education research in 1998 (United States Department of Education, 1998). It is therefore timely and important to recognize factors that might influence the efficiency of carrying out school-based research, particularly with respect to challenges unique to education researchers. The finite academic-year schedule imposed on data-collection efforts is one such challenge. Whereas data-collection delays in a laboratory setting might result in additional weeks or months of data-collection efforts, delays in school-based research can ultimately lead to reduction in sample size if additional data cannot be obtained once the school year has concluded or to significant delays related to waiting for subsequent academic years. As such, successful research in a school setting would be enhanced by efficient participant enrollment. The present report pertains to potential influences on recruitment efficiency for a study that was recently carried out in 23 public school kindergarten classrooms.

Influences on recruitment have been examined in the context of medical research and medical compliance, and in the context of general influences on human behavior. There is long-standing, well-established support for the notion that authority figures exert influence on individuals' decision making across a variety of settings (e.g., Cialdini, 1993; Milgram, 1974). Physicians are one type of authority figure that might influence enrollment in medical research. Findings from two recent studies indicate an increase of approximately 15% to 17% in subject enrollment among parents who were introduced to a genetic screening study by a familiar physician, rather than by an unfamiliar research assistant (Mazzocco, Myers, Harum, & Reiss, 1999; Mazzocco, Sonna, Shapiro, Pinit, & Reiss, 1998). This effect was evident despite minimal familiarity between the parent and physician, who in many cases became acquainted hours before study enrollment. The effect was seen for parents of school-age children and parents of preschoolers, and for parents of children with mild or moderate developmental delays.

There is evidence that these effects generalize to other studies. For instance, primary care physicians' recommendations significantly influenced whether women chose to enroll in a breast cancer prevention study (Kinney, Richards, Vernon, & Vogel, 1998). The magnitude of the effect in that study (1,300%) far exceeded the 15% to 17% increase reported by Mazzocco and colleagues (1998, 1999). However, in the breast cancer prevention study, the participants were free to deliberately seek the advice of their primary care physician, and the nature of enrollment in that study differed significantly from the requirements of enrolling in a genetic screening study. In a study where physician involvement included only an introduction to a study presented to cancer patients, the introduction did not appear to influence enrollment decisions (Sugarman, Regan, Parker, Bluman, & Schildkraut, 1999). Taken together, these findings illustrate how multiple factors interact with the role of an influential authority figure.

Despite the role of interacting factors and limitations, it follows from these findings that endorsement of a research project by a familiar authority figure might enhance the rate of enrollment in the study. The extent to which these findings generalize to a school-based setting, and to parents of children without developmental delays or learning disabilities, has not been established. The aim of the present study is to examine potential support for this notion.

In this report, enrollment rates are compared for a study carried out in seven elementary schools within one public school district. Of interest is whether an overall higher or more rapid enrollment rate results from having the study introduced to parents by the school principal and the principal investigator of the study, rather than by the study's principal investigator alone. Both the school principal and the principal investigator are authority figures; however, the degree of parents' familiarity with these two authority figures differs.

The question addressed through this study might have important implications for carrying out school-based research more efficiently, thus in a more time- and cost-efficient manner. With higher enrollment rates, fewer schools are needed to achieve a target sample size. More rapid enrollment would mean less down time of research assistants, and a more continuous data-collection phase of a study. An increase in the percentage of participants enrolled could yield a less biased sample. These are some potential effects of enrollment rates.

Method

The recruitment rate examined in this study was for enrollment in a school-based, prospective longitudinal study of math learning disability (MLD). Participants for the MLD study were sought from all regular, half-day kindergarten programs in each of seven participating schools, all of which were in one large school district. The MLD study had received approval from the school district's Office of Research, and participation by each school in the district was strictly optional and left to the discretion of each school principal contacted by the researchers. Schools were approached for participation in the MLD study if they met selected criteria, which were established to enhance participant retention for the 4 years of the study and to increase homogeneity of socioeconomic background within the research sample. These criteria included a relatively low rate of mobility and a relatively low percentage of children eligible for free- or reduced-price lunch (Table 1).

Table 1. Description of Participating Schools.

	Recruitment information sent with principal's letter		Recruitment information sent alone
School	A	B	C	D	E	F	G
Prior year mobility (%)	18.00	16.20	18.90	16.20	19.50	20.10	6.80
Free- and reduced-price lunches (%)	12.67	18.70	29.04	14.71	27.69	11.11	1.58
Number of letters actually sent	63	75	57	58	79	50	59
Children enrolled in 2 weeks	61.90% (39)	53.33% (40)	36.84% (21)	3.45% (2)	24.05% (19)	18.00% (9)	37.29% (22)
Children enrolled in 4 weeks	1.59% (1)	4.00% (3)	7.02% (4)	15.52% (9)	12.66% (10)	0.00% (0)	8.47% (5)
Children enrolled in 6 weeks	0.00% (0)	6.67% (5)	5.26% (3)	13.79% (8)	3.80% (3)	8.00% (4)	5.08% (3)
Children enrolled in 8 weeks or more	3.17% (2)	0.00% (0)	5.26% (3)	27.59% (16)	20.25% (16)	12.00% (6)	3.39% (2)
Total enrolled	66.67% (42)	64.00% (48)	54.39% (31)	60.34% (35)	60.76% (48)	38.00% (19)	54.24% (32)

Note. The number of children enrolled appears in parentheses following the corresponding percentages.

A written invitation to participate in the MLD study was sent to 31 elementary school principals. Principals at 9 of these 31 schools agreed to participate in the study. Scheduling conflicts prevented two schools from being included, leaving seven participating schools. Among the 22 school principals who did not accept the invitation to participate, 2 gave no response. For the remaining 20 principals, 7 declined without providing a reason for their decision. Reasons provided by the remaining 13 principals were either that other research studies were being conducted in the school (n = 6), ongoing needs prevented time to participate (n = 4), the school was in the process of redistricting (n = 1), or there was a lack of space for research-based assessment (n = 2). Among the seven schools participating in the MLD study, all were included in the present study.

Participants

Upon agreeing to participate in the MLD study, the principal at one school (hereafter referred to as School A) asked us to include a cover letter addressed to the parents of potential MLD study participants with the recruitment materials. We then asked a principal from another one of our seven schools (School B) to provide a cover letter modeled from Principal A's letter. Thus the same principal's letter was sent from two schools.

As seen in Table 1, the demographics of Schools A and B differed only slightly on the mobility and free- or reduced-lunch criteria. Both schools were in suburban neighborhoods, approximately 20 miles (32 km) apart. Participation in the MLD study was open to all children enrolled in a regular half-day kindergarten program in one of the seven schools, with the exception of children identified as having an insufficient grasp of English for valid study participation.

Materials and Procedure

Identical enrollment materials were distributed to parents of all eligible children, through the child's classroom teacher. The introductory letter from the principal investigator (hereafter referred to as the researcher) of the study was distributed either with or without a cover letter from the school principal. The principal's cover letter was a brief endorsement of the importance of the MLD study. The researcher's introductory letter provided a one-page summary of the objectives of the MLD study, background about the researcher, and procedures for enrollment. A response form attached to the introductory letter was used to inform the researchers whether the child would be enrolled or whether participation was declined.

The same study information, with (at Schools A and B) or without (at Schools C through G) the principal's cover letter, was sent to each parent a variable number of times. Once a parent indicated a decision to enroll or not enroll in the MLD study, no further materials were sent to the parent. Identical sets of materials were sent approximately 4, 6, and 8 weeks after the initial distribution of recruitment materials, only to parents who had not yet indicated a decision about participation.

Results

The hypotheses addressed in the present study concern whether an introductory letter from a school principal would be associated with (a) an increase in participant enrollment, and (b) a more rapid rate of enrollment, relative to enrollment patterns observed without an introductory letter. Frequency rates (as seen in Table 1) were examined using chi-square analyses. Each set of analyses included one analysis to evaluate a possible effect of principal letter on final recruitment counts. Separate analyses were conducted for the initial recruitment mailing and two of the following subsequent mailings. Thus, enrollment figures were examined following the initial mailing, and 4 and 6 weeks later, in order to examine cumulative rates of enrollment, and final enrollment counts 2 months after initial recruitment. The latest plateau in enrollment counts was 2 months, so enrollment up until 2 months was examined in all schools in the study.

Primary Analyses Including All Seven Schools

Enrollment frequencies were calculated as a function of whether materials were sent with or without a cover letter from the child's school principal. Data were collapsed for the two schools where materials included a cover letter, and for the five schools where no cover letter was sent. Following the initial contact concerning recruitment for the study, enrollment was significantly higher among parents who received a cover letter with the enrollment materials, relative to the rate observed among parents who did not receive a cover letter, χ²(1, N = 441) = 46.14, p < .0001. Cumulative enrollment rate differences across these two groups persisted after subsequent distributions of enrollment materials, such that at 4 to 6 weeks after the initial recruitment effort, enrollment remained higher at schools where the principal had included a cover letter with the materials, χ²(1, N = 441) = 28.03, and χ²(l, N = 441) = 21.00, respectively, ps < .0001. The effect size on overall recruitment rate was smaller 2 months following the initial recruitment efforts, but still statistically significant and in the same direction, χ²(1, N = 441) = 4.50, p < .05.

Primary Analyses With a Subset of Five Schools

The analyses completed with all seven schools were repeated with five schools only, excluding the two schools most discrepant from Schools A and B on socioeconomic status or mobility indexes. Among these five schools, within 2 weeks of initial recruitment for the study, enrollment was higher among parents who received a principal's cover letter with the enrollment materials, relative to the rate observed among parents who received enrollment materials without a cover letter, χ²(1, N = 303) = 46.38, p < .0001. Cumulative enrollment rate differences across these two groups persisted after subsequent distributions of enrollment materials, such that at 4 to 6 weeks after the initial recruitment effort, the enrollment rates remained higher at schools where the principal had included a cover letter with the materials, χ²(1, N = 303) = 33.28 and 22.59, respectively, ps < .0001. Over time, there was a decrease in the degree to which cumulative enrollment rates differed, as seen in the statistics. There was a smaller effect size 2 months following initial recruitment efforts, but nevertheless a significant difference in overall recruitment rate, χ²(1, N = 303) = 5.78, p < .05.

Primary Analyses Using Two Subsets of Two Similar Schools

Analyses were repeated using only two schools, one from each of the two groups (with or without cover letter) that were the most similar in size and mobility data (Schools B and E in Table 1, respectively). The pattern of findings was similar to that observed in the initial analyses: Enrollment was significantly higher at School B, relative to School E, following the initial recruitment attempt, χ²(1, N = 154) = 13.96, p < .001; and cumulative enrollment differences diminished but remained significant 4 to 6 weeks after the initial recruitment, χ²(l, N = 154) = 6.75, p < .05; and 8.51, p < .01, respectively. However, in this set of analyses, the gradual decrease in the magnitude of the differences was reflected in overall comparable enrollment counts 2 months following initial recruitment (p = .67).

Analyses were also repeated using two schools (one with cover letter and one without cover letter) that had the most similar mobility and free- or reduced-lunch indexes (Schools B and D in Table 1, respectively). Enrollment was significantly higher at School B, relative to School D, following the initial recruitment attempt, χ²(1, N = 133) = 37.67, p < .0001; and cumulative enrollment rate differences diminished but remained significant 4 to 6 weeks after the initial recruitment, χ²(1, N = 133) = 19.96, p < .0001; and χ²(1, N = 133) = 12.77, p < .001, respectively. As was true for the other pair of schools, in this set of analyses, the gradual decrease in the magnitude of the differences ultimately reflected no significant difference in overall enrollment counts 2 months following initial recruitment (p = .72).

Secondary Analyses to Examine Potential Confounds: Date of Enrollment

The schools included in the study were enrolled at different points during the school year, and thus a potential influence on promptness of parents' responses is the number of weeks remaining in the school year at the time of recruitment. To address this potential confound, an additional set of analyses was carried out using data from the three schools for which enrollment materials were all initially sent to parents in February (Schools A, C, and F). The results were similar to those reported earlier, in that the percentage of children enrolled in the study 2 weeks following initial enrollment was higher in School A, relative to Schools C and F, χ²(1, N = 170) = 18.86, p < .0001. The differences were significant and in the same direction also 4 weeks, χ²(1, N = 170) = 16.23, p < .001; 6 weeks, χ²(1, N = 170) = 10.07, p < .01; and 8 weeks, χ²(1, N = 170) = 6.35, p < .02, following initial recruitment, with a diminished but significant effect size by the end of the 8-week interval.

Secondary Analyses to Examine Potential Confounds: Socioeconomic Status

Within the group of five schools at which recruitment materials did not include a cover letter (Schools C through G), enrollment counts were examined as a function of percentage of students qualifying for free- or reduced-price lunches; this criteria was used as a global index of socioeconomic status (SES). No significant correlation emerged 2 weeks following initial recruitment efforts (Spearman Rho corrected for ties = -.054, p = .35). Although rates of enrollment were correlated with the SES index 8 weeks after the initial recruitment attempt (Rho corrected for ties = .586), the association was positive. In schools with a higher free-lunch index (School E), enrollment counts were higher.

Secondary Analyses to Examine Potential Confounds: Classroom Differences

Finally, to address whether additional main effects influence enrollment rate, final enrollment rates across classrooms were examined within each school. Among the seven schools in the study, interclassroom differences varied from virtually nonexistent (63.9% vs. 64% enrollment rates, overall) to large (37.5% vs. 61%), as seen in Table 2. For Teachers 3-1 and 3-2, even this difference was not statistically significant, at the initial recruitment period (p = .55), nor 2 months later (p = .14). However, this lack of a significant difference might have resulted from insufficient statistical power, given the diminished sample sizes when enrollment rates are examined within single classrooms. Moreover, although teacher-related variables very well might influence enrollment counts, the limitation of this potential variable is illustrated by the discrepant enrollment rates across the morning and afternoon kindergarten student body for the same teacher. In the most extreme case, enrollment was at 85% and 41% for one teacher's two kindergarten classrooms.

Table 2. Total Enrollment Rates by Teacher and Classroom.

Teacher number	Class total	Total number enrolled	Percentage enrolled
1-1a	19	12	63.16
1-1b	18	12	66.67
1-2a	23	17	73.91
1-2b	19	7	36.84
2-1a	21	7	33.33
2-1b	16	10	62.50
2-2a	22	15	68.18
3-1a	21	14	66.67
3-1b	20	11	55.00
3-2a	16	6	37.50
4-1a	17	7	41.18
4-1b	20	17	85.00
4-2a	21	11	52.38
5-1a	18	8	44.44
5-1b	12	4	33.33
5-2a	20	7	35.00
6-1a	22	14	63.64
6-1b	18	11	61.11
6-2a	23	17	73.91
7-1a	21	14	66.67
7-1b	18	11	61.11
7-2a	19	14	73.68
7-2b	17	9	52.94

Note. Teachers are identified by numbers (1-1 to 7-2), and classrooms are identified by letters (a or b). Each teacher had one or two classrooms.

Discussion

In this observational study, inclusion of a cover letter with study recruitment materials was associated with a significantly higher initial pace of successful recruitment. To a lesser degree, the presence of a cover letter also increased the overall number of children enrolled. These findings have important implications for efficient management of school-based studies, in terms of accelerating reaching the plateau in final enrollment counts—in this case, 2 versus 8 weeks following initial recruitment efforts.

A decrease of 6 weeks in recruitment efforts affords more time for carrying out the school-based research, particularly when the study can typically be carried out during approximately 7 months of the academic schedule (the 9-month school year minus several initial and final weeks, holiday periods, and days with special scheduled activities). If recruitment activities continue an additional 6 weeks, this means that approximately 21% of available testing time is lost to recruitment activities. This figure increases if fewer than 7 months of testing time is available at a given school. Staff activities turned sooner to data collection rather than to continued recruitment efforts also enhance research efficiency. The efficiency is beneficial to the school in terms of fewer days (or months) during which the researchers' presence is necessary at the school.

The final enrollment counts were not as significantly affected by the presence of a cover letter as was the rate of enrollment. Final enrollment figures were similar across the schools regardless of whether the school principal sent a cover letter with enrollment materials. However, inclusion of a principal's letter did result in quicker enrollment. For a subset of analyses where potential confounding variables were controlled, there was no significant difference in final enrollment count as a function of including a principal's cover letter. This finding is a positive one in that it suggests that the cover letter facilitated parents in deciding whether to enroll in the MLD study, yet parents who chose not to enroll their child apparently were not persuaded to ignore that choice. This finding is important in view of researchers' desire to avoid coercive practices while improving the efficiency of research procedures.

Despite their importance, these findings have limitations. The overall number of schools included in the study was small, and only two schools provided a school principal's cover letter. However, the total sample size was 441 children from 23 classrooms across seven participating schools. The findings are limited to those schools where the principal agreed to participate in the MLD study. Moreover, the relationship between the school principal and the student body's parents might have differed for the principal who requested including a cover letter, and this relationship might be a confounding variable. This notion is supported by the differential effect on enrollment seen between individual classroom teachers within a school. However, the second school to include a principal cover letter did so at the request of the researchers; outcome data from these two schools were comparable. Finally, factors in addition to those addressed in this report clearly influence a parent's decision regarding research participation, as supported by the discrepant rates of enrollment observed across classes with the same teacher. Other findings of interest were that schools in areas with relatively higher SES ratings did not necessarily demonstrate higher enrollment nor a more expedient rate of enrollment. This finding was contrary to our expectations, and contrasted with the anecdotal observations often made regarding higher enrollment figures among schools in these demographic areas.

Regardless of these limitations, the effects observed in this study warrant the following consideration when preparing for school-based research: Inclusion of a school principal's cover letter does not appear to dramatically increase the number of parents who enroll, and thus does not constitute a coercive practice. Inclusion of the cover letter does significantly decrease the number of weeks required to achieve final enrollment counts. This increases the number of weeks and personnel hours dedicated to data collection versus recruitment, and decreases the amount of time needed for researchers to be present in the school. In this study, including the principal's cover letter had these positive effects on carrying out an important school-based research project.