Reading Groups: A Practical Means of Enhancing Professional Knowledge Among Human Service Practitioners

Abstract

Quality human service provision is heavily dependent on practitioners maintaining up-to-date professional knowledge. We evaluated a monthly reading group process as a practical means of enhancing professional knowledge among educators in a program for adults with severe disabilities. The reading group process was designed to minimize disruptions to participants' direct contact time with students and to promote participant acceptance of the process. The process followed a structured format involving study questions and systematic prompting to ensure active participation. Comparison of pre- and post-quiz responses to short-answer and multiple-choice questions showed consistent improvement in correct answers following each of four reading group meetings. Survey responses indicated all participants reported the process to be highly acceptable. Results are discussed regarding guidelines for behavior analysts interested in enhancing professional knowledge of other practitioners with whom they work, with a focus on obtaining administrative approval for reading group activities, structuring the activities to promote active participation, and using strategies to enhance participant acceptance.

A critical determinant of the quality of supports and services provided in human service agencies is the professional knowledge of practitioners. Human service practitioners must have up-to-date knowledge of relevant information if agency services are to comport with current standards of practice (Reid, 2010). Because the highest quality services are dynamic in nature in terms of evolving in accordance with the availability of new information, practitioners need professional development opportunities to maintain a current knowledge base (Jacobson, 1990). This is particularly the case with evidence-based practices such as behavior analysis because research is ongoing, with corresponding changes in what constitutes state-of-the-art service delivery (Grimes, Kurns, & Tilley, 2006).

Behavior analysts can play a key role in the professional development of human service personnel, both within an agency in which they work full time and when employed in an ongoing consultant basis. In this regard, despite recent growth in the number of practicing behavior analysts (Shook & Favell, 2008), the number of behavior analysts associated with many agencies is still small relative to other practitioners (Schlinger, 2010). It is generally expected that behavior analysts will help disseminate information about effective practices while working with professionals and other caregivers who have limited experience and training in behavior analysis (Lerman, 2009).

If behavior analysts are to assist in professional development activities they must be able to do so in light of practical barriers existing in many human service agencies. To illustrate, the most common means of professional development frequently involves staff attending conferences and workshops (Tillery, Varjas, Meyers, & Collins, 2010). Such activities can be expensive though, and often represent a financial challenge for agency administration (Test, Flowers, Hewitt, & Solow, 2004). During difficult economic times, agency budgets for these types of professional development activities are also a likely priority for reduction or elimination (Wolfensberger, 2010). Other common means of professional development involve periodic consultant services and within-agency workshops (Tillery et al., 2010). However, paying independent consultants exclusively for professional development services can likewise be expensive and time constraints can impede staff participation in agency workshops (Frieder, Peterson, Woodward, Crane, & Garner, 2009). Related obstacles, reported by many administrators, include reluctance to relieve staff from direct contact time with agency clients for professional development and the cost incurred to find temporary relief personnel for these activities (Test et al., 2004).

Due to challenges associated with time and fiscal constraints, there is a recognized need for time- and cost-efficient ways to provide professional development opportunities for human service staff (Reid, 2010; Test et al., 2004). One possibility in this respect is for behavior analysts to conduct reading groups or journal clubs for selected agency personnel involved in applying behavior analytic procedures (Carr & Briggs, 2010). A relevant reading is selected and then a discussion on the target topic is directed by the reading group leader, a behavior analyst. Such a process requires little if any extra cost to an agency and does not involve staff travel away from the work site. The process also represents a means for practitioners to have contact with the scholarly literature (Carr & Briggs), which in turn is considered an optimum way of staying abreast of scientific developments and other information relevant to human service practice (Normand, 2008). However, despite the apparent appeal of reading groups, there has been a lack of research on the effectiveness of this approach to professional development for practitioners in human service agencies.

The purpose of this investigation was to describe a means of conducting a reading group process with human service practitioners and to evaluate the effects on knowledge enhancement. The reading group process evaluated was developed with several practical considerations based on common obstacles to professional development activities as just summarized. Specifically, concern was directed to minimizing the amount of time the activities required to avoid major disruptions in consumer services provided by participating staff and to enhancing the acceptability of the activity to participating staff.

Method

Setting and Participants

The setting was an adult education program serving individuals with severe intellectual disabilities. The primary location of the program's services consisted of classrooms located in one building in which teaching occurred. The program also provided supported work services in the local community and paid work (contract work and retail manufacturing). The participants were seven teachers and one teacher assistant (who was working on a teaching degree). These eight practitioners represented the group who participated in all intervention procedures. Each teacher was responsible for the services provided to adult students within a respective classroom and the teacher assistant worked in one of the classrooms. All teachers were licensed to teach special education, four had a bachelor's degree, and three had a master's degree. Six of the participants were women. Ages ranged from 30 years to 53 years (M = 45 years) and years of experience ranged from 1 year to 30 years (M = 14 years). The participants were not certified as behavior analysts, although all had received some training in behavior analytic procedures within their academic teacher-training programs. All intervention procedures were carried out by the participants' supervisor (first author), who directed the adult education program and had extensive training and experience in behavior analysis.

Dependent Measures

There were two dependent measures, each involving answers to questions related to the reading for a designated reading group meeting (there was one reading and one reading group meeting each month). The first was narrative answers written in response to short-answer questions. There was one short-answer question for the reading for each reading group meeting. Each question was developed when the reading was initially selected for a given group meeting, along with criteria for a correct answer. A participant's answer to each question was scored as correct or incorrect according to the criteria. The second dependent measure consisted of answers to four multiple-choice questions, also for the reading covered within each reading group meeting. There were four possible answers listed for each question, with only one representing the correct answer. The short-answer and multiple-choice questions were developed by the reading group leader (supervisor of the participants) and the second author. Each question pertained to key information presented in a respective reading as determined jointly by the experimenters. Each multiple-choice and short-answer quiz for a respective reading topic remained the same across all quiz administrations for that reading.

Interobserver agreement checks for responses to the short-answer questions were conducted for 94% of the quiz administrations during each experimental condition. Copies of the answers were reviewed independently by the experimenters and then their recordings of correct and incorrect answers were compared. Reliability for occurrence, nonoccurrence, and overall agreement for correct answers was determined on an answer-by-answer basis for each participant's quiz using the formula of number of agreements divided by number of agreements plus disagreements, multiplied by 100%. Overall agreement across quizzes averaged 85% (range, 67% to 100%), occurrence agreement averaged 80% (range, 33% to 100%), and nonoccurrence, or occurrence of an incorrect answer, averaged 83% (range, 0% to 100%). The lower ranges were due to small numbers of disagreements that deflated the averages. There were never more than two disagreements across all participant answers for a quiz for a given reading.

Experimental Conditions

Baseline. The baseline, or pre-reading group meeting condition, involved the following. First, a memorandum was sent to all participants informing them that during the upcoming year, on-the-job professional development activities would focus on holding a reading group meeting on a monthly basis. Three potential topics for respective reading group meetings were listed on the memorandum (based on what the supervisor thought would be useful and of interest to participants): the concept of evidence-based practices, strategies for teaching individuals with autism (with examples provided of pivotal response teaching and discrete trial teaching), and ways to promote generalization of newly taught skills. The memorandum next requested the participant to list three topics at the bottom of the page that s/he was interested in that may be addressed with respective reading group meetings. Participants were requested to complete the listing and turn it in to the supervisor's office (all participants turned in a completed listing).

The second process involved the supervisor reviewing the participants' topics and selecting those topics that were listed by the most participants. The supervisor also ensured that potential topics appeared relevant to the participants' general job expectations (none of the topics listed by multiple participants were deemed irrelevant). The supervisor also selected topics that she considered relevant for her staffs' professional knowledge development. The four topics pertaining to respective reading group meetings that were the focus of this investigation (first four monthly reading group meetings that were conducted) were the concept of evidence-based practices, characteristics of the disability of autism, specific teaching strategies for individuals with autism, and seizures among individuals with severe disabilities (two topics selected by the supervisor and two by the participant consensus).

A reading was then selected for each topic. Readings were selected that provided an overview of the target topic and anticipated to require no more than 1 hour of participant reading time. Each selected reading consisted of a book chapter or specific parts of a chapter (references for readings are available from the experimenters) that provided a review or summary of relevant research or information. Chapters were selected in contrast to investigative journal articles because of the lack of comprehensive training of the participants in behavior analysis and it could not be safely assumed participants had acquired a repertoire for critically evaluating research articles (Carr & Briggs, 2010). Next, six or seven study questions were developed for each reading to guide the participants through the respective reading (Saville & Zinn, 2009). Subsequently, a quiz was developed for each reading, consisting of the four multiple-choice questions and one short-answer question as described previously. The quiz questions and study questions covered the same material but were not identical. For example, one study question for the reading on the characteristics of autism was “How is Asperger's Disorder different from an Autistic Disorder?” whereas the short-answer question on the quiz was “What are two characteristic differences between Autistic Disorder and Asperger's Disorder?” Examples of quiz questions are provided in Table 1.

Table 1.

Samples of Quiz Questions

The next baseline procedure involved providing quizzes to each of the eight participants prior to a scheduled reading group meeting. Two quizzes were combined into one form such that there were eight multiple-choice questions (four for each of two respective readings and corresponding reading group meetings) and two short-answer questions (one for each of two readings and reading group meetings). Hence, two quizzes were always administered simultaneously, although the pairing of the two respective quizzes sometimes varied across administrations.

To administer the quizzes, the staff supervisor sent an announcement to the participants to report to her office during the week to complete assigned quizzes. Upon reporting to the office, the participants were given the quizzes (i.e., two quizzes administered at a time on one form) to complete in her office or the adjoining secretary office. There was a maximum of one administration of quizzes per week, representing pre- and/or post-reading group meeting quizzes depending whether a reading group meeting had occurred for the reading targeted by a respective quiz. No feedback was provided to participants regarding the correctness of their answers.

Reading group meetings. Reading group meetings were held on a monthly basis as indicated previously and were limited to an hour. The reading group leader obtained approval from executive personnel of the agency to conduct monthly reading group meetings as a professional development activity. There was a consensus that the meetings should be limited to 1 hour to avoid major disruptions in the participants' work with their adult students. Reading group meetings were scheduled during times when participants were not assigned to be with students, such as during their 1-hour, daily planning time or on a teacher workday. Participation in the reading group process was voluntary in accordance with agency policy.

Within two weeks prior to a scheduled reading group meeting, the supervisor provided each group participant with a copy of the selected reading and the corresponding study questions (after completion of all pre-reading group meeting quizzes). Subsequently, each reading group meeting consisted of the following. First, participants signed a form to allow for agency-sponsored continuing education units. Second, the group leader provided a rationale for the topic of the reading group meeting in terms of how information in the reading could be relevant for the participants. For example, for the reading on the concept of evidence-based practices, it was explained that intervention procedures with a scientific research base that has documented procedural effectiveness are more likely to be successful relative to procedures without an underlying research base. Third, the leader asked participants for their general impression of the reading and ensured each participant provided a response. Fourth, main points of the reading were discussed by asking a participant to answer a study question and then prompting opinions from other participants as to whether they agreed with the answer and if not, how they would change the answer. This process was used with each study question, again with the group leader prompting respective participants for responses if necessary to ensure each participant partook in the discussion. Fifth, the group leader asked participants to relate a key point from the reading to their personal on-the-job experience. For example, for the reading on autism characteristics, participants were asked to think of someone on their caseload with the diagnosis of autism and then indicate what characteristics described in the reading were applicable to that student. Sixth, after all study questions were addressed, the group leader summarized the main points of the reading, using the study questions as a guide and then asked if there were any remaining questions. Seventh, the group leader thanked participants for their involvement. Finally, a quiz was administered in the manner described previously, which involved the first post-quiz for that reading group meeting combined with a quiz for another reading. The latter quiz represented either a pre- or post-reading group quiz administration in accordance with a multiple-probe format (see Experimental Design).

There were also several procedures designed to enhance participant acceptance of the reading group meetings. Specifically, during two reading group meetings refreshments were provided. During another meeting, a lottery was conducted based on the completed study questions turned in by the participants. One set of questions was randomly drawn and the winner received a $5 cash prize. Another reading group meeting involved providing a free pass to a local movie theater. Additionally, the reading group leader praised and thanked participants for correct answers to questions she posed during each reading group meeting.

Post-reading group meeting. This condition involved administering the same quizzes in the same manner as during baseline (e.g., no feedback was provided regarding the correctness of quiz answers). It should also be noted that the participants had access to the readings previously provided by the supervisor.

Following the last reading group meeting that was part of the investigation, the reading group process continued on a monthly basis as an administratively approved, routine part of the education program's professional development activities. Follow-up assessment of the effects of the process was conducted for monthly reading group meetings that occurred at 1 and 3 months following completion of the study proper. Each of the latter two reading group meetings (with two separate readings) was accompanied by one pre- and one post-quiz administration, with these same participants.

Acceptability Survey

Following the fourth reading group meeting, an acceptability survey was completed by each participant. The survey consisted of three questions with response options on a seven-point Likert scale and one question with “yes” or “no” response options. The first question asked the participants how useful or nonuseful the reading group process was for learning information relevant to their job and profession, with response options ranging from (1) extremely nonuseful to (7) extremely useful. The second question asked how practical or impractical the reading group process was in terms of amount of participant time and work, with response options ranging from (1) extremely impractical to (7) extremely practical. The third question asked how enjoyable or not enjoyable the reading group process was, with response options ranging from (1) extremely not enjoyable to (7) extremely enjoyable. The final question asked participants if they wanted the reading group process to continue. Participants were asked to complete the survey anonymously (i.e., not to sign the form and place it in a folder so all forms were grouped together and the supervisor could not identify who completed each form).

Experimental Design

The experimental design was a multiple baseline design across four consecutive, monthly reading group meetings.

Results

Figure 1 presents the percentage of correct quiz answers averaged across all eight participants for each quiz administration. As indicated on Figure 1, the average percentage correct answers was higher for all post-reading group meeting quizzes relative to all pre-quizzes for each respective meeting for both the short-answer and multiple-choice questions. Average correct answers for the short-answer pre-quizzes for the reading group meetings was 12% (range, 0% to 50%), which increased to 80% (range, 43% to 100%) for the post-quizzes. The averages for the multiple-choice questions increased from 29% (range, 13% to 46%) on the pre-quizzes to 88% (range, 54% to 100%) on the post-quizzes.

Figure 1.

Mean percentage of correct answers for all participants for the short-answer and multiple-choice questions for all quiz administrations during each experimental condition.

Results for individual participants coincided with the group averages (see Table 2). The average percentage of correct answers increased for all participants from pre- to post-quizzes for both the short-answer and multiple-choice questions. The average percentage of correct short answers increased for all participants by at least 33 percentage points from pre- to post-quiz administrations. The average percentage of correct answers on the multiple-choice questions for all participants increased by at least 46 points from pre- to post-administrations.

Table 2.

Average Percent Correct Participant Answers to Pre- and Post-Reading Group Meeting Quizzes

Follow-up results indicated the reading group meetings continued to be accompanied by improved quiz answers. For the reading group meeting that occurred 1 month after the study concluded, answers for the short-answer questions averaged 0% correct across participants on the pre-reading group meeting quiz administration and 75% on the post-reading group administration. For the reading group meeting that occurred 3 months after the study, averages increased from 13% to 88% from pre- to post-reading group meeting quiz administrations. For the multiple-choice questions, average correct answers increased from 81% to 100% and from 55% to 89% for the two separate reading group meeting pre- and post-quiz administrations.

Results of the acceptability survey indicated participants reported the reading group process to be acceptable. For the question regarding the usefulness of the process, all participants indicated the process was extremely useful (rating of 7) or very useful (rating of 6), with an average rating of 6.4. For the question regarding the practicality of the process, all participants rated extremely or very practical, with an average rating of 6.4. Regarding the question pertaining to enjoyment, all participants rated extremely or very enjoyable, with an average of 6.3. For the question about desiring the reading group process to continue or not, all participants reported they preferred the process to continue.

Discussion

Results suggested the reading group process increased knowledge among the participants based on improvements in correct answers to questions pertaining to the reading topics. Consistent increases occurred in correct answers to both the short-answer and multiple-choice questions following each reading group meeting relative to pre-reading group meeting answers. The improvement should be qualified for the multiple-choice answers in that for the three reading groups with repeated administrations of the post-quizzes there were decreases in correct answers across consecutive post-quizzes. However, the averages for all multiple-choice post-quizzes were still always well above the averages for all pre-quizzes for respective reading group meetings. The decreases that did occur were somewhat expected given that no feedback was provided regarding the correctness of answers. Feedback was not provided to allow an evaluation of the effects of the reading group process. It is recommended that feedback be provided to participants regarding their answers when using reading group processes as a professional development activity in routine practice.

The reading group meeting process was well received by participants. All participants indicated acceptance of the reading group process based on answers to questions regarding its usefulness, practicality, and enjoyment and all reported they preferred the reading group process to continue. However, these acceptability data should be considered with some caution. Despite the common use of surveys to assess acceptability in behavior analysis research, survey data are not always valid indicators of participant acceptance (Parsons, 1998). Some additional support for participant acceptance of the reading group process may be reflected in their consistent attendance at the reading group meetings, which was voluntary. All participants who were present at work on the days of reading group meetings attended all meetings. The only absences for reading group meetings occurred when a participant was not at work due to agency-approved medical leave (across all reading group meetings attendance averaged 87%).

As indicated previously, the intent of the reading group process was to enhance professional knowledge. Although professional knowledge among practitioners is an important determinant of the quality of an agency's services, knowledge is only one necessary component in this regard. Practitioners must also display competent skills in implementing recommended practices. Although this investigation did not focus on participant skills in implementing various procedures, there is a well-documented technology for training practitioners in relevant skills (see Reid & Fitch, 2011, for a recent review of behavioral staff training research). Future research could further examine how to train new skills to human service practitioners as one aspect of professional development. In particular, how to provide such training in a time- and cost-efficient manner seems warranted in light of the noted time and fiscal constraints in many human service agencies (Frieder et al., 2009; Test et al., 2004). One possibility in this regard would be to alternate professional development endeavors between activities focused primarily on knowledge enhancement and skill acquisition.

Simply giving practitioners selected readings and administering quizzes may suffice for enhancing relevant knowledge, without the extra time involved with the reading group meetings. The meetings may be helpful in other ways though. For example, the meetings allow opportunities to potentially expand knowledge enhancement, such as by the questions posed by the reading group leader regarding how information in respective readings relates to the participants' own work. Additionally, the meetings may enhance acceptability of the overall reading group process. To assess the latter possibility, during the follow-up period the supervisor asked each participant if s/he would prefer in the future to just receive the readings and quizzes or continue the existing process that included the reading group meetings. All eight participants indicated they preferred to continue the reading group meetings. Future research could address the relative effects of providing readings and quizzes alone versus in conjunction with the reading group meetings on quiz scores, acceptability, and time efficiency.

Even with the reading group meetings, the reading group process appeared at least relatively efficient for enhancing professional knowledge when compared to more traditional professional development activities summarized earlier. For the participating practitioners, the reading group process encompassed only 1 hour monthly for the reading group meeting, generally less than 1 hour to read the selected reading, and a maximum of a half hour to complete an evaluative post-quiz. However, the time investment for the reading group leader was more significant (selecting and retrieving a reading, making quiz questions, etc.). When considering the apparent effectiveness of the reading group process though, the latter time investment would seem worthwhile pending future research on means of streamlining this type of professional development activity.

Guidelines for Practitioners

In light of the results and accompanying qualifications, the following guidelines are offered for behavior analysts interested in enhancing professional knowledge among practitioners with whom they work. First, obtain approval from agency administrators prior to initiating a reading group process. Such approval can be important to help overcome contingencies operating within some agencies that mitigate against time spent in professional development (Carr & Briggs, 2010). The likelihood of obtaining such approval would seem to be enhanced if efforts are made to minimize the amount of time to participate in the reading group meetings, with corresponding efforts to schedule the meetings to avoid significant interruptions in direct contact time with agency consumers.

A second guideline is to conduct reading group meetings in a structured manner to enhance individual participation. In the current investigation, the reading group process involved study questions to guide participants through each reading and relatively short quizzes to assess their knowledge of reading content. The group meetings were also conducted in a structured and consistent manner, such as by the group leader initially summarizing the intent of the reading and requiring participant responses to each study question.

A third guideline for behavior analysts is to actively strive to make the reading group process acceptable to participants. Generally, the more acceptable a given process is to staff participants, the more likely the process will be viable over time within an agency (Parsons, 1998). A number of procedures were included to potentially enhance participant acceptance of the reading group process. These included offering continuing education credits for participation, soliciting participant input into the reading topics, and procedures associated with respective meetings (e.g., providing refreshments, free passes to a movie theater). There is a relatively well-established evidence base indicating that involving participants in decisions affecting their on-the-job activities, such as by soliciting input into reading topics, can enhance their acceptance of the activities (see Phillips, 1998, for a review). Hence, it is strongly suggested that reading group leaders involve participants in selection of the target readings.

In summary, it is recommended that behavior analysts interested in a reading group process as a professional development activity to enhance knowledge among human service practitioners consider the following guidelines: (a) obtain administrative approval for the process, which seems likely to be enhanced if the process is developed to minimize staff time involvement and disruption to client services, (b) structure the process to require active participant involvement centered around identifying key information in each reading, and (c) include procedures to make the process acceptable to participants and especially by involving them in decisions affecting the reading topics. Following these guidelines would appear to be a relatively practical means of helping practitioners enhance job-related knowledge and have contact with relevant professional literature.