Increasing Fiction Writers' Productivity through an Internet-Based Intervention

Abstract

Ten participants from an online community of fiction writers were exposed to a treatment package intended to increase their writing productivity. The package consisted of graphic feedback provided through a Web page, praise for goal completion delivered via e-mail, and editing of manuscripts by other members of the community dependent on the completion of word-count goals. A multiple baseline design across groups was used to evaluate the effects of the intervention, which was successful in increasing the number of words written.

A potentially important application of computer technology involves Internet-based treatment of behavior problems. Coupled with an Internet connection and appropriate software, personal computers allow behavior-change professionals and research participants or clinical clients to interact with little delay, albeit indirectly, regardless of the spatial distance between them. The feasibility of this approach is evident in a recent review of Internet-based interventions conducted by Ritterband et al. (2003), who described 12 studies in which such interventions were demonstrated to be useful in treating numerous problems such as cigarette smoking, obesity, diabetes, headaches, and encopresis. Interestingly, these studies focused on behavioral medicine or health psychology issues, perhaps because such problems have been effectively treated with highly structured approaches that can be readily adapted to the Internet. Consistent with this notion, the only study published in the Journal of Applied Behavior Analysis that describes an Internet-based intervention is one in which cigarette smoking was reduced using the Internet to monitor behavior and to deliver consequences (Dallery & Glenn, 2005).

Although health-related behaviors may be especially good targets for Internet-based interventions, such interventions should be appropriate for changing any form of behavior, as long as the behavior produces an electronic permanent product and the intervention can be arranged without direct therapist–client interactions. For example, interventions designed to increase creative writers' productivity (e.g., Boice, 1983; Dillon, Kent, & Malott, 1980) may be amenable to Internet-based delivery. The purpose of the present study was to extend work on Internet-based behavioral interventions to creative writing. In this investigation, the impact on words written by fiction writers was evaluated in response to Web-page-based graphic feedback, praise for goal completion delivered via e-mail, and online editing of manuscripts.

Method

Participants

Ten participants were recruited from the population of an international online fiction writing community comprising approximately 4,000 people. Eight participants were located in the United States, 1 was located in the United Kingdom, and 1 was located in Ireland. The research project was briefly described in a weekly newsletter e-mailed by the Webmaster to all members of the community. Those who were interested downloaded a packet that contained consent forms and detailed information about procedures. The packet asked individuals who wished to participate in the study to complete an online survey, the results of which were used to select 10 participants, none of which refused to participate. Participants were the first 10 people who reported that they (a) were working on a manuscript and were dissatisfied with their progress towards completing it, (b) were willing to participate in all aspects of the treatment package, and (c) had participated in the community for at least 1 year. After selection, each participant was asked to complete and return an informed consent form via mail or fax. He or she also was asked to e-mail the experimenter a beginning copy of his or her manuscript. Thereafter, throughout the baseline phase, each participant was asked to e-mail an updated copy of this manuscript each day of the week. Word counts were taken via Microsoft Word®, and the relevance of content was checked for all manuscripts received.

Interrater Agreement

Relevance checks involved having the experimenter read manuscripts and determine whether newly added material was thematically related to prior material and not literally repetitive. This primary rater evaluated all submissions and deemed each acceptable. Each week, a research assistant also independently evaluated one submission from each participant, selected at random (N  =  76). All of these submissions were rated as acceptable; therefore, interrater agreement with respect to relevance of content was 100%.

Procedure

Baseline

During baseline, participants self-selected daily writing goals and e-mailed their manuscripts to the experimenter each day a goal was to be met. Whether or not goals were reached was determined by ascertaining the number of additional words that a manuscript contained each time it was submitted, but no feedback was provided to the participant.

Intervention

From the day Internet-based intervention began, participants were allowed to access a personalized Web page that included the goal statement and provided graphic feedback on progress. Graphic feedback involved a figure depicting words written per goal day, number of goals attained, cumulative words written during treatment, and number of “crit” points (see below) earned. Each week during intervention, an e-mail was sent to all participants. This e-mail publicly acknowledged each participant who accomplished all of his or her goals for that week. A personal e-mail also went out to each participant, informing him or her of individual goals missed or congratulating the participant if all goals were made. If a participant attained his or her goals for 3 consecutive weeks, that person received special recognition in a community-wide e-mail that was sent to all members of the online writing community.

Prior to and throughout the study, members were awarded points, called “crits,” when they critiqued other members' manuscripts. They earned one point for each critique of another member's manuscript. These critiques were submitted to the Webmaster, who forwarded them to the writer of the manuscript. These points could be used within the online community to obtain critiques of manuscripts submitted by the holder of the crits. That is, crits were exchangeable for editorial input from other members of the online community. A person was eligible to have his or her manuscript critiqued if the cumulative number of crits he or she had earned divided by the number of weeks that he or she had been a member of the community was 0.75 or higher. To receive a critique, manuscripts were sent to the Webmaster, who made them available to the online community in the order that they were received, and members of the community selected the manuscripts they wished to critique.

During intervention, participants also were able to earn crits when they met personal writing goals. One point was awarded for each daily goal met. Bonus points were awarded for weeks in which all goals were met; additional bonuses were awarded for consecutive weeks in which all goals were met. Missing a goal was penalized with the removal of 10% of a participant's crits earned in the program (rounded up to the nearest whole number). Nine participants submitted manuscripts via the personalized Web by sending coded e-mails with attachments. Submitting manuscripts in this way ensured contact with the feedback from the Web page and with the goal statement. The 10th participant sent manuscripts via standard e-mail during both baseline and intervention.

Design

A multiple baseline across groups design was used in the study. Six participants (Group 1) began intervention 1 week before the other 4 (Group 2). Groups were determined through participant preference with respect to when he or she wished to begin the intervention or, in the case of no preference (8 participants), through random assignment. If, for 2 weeks in a row during intervention, a participant completed all of the goals, the experimenter suggested via e-mail that he or she increase the word-count goal for the following week. Suggested goal increases were always above the lowest submission from the previous week. Participants could decline or revise a suggested goal increase without penalty, and the goals set for a given week ultimately were set by the participants. Participants earned one extra crit point for each week of completed goals at a higher goal level. This increase was cumulative across all goal changes. Intervention lasted 6 weeks.

Results and Discussion

Figure 1 shows the cumulative number of words submitted each day by each group during baseline and the Internet-based intervention. Trend lines were fitted to the baseline data through the least squares method. The degree to which data points during the Internet-based intervention fall above the trend lines provides an indication of the effectiveness of the intervention. Results show that with the sequential introduction of the Internet-based intervention, both groups increased the number of words submitted per day. Group 1 showed a gradual increase in number of words submitted, whereas Group 2 showed a substantial increase. Both groups maintained an increased level of submission across the 6-week intervention. A survey completed by participants indicated that they were happier with their writing productivity during the program than prior to it and that the quality of their writing was as good or better during the program than prior to it. They were generally satisfied with the treatment package, although some components (setting goals, earning crits) were rated more favorably than others (bonus points, group e-mails). One participant completed a novel during the program and, upon invitation, submitted it to a potential publisher.

Figure 1. Cumulative number of words submitted across days by participants in each group.

The solid line represents the trend established during baseline.

Although the number of words submitted per day clearly increased from baseline to intervention in both groups, this does not necessarily indicate that the participants' actual writing rates increased. It is, for example, possible that some or all of the writing was done by persons other than the participants, or by the participants prior to the onset of intervention. Given that increasing the writing proficiency of its members was the primary purpose of the online community and all participants had been members of that community for at least 1 year, it appears unlikely that participants would be motivated to cheat, especially given that no tangible rewards were associated with meeting stated goals. Moreover, goals were set by the participants, who could have received most of the consequences that were available for meeting goals (crits, praise from the experimenter) by setting very modest goals and writing a few words, rather than by cheating. Nonetheless, the permanent-product measure did not directly measure writing, and this is one limitation of the study. This limitation could be overcome by using videocameras to provide real-time recordings of participants while they wrote submitted passages, but such a procedure would be relatively complex and costly and might decrease the acceptability of the intervention. Be that as it may, videocameras provide a means of obtaining direct measures of target behaviors and are likely to be useful in many Internet-based interventions. The value of videocameras is evident in the seminal work of Dallery and Glenn (2005), whose participants videotaped themselves while blowing into carbon monoxide detectors used to detect exposure to cigarette smoke.

As the Dallery and Glenn (2005) study illustrates, adequate data collection in Internet-based studies may require technological sophistication in areas unfamiliar to some behavior analysts. For example, software that identifies individuals in terms of their keystroke dynamics could have been used in the present study to ensure that the participants actually typed submitted passages (e.g., Bergadano, Gunetti, & Picardi, 2002). Unfortunately, we lacked the sophistication needed to use such programs, which in any case would not prevent the possibility that the typed passages were being transcribed from text written by someone else. Rapid advances are being made in identification and recognition hardware and software in general, and the availability of easy-to-use commercial products may soon make it easier to collect adequate data in Internet-based studies.

A second limitation of the present study is that the quality of the submitted writing was not evaluated, except for having participants rate it. Although they reported that the quality of their writing was as good or better during the program than prior to it, there is no independent evidence that this is the case. In addition, participants in the present study were not selected at random, and the present results may not generalize to other fiction writers. Nonetheless, the present results suggest that such interventions are promising. The most significant aspect of the present study was that experimenter–participant interactions were indirect, through the Internet. This format allowed multiple, geographically distant people to participate in the study and to benefit from the intervention. It also permitted automated, albeit less than optimal, data collection. These are important benefits that illustrate the potential value of computer-based behavior-analytic treatments. Although Internet-based interventions are becoming more popular, the relevant literature is limited and few outcome studies have appeared (Ritterband et al., 2003). Additional work in the area is certainly warranted.