Scenario-based Participatory Design of A Collaborative Clinical Trial Protocol Authoring System

Abstract

We present our experience of using prototype scenarios to actively involve users in the design of a collaborative clinical trial protocol authoring system. This method enables us to do usability testing and elicit prompt user feedback at the early phase of design. We conclude that it is an effective approach to the design of complex medical information systems.

Background

Many clinical trial protocols contain ambiguities, or inconsistencies. Such protocol problems can lead to inappropriate patient care or even endanger a patient’s life. We are designing a collaborative clinical trial protocol authoring system to enhance communication among collaborative protocol writers and to evaluate whether communication support during the writing process can improve protocol quality.

Prior research on quality improvement for clinical trial protocols has focused on using a formal knowledge model to guide protocol development. As far as we know, no prior work provides collaboration support for protocol authoring. Recently, we studied the clinical trial protocol writing processes at the Southwest Oncology Group (SWOG), a cooperative cancer protocol development group under NCI. In this study, we interviewed nine protocol writers, including three study coordinators, two statisticians, and four protocol coordinators. We also participated in their weekly protocol review committee meetings for almost one year. We found that authoring a clinical trial protocol is a highly collaborative process. A team of distributed protocol writers with varied expertise is assigned to develop a clinical trial protocol. We believe these protocol writers need improved communication support. Specifically, they need groupware that provides group-writing support features such as version control, change representation, progress notification, and other awareness support. Our background study with SWOG has motivated our research in collaborative clinical trial protocol authoring systems.

Method

Because our groupware will have users with different backgrounds and different needs, usability testing will be difficult. Thus, we need to involve potential users early during system design. Participatory Design (PD) has been a successful software design methodology that includes users into the system design process.^1,2 A scenario can be narrative text or a set of prototype screenshots that describes a user’s interaction with a system situated in a work setting. Scenarios should provide sufficient details so that design implications can be inferred and reasoned about.¹ It has been advocated as an effective means of communicating a system design and “use cases” between users and stakeholders in an intuitive way.

We designed our scenarios for a collaborative protocol authoring system in Microsoft Visio 2002 and presented them to protocol writers during design meetings. We iteratively solicited user feedback on system usability and refined the design and the scenarios according to the feedback.

Lessons learned and conclusions

We found that protocol writers were enthusiastic about this design process. The scenarios encouraged them to imagine potential uses and suggest some novel use cases that we did not uncover through interviews or other system requirement analysis methods. Through this process, protocol writers contributed some important design ideas and shaped our decision making for implementation choices. For example, originally we thought that the text formatting of the protocol was complicated and important, and therefore the rich editing features in Word were indispensable to a protocol writer. However, when we proposed both a web-based system scenario and a Word-based system scenario to our users, we were surprised that they preferred the web-based system. They prioritized content over format and felt that a web-based system would be more helpful in solving cross-platform formatting problems.

Tangible scenarios enabled our users to think about the constraints on system design in the situational context of their day-to-day work. We conclude that scenario-based participatory design is an effective approach to involving users in design at the early stages of system development.