An Internet Portal for the Development of Clinical Practice Guidelines

Abstract

Background

The complexity and quality requirements for the development of clinical practice guidelines steadily increase. Internet technologies support this process by optimizing the development process.

Objective

The aim of this internet based solution was to facilitate the development of clinical practice guidelines.

Methods

An internet portal was developed allowing for a shared workplace to support clinical practice guideline authoring. It is based on a Content Management System and combines different tools for document handling and editing, communication as well as process and team steering.

Results

Until now, the internet portal has been successfully implicated in the development of six evidence- and consensus-based clinical practice guidelines. Additional German and European clinical practice guidelines are currently generated with support of the internet portal. The available tools allow for a flexible design of the scheduled workflow, depending on the requirements of the respective group. An additional strength of the platform is the advantage to transfer all data from a previous version of a guideline into the next ‘life-cycle’.

Conclusion

The application of the portal results in a considerable reduction of costs and development time of the resulting clinical practice guidelines.

1. Background

Clinical practice guidelines (CPGs) play an important role as treatment guidelines in daily clinical practice. A prerequisite of this trend was the development of rigorous quality criteria for such guidelines, which has been promoted by different national and international organisations. As a result, the requirements for the development process of clinical practice guidelines have been sharpened steadily. A modern evidence- and consensus-based guideline [1, 2], in Germany known as a S3 CPG [3], requires both the evaluation of highest possible evidence and the establishment of a consensus between all participants, from the specialized medical professional to patient organizations. To obtain acceptance, the finished guideline has to pass through an intensive reviewing process and assessment by instruments such as AGREE [4] or its German counterpart, DELBI [5]. Prerequisite for these steps is a comprehensive method report. Because of its limited lifetime moreover, the CPG also requires a cyclic update.

As a consequence, typical development time-periods for such guidelines quickly reach up to three years. At the same time, the development is expensive. There have been reported costs per guideline of € 150.000 in the Netherlands or of £ 400.000 for British NICE guidelines [6]. Consequently, reduction of expenses represents one important objective in CPG methodology [7] as well as simplification of time and cost intensive steps in the development of CPGs should be considered.

Modern Information Technology (IT) techniques such as internet portals, content management systems or communication platforms have the potential to contribute to the solution of these problems. More remarkable, no specialized IT solutions have been described for accompanying support of the CPG development so far.

2. Objective

Based on the schedule, defined by the German Association of the Scientific Medical Associations (AWMF), our objective was to develop an IT solution for the support of as many as possible steps within the CPG lifecycle (►Figure 1). In order to substantially reduce time and expenses, especially such activities as voting or collaborative work with documents had to be facilitated. A critical point was the management and surveillance of the development process by steering committees as well as workgroup leaders.

Fig. 1.

Lifecycle of a clinical practice guideline according to German AWMF schedule

For the users the resource requirements should be as low as possible and limited to standard tools such as browsers. Platform independence (Windows, Mac OS/X, Linux/Unix) of the clients had to be guaranteed, while application of proprietary standards had to be avoided. As a consequence and to prevent licence conflicts and incalculable operational costs, the solution had to be limited to open-source and freeware components. At the same time, it was not the aim to postulate a rigid workflow, but the users should have the possibility to freely combine available tools and to scale them to their defined demands.

The main components, activities and resources, involved in CPG development are shown in ►Figure 2. The following active parts can be identified:

Fig. 2.

Model of guideline development

• 1.Clinical Practice Guideline: It is the main object of development, which has to be processed, stored / archived and communicated between the developers and later to the public. CPG self is accompanied by other documents, e.g. older versions and the CPG-specific literature.
• 2.Guideline Developers: They build the component, which drives all activities within the development process and represent the “central processing unit”. Main activities are adapted according to evidence assessment, consensus finding and formulation of the CPG and its recommendations. In parallel, the developers coordinate and organize all activities. These are highly interactive tasks, which require powerful instruments for communication and data exchange.
• 3.Development process: It comprises all activities required for the development during the guideline generation. This includes in particular all processes to achieve evidence and consensus such as literature evaluation, voting or consensus conferences. The documentation of this process is one of the main parts of quality management. In addition, there is the need for tools to supervise and manage the process by authorized developers.

On the basis of this model one can identify the tasks which the IT solution should fulfill:

• •management, backup and retrieval of documents;
• •collaborative processing of documents;
• •efficient tools for communication between members of development teams;
• •instruments for evidence assessment of documents, recommendations etc.;
• •instruments supporting consensus generation, e.g. different forms of voting;
• •efficient management of participants, e.g. of their roles and contact data, and secure authentication and access;
• •management instruments for monitoring the development process;
• •simple, but effective instruments for documentation of the development process, generation of audit trails and of method reports.

None of these components is limited to one step of the development process or is restricted to a defined workflow. Accordingly, the fundamental design principle was not the development of an integrated tool, possibly limited to a single workflow, but rather a universal toolbox for the support of frequently appearing tasks.

3. Methods

To fulfill these requirements, a content management system (CMS) with powerful web interface [8] and a large degree of configurability and extensibility has been chosen. As an optimal framework, the Plone CMS has been selected [9]. Plone, as a portal solution, already includes a full-developed subsystem for accounting and access control, which is relatively simple to extend to a full-fledged, LDAP-based participant management.

A decisive advantage is the application of the universal scripting language Python as an implementation language. This offers a high flexibility for the programming of server-side business processes, compared with solutions based on web-only languages such as PHP or on more complex Java environments. Accordingly, it is an excellent platform for rapid reaction on emerging user requirements, and it comes with a multitude of extensions.

One key component of the portal is a tool for the preparation, execution and evaluation of voting via Internet [10]. It applies a formalized description of questionnaires, based on XML Schema. Various scripts for automated generation of HTML implementations of questionnaires, for monitoring of voting and finally, the statistical analysis of results simplify online voting at a considerable rate. Some features, e.g. the possibility to comment answers, are specially tailored to the demands of consent establishment.

All data, describing a CPG and generated during its development are handled by an external database, using the relational database management system MySQL, with the exception of text documents, which are managed within Plone’s internal repository. To ensure a high degree of availability, the portal is hosted on a virtualized Linux server on a VMware ESX cluster.

The client-side part can be kept very simple. As software configuration, it only requires a browser, possibly supplemented by some support for the handling of PDF and office documents. But the latter are optional, since the text editing mostly takes place with internal editors, supplied by the portal. To simplify the access to the Internet, all client-server communication exclusively uses HTTP/HTTPS. Furthermore, all tools are server-side applications, and no download of any executable code is required. Thus, even in secure environments like hospitals the access does usually not raise problems.

4. Results

4.1 General usage of the portal

A view of a typical start page, as seen by a portal user, is shown in ►Figure 3. The middle part of screen represents the working area. At the left side, the main menu is located. There, the user finds all CPGs listed, for which she/he is registered. Thus the user is flexible in switching in between these CPGs. At the right column, the developer finds news and appointment information. The upper part contains general functions such as language selection (currently English / German). With the ‘Preferences’ function, each user can maintain the personal contact information.

Fig. 3.

Start page with index of contents of a typical CPG workspace

The available functionality depends on the development process, defined by coordinators of the concrete CPG. Default functionality is organization including internal news and appointments, communication tools and a shared workspace for document editing and exchange. Normally, tools for consensus finding, online voting as well as for literature work and evidence assessment are available.

4.2 User management

For the management of members of development groups, a detailed, CPG-specific system of roles has been implemented. A role defines the rights of a user and also his personal view of the portal. From a total of ten guideline specific roles, the most important are:

• •coordinator: responsible for organization, monitoring, quality management, final decisions; no access to workgroup level data; no right to vote
• •workgroup leaders: responsible for management of their workgroup (WG); WG-local decision making; full access to WG local data; right/obligation to vote
• •workgroup member: full access to WG local data; right/obligation to vote
• •patient: WG member; right, but no obligation to vote
• •expert: WG member; no right to vote

Each participant can represent different roles within a CPG development. Users, participating in different CPG developments, authenticate themselves with one single account, allowing them after login to access to all guidelines, for which they are registered. With registration of a participant, usually performed by a coordinator, the user database will be automatically updated with contact data, mailing lists etc.

4.3 Communication and collaboration

The user database builds the basis of a widespread communication subsystem. The most elementary function is the allocation of mailing lists: They can map the role scheme, but also the workgroup structure and are supplemented by efficient searching mechanisms for contact data.

For bi- and multidirectional communication discussion forums are the preferred form of communication. Each user can create a discussion thread and invite other members to participate. Compared with e-mail, these forums have some essential advantages. Firstly, they are public, either for a whole CPG team or for limited groups, e.g. coordinators or individual working groups. Thus, each member interested in a specific problem can track the discussion and contribute to the topic. Secondly, the centralized recording of a discussion thread is an effective documentation of decision processes. To perform short-time communication, chat-rooms and telephone conferences are available.

For collaborative document processing, the portal supplies some more complex tools. Within dedicated workspaces, working groups can create and edit complex documents. With FCKEditor a powerful text processor is available for this purpose (►Figure 4). In combination with version-control and history functions, including a fallback option, this tool is the core of document processing.

Fig. 4.

Handling text with embedded FCKEditor

The portal also supplies a wiki tool, whose main scope is the development of dictionaries and ontologies. But it can be applied to compile definitions of any kind. Each developer can request the definition of some item, whose meaning appears unclear. In this and all other cases, a wiki process will be initiated, which leads to a harmonized definition. The definition then can be declared mandatory by coordinators and adopted into the final CPG text.

4.4 Evidence assessment

The portal supplies an interface to myNCBI keyword search. CPG developer groups have access to private workspaces to perform a preselection of references on the basis of abstracts and metadata. Full texts of preselected sources can be uploaded into the portal. Because of German copyright regulations the portal guarantees an exclusive access to these documents only by authorized group members. This is part of user and session management of the portal. The documentation of evidence assessments occurs by standardized evidence tables. The assessments and also all full text documents, reported in the CPG, are archived for documentation purposes.

4.5 Consensus finding

Activities in consensus gathering are very complex, time-consuming and expensive tasks. This lies in the philosophy of the Nominal Group Process with its numerous meetings, voting and other harmonizing steps. Although this is one prerequisite for high-quality CPGs, its practical implementation often ignores modern technical possibilities, especially those of Internet.

From there, the portal provides a powerful tool to support those tasks. Principal part is a comprehensive system to generate, answer and analyze guideline-specific questionnaires. These questionnaires (►Figure 5) support all essential types of answers, e.g Likert scales, numerical scales with freely definable endpoints, single- and multiple-choice answers or selection of percent value. In each case, the respondent has the possibility, in some situations also the responsibility, to comment on the decision. In well-founded situations a respondent can declare himself as ‘not qualified’. Then she/he can decline the answering of a question.

Fig. 5.

Typical question layout

The decision process can be supported by considerable contextual information, embedded into the questionnaire. The concrete approach depends on the policy, forced by coordinators.

A typical voting period spans over up to two months. Within this time period, each participant has the opportunity to answer the questions or to adjust them, whenever it is required.

The status of the voting can be supervised by coordinators at any time. Moreover, at any time they can request preliminary reports. For all participants, the final report of a vote (►Figure 6) is available immediately after its termination. Of special value are the comments submitted. They contain significant information for the commentary part of the CPG.

Fig. 6.

Evaluation report for question from ►Figure 5

Another useful instrument, supported by the portal, is the televoting system in the context of consensus conferences and meetings. The application of these systems considerably reduces the duration and error rate of ballot counts. The results are immediately available for further processing like those of questionnaire surveys. Another important aspect for both procedures, questionnaire based voting and televoting during a meeting, is the fact, that the results are immediately available for the later generation of quality documentation, e.g. method report.

4.6 Usage results

So far, the portal has been applied to finish six evidence- and consensus-based CPGs, four new developments and two updates, both German and European. Two are already published [11, 12], the others are in editing or with the reviewing process. Currently, five additional CPG developments are hosted.

The experience from these applications indicates a considerable reduction of development time and cost. Without the portal, the duration from the designation of participants to the moment when a complete consensus is reached has been on average 2 years for new developments and one year for updates. Taking advantage of the portal, this period could be reduced to less than one year for all new developments and three respectively four months for the updates. Although there are no statistically significant data about usage characteristics up to now, at least a halving of development time seems to be representative.

Remarkable is the increasing shift from meetings toward questionnaire-based surveys for gathering consensus. In doing so, one trend is to pre-vote the key statements of the CPG in the run-up to the consensus conference. The participants of the consensus conference can then focus on really sophisticated questions, which considerably enhances the efficiency of the meeting. Questions, not answered during the meeting, can even be passed back to working groups for revision. Later, the revised version can be submitted to a short, new online voting, which further increases the flexibility.

5. Discussion

The experience, gained from different CPG developments hosted so far, indicates a high flexibility in adapting to individual methodology. Some early expectations, e.g. to fully substitute Nominal Group Processes by pure online votings, have been revised. On the contrary, skillful combination of online votings about CPG recommendations with their subsequent discussion and finalization during a consensus conference has remarkably increased the quality (accuracy, completeness) of their formulation. To optimize the interplay between those two steps, the voting tool has been completely redesigned during the test phase to improve the possibilities to comment answers.

So far, the full integration of tools, required for evidence appraisal, into the portal is not solved satisfactory. Thus, it is planned to extend the portal by a powerful Document Management System, which will support all aspects of literature work without the need of external tools.

The portal is available for further users at anytime. Detailed information can be found on the start page of the portal [8] and in an information brochure [13] available for download from the portal. Terms of portal use and prices can be inquired. The latter depend on the extent of the required tools. Referring to a CPG with up to 10 workgroups and a total of 50 participants, the cost of portal usage normally is below 10.000 €, which matches the minimum expense of a single one-day meeting of such a team.

6. Conclusions

The portal has shown its value in the development of CPGs. For developer teams it offers a high degree of flexibility in the organization. Especially, considerable parts of consensus finding, as yet often connected with numerous meetings, which are difficult to organize and very expensive, can be replaced by platform usage. In addition, the management and control of the development process can be simplified significantly. This includes measures for quality management, which are profoundly supported.

Conflict of Interest Statement

The authors have not to report any Conflicts of Interest with respect to the content of this manuscript.

Protection of Human Subjects and Animals in Research

The present manuscript does not report results from research including experiments on human subjects or animals.

Clinical Relevance Statement

As evidence-based recommendations for health professionals, clinical practice guidelines play an important role in their daily work. The aim of the portal, presented here, is to optimize the quality of these recommendations, by supporting their development. This increased quality also results in increased confidence in CPGs, connected with improved implementation into clinical practice.

Glossary

Abbreviations

AGREE

Appraisal of Guidelines Research and Evaluation

AWMF

German Association of the Scientific Medical Association

CMS

Content Management System

CPG

Clinical Practice Guideline

DELBI

German Instrument for Methodological Guideline Appraisal

HTML

Hypertext Markup Language

HTTP

Hypertext Transfer Protocol

HTTPS

Hypertext Transfer Protocol Secure

IT

Information technology

LDAP

Lightweight Directory Access Protocol

NCBI

National Center for Biotechnology Information

NICE

British National Institute for Health and Clinical Excellence

PDF

Portable Document Format

PHP

PHP Hypertext Processor

TMF

German Technology, Methods, and Infrastructure for Networked Medical Research Association

WG

Workgroup

XML

Extensible Markup Language