Desiderata for Personal Electronic Communication in Clinical Systems

Abstract

Electronic communication among clinicians and patients is becoming an essential part of medical practice. Evaluation and selection of these electronic systems, called personal clinical electronic communication (PCEC) systems, can be a difficult task in institutions that have no prior experience with such systems. It is particularly difficult in the clinical context. To directly address this point, the authors consulted a group of potential users affiliated with a nationally recognized telemedicine project, to determine important characteristics of a hypothetical PCEC system. They compiled a list of these characteristics and produced a desiderata, or list of desired features, for PCEC systems. Two conventional e-mail implementations and three Web-based PCEC systems were evaluated with respect to the features. The Web-based systems all scored higher than conventional e-mail. It is the hope of the authors that this paper will initiate further discussions about the features of PCEC systems and how to evaluate them.

Communication among care providers and between care providers and their patients is an essential component of medicine.¹ Systems that utilize information flow across a modern electronic medium, such as the Internet, have been developed to facilitate communication among various persons involved in patient care, such as clinicians, administrators, and patients. These systems are the latest trend in health care's quest to improve the transfer of clinical information. A recent medline search on the term “e-mail” produced more than 200 references.

Such systems do not yet have a universally accepted name. Labels range from the very narrow “Web-based doctor–patient communication services”² to the broader “electronic communication with patients,”³ “physician practice–patient communication network,”⁴ and “electronic patient-centered communication system”⁵ to the very nonspecific “electronic messaging.”⁶ None of these terms conveys both the clinical content of the communication and the frequent use of such systems for direct provider–provider communication. To capture these characteristics, we chose “personal clinical electronic communication” (PCEC) as the best descriptor.

Medicine is historically quick to adapt new technologies to facilitate communication.⁷ There are many anecdotal stories of health-care professionals using whatever means they have available to communicate medically relevant items. For example, in the 1800s, clinicians tied yellow kerchiefs on patients' doors to communicate that the patient residing in that house was infected with a communicable disease and was in quarantine.⁷ In modern times, care providers often use the telephone to communicate with their patients, to consult or share information with colleagues, or to arrange many aspects of care.^6,8–10 Many health care providers already use some form of electronic communication, such as electronic mail or instant messaging.^6,11–14

An individual practice or institution attempting to select a PCEC system has a bewildering array of options, including conventional e-mail, messaging components embedded in electronic medical record systems, and specialized stand-alone systems. Studies have explored guidelines for the use of electronic mail and other electronic communication with patients^3,11,15,16 and the various other benefits of technical programs and their effects on patient care.^{8–10,12,14,17–19} Typically, these studies have focused on a specific messaging technology,^11,12,15 the clinical needs arising from a specific disease,^{1,8–10,14,17,18,20} or anecdotal experience with a specific system.^3,6,19 Formal studies addressing the necessary features of a PCEC system that functions across diseases and specialties are lacking.

The Informatics for Diabetes Education and Telemedicine (IDEATel) project is a large home-telemedicine project that uses informatics technology to bring maintenance care into the homes of patients who have diabetes mellitus.^21,22 A PCEC system was essential to IDEATel as a means for clinicians to communicate with one another and with their patients. Early in the selection process, it became clear that clinicians were not in unanimous agreement on the features of an ideal PCEC system. It also became clear that no current PCEC implementation contained all the features desired by even one of the clinicians. To rationalize the process, it was necessary to develop a prioritized desiderata (a list of desired features). This paper describes the process of creating the feature list, the resulting desiderata, and the use of the features list to evaluate several existing PCEC systems.

Methods

We used an electronic-mail version of the Delphi technique.²³ This was done in part because of our panelists' disparate geographic locations and the convenience of electronic mail messaging.²³ We contacted 16 people who were affiliated with the IDEATel project; 10 (one nurse, one administrator, and eight physicians) agreed to participate. These 10 people made up our expert panel. Four of the panelists had significant experience in medical informatics, in that they were either faculty in informatics departments or were actively involved in informatics projects. Neither author was a member of the panel.

The study was part of a system selection process. It was begun prior to selection of the IDEATel PCEC system and was completed after selection, but prior to installation, of the system.

The elicitation process began with a request that each panelist, independent of all the others, present a list of features that he or she believed were important in the selection of a PCEC system, based on their personal experience. To maximize the generalizability of the resulting features, we asked the panelists to consider general clinical practice and to not limit themselves to the context of the IDEATel project. We also asked the panelists to not worry about any of the technical limitations that may currently exist for selection of what they thought of as an ideal system.

For this and each subsequent round, the panelists had two weeks to respond, and all responded on time for all rounds. We intentionally did not “seed” the list with features from existing products or prior published guidelines. The goal was to product a user-centric list that could be compared with lists derived by different methods.

We combined all the responses into a single, unaccredited list, which we then returned to all the panelists. We asked the panelists to add new features—and refine any of their own features—that they felt needed to be included on the list. We recompiled the results and again returned them to the panel.

During the third round, we asked the panelists to rank the features. Using a ten-point Likert scale,²⁴ ranging from −5 (most detrimental) to +5 (most essential), they were asked to quantify each feature in the set in terms of its benefits or detriments to a hypothetical, ideal PCEC system.

In the fourth round, we sent the panelists a list that included, for each feature, a mean score and that panelist's individual score; we asked them to adjust their individual scores if they wished, based on the overall means. As is standard in the Delphi method, direct discussion among panel members was discouraged, to mitigate the influence of forceful or highly respected persons. After we updated the mean scores, we again sent the list to each panelist, with the mean and most recent individual score and a request that he or she update the score, if desired. No panelist changed his or her individual score in this round, so the collection process ended.

We removed duplicate items from the final list. The criteria for concluding that two items were duplicates included not only that they appeared to describe the same feature but also that the were scored the same by the panel. Remaining features were grouped into thematic categories. Within each category, the items were ranked in descending order first by mean, then by standard deviation. The categories were then collectively ranked also by mean and standard deviation.

Further inspection of the feature set lead to the development of three categories of features, based on mean rank and inter-rater agreement. Inter-rank agreement was ascertained by examining the standard deviations of each feature. The “critical” features shared a mean greater than or equal to 3.5 and a minimum score greater than or equal to 0. The equivocal features shared mean less than 2.25 and a minimum score less than 0. Between these two groups are the “desirable” features.

We made no attempt to enforce uniform syntactic structure on the submitted features; i.e., some features were phrased in the positive (“Strong Authentication) and some in the negative (“No SPAM”). We considered the effect of such phrasing on the scoring by looking at the relative scoring of positive and negative features. In particular, we looked for phrases in which a statement was negatively set and then the grouping it was placed in. Because of the nearly bimodal distribution of scores for one negatively phrased feature, “Recipient Disclosure,” we suspected possible misunderstanding. We contacted our panel for written descriptions of their answers. Eight panelists responded, and the “Recipient Disclosure” scores were recalculated on the basis of just these responses.

The features and associated scores were converted to a numeric evaluation method for PCEC systems. The evaluation method was based on two components—a weighting value, based on the median score from the panel, and an implementation value from 0 to 1.

Each feature was individually considered for a candidate PCEC system. If a feature was fully supported, an implementation score of 1 was assigned. Those that were partially supported were given an implementation score of 0.5. The score for each feature of the candidate system was determined by the implementation score (0 to 1) multiplied by the weighting score (0 to 5). Only critical and desirable features were used for the score.

As a proof of concept, we applied the evaluation to two potential implementations of conventional electronic mail and to three existing Web-based PCEC systems—Healinx² (Healinx Corporation, Alameda, California), the PCEC supplied by the American Medical Association to its members⁴ (Medem Inc., San Francisco, California), and the PCEC system selected for the IDEATel project (Siemens Health Services, Malvern, Pennsylvania).

Results

In the initial round of the Delphi method, our panelists submitted 29 criteria; they added 5 criteria in the second round. We then removed two duplicate entries, for a total of 32 criteria. These criteria were sorted into 21 categories, which are shown in Table 1▶.

Table 1 .

Desiderata for Personal Clinical Electronic Communication

Item	Sub-item	Median	Mean ± SD	Description
	1	5	5 ± 0.7	Security/Reliability
	1.1	5	4.9 ± 0.3	Backups—Backups should be maintained such that adequate and effective recoverywill be efficient and accurate.
	1.2	5	4.8 ± 0.4	Audit Trail—A record should be kept of every time anyone writes, reads, forwards, or responds to a message. This should automatic, regardless of what record was record was accessed or by whom.
	1.3	5	4.8 ± 0.4	Encryption—All messages should be stored and transported in an appropriate manner such as to deter unwanted intrusion.
	1.4	5	4.8 ± 0.4	Security Policy—A formal security policy should be established and implemented.
	1.5	5	4.8 ± 0.6	Strong Authentication—Secure methods should be implemented such that authorization is restricted to appropriate users. This will include utilization of user password validation, and may also incorporate techniques such as digital signatures, cryptography, secureID, etc.
	1.6	5	4.0 ± 1.3	Digital Signature—Through the use of software tools, a digital “signature” should be used to sign every message indicating who the message is from. Every user will be assigned a unique encryption key, such as a large number carried on a smart card. This key is then used to uniquely identify any message that is sent by that person.
	1.7	3.5	3.2 ± 1.9	Automatic Timeout—The user interface should be such that a timeout mechanism is in place to prevent unwanted use due to negligence.
2		5	4.7 ± 0.5	Cross Coverage—Communications should be in place such that any lack of response should be automatically forwarded to another caregiver so that the issue can be dealt with
3		5	4.65 ± 0.5	E-mail Directing—The system should be able to direct messages to more than one addressee, identified as “To:” and “CC:,” in the message header.
4		4.8	4.35 ± 0.8	Footprint/Thin Client—The communication system should not require any special software for accessing messages. Messages will be dealt with through commonly available applications such as a Web browser.
5		5	4.1 ± 1.2	No SPAM—Unsolicited e-mails (SPAM) should be kept to a minimum.
6		4.5	4.0 ± 1.6	Convenient—The user interface should be easy to use and not foreign to the users.
7		4	3.9 ± 1.0	Copy-and-paste Features—The system should allow pasting from the Windows clipboard to facilitate the addition of pertinent information into a message.
8		3.5	3.7 ± 1.1	Receipt System
	8.1	4.5	3.9 ± 1.4	Receipt Record—A record of whether the recipient has received an e-mail and responded to it in a timely fashion—i.e., a temporal tracking system with a reminder to the appropriate party (or parties)—will be important if the volume of messages is high.
	8.2	3.5	3.7 ± 0.8	Receipt Management System—There should be a centrally located system that maintains a record of the status of each message, such as whether the message has been “received,” “received and read,” or “read and responded.”
	8.3	3.5	3.6 ± 1.2	Acknowledgment—Senders should know that receivers have not only received but have read and responded to an e-mail. Acknowledgments need to be managed so that at any given time the status of all messages sent, whether read or not, can be seen.
9		3.5	2.9 ± 1.5	Escalation
	9.1	4	3.8 ± 1.0	Triage/Automatic Escalation—Urgent items should be automatically routed to the appropriate person so that they can be processed in an expedited fashion. In the event of no response, the message should be sent to someone else, until the issue contained in the message is dealt with.
	9.2	3	2.0 ± 1.9	Due-date Escalation— As a particular date approaches, the level of priority increases proportionally.
	10	3.5	3.4 ± 1.4	Attachments—The system should be able to handle encoded attachments for file transfers between different entities.
11		4.5	3.4 ± 2.1	Recipient Disclosure/No BCC (blind copying)—The system should not allow blind copies; all users to whom a message is sent to should be made known.
12		3.5	3.2 ± 1.6	Alerting Reminders—The system should be able to individualize alerts and should have a reminder system on both the patient and the provider sides.
13		3	3.2 ± 1.0	Group Addressing—The ability to send messages to various groups (such as all the patients of Dr. C) should be in place to create a more efficient atmosphere for communication.
14		3	2.9 ± 1.5	Threading Multiple Messages of Same Case—This is the ability to have threaded messages, so that particular issues can be dealt with in a linear fashion. That is, when somone replies to a message, the reply is linked to the original message. This allows for ease in the review of related messages.
	15	3	2.6 ± 1.3	Structured Messaging
	15.1	3	2.6 ± 1.3	Beyond Standard Text—This is the ability to go beyond standard e-mail format, such as structured messaging.
	15.2	3	2.6 ± 1.3	Check Lists—Check lists give users options to respond to a query rather than write a narrative account.
16		2.5	2.3 ± 1.6	Chat/Instant Communication—A feature of instant communication, possibly with other members of a health care team, in the care of a particular patient.
17		2	2.1 ± 1.2	Calendaring—This function would assist in the scheduling of events by use of the messaging system.
18		1.8	1.4 ± 2.7	Prioritization
	18.1	3.5	2.2 ± 3.1	Levels of Importance—Each message should be clearly established at a level of importance set by the user.
	18.2	0	0.6 ± 2.2	Priority Preservation—The priority of a sent message should maintain its level of given priority throughout a given thread
19		2	1.9 ± 2.0	Separation of Roles—The accounts of certain persons should be maintained separately from accounts for their roles, to allow for the maintenance of better audit trails.
20		1	1.0 ± 1.7	Asynchronous Alerting—A mechanism should be in place that gives notification out of band; e.g., a page is sent each time a message is received.
21		0.5	0.6 ± 2.2	VIP Messaging—Messages of high importance should be given individual passwords to add another level of security.

Note: Features are ordered by mean score. When several features are grouped into a single item, the overall mean is used for ordering and sub-items ordered within the item.

A total of 318 scores were received for the 32 criteria, across the full possible range (−5 to +5). The overall mean, median, and mode were 3.3±0.7 (SD), 4, and 5, respectively. Using the rank definitions and the inter-rater agreements, our final list consisted of 16 critical features (mean, 4.3±1.0), 13 desirable features (mean, 2.7±1.8), and 3 equivocal features (mean, 1.5±2.1). The scoring for each feature is summarized graphically in Figure 1▶.

Figure 1 .

Scores for features of personal clinical electronic messaging systems. Features are ordered vertically by mean score. The order of features is not completely monotonic, because of thematic grouping of features, as shown in Table 1▶. Center diamond indicates median score; gray bar, one quartile above and below the median; thin whiskers, range of scores.

The numeric evaluation based on the feature list revealed that conventional e-mail scored between 19 and 33 percent overall. The range of scores for conventional e-mail was dependent on the number of features that were available for different implementations of e-mail. This scoring assumed partial credit for features, like digital signature, that are not supported by all e-mail readers. Without these features, the electronic mail scores dropped to 19 percent, for both the critical and overall scores. In comparison, all three Web-based PCEC systems scored markedly better. Scores ranged between 43 and 62 percent for all features and between 49 and 75 percent for critical features.

Discussion

The most striking finding was the level of consensus among panelists in different specialties from different institutions. Across all the features, the average standard deviation of the group was 1.3 units on a 10-point scale. This consensus was especially strong for the 16 features in the “critical” group (1.1 to 1.6, 2 to 8.3, and 9.1), in which the average standard deviation was 1.0. Nine features had median scores of +5.

Further evidence of consensus was the fact that no feature submitted by an individual panelist was ranked as undesirable (average score less than 0) by the panel. In spite of this agreement among clinicians, the highest-rated PCEC system scored only 75 percent for these features. More important, conventional e-mail, which is probably the most widely used PCEC tool today, scored only 40 percent for critical features in an optimal implementation and only 19 percent in a more typical implementation.

Although the number of implementations evaluated was limited, every Web-based PCEC evaluated out-scored conventional e-mail.

These features are not the first collection of recommendations for electronic messaging. The features selected by the panel compare closely with those included in previous guidelines on e-mail use, including the importance of message archiving, encrypted transport, and documentation of when messages are actually read.¹¹

Of the 16 criteria ranked as critical in our study, 13 were also relevant to the HIPAA requirements.²⁵ These criteria (1.1 to 5, and 8.1 to 8.3) all deal with the security and reliability of messaging between various health care players, such as clinicians among themselves and with their patients. This was a reflection of how the critical elements, on the whole, are essential to a PCEC system.

Strikingly absent from the list of features are many that are typically discussed in relation to Web-based sites for patients. These include statements regarding the limits of liability³ or limits on the types of messages that are permitted.¹⁶ Also absent are features relevant to developers and implementers. For example, there is no mention of open application program interfaces or the use of standards, even though both are critical for the integration of PCEC into complex clinical computing environments.²²

Similarly, disclaimers and controls to limit legal liability are not mentioned.¹⁶ This is because these features reflect the views of individual clinical users rather than the views of developers or institution administrators. It also reflects the focus on messaging among individuals, not on broad communication (e.g., Web sites). As a result, these features complement, but are not a substitute for, good system and user interface design. The requirements of local clinical practice, pre-existing electronic medical record systems, HIPAA, and other regulations (such as federal Section 508 guidelines²⁶) will influence the final design of a PCEC implementation.

Although this list of features represents a start at addressing criteria that should be used in the selection of a PCEC system, this study is not without its limitations. The number of panelists included in the study was small. With such a small sample, it is conceivable that areas of disagreement were not fully expanded and areas of agreement were not properly extracted from the data.

The panelists came from two separate institutions but were all affiliated with a similar project, and most had some experience with electronic medical record systems. However, because the IDEATel PCEC was not functional at the time of this survey, this is not likely to have induced significant bias. Also, this study focused on clinician users. Further studies will need to assess the preferences of other groups, such as patients, vendors, and other non-clinician users.

Finally, in the numeric evaluation, the weighting values for individual features may not translate directly to the aggregate value of classes of features. For example, the large number of security features (6 of 16 critical features) may lead to overestimation of the aggregate importance of security in PCEC systems.

The “critical” features may be the most important for system developers. However, equally interesting are the “desirable” and “equivocal” features, which elicited more disagreement among the panel. The desirable group (features 1.7, 9 to 17, and 18.1) is distinguished from the critical group by both lower mean scores and higher average deviations. Com-pared with the critical features, these features share the characteristic that their value to a clinician may be dependent on the functionality of other computer systems at the institutions.

For example, a calendaring system may be essential at one institution but of no particular use at another, where an efficient scheduling system is already in place. Structured messaging, which provides templates for certain message types (e.g., medication renewal and appointment request), could act as a surrogate for a forms or order entry system if one did not already exist. Threaded message viewing would allow the messages pertaining to a specific patient to act almost as an electronic medical record. Thus, the importance of these features could be expected to vary among individuals and among institutions.

Institutions selecting PCEC systems will want to pay particular attention to those features that receive both beneficial and detrimental ratings from the panel. This group included all three equivocal features, as well as five features from the desirable group, including automatic timeout, due-date escalation, chat, levels of importance, priority preservation, separation of roles, asynchronous alerting, and VIP messaging. Each of these features was also rated +4 or +5 by at least one panelist, indicating that some panelists felt these features to be highly desirable. One feature, levels of importance, received scores ranging from +5 (must have) to −5 (must never have). This level of disagreement did not decrease in subsequent Delphi rounds.

Overall, these features tended to involve additional effort, interaction, or liability by clinicians in the way they practice medicine. Both chat and asynchronous alerting involve the use other communication means, such as paging or pop-up windows, that may interrupt the clinician. Paging systems are often seen as the bane of life by hospital clinicians, and many clinicians may be hesitant to permit computers to page them directly. Computerized paging may also affect clinical care, when the sensitivity is set too high and an excessive number of alerts are sent to clinicians.²⁷

Four features in this subset involved the creation of special high-priority messages (due-date escalation, levels of importance, priority preservation, and VIP messaging). Levels of importance and priority preservation involve the ability of a sender to flag a message as high priority and retention of the flagging by the reply message. Most clinicians have encountered coworkers who label everything “STAT.” Consequently, some may wish to limit that behavior in a PCEC system.

With the VIP feature, messages pertaining to persons designated VIPs are associated with additional security and auditing procedures. In practice, most hospitals already have special procedures for protecting the security of both the paper and the electronic records of VIP patients. These may include admitting the patient under an alias or removing the name label from the paper chart. This practical need conflicts with the ethical imperative to treat all patients equally, regardless of wealth or status.

The case of receipt disclosure/no BCC (blind copying) is also of note. Originally, the panel generated a nearly bimodal distribution of strongly positive and negative scores. Subsequent queries to the panel revealed that there was confusion about whether a negative score meant that blind copying was bad or that preventing blind copying was bad. After clarification, the scores still had a somewhat bimodal distribution, but they were now clustered about 0 and +5. Panelists who gave high scores indicated that full disclosure was imperative if patients were to trust the system. Panelists who gave low scores indicated a conflict between the need for disclosure and the fear that full disclosure might also disclose patient names and thereby violate patient privacy.¹¹ This is an example of a feature that is very different when viewed from clinician and patient perspectives.

It is clear from these desiderata that no single PCEC system will satisfy all the wants and needs of all clinicians or institutions. However, they also show that a high level of consensus about the critical features that should be included in future PCEC systems.