Tool to assess the quality of consultation and referral request letters in family medicine

Good communication is essential to a safe and high-quality consultation and referral process. With the move to providing more patient care on an outpatient basis, there is now often little direct contact between primary care physicians and other specialists. As a result, written communication, in the form of consultation or referral requests and reply letters, is the most common—and sometimes sole—means by which doctors exchange information pertinent to patient care.^1–5 Poor communication in the consultation and referral process can have adverse effects on patient care by delaying diagnoses, leading to the repetition of investigations, and increasing polypharmacy, which might also increase health care costs.⁶ Studies have shown that both primary care physicians and other specialists are often dissatisfied with the quality and content of written communication between colleagues.^4,7,8

Although competency in written communication is essential, most Canadian physicians have not received any training or feedback about their letters.^9,10 Although there exists a considerable body of literature concerning effective consultation and referral, surveys of communication skills programs show that written communication is seldom the focus of formal instruction in medical education.¹¹

In 1993, the College of Family Physicians of Canada and the Royal College of Physicians and Surgeons of Canada developed a joint task force to examine the relationship between family physicians and specialist consultants. Both its 1993 report and its 2006 follow-up report outlined a number of themes relevant to medical education, including the recommendation that all medical trainees receive instruction and feedback on the preparation of medical reports for other physicians.^12,13

The evaluation of written communication should be part of any ambulatory-based rotation, but the lack of validated assessment tools has been identified as a barrier to effective teaching of these skills.^14,15 The use of a rating scale as an assessment tool has been shown to facilitate delivery of detailed formative feedback to learners in specialty areas.^16–18 At present no tools exist for the assessment of letters written by family medicine learners. As part of a larger educational intervention to improve the ability of family medicine residents to write effective consultation and referral request letters, an 18-point assessment scale was developed.

Development of an assessment tool

Based on a systematic literature review of articles published between 1970 and 2005, the current gaps in referral and consultation request letters and their desired content were identified.^{1,7,8,19–22} This literature provided the basis for the construction of an assessment tool with face validity for consultation and referral request letters. In addition, writing-style items such as sentence and paragraph length were included. These were based on consensus literature on good practice in correspondence, as well as previous work on the development of evaluation tools for specialty residents’ letters.^{17–19,23,24}

The initial tool included dichotomous scales for 13 content and 3 style items, and it also included a 5-point Likert scale with anchors used to provide an overall rating for each letter.

To ensure clarity and reproducibility, the prototype assessment instrument was then pilot-tested by 10 family physician educators from the University of Manitoba’s Family Medicine Residency Training Program using the tool to score 2 standardized letters.

After analysis, the questions used in the tool appeared to form a reliable scale, as there was a high level of agreement between the global rating of a letter and the sum of the checklist items (Pearson coefficient = 0.88). Some items had very high rates of endorsement (raters all scored yes or no), suggesting that dichotomous items did not individually contribute to the discriminating power of the instrument. However, dichotomous items (yes or no items) do play an important role in facilitating the work of raters in assigning global scores and are also believed to be valuable in the provision of formative feedback.^18,25 Interrater reliability of the tool was assessed by analyzing the variance between raters’ scores in both the individual items from the tool and in the total score. The intraclass correlation coefficient for dichotomous items was 0.36, and for the overall rating scale it was 0.67. Simple interrater reliability for the overall score was 0.83.

Based on these results and on feedback from raters, the assessment tool was then modified to its final format (Table 1) and it has subsequently been used within the residency program.

Table 1.

A tool to assess the quality of consultation and referral request letters

Consultation and Referral Request Letter Assessment Tool
Date of letter: ______________________________________
Discipline letter directed to: ________________________
A. Content
1. Patient demographics			YES	NO
2. Initial statement identifying the reason for the referral			YES	NO
3. Description of chief complaint			YES	NO
4. Description of associated symptoms			YES	NO
5. Description of relevant collateral history			YES	NO
6. Past medical history			YES	NO
7. Past surgical history			YES	NO
8. Relevant psychosocial history			YES	NO
9. Current medication list			YES	NO
10. Allergies			YES	NO
11. Relevant clinical findings			YES	NO
12. Results of investigations to date			YES	NO
13. Outline of management to date			YES	NO
14. Provisional diagnosis or clinical impression			YES	NO
15. Statement of what is expected from the referral			YES	NO
B. Style
16. One topic per paragraph			YES	NO
17. Paragraphs with fewer than 5 sentences			YES	NO
18. One idea per sentence			YES	NO
C. Overall appreciation
Letter unhelpful to consultant			Informative helpful letter
1	2	3	4	5

Use of the tool

Using the tool requires very little time or effort, and it is easily integrated into the clinical teaching context. Because the assessment process uses routinely generated data (dictated letters), it does not generate much extra work for residents or family physician teachers; however, there is some concern that this might generate extra work for typists, as residents might do extensive revisions of their letters.

The tool can be used in different ways. Residents can use the tool as a guide when writing letters or as a self-assessment activity when reviewing their letters before sending them out. Family medicine teachers can also use the tool to structure their feedback of residents’ letters.

Conclusion

The described tool appears to meet criteria for a good assessment instrument: it has validity, reliability, and feasibility. The content and style items of the instrument were derived directly from a framework for good practice in written communication. Family medicine teachers provided input to its final composition, and a consensus was achieved. The final product is consistent with the literature on good practice in written communication.

More important, this tool addresses a gap in our curriculum and helps learners to successfully complete one of the most commonly performed writing tasks required of primary care physicians.