A Model of Ambiguity and Vagueness in Clinical Practice Guideline Recommendations

Abstract

Ambiguity and vagueness in clinical practice guidelines reduce the likelihood of clinician adherence. They lead to inconsistent interpretation and, in turn, to inappropriate practice variation and medical errors.

Resolving ambiguity and vagueness is an essential step in the computerized implementation of clinical practice guidelines. Successful resolution of ambiguity and vagueness requires an understanding of their characteristics, yet ambiguity and vagueness have not been differentiated, classified and described in medical context.

In this paper, we propose a tri-axial model to describe ambiguity and vagueness in clinical practice guidelines: differentiation of true ambiguity from vagueness, classification of ambiguity and vagueness, intentionality and components involved. Our goals in introducing this model are: (a) to provide guidance to guideline authors to enable them to reduce inadvertent use of ambiguous or vague language, (b) to improve transparency when vague language is used deliberately and (c) to create a framework for the development of tools to apply the model during authoring and implementation of clinical practice guidelines.

INTRODUCTION

Ambiguity and vagueness have been the subject of considerable attention in linguistics and philosophy^1–4, but despite the significant detrimental effect of ambiguity and vagueness on CPG adherence and implementation, these concepts have not been explored and differentiated in a medical context. The term “ambiguity” is often used indiscriminately in the medical literature to denote both vagueness and ambiguity and the term “disambiguation” has been used to describe the resolution of both vagueness and ambiguity.

Ambiguity exists when a term can reasonably be interpreted in more than one way, for example, the word “bank” can refer to a financial institution or a riverside. Vagueness occurs when the boundaries of a word’s meaning are not well defined, as in the word “tall”⁵. Vagueness also exists when a word or phrase reduces the level of information contained in a statement, as in stating that an action “may be appropriate”; this phrase reduces the clarity about whether or not the action should be performed.

CPGs are promoted as a means to decrease inappropriate practice variation and to reduce medical errors. For these goals to be achieved, clinicians must adhere to the CPG recommendations in a consistent manner. Use of ambiguous and vague terms hampers communication and leads to uncertainty and to variable interpretation. This uncertainty is an established cause of decreased adherence to clinical practice guidelines (CPGs)^6–8.

The interpretation of vague terms by clinicians varies greatly^{9, 10}, leading to reduced adherence or to increased practice variation despite apparent CPG adherence.

In this paper, we propose a model for ambiguity and vagueness in CPG recommendations. Our goals in introducing this model are to provide guidance to CPG authors to enable them to reduce inadvertent use of ambiguous and vague language, to improve transparency when vague language is used deliberately, and to create a framework for the development of tools to diminish the various forms of ambiguity and vagueness during the CPGs authoring and implementation processes. We believe that achieving these goals will facilitate CPGs implementation and lead to a reduction in variable interpretation of CPGs, ultimately diminishing inappropriate practice variation by clinicians.

BACKGROUND

Previous approaches have focused largely on clinicians’ assignment of numerical values to vague terms. Kong et al. studied the “use of qualitative expressions of probability” among medical professionals, asking physicians to assign a numerical value to twelve vague terms which the authors called “probabilistic terms”¹⁰. Their results show great variability of interpretation of these terms among observers. Other researchers showed comparable results applying similar methodology to judgments by radiologists, pathologists, surgeons and non-physician scientists.

The major shortcoming of this research is that subjects were not provided with a context for interpreting the vague terms. The importance of context on the interpretation of vague terms was shown by Mapes who conducted a study in which physicians were asked to assign a numerical value to the phrase “side effects with [this drug] are rare” in two different contexts: beta adrenergic blocking agents and antihistaminic drugs¹¹. The average numerical estimate was much lower for the beta-blocker agents than for the antihistaminic drug. Mapes suggests that these different interpretations reflect the perceived severity of side effects for the different drug classes. Further complicating matters is the effect of framing. Tversky and Kahneman demonstrated that formulating identical statements in terms of “lives lost” or “lives saved” yielded different preferences, despite identical overall outcomes¹².

Previously suggested approaches to resolving vagueness have included: assigning numerical values to vague terms¹⁰, ranking vague terms in relation to one another¹⁰, maintaining a “black-list” of terms that have been found to be particularly prone to variable interpretation¹³ or that have particular legal implications¹⁴, creating a controlled vocabulary of vague terms from which authors could choose, mapping vague terms to specific actions¹⁵ and applying fuzzy logic methods¹⁶.

Creating a controlled vocabulary of vague terms has the potential to reduce variable interpretation. Researchers have found hundreds of vague terms used in the medical literature, and reducing these to a manageable set of terms is a viable solution. A closely related solution is to require users to apply a limited structured vocabulary. This approach has been implemented in radiology where a lexicon of terms was created for the purpose of reporting mammography results ¹⁵. Although some of the six choices are worded in a highly vague manner (“Suspicious. Biopsy should be considered”), clinical uncertainty is diminished since there are only 6 categories.

Researchers have noticed that a rank order of vague terms relative to one other was maintained across studies, despite wide variability in the interpretation of specific terms in the individual studies. This finding indicates it might be possible to create an ordinal scale of ranked vague terms from which authors could select when deliberately using vague terms.

Certain terms yield a wider range of interpretation than others, and should be avoided when possible; examples of such terms are “probably”, “normally” and “possibly”. The Agency for Health Care Policy and Research suggested that words such as “should” be avoided in guideline recommendations because of concern over potential legal implications ¹⁴.

Areas not addressed in previous research include the various forms of ambiguity and vagueness, the source (deliberate or inadvertent) and rationale for using them, and the component of the CPG recommendations affected by them, e.g. the condition, the action or the explanation.

METHODS

We reviewed the literature on ambiguity and vagueness in medical communication with the goals of determining the scope of previous research in this field and identifying previously recognized forms of vagueness of ambiguity.

The term “vagueness” is not mapped by Medical Subject Headings (MeSH). “Ambiguity” is mapped to the keyword “fuzzy logic”, which does not sufficiently describe the scope of ambiguity as we define it in this work. We therefore performed a text search of the terms “vague*” and “ambigu*”. This search yielded over 3000 results for each term, with very few of the results pertaining to the linguistic concepts of ambiguity or vagueness.

We next identified relevant terms, by traversing pertinent branches of the 2005 MeSH. Potentially relevant terms appeared in several main headings in MeSH, such as Humanities (logic), Information science (linguistics, semantics, terminology and fuzzy logic), and Psychiatry and psychology (comprehension, decision making, judgment and uncertainty). MEDLINE was searched with each of the MeSH keywords alone, and in combination with either “vagueness” or “ambiguity” using (MeSH term)[mesh] AND (vague* OR ambigu*). We further expanded our literature search by searching for the MeSH terms identified above with each of the MeSH terms “guidelines” and “physicians”.

Bibliographic lists of articles identified were searched for additional relevant articles. In addition, we searched the Web of Science database for articles citing the relevant articles identified so far. This process was performed iteratively, until no additional relevant articles were found.

Articles retrieved by this process provided the background for our model. Lists of vague terms from these articles were compiled, as were forms of vagueness that were previously described. Classes of ambiguity and their definitions were derived from the linguistics literature.

PROPOSED MODEL

Our proposed model comprises three axes (Figure). The first axis explores semantics, differentiating ambiguity from vagueness and classifying each of them. The second axis describes authors’ intent: the source of ambiguity or vagueness, i.e. whether it is deliberate or inadvertent, and when deliberate, the specific rationale for its use. The third axis defines what recommendation component is affected.

Figure.

Model of vagueness and ambiguity in CPG recommendations

First axis: differentiation and classification

Ambiguity is classified into syntactic, semantic and pragmatic ambiguity, paralleling the three aspects of information¹⁷. Syntactic ambiguity arises when ambiguity is caused by the structure or syntax of a statement. This can occur when punctuation (or lack thereof) or Boolean connectors in a statement leave its meaning unclear. An example is “A or B and C” without clarifying whether this means “(A or B) and C” or “A or (B and C)”. Semantic ambiguity is the “classic” form of ambiguity in which a term can be interpreted in more than one way, such as the word “bank” in the sentence “I will meet you at the bank”. A special case of semantic ambiguity lies in the use of abbreviations whose reference is unclear. Pragmatic ambiguity refers to usage; as in saying on Wednesday: “See you next Friday”. Does the speaker mean to meet you in two or in nine days?

Pragmatic ambiguity can be created when two or more recommendations within a CPG are inconsistent or conflict with one another. Guideline recommendations that do not include instructions for all clinical scenarios are not comprehensive and lead to pragmatic ambiguity because clinician users of the CPG can reasonably act in more than one way ¹⁸.

Underspecification is a form of vagueness that occurs when terms are used with insufficient details for definitive interpretation. Examples of underspecification are terms such as “moderate”, “elderly” and “adequate”. Some groups of underspecified terms have shared characteristics and can be represented on an ordinal scale of terms. The temporal vagueness scale ranges from “never” to “always” and includes intermediate terms such as “rare” and “common”. Probabilistic terms, range from “impossible” to “certain”, with terms such as “unlikely” and “probable” in between. Quantitative terms range from “none” to “all” with intermediate terms such as “few” and “many”. The terms at the extremes of each scale are not vague; they represent the only non-vague terms in each scale. Other forms of underspecification, such as implicit statements and incomplete information, can not be classified using ordinal scales, and are classified as non-ordinal underspecification.

Phrases that qualify the strength of CPG recommendations make up another category of vague terms. Examples of such terms are “it is prudent to recommend” and “is recommended as probably effective”. Vague terms in this class overlap with the other forms of vagueness, which can also be used as strength qualifiers as in “[a certain action] may be beneficial” and “[a certain action] is often beneficial”. We found it impossible to rank these qualifiers relative to one another.

The use of the passive voice as in “should be performed” is a form of vagueness¹⁹. Such usage has long been considered the norm in scientific writing but it obscures who is expected to perform the action. The actor may be a critical factor in some CPG statements.

As with ambiguity, terms can be considered vague because of the context in which they are used. One guideline recommendation includes both of the following statements: “present data are insufficient to support the use of [drug A]” and “there are no data to support the use of [drug B]”. Either of these, on its own, would be considered clear. When found together, however, the reader cannot determine whether there is more evidence supporting the use of drug A or B, or that both are equally not recommended.

Second axis: source and rationale

Ambiguity or vagueness may be deliberately or inadvertently introduced in CPG recommendations. Inadvertent ambiguity or vagueness generally reflect insufficient editing by the authors, reviewers and publishers of CPGs, and should be addressed by careful attention to the issue of ambiguity and vagueness during CPG authoring.

Deliberate use of vagueness or ambiguity poses a significant problem to the CPG audience. Attempts to resolve the vagueness might contradict the authors’ intention. Successful resolution of deliberate vagueness requires an understanding of its rationale, and of the range of possible interpretations the authors consider appropriate. The process by which CPG authors use vague terms has not been studied.

Insufficient scientific evidence and failure to reach a consensus among CPG authors are probably a common cause of deliberate vagueness. Reporting the quality of evidence and strength of recommendations partially address this cause of deliberate vagueness, but fail to provide clinicians or implementers with the range of acceptable interpretations.

Liability concerns have been reported in connection with CPG authoring, and have included advice on the naming of policy statements by professional societies, as well as advice on the choice of vague terms¹⁴. Ethical and religious concerns have been reported regarding the deliberate use of vague terms such as the term “futility”. Economic considerations are also likely to play a role in the choice of vague terms when a CPG recommendation presents an unusual financial burden.

Third axis: recommendation component affected

Clinical practice guideline recommendations typically describe an action (what should be done) and the conditions under which this action should take place (when it should be done). It is rare for CPG recommendations to be formatted as a pure if-then structure; there is often additional text qualifying or explaining the condition or the action (e.g. why it should be done).

Ambiguous and vague terms can occur within any or all of these three components: the condition(s), the action(s), and the explanation(s) for the recommendation. When the conditions are affected, decidability of the recommendation is undermined. When the actions are unclear, executability is hampered. Vagueness affecting the explanation segment has the least impact on implementability. For example, in the statement “If a 20-foot visual acuity measurement is required, a projected standard Snellen chart should be used because acuities… may not correlate to…” the probabilistic vague term “may not” has minimal effect on implementability, because it is not part of the condition or the action.

DISCUSSION

Vagueness and ambiguity in CPG recommendations are a significant barrier to widespread adherence and impede computer-based representation and implementation. Computerized implementation of CPG recommendations requires the resolution of ambiguity and vagueness. The product of this resolution process is a revised recommendation statement that clearly states what action is to be taken and under what circumstances. The resolution process is successful when the intended audience of the CPG interprets the revised CPG recommendations in the manner intended by the authors, with minimal variation between readers. The best ways to ‘disambiguate’ ambiguity and ‘resolve’ vagueness are not known. We propose that the three axes in our model can serve as a basis for addressing the problem.

The ideal timing for remedying ambiguity and vagueness is during the authoring of CPGs. At this stage careful editing can remove inadvertent ambiguity and vagueness. When authors deliberately choose to use vague terms, they should take into consideration the average and range of values that have been assigned by clinicians in interpreting the terms (axis 1). Vague terms can be categorized according to the first axis of our model, ranked in relation to one another and made available to authors. Where deliberate vagueness is deemed necessary because of insufficient evidence or lack of consensus the reasoning is most appropriately placed in the explanation part of the recommendation, as this part least affects the implementability of the recommendation (axis 3).

Once the CPG has been published, resolving the remaining ambiguity and vagueness is left to clinicians and to informaticians attempting to represent or implement the CPG recommendations. Attempting to understand the source and rationale for vague terms is an essential step in resolving the vagueness as attempting to resolve vague terms which are deliberate runs the danger of significantly altering the authors’ intention (axis 2). Implementers should also be aware of the classification of vague terms, and can use numerical data and the order in which terms are ranked in the ordinal scales in deciding how to interpret vague terms (axis 1). Finally, for recommendations that can be parsed into the three components of axis 3, implementers can concentrate on interpreting terms in the condition and action components.

Our model can be used to develop software tools that would be of help during the authoring and implementation stages. Such software tools could identify and classify vague terms, inform authors (or implementers) of numerical values associated with the term, display an ordinal scale of similar terms, and suggest alternative terms.

Our next step will be to validate this model by examining a random selection of 100 CPG recommendations, classifying vague and ambiguous terms (axis 1), and determining the affected component (axis 3). To assess axis 2 will require polling guideline authors as to their reasons for introducing vague statements. The model will be iteratively refined and reevaluated with an additional set of random CPG recommendations.

CONCLUSION

We propose a novel model of ambiguity and vagueness in CPG recommendations. This model can be used during the development of CPGs to identify and eliminate unnecessary ambiguity and vagueness. When deemed necessary, the use of vagueness can be facilitated by our model aiming to reduce variable interpretation and inappropriate practice variation.