Measuring Communicative Participation: A Review of Self-Report Instruments in Speech-Language Pathology

Abstract

Purpose

To assess the adequacy of self-report instruments in speech-language pathology for measuring a construct called communicative participation.

Method

Six instruments were evaluated relative to (a) the construct measured, (b) the relevance of individual items to communicative participation, and (c) their psychometric properties.

Results

No instrument exclusively measured communicative participation. Twenty-six percent (n = 34) of all items (N = 132) across the reviewed instruments were consistent with communicative participation. The majority (76%) of the 34 items were associated with general communication, while the remaining 24% of the items were associated with communication at work, during leisure, or for establishing relationships. Instruments varied relative to psychometric properties.

Conclusions

No existing self-report instruments in speech-language pathology were found to be solely dedicated to measuring communicative participation. Developing an instrument for measuring communicative participation is essential for meeting the requirements of our scope of practice.

In 2001, the World Health Organization (WHO) proposed a model that integrates the concepts of health and functioning into the International Classification of Functioning, Disability and Health (ICF; WHO, 2001). The purpose of the ICF is to offer clinical providers a common language for describing human functioning and disability, as well as to provide a conceptual framework for gathering data and measuring clinical outcomes. The ICF framework also is useful for speech-language pathologists (SLPs) for describing the consequences of communication disorders at several levels, including communication in social settings.¹ For this reason the American Speech-Language-Hearing Association (ASHA) adopted the ICF as the framework for assessment and intervention in the 2001 Scope of Practice in Speech-Language Pathology (ASHA, 2001) and in the Preferred Practice Patterns for the Profession of Speech-Language Pathology (ASHA, 2004).²

The ICF defines a person's functioning and disability in relation to health condition and contextual factors. It has three levels: (a) “Body Functions and Structures,” which include the physiological functions of body systems or anatomical elements such as organs, limbs, and their components; (b) “Activities,” which are the execution of specific actions; and (c) “Participation,” which encompasses involvement in life situations (WHO, 2001). It is well recognized that difficulties in body functions and structures (“Impairments”) are the most studied outcomes in communication disorders (Eadie, 2001; Threats, 2000). Impairment measures of speech, language, voice, and fluency include measures of articulatory accuracy, grammatical complexity of language, physiological functioning of the vocal folds, and number of disfluencies, to name but a few. Although many instruments exist at the level of body functions and structures, there is a paucity of instruments as performance challenges become less “biomedical” in nature (Eadie, 2001; Simmons-Mackie, Threats, & Kagan, 2005). This is related to the nature of the problem. That is, it is easier to measure the regularity of vocal fold movement, the accuracy of naming objects, or the percentage of words understood by a listener than it is to measure an individual's ability to participate in valued activities or situations such as negotiating a medical plan with a physician, applying for a job, or persuading friends or family members to change their minds about an issue.

Adopting the ICF as the framework for assessing and providing intervention for individuals with communication disorders suggests that SLPs assess communication in social contexts, or communicative participation. The overall purpose of this article is to review self-reported instruments of communicative functioning to determine how well we are addressing the participation component of the ICF model in the field of speech-language pathology. In order to review current instruments relative to communicative participation, it is first necessary to examine issues related to measuring and defining this construct.

Considerations Related to Measuring Participation

Participation is increasingly being recognized as an important outcome domain in rehabilitation research, and as a consequence, there are an increasing number of instruments available for measuring this construct (Brown et al., 2004; Perenboom & Chorus, 2003). A number of factors must be considered when evaluating how well an instrument measures a construct such as participation. First, and most importantly, the construct to be measured must be defined, and individual items within each instrument must be evaluated to determine whether they reflect the intended construct. The domains that encompass a particular construct reflect its breadth. For example, Perenboom and Chorus (2003) reviewed how existing survey instruments assessed participation using the ICF taxonomy. They identified 11 survey instruments for evaluation, with 9 instruments meeting their inclusion criteria. The results indicated that while all 9 instruments measured participation to some extent, most instruments were not solely dedicated to measuring participation, and the majority of instruments included items that were consistent with either the activity or the body functions and structures levels of the ICF. The researchers concluded that development of a health-related instrument of participation was required.

A second consideration related to how well an instrument measures a construct concerns the type of information requested of respondents. For example, Johnston and Miklos (2002) introduced the concept of “activity-related quality of life (QOL)” to assess quality of everyday life. They suggested that to measure activity-related QOL, one must first assess what the person does (activities) objectively and then assess that person's appraisal of the performance of that activity in order to determine whether a successful outcome was attained. Thus, not only must one assess the frequency or intensity of one's performance of an activity (i.e., “objective” measures), but one must also focus on a person's satisfaction or importance for performing that activity (i.e., “subjective” measures). These types of response metrics also are appropriate for measuring the construct of participation or social outcomes (Brown et al., 2004; Dijkers, Whiteneck, & El-Jaroudi, 2000).

Third, one must consider the source of the information when a construct is measured. Information can be self-reported by the individual with the disability or provided by a significant other, a clinician, or an outside observer (i.e., proxy measures; Johnston & Miklos, 2002). Checklists and newer clinical procedural manuals derived from the WHO ICF classification system are completed by the clinician, often with input from the individual with the health condition (Threats, 2000, 2006). However, the nature of a person's response may be affected by how the information is obtained; for example, a question asked during a face-to-face interview might elicit a different response than if the same question is presented within a self-administered questionnaire (Chan, Orlando, Ghosh-Dastidar, Duan, & Sherbourne, 2004).

We believe that information provided by the person with the health condition (i.e., insider perspective) is an essential component for measuring participation because it provides unique and important information not addressed by other types of measures. For example, subjective measures of participation (e.g., satisfaction, importance) are by definition restricted to judgments made by the person with the health condition. Obtaining subjective responses necessitates inclusion of a self-report format.

Participation occurs in social contexts with multiple partners; therefore, only the individuals with the health conditions are privy to all information. For example, if a clinician was asked to provide the number of communication partners an individual had in the past week as a measure of communicative participation (i.e., an objective measure), the clinician would still (in all practicality) have to ask the person for his or her best estimate of this performance because the clinician would not have observed all instances of communication during the week. In addition, responses obtained from proxies, including family members, may yield different responses than those derived from the individual with the health condition. For example, researchers have found that proxies and clients respond differently to questions associated with cognitive, communicative, and emotional status (Cruice, Worrall, Hickson, & Murison, 2005; Duncan et al., 2002; Sneeuw, Aaronson, de Haan, & Limburg, 1997). Thus, information derived from both sources may address different questions.

A final reason why self-reported information is important is that it may help individuals and their health providers make collaborative treatment decisions and measure outcomes. In this vein, patient-reported outcome measures are receiving considerable attention. For example, the National Institutes of Health have selected the Patient-Reported Outcomes Measurement Information System as one of their Roadmap Initiatives (http://www.nihpromis.org/). This project has the goal of developing large banks of self-report items in domains such as pain, fatigue, and social participation.

Communicative Participation as a Primary Outcome Measure

One method of describing the breadth of a construct such as participation is to frame it within the context of the ICF classification of “Activities and Participation,” which includes nine chapters: 1. Learning and Applying Knowledge; 2. General Tasks and Demands; 3. Communication; 4. Mobility; 5. Self-Care; 6. Domestic Life; 7. Interpersonal Interactions and Relationships; 8. Major Life Areas; and 9. Community, Social and Civic Life (WHO, 2001). Although specific communication acts are found in the third chapter of the ICF manual, communication is a construct that is pervasive and is required to fulfill other aspects of participation found in most of the other chapters (e.g., communication involved in job performance, academic performance, self-care, community roles, establishing and maintaining relationships). Thus, while general communication functions are included in this chapter, the coverage of domains outlined in additional chapters of the ICF may provide a preliminary measure of the adequacy of the breadth of communicative participation.

For the purposes of this review, we define communicative participation as taking part in life situations where knowledge, information, ideas, or feelings are exchanged. It may take the form of speaking, listening, reading, writing, or nonverbal means of communication. Communicative participation may occur in multiple life situations or domains and includes, but is not limited to, personal care, household management, leisure, learning, employment, and community life (WHO, 2001). Because communicative participation is measured in a social context, by definition it involves more than one person and must involve a communicative exchange (i.e., a message and the opportunity for a response from a communicative partner). Communicative participation may take place for a defined social goal (e.g., establishing relationships), for a function/role (e.g., job-related), and/or in a particular context (e.g., in a restaurant).

It is important to measure communicative participation for a number of reasons. First, participation reflects the final common pathway for many aspects of functioning/disablement (Yorkston, Klasner, & Swanson, 2001). Participation certainly can be altered by the presence of an impairment, limitation in activity, the environment, and personal context. Thus, participation is central because every aspect of disablement and contextual factors may directly influence it.

Second, communicative participation may be considered a universal outcome that is common to both individuals with and without communication disorders. For example, communicative participation may be limited because of a problem in motor speech execution such as dysarthria or a problem with handling language symbols such as aphasia (i.e., in the presence of a communication disorder). Likewise, communicative participation may be limited because of a mobility problem that restricts preferred social contacts (i.e., when a person does not have a communication disorder).

Third, it is important to measure communicative participation to advance our understanding of the impact of intervention and to develop and revise multidimensional models of rehabilitation (Threats, 2006). A socially valid measure of treatment outcome also would allow us to better document the effectiveness of intervention programs. In this context, a socially valid measure is one that reflects the client's concerns and values, and provides a method for prioritizing potential interventions. In clinical research, an instrument would help to identify differences in the severity and pattern of restrictions in participation across and within various populations (e.g., dysarthria vs. cognitive-communication disorders, traumatic brain injury vs. multiple sclerosis, and mild vs. moderate severity of impairment). Finally, if communicative participation was adequately measured, we could study its relationship with other important variables such as QOL (Threats, 2006).

Purpose Statement

In the field of rehabilitation, outcome instruments have been the focus of an increasing number of critical reviews (Cardol et al., 1999; Cohen & Marino, 2000; Dijkers, 2004). The aim of our study was to conduct a review of existing self-report instruments in speech-language pathology that may measure communicative participation in order to reveal the adequacy of current instruments for evaluating communicative participation, and to identify whether new instruments need to be developed to meet the requirements of our scope of practice (ASHA, 2001). Specifically, the research questions were the following:

1. How adequately do self-report instruments in speech-language pathology measure the construct of communicative participation?

2. What is the relevance of individual items found in these instruments to the construct of communicative participation?

3. What are the psychometric properties of instruments that measure communicative participation, or some aspect of communicative participation?

Method

Identification of Instruments: Inclusion/Exclusion Criteria

To identify instruments that measured communicative participation, we convened an interdisciplinary team that consisted of rehabilitation experts in the fields of speech-language pathology, occupational therapy, and educational psychology. Team members first identified familiar commercial instruments or instruments published in the literature that were primarily focused on measuring communication and could potentially contain items measuring communicative participation. In addition, an electronic search was performed using PsycINFO, Medline, and CINAHL, using the following key words: clinical assessment tools, outcome assessment, instrument construction, quality of life, handicap, handicapped, participation, communication disorders, aphasia, dysarthria, voice disorders, scales, instruments, questionnaires, and measurement.³ Edited books and published scales also were hand-searched.

To address our specific goals, we targeted instruments whose primary purpose was to measure communicative functioning. These instruments were targeted for two reasons. First, instruments that measure communicative functioning are used most often by SLPs (Simmons-Mackie et al., 2005). Second, most disease- or disorder-specific instruments of health (e.g., the SF-36 Health Survey; Ware, 1993) or broad instruments of participation (e.g., Craig Handicap Scale and Reporting Technique [CHART]; Whiteneck, Charlifue, Gerhart, Overholser, & Richardson, 1992) include only one or two items related to communication and do not usually include communication domains or subscales. Interpretation of items within these instruments would be difficult when items were originally developed to measure other constructs. For example, although the CHART includes items that could be classified as communicative participation (e.g., “How many relatives do you visit, phone, or write at least once a month?”), these items cannot be separated into a single subscale to measure the communicative aspects of participation independent from other aspects. Other instruments such as the Community Integration Questionnaire (Willer, Rosenthal, Kreutzer, Gordon, & Rempel, 1993) contain the following items: “Who usually plans social arrangements such as get-togethers with family and friends?”; “Do you have a best friend with whom you confide?”; and “Who usually cares for children in your home?” Although communicative skills are required for performance in each of these situations, communication is not explicitly addressed.

Only instruments that were initially published in English were considered because this reflected the linguistic expertise of the team of researchers conducting the review. When instruments were not published in the professional literature, every effort was made to locate the manual of the instrument or to contact the authors to obtain the original questionnaire. Finally, only self-report instruments were included in this review because they offer the potential for obtaining subjective responses and because information garnered from proxies may be different from information provided by individuals with health conditions (Jette, Haley, & Kooyoomjian, 2003; Perenboom & Chorus, 2003).

Two team members reviewed each of the identified instruments. Each team member evaluated the instrument independently and then compared results with the other team member. If any questions or disagreements arose, they were discussed and a consensus decision was made with the entire team.

Evaluation of Instruments for Communicative Participation

In order to identify instruments that measured the construct of communicative participation consistent with our definition, we first compared the information provided by the instrument developers about what the instrument was intended to measure with the communicative participation definition used in this study. If the authors’ statements appeared to be consistent with communicative participation, then all individual items from the selected instruments were reviewed and evaluated. For items to reflect communicative participation, they were required to meet one essential criterion: including a communicative exchange between at least two communicative partners (i.e., a message with the opportunity for a response) in the context of a life situation. This criterion excludes items that are consistent with body functions and structures (e.g., physical effort in producing voice, or intelligibility of speech sounds), as well as some activities (e.g., picture naming), because there is no exchange or opportunity for a response involved in these tasks and because some of these tasks do not ordinarily take place outside of a clinic room (i.e., not in the context of a life situation). This criterion was included in an effort to separate basic activities from those elements that are more reflective of participation (WHO, 2001).⁴ For example, an item such as “reading” would be excluded for several reasons. First, in reading, only one person is involved. Second, the “message” is not really tailored to a specific communicative partner, and finally, there is no opportunity for a response. In contrast, “writing an e-mail” would be consistent with the definition because it involves two communicative partners and affords both partners the opportunity to respond to the message. Likewise, “reading to one's spouse” would be considered communicative participation because the spouse has the opportunity to respond to the message. Using these criteria, items such as “listening to the radio” or “watching television” also would not be considered communicative participation because there is no communicative exchange between two communicative partners and there is no opportunity for a response. The criteria used to create this definition were consistent with recent data which suggest that items related to the use of written communication (e.g., “Write your name”) on instruments of communicative functioning appear to be measuring different constructs than other communication items (Doyle, Hula, McNeil, Mikolic, & Matthews, 2005).

In order to assess how adequately current instruments measured the breadth of communicative participation, all items considered communicative participation were then classified into domains outlined in Table 1. For example, an item might include communication related to a specific task such as making an appointment, or it might be to maintain a relationship and enjoy a social situation. In addition, an item might specify a particular context (e.g., indicating what you would like to eat in a restaurant). These domains included (a) personal care, (b) household management, (c) work/ education, (d) leisure/recreation, (e) relationships, and (f) community. If a domain was not stated specifically and the communication could cross multiple domains, the item was classified as (g) general communication. For example, an item such as “greeting a person by name” would be included in the “general communication” domain because it could take place across many life situations. These items are consistent with global communication items found in Chapter 3 of the ICF.

TABLE 1.

Definitions of communicative participation domains used in the review.

Domain	Definition and examples
Personal care	Communicating in situations related to self-care (e.g., providing instructions to assistants for personal services or health care; asking questions about medications).
Household management	Communicating in situations related to performing the routine duties of managing a household and those living together in it (e.g., giving instructions to a repair person; indicating to a family member what one would like for dinner).
Work/education	Communicating in situations related to paid or unpaid (volunteer) employment or school work (e.g., explaining a problem at work or school; giving or accepting feedback about work or school).
Leisure/recreation	Communicating in situations related to discretionary activities not related to work or other duties; may be either quiet or active activities (e.g., taking part in a book club meeting; playing cards or other games with friends; discussing or debating current events).
Relationships	Communicating in situations that connect or bond participants, including family, friends, and romantic relationships (e.g., writing a text message to a friend; getting one's point across when upset).
Community	Communicating in situations related to community integration (e.g., taking part in a religious service or activity; ordering a meal in a restaurant; asking a bus driver for information).
General communication	General communication items that could cross multiple domains (e.g., making small talk; resolving a conflict; talking on the telephone).

Psychometric Properties

Once the instruments were identified, two interdisciplinary team members reviewed all available primary literature sources and instrument manuals and noted what populations were tested during scale development, the intended target population of the scale, and psychometric properties using the definitions provided in Table 2 (Andresen, 2000). The frequency of literature citations for each instrument also was reported. The number of published studies associated with an instrument is not a psychometric property, but it is important to consider in the context of evaluation of an instrument's psychometric properties. It is for this reason that this information was provided in this review.

TABLE 2.

Definitions of characteristics of the evaluated instruments.

Characteristic	Definitions/considerations for evaluation
Construct measured	What is measured by the instrument and how is it defined?
Population	Who is the target population of the instrument (for whom was the instrument intended?). Who is the actual tested sample (i.e., who was tested identifying details such as medical diagnosis and communication disorder). What is the size of tested sample?
Reliability	Does the instrument give a consistent answer between 2 test times (test–retest reliability)? What is the instrument's internal consistency (i.e., the extent to which items measure various aspects of the same characteristic as measured by correlations among all items in a scale or measure the same construct in a scale using Cronbach's coefficient alpha or item-to-total correlations)?
Validity	Does the instrument measure what it purports to measure? Includes face validity (do consumers of the instrument help verify the importance of items?); content validity (do experts in the field help verify the theoretical domain sampled by the test?); concurrent/convergent validity (how closely does an individual's test score correlate to his/her score on a criterion variable measured at about the same time the test score was obtained?); divergent validity (are scores from unassociated instruments related to scores on the reviewed instrument?); predictive validity (how closely does an individual's test score predict future performance on a criterion measure?); and construct validity (how well does an instrument measure an abstract or theoretical concept?).
Frequency of instrument use	How many times has the instrument been used in the peer-reviewed research literature?

Reliability

Test–retest measures score consistency over a brief time period, typically no more than several weeks (Allen & Yen, 1979). They show how much a score is likely to change when the scale is readministered in a brief period of time. A different type of reliability assesses to what extent each item on an instrument consistently contributes to the total score for that instrument; this internal consistency is usually calculated using Cronbach's alpha statistic (Streiner & Norman, 2003). Measures of internal consistency are expected to be high because items presumably measure one related construct. Although not as typical, some test developers also calculate the relationships of scores between subdomains using Pearson correlation coefficients (Streiner & Norman, 2003). These values are not expected to be as high as measures of internal consistency because they presumably measure different aspects of a construct.

Validity

An instrument's validity can be evaluated in several different ways. Face validity and content validity are the logical examination of the content of instrument items. Concurrent validity examines the relationship between the instrument of interest and other instruments that purport to measure the same construct. To demonstrate construct validity, the developers should demonstrate a clear conceptual framework (see Table 2 for specific definitions).

Frequency of instrument use

A final description of the instrument included the number of times it was cited in the peer-reviewed research literature in the PsycINFO, Medline, and CINAHL databases from 1980 to 2005.⁵

Results

Fifteen instruments were identified for possible review. Of these, three of the most frequently used instruments of functional communication were excluded because they used clinicians’ judgments of an individual's behavior in lieu of self-reports. These included the ASHA Functional Assessment of Communication Skills for Adults (FACS; Frattali, Thompson, Holland, Wohl, & Ferketic, 1995), the Communication Activities of Daily Living (Holland, 1980), and the Therapy Outcome Measures (Enderby & John, 1997). In addition, the Communication Effectiveness Index (CETI; Lomas et al., 1989) was excluded because it involved a spouse or family member's judgment of an individual's ability to communicate. Four instruments of hearing performance—the Hearing Handicap Inventory for the Elderly (Ventry & Weinstein, 1982), the Communication Profile for the Hearing Impaired (Demorest & Erdman, 1986), the Hearing Participation Scale (Hawthorne & Hogan, 2002), and the Speech, Spatial, and Qualities of Hearing Scale (Gatehouse & Noble, 2004)—were excluded because they focused on measuring hearing and auditory processing, rather than communicative exchange. Finally, although the Communication Effectiveness Survey appeared to measure a construct consistent with communicative participation, it was excluded because it was considered too early in its development to be evaluated and was not yet available in any published format (Donovan, Velozo, Rosenbek, Okun, & Sapienza, 2004).

Six instruments underwent full review. These were the ASHA Quality of Communication Life (QCL) scale (Paul et al., 2004), the Burden of Stroke Scale (BOSS; Doyle, McNeil, & Hula, 2003), the Voice Activity and Participation Profile (VAPP; Ma & Yiu, 2001), the Voice Handicap Index (VHI; Jacobson et al., 1997), the Voice-Related Quality of Life (V-RQOL) scale (Hogikyan & Sethuraman, 1999), and the Voice Symptom Scale (VoiSS; Deary, Wilson, Carding, & MacKenzie, 2003). Characteristics of the reviewed instruments, including instructions and response metrics, are included in Table 3. As indicated in Table 3, the ASHA QCL and BOSS measure the level or difficulty of performance, the VHI and VoiSS measure the frequency of interference, and the VAPP and V-RQOL measure the intensity of interference.

TABLE 3.

Reviewed instruments, instructions, response metrics, and items from each instrument consistent with domains of communicative participation.

Instrument	Domain	Instructions/item	Metric
ASHA QCL		Even though I have difficulty communicating... (1 = poorest score, does not describe you well; 5 = highest score, describes you well):	1−5 points
	Household	...I have household responsibilities.
	Work/education	...I meet the communication needs of my job or school (typing, giving, and following directions, reading).
	Leisure/recreation	...I follow news, sports, and stories on TV or in movies.
	Relationships	...I stay in touch with family and friends.
	General	...I use the telephone.
	General	...people include me in conversations.
	General	...people understand me when I talk.
	General	...it's easy for me to communicate.
BOSS		Because of your stroke, how difficult is it for you to... (1 = not at all; 5 = cannot do)	1−5 points
	General	...find words you want to say?
	General	...understand what people say to you?
	General	...understand what you read?
	General	...write a letter?
	General	...talk with a group of people?
	General	...be understood by others?
	General	How much do difficulties communicating prevent you from doing the things in life that are important to you?
VAPP		Please answer the following questions by putting a cross (“X”) on the line that best represents your answer. A cross toward the left side means you are never affected, while a cross toward the right side means you are always affected. In the last 6 months, have you...	10-cm visual analog scale
	General	...reduced the use of the telephone because of your voice problem?
	General	...ever avoided having conversations in quiet environments because of your voice problem?
	General	...ever avoided having conversations in noisy environments because of your voice problem?
	General	...ever avoided having conversations in a group because of your voice problem?
	General	...ever avoided communicating with your family, friends, or coworkers because of your voice problem?
VHI		These are statements that many people have used to describe their voices and the effects of their voices on their lives. Circle the response that indicates how frequently you have the same experience (0 = never; 4 = always):	0−4 points
	Relationships	I speak with friends, neighbors, or relatives less often because of my voice.
VoiSS		On a 5-point scale, rate how frequently the following occurs (1 = never; 5 = all of the time):	1−5 points
	Relationships	Does your voice problem put a strain on your family and friends?
	General	Do you have trouble attracting attention?
	General	When talking in company, do people fail to hear you?
	General	Do you have problems talking on the telephone?
V-RQOL		How much of a problem is this? (1 = none, not a problem; 5 = problem is as “bad as it can be”):	1−5 points
	Work/education	I have trouble doing my job or practicing my profession (because of my voice).
	Relationship	I avoid going out socially (because of my voice).
	General	I have trouble using the telephone (because of my voice).

Note. ASHA QCL = American Speech-Language-Hearing Association Quality of Communication Life (Paul et al., 2004); BOSS = Burden of Stroke Scale (Doyle et al., 2003); VAPP = Voice Activity and Participation Profile (Ma & Yiu, 2001); VHI = Voice Handicap Index (Jacobson et al., 1997); VoiSS = Voice Symptom Scale (Deary et al., 2003); V-RQOL = Voice-Related Quality of Life (Hogikyan & Sethuraman, 1999); General = general communication; Household = household management.

Items Considered Consistent With Communicative Participation

Individual items considered consistent with communicative participation are listed in Table 3. Items that included general communication in life situations (e.g., talking on the telephone, or having a conversation with a group of people) that could cross domains were classified as “general communication.” This information is summarized in Table 4 as the numbers and percentages of items from each instrument that were considered consistent with communicative participation as well as their classified domains.

TABLE 4.

Total number of items consistent with communicative participation.

				Domains of communicative participation
Instrument	Total items	Total CP	% CP	Personal care	Household management	Work/education	Leisure recreation	Relationships	Community	General communication
ASHA QCL	19	8	42	0	1	1	1	1	0	4
BOSS	15	7	47	0	0	0	0	0	0	7
VAPP	28	5	18	0	0	0	0	0	0	5
VHI	30	7	23	0	0	0	0	1	0	6
VoiSS	30	4	13	0	0	0	0	1	0	3
V-RQOL	10	3	30	0	0	1	0	1	0	1
Total	132	34	26	0	1(3)	2(6)	1(3)	4(12)	0	26(76)

Note. Includes instrument name, total number of items on the scale, total number of items consistent with criteria for communicative participation (CP), percentage of items consistent with CP, and items consistent with personal care, household management, work/education, leisure/recreation, relationships, community, or general communication domains.

In total, 26% (34 items of 132 possible) of all items across the six instruments were consistent with communicative participation. Of the 34 items that were considered measures of communicative participation, 76% (26 of 34) fell within the general communication domain (i.e., the item could cross multiple domains but did not specify any other participation domain).

Psychometric Properties of Reviewed Instruments

A summary of the reviewed instruments is provided in Table 5. For each of the instruments, this summary includes the conceptual definition of what the instrument purports to measure, the target populations indicated by instrument developers, the actual populations included during instrument development, and number of times the instrument was cited in the peer-reviewed research literature. Although the selected instruments did not exclusively measure communicative participation, a summary of additional psychometric properties was included (see Table 6) because this information is valuable for understanding the content and structure of instruments. Such information may be useful when selecting the most appropriate measurement tool for clinical or research purposes.

TABLE 5.

Instruments evaluated for communicative participation, including construct measured, intended target population, the population the instrument was developed on, sample size, and frequency of instrument use.

Instrument	What is measured by the instrument?	Target population	Development population	Frequency of instrument use
ASHA QCL	Communication-related quality of life: the extent to which a person's communication acts as constrained within the boundaries drawn by personal and environmental factors, and as filtered through this person's perspective, allow meaningful participation in life situations.	Adults with communication disorders	Individuals with fluent aphasia (n = 21), nonfluent aphasia (n = 40), TBI/cognitive communication disorders related to right-sided cerebrovascular accidents (n = 14), and individuals with a variety of dysarthrias (associated with PD, ALS, and MS; n = 11)	0
BOSS	Burden of stroke and underlying dimensions representing the physical (cognitive) burden of the condition.	Adults with communication disorders (stroke)	Community (n = 135) and non-community dwelling (n = 134) stroke survivors, and healthy controls (n = 251)	4
VAPP	Perception of voice problem, activity limitation, and participation restriction related to voice.	Adults with voice disorders	Adults with a variety of voice disorders (e.g., voice disorders associated with vocal nodules, polyps, thickened vocal folds, vocal fold paralysis, laryngitis; n = 40) and healthy controls (n = 40)	2
VHI	The psychosocial handicapping effects of voice disorders, including voice disability (restriction or lack of ability manifested in the performance of daily tasks) or voice handicap (social, economic, or environmental disadvantage resulting from an impairment).	Adults with voice disorders	Adults with voice disorders (e.g., voice disorders associated with mass lesions, neurological origins, muscle tension dysphonia, inflammation, and atypical origins; N = 63)	37
VoiSS	Voice symptoms, including communication problems, throat infection, psychosocial distress, voice sound and variability, and phlegm.	Adults with voice disorders	Adults with voice disorders (e.g., functional voice disorders, and those associated with mass lesions, neurological origins; N =180)	5
V-RQOL	How a voice problem interferes with day-to-day activities.	Adults with voice disorders	Adults with voice disorders (n = 109) and healthy controls (n =22)	6

Note. ALS = amyotrophic lateral sclerosis; MS = multiple sclerosis; MTD = muscle tension dysphonia; PD = Parkinson's disease; TBI = traumatic brain injury.

TABLE 6.

Psychometric properties for evaluated instruments.

	Reliability		Validity
Instrument	Test-retest reliability	Internal consistency	Concurrent	Construct
ASHA QCL	Mean Pearson correlations; 25% subjects retested 2−7 days later, r = .63	Mean Pearson correlations within domains: (1) soc/act = .49, (2) conf/self-conf = .54, (3) roles/resp = .59	NA	Factor analysis did not support 3 domains.
BOSS	NA	Cr. alpha: total = .97; domains: (1) phys = .92, (2) psych = .97, (3) cogn = .93; subdomains: (1) communication difficulty scale = .89, (2) communication-associated psychological distress scale = .86, (3) negative mood scale = .80, (4) positive mood scale = .73	Spearman rho BOSS communication difficulty & BDAE = −.55; BOSS & RTT = −.45	Factor analysis of composite scales accounted for 76% of total BOSS score
VAPP	Mean Pearson correlations; 25% subjects retested 2 weeks later, r = .86	Cr. alpha: total = .98; mean Pearson correlations within domains: (1) voice/job = .74, (2) voice/daily comm = .82, (3) voice/social comm = .69, (4) voice/emtn = .74, (5) job/comm = .79, (6) job/social comm = .80, (7) job/emtn = .83, (8) social comm/emtn = .82	Spearman rho VAPP and VHI = .89	NA
VHI	Mean Pearson correlations; 100% subjects retested 6−71 days later; (1) func = .84, (2) emtl = .92, (3) phys = .86, total = .92	Cr. alpha: total = .95; mean Pearson correlations within domains: (1) func/phys = .70, (2) func/emtl = .79, (3) phys/emtl = .72, (4) func/total = .91, (5) phys/total = .88, (6) emtl/total = .93	Mean Pearson correlation; total VHI with self-assessment of voice severity = .60	Factor analysis did not support 3 domains; suggested 2 factor solution
VoiSS	NA	Cr. alpha: total = .92; domains: (1) impt = .89, (2) emtl = .73, (3) phys = .90	NA	Principal component analysis of 3 domains (impt, emtl, phys) accounted for 48.9% of total item variance
V-RQOL	Mean Pearson correlations; 63% subjects retested 1−13 days later; (1) phys = .93, (2) soc/emtl = .86, total = .93	Cr. alpha: total = .89; (1) phys = .80, (2) soc/emtl = .81	Significant difference in V-RQOL scores across 3 severities of self-assessed voice quality V-RQOL and mean GRBAS score by clinicians Spearman rho = −.44	NA

Note. BDAE = Boston Diagnostic Aphasia Examination; cogn = cognitive; comm = communication; conf = confidence; Cr. alpha = Cronbach's alpha; emtl = emotional; emtn = emotion; func = functional; GRBAS = Grade, Roughness, Breathiness, Asthenia, Strain (Hirano, 1981); impt = impairment; NA = not available; phys = physical; psych = psychological; roles/resp = roles/responsibility; RTT = Revised Token Test; soc/act = socialization/activities.

The amount of information about psychometric properties of the individual instruments varied considerably (see Table 6). Information in the table was derived from all sources, including both the instrument developers and other literature sources. The instruments were validated on samples that ranged in size from 63 to 800 and included participants with many different types of communication disorders.

Reliability

Test–retest correlations between two administrations of the instruments were reported for four out of the six reviewed instruments and ranged from .63 to .93 (see Table 6). Five instruments provided Cronbach's coefficient alpha (Cronbach, 1951) as a measure of how well each individual item correlates with the sum of the remaining items in the instrument. For all instruments except the ASHA QCL, the reported Cronbach's coefficients for the summary scores were high and ranged from .89 to .98. The manual for the ASHA QCL did not list the coefficient alpha or item-to-total correlations.

Validity

Concurrent validity was reported for four of the six instruments (excluding the ASHA QCL and VoiSS); however, the studies were not always conducted by the instrument developer. For a few instruments, results of analyses of discriminant, convergent, and divergent validity also were reported. Responsiveness or sensitivity to change was reported for one instrument (VHI), and gender or cultural differences were investigated for only one instrument (ASHA QCL).

For four of the instruments, factor analyses also were conducted. For these instruments (ASHA QCL, BOSS, VHI, VoiSS), a hypothesized domain structure was examined by factor analyses. In most cases, the domain structure was evaluated using principal component analysis. In some cases the hypothesized structure was supported; in others, the analyses suggested different numbers of domains or found that some items did not load on the expected factor.

Frequency of instrument use

By far the most frequently used instrument in the published literature was the VHI (used 37 times in the reviewed literature), undoubtedly because it was the first scale of this type (in voice disorders) to be published (1997). At the time of the publication of the VHI, little was known about its psychometric properties, but a later study (Wilson et al., 2004) provided more information. The V-RQOL was reported in the literature six times, followed by the VoiSS (five times) and BOSS (four times).

Discussion

The overall purpose of this article was to provide a review of existing self-report instruments in speech-language pathology for their breadth, depth, and adequacy of evaluating the construct of communicative participation (WHO, 2001; Yorkston et al., 2001). Results indicated that none of six existing instruments were wholly adequate for measuring communicative participation and for meeting the requirements of the current scope of practice in speech-language pathology (ASHA, 2001). These results are consistent with those found in the general rehabilitation literature and provide support toward the development of new instruments that measure communication in social contexts (Cardol et al., 1999; Perenboom & Chorus, 2003). In the next section, the development of new instruments will be discussed relative to recent trends in the literature, followed by implications for future research and clinical applications.

Trends Supporting the Development of Socially Valid Outcome Measures

There are three trends that lend support toward the selection or development of appropriate outcomes for measuring communicative participation: (a) the stabilization of terminology and the conceptual framework related to health, functioning, and disability; (b) the growing appreciation of client-centered decision making; and (c) advances in psychometric methods and instrument development.

Stabilization of terminology and conceptual framework

Instruments that assess participation restrictions (WHO, 2001) are limited in number in the general rehabilitation literature (Cardol et al., 1999; Dijkers et al., 2000; Perenboom & Chorus, 2003). Of those instruments that measure some aspect of communicative participation, definitions of communicative functioning are variable, as are the theoretical frameworks underlying these definitions (see Tables 5 and 6). One possible reason for the limited inclusion of the concept of communicative participation may relate to ever-changing terminology and multiple theoretical frameworks of disablement that historically have been part of the rehabilitation literature (Jette, 2005).

Although the WHO ICF is finding more widespread acceptance, and terminology is becoming more consistent, a few challenges remain. One difficulty related to taxonomy is founded in the inconsistent (and sometimes interchangeable) use of terms and conceptual frameworks of several levels of functioning, including so-called “functional skills,” activities, participation and handicap, and QOL. However, there is increasing support for participation as a distinct construct (Jette et al., 2003). Studies such as the present one highlight the need for instruments that independently measure this construct.

With an increased understanding and acceptance of the model of health underlying the ICF, instrument developers now can review individual items of instruments and evaluate these instruments for construct validity. These considerations apply to constructs such as communicative participation. Results from the present study suggest that the reviewed instruments focused on general communication activities that could cross multiple domains. To ensure that all domains of communicative participation are measured (i.e., within each participation domain that includes communication), one would also want items that reflect a broader sample of social goals, functions, or contexts. While items that fit within the general communication category are important, their breadth is difficult to assess because an individual's abilities could vary for a particular item depending on the domain. For example, one person might be extremely competent when conversing with a family member in his or her home but might not do as well when having the same conversation with a grocery clerk in a noisy store or with a supervisor at work. Thus, specifying a social function or context to the communicative exchange ensures that the breadth of the construct is adequately represented. These suggestions are consistent with evaluating individual participation items in the context of different environments, as outlined in the ICF (WHO, 2001). These considerations are important as new instruments are developed, and as these measures gain widespread use in clinical practice.

Client-centered decision making

The importance of participation level outcomes to individuals with disabilities is supported by the observation that participation is probably the strongest correlate of subjective QOL, above and beyond activities or impairments (Dijkers, 1997). Holland and Thompson (1998) noted the importance of measuring social outcomes in the study of aphasia by suggesting the “overriding issues are not changes in aphasia test scores” but rather participation in life situations such as “getting another job” or “being comfortable going out to dinner again” (p. 262). These suggestions highlight the importance of not only documenting changes in scores from instruments that measure “impairments” but also obtaining evaluations from the individual with the communication disorder to ensure client-centered decision making.

In this review, we only included instruments that were self-reports of communicative function. Instruments such as the ASHA FACS (Frattali et al., 1995) and the CETI (Lomas et al., 1989) were not included because they are completed by clinicians and/or family members instead of the person with the communication disorder. We analyzed the ASHA FACS and the CETI to determine whether the inclusion of these instruments would have altered our results. Overall, a large proportion of items on these two instruments were consistent with communicative participation. Most of the items fell within the general communication domain. For example, 14 of 16 items (87%) on the CETI were found to be consistent with communicative participation, although 86% (12 of the 14) were general communication items and did not specify whether communication was used for other social goals, functions, or contexts (e.g., communication at work, home, or in the community). Likewise, 33 of 43 items (77%) on the ASHA FACS were found to be consistent with communicative participation, and 91% (30 of the 33) were general communication items. Although the ASHA FACS and CETI appear by this tally of items to be more representative of communicative participation than the reviewed instruments in terms of item content, information is garnered from proxies. Therefore, the information obtained from these instruments may be different than that obtained from the person with the communication disorder (i.e., may or may not represent a person's actual level of performing a task). In addition, responses are limited to objective measures of communication. Like the other instruments reviewed in this study, the ASHA FACS and CETI also focus on items that are general communication items and do not specify communication in other participation domains.

In the field of speech-language pathology, research and clinical practice have progressed from focusing on performance components to emphasizing the person's experience of participation in daily life (i.e., client-centered practice). The exclusion of subjective responses is a weakness of the classificatory system of the ICF and adapted clinical manuals (Hemmingsson & Jonsson, 2005). The six instruments in this review were selected because self-report formats allow the collection of both objective and subjective responses (see Table 3). Specifically, the ASHA QCL and BOSS measure the difficulty or level of performance, and the other four instruments measure some aspect of interference with performance. These responses do not include other evaluations of those same activities that may be important to client-centered evaluation. For example, in addition to asking individuals to judge the frequency or intensity of performing particular communication activities, one also might ask individuals to evaluate satisfaction with performance, or the importance of a particular communication activity (Johnston & Miklos, 2002). These issues are considered in instruments developed in other rehabilitation disciplines (e.g., the Canadian Occupational Performance Measure; Law, Baptiste, Carswell, Polatajko, & Pollock, 1998), and they need to be considered when developing or evaluating instruments that measure participation.

Advances in psychometric methods and instrument development

One goal of this review was to evaluate the psychometric properties of instruments in speech-language pathology that measured the construct of communicative participation, or aspects of this construct. Although we did not find any instruments that were wholly consistent with this construct, we proceeded to summarize the psychometric properties as a resource for SLPs who select instruments for assessing and/or for measuring treatment outcomes. Results revealed considerably different levels of psychometric properties of the instruments. For example, Cronbach's coefficient alphas reported were very high (e.g., .89 to .98), suggesting a high level of item redundancy; that is, several items may ask the same question in slightly different ways (Streiner & Norman, 2003). In contrast, construct validity was evaluated for four instruments (ASHA QCL, BOSS, VHI, VOISS). In most cases, the domain structure was evaluated using principal component analysis, and results did not always support the hypothesized structure.

All reviewed instruments were developed based on classical test theory. The need for psychometrically sound measures calls for better analytical tools beyond the methods available from traditional measurement theory. Applications of item response theory (IRT) modeling have increased considerably in the last 10 years in medical rehabilitation because of its utility for instrument development and evaluation, development of short and alternate forms of instruments, instrument linking, and computerized adaptive testing (Streiner & Norman, 2003). IRT models the relationship, in probabilistic terms, between a person's response to an item and his or her standing on a construct such as communicative participation. The main strengths of IRT lie in the ability to address psychometric and substantive issues that would otherwise be difficult to solve using traditional procedures (e.g., the study of cultural differences in item functioning; Embretson & Reise, 2000). In addition, instruments developed or scored using IRT methodology have been found to be more responsive to treatment than those developed using classical test theory (Kosinski, Bjorner, Ware, Batenhorst, & Cady, 2003). These issues should be the focus of future research efforts if we are to develop measures of communicative participation to demonstrate the effectiveness of our interventions (Donovan, Rosenbek, Ketterson, & Velozo, in press; Doyle et al., 2005).

Conclusions and Future Directions

Simmons-Mackie et al. (2005) examined the practice patterns of SLPs and concluded that very few clinicians reported using outcome measures above and beyond impairments. They suggested that the limited use of such measures could be due to an inadequate number of instruments that capture these outcomes; however, they also conceded that their data did not answer this question. The results of this review appear to support several conclusions. First, instruments that measure communicative participation are extremely limited in the field of speech-language pathology. Second, instruments that do measure aspects of communicative participation also include items that assess other constructs. Therefore, no instrument appears to exclusively assess communicative participation. Finally, items consistent with communicative participation typically include global communication items that may cross multiple life domains. Although these general items may be essential in measuring communicative participation, they alone may be restricted in scope and may not reflect the breadth of the construct of communicative participation in the multiple domains in which it occurs. Because we do not appear to have an instrument that focuses wholly on communicative participation, we are not fully meeting the challenge of adopting the ICF framework for measuring treatment outcomes in speech-language pathology (ASHA, 2001).

Dijkers et al. (2000) found that the psychometric properties of current social outcome instruments were adequate to justify their use. Techniques for measuring social outcomes can be applied to measuring constructs such as communicative participation. For example, the development of instruments that measure communicative participation could begin with items derived from the reviewed instruments, as well as items constructed with input from individuals with and without communication disorders. These considerations would ensure that items reflect the broad range of communicative functioning that is universal to all individuals. General communication items would form the core of such an instrument; however, items from these instruments would also need to be revised to include multiple social goals, functions, or life situations to ensure adequacy in measuring the breadth of this construct.

In addition, instruments need to meet criteria for tools of disability outcomes such as (a) sound psychometric properties; (b) consideration of objective and subjective response metrics (e.g., is it ability to perform a task or a person's satisfaction that is the best metric of communicative participation?); (c) reduction of burden; (d) focus on clinically meaningful differences (e.g., as opposed to statistically significant differences pre- and postintervention); (e) development of alternate forms and versions accessible to individuals with disabilities, in particular those who use assistive technology; and (f) consideration of linguistic or cultural adaptations (Andresen, 2000; Polgar, 1998). We challenge ourselves and others to develop outcome measures that support objectives and treatment goals that are meaningful to clinicians, administrators, researchers, and, most importantly, individuals with communication disorders.