Item Bank Development for a Revised Pediatric Evaluation of Disability Inventory (PEDI)

Abstract

The Pediatric Evaluation of Disability Inventory (PEDI) is a useful clinical and research assessment but it has limitations in content, age range and efficiency. The purpose of this article is to describe the process used to develop the item bank for a new computerized adaptive test version of the PEDI (PEDI-CAT). An expanded item set and response scales were reviewed by clinician experts and examined at parent and clinician focus groups. Eleven parents participated in 32 cognitive interviews to examine content, format, and comprehension of items and responses. A revised set of self-care (n=76), mobility (n=78; walking aids n=13; wheelchair n=14) and social function (n=64) items with pictures and a 4-point ‘Difficulty’ scale were developed. Also, the PEDI’s Caregiver Assistance Scale was replaced by a ‘Responsibility Scale’ with 53 items. The new PEDI-CAT item bank covers a broader range of functional activities for children of all ages and abilities.

The Pediatric Evaluation of Disability Inventory (PEDI) is a comprehensive functional assessment designed for use by physical and occupational therapists, as well as other rehabilitation and educational professionals. It is administered by structured interview with the child’s caregiver(s) and/or via observation of the child. The original version of the PEDI measures self-care, mobility and social function capability in daily activities with 197-items in three Functional Skills scales. The 20 items in the PEDI’s Caregiver Assistance scales measure the amount of caregiver assistance provided when the child is performing multi-step self-care, mobility or social function activities. Standardized on a sample of 412 children between the ages of six months and 7.5 years who were typically developing, the PEDI can also be used to assess capability and performance of older children if their functional abilities fall below that expected of a 7.5 year-old child with no disability. Thus, the PEDI can be used for the determination of eligibility for disability-related services (with norm-referenced standard scores) and for evaluating change following intervention (with criterion referenced scores) (Haley, Coster, Ludlow, Haltiwanger, & Andrellos, 1992).

Since its publication in 1992, the PEDI has been used with children with a wide variety of disabling conditions (Danielsson et al., 2008; Dolva, Lilja, & Hemmingsson, 2007; Dumas, Haley, Ludlow, & Rabin, 2002; Dumas, Haley, & Steva, 2002; Eisenberg, Zuk, Carmeli, & Katz-Leurer, 2009; C. Ho & Karol, 2008; E. Ho, Curtis, & Clarke, 2006; Rodger et al., 2003; van der Net et al., 2008; Van Empelen et al., 2005; Verhoog et al., 2008), in varied clinical settings (Ahl, Johansson, Granat, & Brogren Carlberg, 2005; Daichman, Johnston, Evans, & Tecklin, 2003; Dumas, Haley, Ludlow et al., 2002; Novak, Cusick, & Lowe, 2007; Stiller, Marcoux, & Olson, 2003), and to assess the effects of surgical (C. Ho & Karol, 2008; Nordmark et al., 2008; Van Empelen et al., 2005), pharmacological (Daichman et al., 2003; Löwing, Aström, Oscarsson, Söderhäll, & Eliasson, 2007; Moore et al., 2008; Scheinberg, O’Flaherty, Chaseling, & Dexter, 2001), and rehabilitation interventions (Casady & Nichols-Larsen, 2004; Daichman et al., 2003; Dumas, Haley, & Steva, 2002; Fragala-Pinkham, Dumas, Barlow, & Pasternak, 2009; Jones, McEwen, & Hansen, 2003; Kelly, MacKay-Lyons, Berryman, Hyndman, & Wood, 2008; Ketelaar, Vermeer, Hart, van Petegem-van Beek, & Helders, 2001; Knox & Evans, 2002). The PEDI has also been studied extensively to examine its psychometric properties including its reliability (Berg, Jahnsen, Froslie, & Hussain, 2004; Erkin, Elhan, Aybay, Sirzai, & Ozel, 2007; Haley et al., 1992; Nichols & Case-Smith, 1996), validity (Bourke-Taylor, 2003; Erkin et al., 2007; Feldman, Haley, & Coryell, 1990; Gannotti & Cruz, 2001; Haley, Coster, & Faas, 1991; SM Haley et al., 1992; Nichols & Case-Smith, 1996; Wright & Boschen, 1993; Ziviani et al., 2001), and responsiveness (Dumas, Haley, & Ludlow, 2008; Dumas, Haley, & Steva, 2002; Haley et al., 1992). The “Minimally Important Difference” (MID), the smallest difference in score perceived as beneficial, has been established based on clinician report for each of the six PEDI scales and is on average, 11 points (Iyer, Haley, Watkins, & Dumas, 2003).

While the PEDI continues to be a preferred clinical and research assessment used worldwide, it has limitations. With more than 200 items, it is time-consuming to administer all three domains for the Functional Skills and Caregiver Assistance Scales and thus, its use is limited in many practice settings (Custers et al., 2002). The other major limitations of the PEDI are that the normative scores cover a small age range (six months – 7.5 years) and the content may not be broad enough for children with minimal restrictions in activity or participation (Kothari, Haley, Gill-Body, & Dumas, 2003; McCarthy et al., 2002; Ostensjo, Bjorbaekmo, Carlberg, & Vollestad, 2006). The ideal functional measure would cover a broad range of functional activities and content area for children of all ages and abilities while at the same time minimizing respondent burden.

To address these limitations, work has begun to revise the PEDI by developing a computerized adaptive test (CAT) version, called the PEDI-CAT (SM Haley et al., submitted). CATs can be completed by parents or clinicians based on observation of the child. CAT provides an individualized and meaningful assessment for each child because it uses a computer algorithm to choose which items are appropriate and the minimal number of items needed to insure an accurate score. CAT applications therefore require a large set of items (item bank) that consistently scale along a dimension of low to high functional proficiency in order that every test administration can be adapted to the unique ability level of the child and essentially mimic what an experienced clinician would do during an assessment. For example, if a parent answers that the child walks up one flight of stairs without difficulty, then questions about crawling up stairs would not be asked as the computer-based algorithm would account for walking being a higher functional skill than crawling. This approach minimizes the number of items administered but still allows the clinician to obtain an estimate of functioning in a particular content area.

The process for revising the PEDI has involved the development of new items to create an item bank in each of the three functional domains of self-care, mobility and social function and for a newly developed Responsibility Scale. Two important methods for item bank development are focus groups and cognitive interviewing (Cella, Gershon, Lai, & Choi, 2007; Walsh, Irwin, Meier, Varni, & DeWalt, 2008). Focus groups help to determine both general themes and specific ideas of participants about particular issues, products and/or services and can thus be used to identify important functional activities for new items during test development (Krueger & Casey, 2008; Walsh et al., 2008). Cognitive interviewing is a specialized form of structured individual interview used to provide insights into the respondents’ thought processes as they read or hear and respond to test questions. The primary goal of cognitive interviewing is for test developers to know whether or not the respondents understand questions consistently, easily and as intended (Willis, 2005). The purpose of this article is to describe the process, including focus groups and cognitive interviewing, used to develop the item bank for a new computerized adaptive test version of the PEDI (PEDI-CAT).

METHODS

Procedures

Institutional Review Board approval was obtained from the Boston University Medical Campus, Boston, MA, New England IRB, Boston, MA and Franciscan Hospital for Children, Boston, MA. The process of developing an item bank for a new computerized adaptive test version of the PEDI included two phases. Phase I, Development of New Items and Response Scales, included generating an expanded set of items and corresponding response scales using a review of existing measures, expert review and parent and clinician focus groups. Phase II, Item Revision and Finalization, included a series of cognitive interviews. (Figure 1) The project team included three of the PEDI’s original authors as well as additional physical and occupational therapists, a nurse, and an educator.

Figure 1.

Process for development of PEDI-CAT item bank

Phase I – Development of New Items and Response Scales

A comprehensive review of more than 60 existing pediatric and adult measures of function in the content domains of self-care, mobility, and social function was conducted. The project team examined measures for content not contained in the original PEDI, age equivalents, item wording, response scale type (e.g. frequency, level of difficulty, amount of assistance) and number of response scale points. We recognized that if we wanted to expand the age range, then new content areas had to be addressed that were relevant for older children such as cooking, managing health needs, and time management.

The project team developed an expanded set of items in each of the PEDI’s existing three functional domains (self-care, mobility and social function) and revised the response scale from a two-point (unable/capable) scale to a 4-point difficulty scale. In addition, the project team replaced the Caregiver Assistance Scale with a ‘Responsibility Scale’ to assess the extent to which a young person is managing life tasks that enable independent living. This new scale indicates the amount of responsibility a parent and child each take for carrying out particular multi-step tasks versus the existing scale that measures how much assistance is needed.

The expanded set of items and response scales were sent via electronic mail to 12 physical and occupational therapy clinicians with expertise in child development, measurement of children’s daily activities in home and community contexts, and instrument design, validation, and score construction. In addition, these experts were experienced national and international users of the PEDI. We collected feedback regarding content coverage, content relevance, and item clarity.

Focus groups were also held with stakeholders to review the initial expanded item bank. Parents of children with disabilities (n=6) ages infant through 21 years of age and who were English-speaking were recruited through personal contacts to participate in one of two focus groups by telephone to provide feedback on the expanded set of PEDI items and response scales provided via e-mail. During the group conference calls, parent participants were asked if there were other important functional skills in each of the PEDI’s three content domains that should be addressed. Parents were also asked if items were written clearly for parents to understand and respond to. For the response scales, parents were asked: a) Are the definitions of each rating scale point clear?; b) Do the rating scale points reflect meaningful distinctions in management of daily life tasks?; c) Will parents be able to make these distinctions in their child’s management of daily skills?; and d) Do you have a preference for a “Difficulty” (respond with how difficult) versus an “Easy” (respond with how easy) scale? Calls were facilitated by one member of the project team who took notes to record the parents’ input. In addition, telephone calls were audiotaped. Each call lasted approximately 60 minutes. Participants provided written informed consent prior to the calls and were provided with an honorarium by mail following the call. Table 1 provides a description of the participants.

Table 1.

Demographics of participants

Phase I-Parent Focus Groups (n=6)
Relationship to child	100% Mothers
Race/Ethnicity	83% White/Non-Hispanic
Children’s ages	3 – 12 years
Children’s Diagnoses	Joubert’s Syndrome/Blindness Cerebral Palsy with hemiplegia Mitochondrial Disorder Asperger’s Syndrome (n=2) Myelomeningocele
Phase I-Clinician Focus Groups (n=11)
Clinical Discipline	36% Physical therapists 27% Occupational therapists 27% Speech-language pathologists
Average Years of Experience	16.3
% Female	88%
Phase II-Cognitive Interviews (n=11)
Relationship to child	100% Mothers
Education	27% Some College 46% College Graduate 27% Graduate School
Race/Ethnicity	91% White; 100% Non-Hispanic
Children’s ages	4 – 17 years
Children’s Diagnoses	Learning Disabilities and Dyslexia ADHD Asperger Syndrome Mitochondrial Disorder Spina Bifida Chromosome Deletion/Developmental Delay/Vision and Hearing Impairment

Two focus groups of four to five clinicians were held at a local pediatric hospital. Clinicians with four or more years of experience evaluating children with disabilities and who were English-speaking were recruited through personal contacts. Each group lasted up to 90 minutes and was led by two facilitators from the project team experienced in conducting focus groups. Clinicians were provided with the expanded list of items and asked to respond to the following questions: a) Are there other important functional skills in each domain that should be addressed?; b) Are the items written clearly for parents and/or clinicians to understand and respond to?; and c) Given the expanded content of the item bank, what do you think would be the clearest and most appropriate title for each scale? For the response scales, clinicians were asked: a) Are the definitions of each rating scale point clear?; b) Do the rating scale points reflect meaningful distinctions in management of daily life tasks?; c) Will parents and/or clinicians be able to make these distinctions in a child’s management of daily skills?; and d) Do you have a preference for a “Difficulty” (respond with how difficult) versus an “Easy” (respond with how easy) scale? The discussion was audiotaped and the facilitators took notes throughout. The clinician participants provided written informed consent before participation and were provided with an honorarium following the group. Table 1 provides a description of the participants.

Phase II – Item Revision and Finalization

Following additions and revisions to the expanded item bank based on the Phase I feedback, a convenience sample of 11 parents (6 parents of children with disabilities and 5 parents of children without disabilities) were recruited through personal contacts and the clinical services departments at a pediatric hospital to participate in cognitive interviews. To be eligible for inclusion, participants had to be the parent of a child (infant to 21 years of age), over 18 years of age themselves and able to speak and read English. Once the participant was identified as having met the eligibility criteria and provided informed consent, the cognitive interviewing session(s) were scheduled. Each parent was eligible to participate in up to three interviews and a total of 32 interviews to examine content, format, and comprehension of items and responses were completed. Table 1 provides a description of the participants.

Project staff had been trained in previous projects to conduct cognitive interviewing through collaboration with the Center for Survey Research at the University of Massachusetts, Boston. Cognitive interviewing sessions were conducted individually in a quiet, distraction-free environment with adequate space for the interviewer to record notes. Participants were instructed in the procedures and reminded not to focus on answering the question as it relates to their own child/children but rather to discuss what they understood the item to mean, their opinion on the wording of the item and whether the response choices were adequate and appropriate for the question. Respondents were reminded of these purposes throughout the interview.

Notes were taken throughout the interview by the interviewer and included the participants’ thoughts, questions and/or concerns about the items. Each interview consisted of examining approximately 30 items and lasted up to 60 minutes. All interviews were conducted in English and an honorarium was provided to all participants at the conclusion of each interview. Feedback for each item was compiled on a master form. After 15 interviews, all feedback and items were reviewed by the project team and all items (including line drawings for the self-care and mobility items) were re-tested in a second set of interviews with new parent participants. Final revisions to the items were made following this second round of 17 cognitive interviews.

Data analysis

For Phase I, data analysis was conducted at a series of project team meetings by analyzing the feedback from the expert clinicians and focus groups and making revisions related to domain content, redundancy, item tense or wording and response scale options. For Phase II, after each round of cognitive interviews, the project team compiled a summary of feedback for each item and domain to determine what revisions were needed due to issues with clarity, redundancy, or comprehension.

RESULTS

Phase I- Development of New Items and Response Scales

In Phase I, the expert reviews and parent and clinician focus groups yielded similar results. Participants indicated the items encompassed a wide range of daily activities and the clinicians also indicated that the items encompassed a much wider range of functional activities than the original PEDI. The feedback yielded valuable information for clarifying content with suggested formatting and wording changes within items. Changes to quantify capability or clarify expectations were added (e.g. “thoroughly” was added to “Wipes nose with a tissue”). All participants indicated that separate items for wheelchair and walking aid use were needed in the mobility domain. All participants responded very favorably to the concept and content of the Responsibility scale.

In general, parents, clinicians and experts preferred the use of a “Difficulty Scale” to an “Easy Scale” for responding to items, as it was reported to be a more typical way that parents and clinicians report on children’s function. All were in favor of expanding the scale to four responses from the two response options (unable/capable) of the original PEDI Functional Skills scales. All participants reported that the distinctions between the “Difficulty” responses were clear and all participants reported favorably on the response options for the Responsibility Scale. Based on the feedback from Phase I, a set of 76 self-care, 104 mobility (including walking aid and wheelchair items), 69 social function and 50 responsibility items were prepared for cognitive interviewing.

Phase 2– Item Revision and Finalization

The results of the cognitive interviews indicated that changes were needed for item clarity and comprehension including changes in item wording, the addition of examples, clarifying ambiguous items, and adding qualifiers (e.g. time, distance). In addition, item redundancy and domain fit were addressed. Parents participating in the first round of interviews strongly suggested the addition of pictures to assist with comprehension of item meaning. Table 2 provides examples of common issues identified and the suggested changes. In addition, parents’ feedback indicated that the ‘Difficulty’ response scale was too wordy as parents had to re-read the response choices for each item. Lastly, we received feedback on the presentation of the items indicating that the response scale should accompany each item on each page and that key words in the response scales should be underlined or bolded. Based on the results of the cognitive interviewing, a final set of 76 self-care, 105 mobility (including walking aid and wheelchair items), 64 social function and 53 responsibility items were developed. Table 3 provides examples of final items and Table 4 provides the final response scales.

Table 2.

Examples of concerns and suggested changes during cognitive testing

	Problematic Item(s)	Suggested Change(s)
Item Clarity	Prints or writes own name legibly	Specify first and/or last name
	Opens cardboard food boxes	Specify if box is sealed or has already been opened
Item Wording	Stirs to mix up foods when cooking	Change “foods” to “ingredients”
	Recognizes numbers on everyday objects such as a clock or phone	Remove “everyday objects such as ”
Need for (Additional) Examples	Can handle stimulating situations like a shopping mall for 1-2 hours without losing control	Add another example such as birthday party to broaden applicability
	Fixing snacks and simple meals that do not involve cooking (Includes: identifying what is available to eat; selecting the needed food and utensils; preparing by mixing, pouring, etc)	Add examples such as cereal and/or a sandwich
Ambiguous Items	Rides tricycle or bicycle with training wheels	Separate into two items
	Walks and carries a full glass or plate without spilling	Separate into two items
Need for Qualifiers	Stands in a moving vehicle (bus, train, trolley, boat/ferry)	Stands while holding on in a moving vehicle (bus, train, trolley, boat/ferry)
Item Redundancy	Making healthy choices to maintain health and well-being (Includes: nutrition; exercise; avoiding substance use or exposure to environmental hazards) and Eating and drinking appropriate foods to maintain health and energy (Includes: regulating amount of food eaten; avoiding undernourishment and dehydration	Remove “Nutrition” from first item and focus on other lifestyle choices
Domain Fit	Lets home, school or job know when he or she will be late or absent	Move from Social Function to Responsibility Scale
Item Comprehension- Need for Pictures	Fastens belt buckle	Add picture to indicate pants buckle and not car seat belt
	Sits on infant playground swing while swing is pushed	Add picture to show what an infant swing looks like

Table 3.

Examples of final items in all domains

Domain
Self-Care	Removes a single bill from wallet
Self-Care	Inserts laces into sneakers or boots
Self-Care	Tightens loose screws using a screwdriver
Mobility	Sits on infant playground swing while swing is pushed
Mobility	Gets on and off an adult-sized toilet
Mobility	Moves across monkey bars
Mobility-Walking aid	Opens and closes door to enter and exit home, while walking with walking aid (e.g. cane, crutches, walker)
Mobility-Wheelchair	Goes up and down curbs with wheelchair
Social Function	Participates in role-playing activities such as playing school or acting out famous characters
Social Function	Greets new people appropriately when introduced
Social Function	Writes short notes or sends text messages or email
Responsibility	Putting items and objects away after use Includes: Knowing where objects are stored; Organizing belongings and objects so they can be found when needed
Responsibility	Planning and following a weekly schedule so all activities get done when needed Includes: Identifying what needs to be done during a week; Determining how much time each activity will need and when it should be done; Carrying out plan; Making necessary adjustments due to unexpected delays or events
Responsibility.	Buying clothing at a store, from a catalog or online Includes: Purchasing clothing, including outerwear and undergarments

Table 4.

Final Response Scales

Difficulty Scale

To be used for Self-Care, Mobility and Social Function Scales

□ Unable = Can’t do, doesn’t know how or is too young.

□ Hard = Does with a lot of help, extra time, or effort.

□ A little hard = Does with a little help, extra time or effort.

□ Easy = Does with no help, extra time or effort, or child’s skills are past this level.

□ I don’t know.

Responsibility Scale

□ Adult/caregiver has full responsibility; the child does not take any responsibility.

□ Adult/caregiver has most responsibility and child takes a little responsibility.

□ Adult/caregiver and child share responsibility about equally.

□ Child has most responsibility with a little direction, supervision or guidance from an adult/caregiver.

□ Child takes full responsibility without any direction, supervision or guidance from an adult/caregiver.

DISCUSSION

The purpose of this article is to describe the process used to develop an expanded set of items (item bank) for a new computerized adaptive test version of the PEDI (PEDI-CAT). A comprehensive approach was taken to develop this new set of items. The measure review, input from practicing clinicians, parents of children with and without disabilities, and experts in the field of rehabilitation, along with cognitive interviewing served to establish preliminary content validity.

This revised instrument is being designed to fit an expanded age range, encompass expanded content within each domain and reduce respondent burden. The content of the PEDI-CAT is intended to measure functional skills in the domains of self-care, mobility and social function as well as assess the extent to which a young person can manage life tasks independently. Items now represent functional skills throughout the age range from birth to young adulthood. For instance, in the self-care domain, the item “Puts on socks” remains from the original PEDI however, a new item “Puts on tights or pantyhose” has been added which is applicable to older girls. Additional self-care items for older children and young adults include “Puts on and ties a tie”, “Shaves face (or legs) using electric or safety razor” and “Opens childproof medicine or vitamin containers”. In the original PEDI social function domain, items about safety included “Knows not to accept rides, food or money from strangers” and “Crosses busy street safely without an adult”. Here we have expanded the items to include content applicable to younger children such as this, “Keeps unsafe objects and household materials out of mouth”.

Participants confirmed that the items included in each functional domain were current and meaningful in the context of modern daily life. The expanded content includes items related to use of a computer, the internet, videogames, automated teller machines (ATM) and cell phones, most of which were not applicable in 1992. Also, for example, in the Mobility domain, there remain items related to stair-climbing with and without a handrail, but items related to escalator use, climbing up the ladder of a slide, and walking up and down bleacher stairs in a gym or stadium have been added. In the Social Function domain, new item content includes friendship (e.g. Maintains friendships that involve give-and-take, compromises and loyalty), functional reading and writing (e.g. Prints first and last name legibly), behavior regulation/behavior management (e.g. Accepts advice or feedback from a teacher, coach, or boss without losing temper), and interaction with adults and people in authority (e.g. Uses language appropriate to the situation such as formal language at a job interview or informal language when hanging out with friends). Additional items have been added to expand the age range and content in areas such as functional communication, interaction with peers, time orientation, safety, problem-solving, and play with objects.

The Responsibility scale was created to meet the growing need for assessments that can plan for and track a young person’s growing independence and successful transition to adulthood. Where the Caregiver Assistance scale measured amount of assistance, the Responsibility scale measures the extent to which the parent or child takes responsibility for managing complex, multi-step life tasks. For example, the item “Traveling safely within the community includes: identifying and following a safe route; using available methods of transportation (e.g. walking, driving, public transportation)”. This is a multi-step functional task aimed at older children and adolescents that requires capability in multiple functional domains. All items in this new scale combine gross and fine motor, cognitive and social skills in one multi-step task that is relevant for independent daily living or life situations.

Using focus groups to inform the item development process allowed us to get input from the stakeholders, the parents and clinicians, who will be responding to the items in the new PEDI-CAT. Parent participants were those whose children had a wide variety of diagnoses, helping us to incorporate content for children with physical, intellectual and behavioral disabilities. The use of cognitive interviewing was particularly critical for the development of the items since fewer items are administered in a CAT than in a traditional assessment and thus, each item must be interpreted with as much clarity and consistency as possible. Cognitive interviewing participants assisted in making the items understandable and easy to respond to. The addition of pictures to the self-care and mobility items is anticipated to be a major advancement in the PEDI-CAT adding to the reliability, validity and accuracy of results. The research team determined however, that it would be difficult to depict the Social Function and Responsibility items in a line drawing, so drawings were not created for these scales.

This work does have several limitations. First, the number of cognitive interviews per item may be considered small. While we tested each item two to three times, it has been suggested that new items be tested at least 10 times (Willis, 2005). Based on the consistent feedback from respondents and the previous clinical, measurement design and cognitive interviewing experience of the project team, we felt comfortable proceeding with this limited number of cognitive interviews. Our parent and clinicians participants were primarily Caucasian and highly educated, which may have influenced our results. We also were forced to limit the final number of items in all domains in order for future field testing to be feasible.

Conclusions and Future Work

We believe that we have expanded and improved the PEDI item bank by broadening the age range and content measured by each scale. CAT is an ideal application for solving some of the limitations of the original PEDI by expanding the applicable age range and item content while minimizing respondent burden. Normative-based item calibrations for the PEDI scales will be developed by collecting general population data for the new PEDI and scoring algorithms and score reports will be created for a fully operational PEDI-CAT system. The full PEDI-CAT will be evaluated in terms of reliability, validity, and precision to create an assessment of disability in children and youth that will improve measurement in clinical research and practice. A practical and feasible PEDI-CAT will greatly improve the information used in making decisions about disability status, service eligibility, program needs, outcomes of rehabilitation interventions, and the effectiveness of clinical research. The PEDI-CAT is expected to be fully operational on both desktop applications and on the Internet.