Abstract
Background
Children's activity level is commonly assessed in clinical research, but rigorous assessment tools for children are scarce. Our objectives were to improve pediatric activity self-report measures using qualitative methods to develop item pools that measure these concepts.
Methods
Based on the items generated from our conceptual framework development, we applied cognitive interviews and comprehensibility reviews to ensure children readily understood the items.
Results
Our methods resulted in 129 unique items, physical activity (80 items), sedentary behaviors (23 items), and strengthening activities (26 items), that were comprehensible to children between the ages of 8 – 18 years. Comprehensibility review resulted in the deletion of 4 items.
Conclusions
The resultant item pools reflect children's experiences and understanding of the concepts of physical activity, sedentary behavior and strengthening activities. The item pools will undergo calibration using item response theory to support computer adaptive test administration of self and proxy reported outcomes.
Keywords: adolescent; child; female; health status; humans; information systems; male; outcome assessment (health care)/methods; parents; pediatrics/methods; physical activities; sedentary lifestyle; strengthening, quality of life
Background
The development of valid and reliable self-report measures of physical activity (PA) and sedentary behavior (SB) continues to be a high priority within the clinical research community. Pediatric physical therapists are engaged in clinical practice to change these behaviors as well as clinical effectiveness research related to these domains. Inclusion of person reported outcomes of these behaviors is considered necessary and augments the information gained from other clinical tests and measures. The development of outcomes instruments should be based on a conceptual framework that is consistent with the intended use, current available knowledge from experts and the literature, and also reflects the respondents’, in this case children’s, perception and understanding of the underlying concept. In the previous paper,1 we reported on development of child-informed domain frameworks for 3 health related PA concepts – PA, SB and strengthening behaviors (Strength). In this paper, we report the development and refinement of item pools for each of these concepts, the next stage in the development of self-report measures of PA, SB and Strength.
A self-report measure should be composed of items with meanings that are explicitly clear to the respondents, applicable for a wide range of ages, and a wide range of skills. However, a self-report instrument that is comprehensive in content and applicable to a wide age range would be lengthy and burdensome for both clinicians to administer and for respondents, particularly children, to complete. Self-report computer adaptive testing (CAT) has been proposed as an alternative to the current fixed-format instruments to alleviate these limitations.2 Currently there are no publically available CATs for assessment of PA, SB or Strength in children.
The National Institute of Health’s Patient Reported Outcome Measurement Information System (PROMIS®) initiative3,4 has supported the development of CATs for health outcomes, and this work expands the coverage of pediatric health concepts in PROMIS. PROMIS methods for instrument development,5 in particular the PROMIS qualitative methods were applied in this study to refine questions or “item pools” that are theoretically based in children’s and experts’ views. The purpose of this study was to develop item pools for each of the 3 conceptual domains we identified (PA, SB & Strength) that can subsequently be used to develop CATs and related instruments. Our long-term goal is to develop CAT instruments to enable efficient administration of our measures for children from 8 – 18 years of age.
Methods
The following activities were performed to develop item pools that mapped to our previously identified conceptual framework1 (see Table 2 in our companion manuscript) for each of our 3 domains (PA, SB and Strength). The items then were used in cognitive interviews, and underwent translatability and comprehensibility review to ensure each item’s meaning and responses were clear to children (Figure). Parental informed consent and child assent were obtained in accordance with protocols approved by the institutional review boards of Temple University and The Children’s Hospital of Philadelphia.
Table 2.
Summary of item revisions resulting from the cognitive interviews
| Physical Activity |
Sedentary Behaviors |
Strengthening Activities |
|
|---|---|---|---|
| Number of Items | 80 | 23 | 26 |
| # of items revised | 22 (27%) | 20 (87%) | 20 (77%) |
| # number of items dropped | 1 (1%) | 2 (9%) | 1 (4%) |
| Final number of items | 79 | 21 | 25 |
Figure.
Overview of the Qualitative Development of the PROMIS Pediatric Physical Activity Domain Item. These steps were used for conceptualization, (expert and child semi-structured interviews), item generation (literature review) and item refinement (cognitive interviews to improve construct validity of item pool, and comprehensibility review).
Item classification, selection, and generation
Item concepts (i.e., the primary focus of the item) from the 3988 items that were identified in our instrument literature review were classified independently by 2 subject matter experts (CAT and RT) into the domains and component facets identified through the earlier expert and child interviews, as described in our companion paper.1
Based on this resultant set of unique item concepts, we generated new items including contexts, stems and responses. The item expressions were written to adhere to PROMIS standards for item construction.5 Items needed to stand-alone without reference to any other item (to facilitate presentation of items individually and in different orders using computerized adaptive testing). The recall period of all items was “In the past 7 days, …”. Item response options (Never – Rarely – Sometimes – Often – Always) were compatible with existing PROMIS items. Finally, items were written as concisely and simply as possible.
Comprehensibility Reviews
The Flesch-Kincaid Grade Reading Level of each item was estimated using the reading level function in Microsoft Word. This readability score is based on the average number of syllables per word and words per sentence, is derived from analyzed and rated text, and is based on a U.S grade-school level.6 For example, a score of 8.0 means that the typical eighth grader would be expected to understand the text.
Translatability review, part of the linguistic validation process of PROMIS instruments, was used during item development to identify possible language and concept translation issues and allow for early modifications of the items.7 A team of 4 translation experts and 2 subject matter experts reviewed each item (item expression) to identify idiomatic expressions, complex sentences, and concepts that would not be easily translated. The translatability of each item was rated according to the existing rating system.7
Cognitive Interviews
Cognitive interviewing is a specific qualitative research approach that is used to study the manner by which a respondent understands, mentally processes and responds to materials that are presented in survey instruments or data collection questionnaires.8,9 To ensure that each item was comprehensible we conducted cognitive interviews in accordance with PROMIS standards5 with a convenience sample of children. Participants were recruited from primary care practices and school settings in the Philadelphia, Pennsylvania region. Cognitive interviews were conducted with 53 children aged 8–7 years (grades 3–12) recruited from primary care practices and school settings. Cognitive interviewees were 38% male, 11% Asian, 57% African American, 24% White, 8% of another race, 2% Hispanic, and 55% had a chronic health condition (e.g. asthma, renal disease, inflammatory bowel disease).
Up to 35 items were tested with each child, and each item was reviewed by minimum of 5 participants, including at least two 8–11 year old children. This approach and sample size per item is consistent with PROMIS standards.5 Prior to the cognitive interview participants completed a paper-and-pencil questionnaire of the subset of approximately 35 items assigned to them. Once the child completed the questionnaire, a trained research assistant interviewed the child following a standard interview procedure. For each item, interviewees read the question aloud and stated the meaning of the question using their own words. The interviewer then asked, “How did you answer that question?” and “Why did you choose that answer?” Responses to cognitive interview probes were coded for the degree to which children’s understanding of the question was consistent with the item definition (1 = poor/different than intended; 2 = partial; 3 = fully consistent). Items with average ratings of less than 2 were removed or revised. Revised items were re-evaluated using the same cognitive interview procedure.
Results
Formation of Item Pools
Item expressions (the specific wording of items including context, stem and responses) were generated from the domain-facet concept maps refined through the expert and child interviews. The proposed item pools (sets of item expressions within a defined domain) for each of the domains were reviewed and iteratively revised to ensure coverage of key concepts and remove redundant item concepts. Similar item concepts were grouped, and each set of similar items reviewed with the more complex worded items removed while retaining adequate concept coverage with the remaining items. We developed a final set of 129 items for cognitive interviewing consisting of 80 PA items, 23 SB items, and 26 Strength items.
Comprehensibility Analyses
The average Flesch-Kincaid readability estimate across all 3 domains was Grade 5.4, with variation noted across the 3 domains (Table 1). The concerns identified by the translatability and cross cultural review primarily related to the use and ordering of adverbs and adjectives in activity concepts in the English language. For example, the phrase an "active game" may be problematic in some languages as the word "active" may be used only to qualify a person, not to describe the type of game or level of physical effort exerted during a game. Another example was our use of the common English phrasing “physically active” as an adjective (e.g. "how physically active were you…") in construction of some items. For such items, other languages (e.g. Spanish) often use the phrasing "to what extent/point/degree were you physically active?” In these cases, the English item was revised or slight grammatical differences were considered acceptable. Many of the more common childhood activities and sports in the United States differ in prevalence in other countries and cultures. In these cases, the general movement aspects of the activity (e.g., team game played with intermittent rests on a large field outside; or team games that individuals get up to bat/kick) were used to provide a basis for comparison in selecting more culturally appropriate sports or activities with similar movement requirements.
Table 1.
Summary reading level of item pools
| Average Rating | Range | SD | |
|---|---|---|---|
| Physical Activity | 5.4 | 1.8–9.7 | 1.6 |
| Sedentary behaviors | 5.6 | 2.8–8.1 | 1.5 |
| Strengthening Activities | 5.6 | 1.8–6.9 | 1.1 |
Based on the initial cognitive interviews, 64 items were revised and re-tested. The primary reason for revision was the complexity of item wording and difficulties with item comprehension. For example, to be as clear as possible, qualifiers had been included (e.g. “early morning means the time between getting out of bed to when you go to school” or “games that are played by teams and have the players run bases like baseball, softball or cricket”) to each item expression. Such descriptors were removed to improve the readability and brevity of the items. A summary of the item revisions based on cognitive interviewing is provided in Table 2.
At the end of these revisions, we retained 134 items (Tables 3, 4 & 5) that reflect the child’s perspective of the full range of item concepts identified in this study. The majority of items had responses indicating how many days (0 to 7 days) the activity in the item stem was performed. The PA items (Table 3) included common activities (e.g. walking, running); common child sports and leisure activities (baseball, exercise), body sensations (sweating, heart racing), as well as sets of items that asked about frequency of simply being “physically active” for specific durations (10, 30 or 60 minutes) or during parts of the day (e.g., morning, evening). The SB items (Table 4) primarily reflect the performance of activities based on media type (video/gaming, watching TV, or using computer) across different times of day (morning, afternoon, evening). Finally, the Strength items (Table 5) reflect the children’s grouping of strengthening and aerobic activities as a singular “exercising” concept, as well as common physical functioning activities that require strength to perform and can be challenging for children with poor health. We retained some sets of items that captured similar concepts (e.g., breathe hard, out of breath) or appeared to be locally dependent (when a response to 1 item can be predicted from another, for example - such as walk for 60 minutes, implies affirmation of walk for 30 and walk for 10 minutes) for the subsequent larger scale administration and quantitative evaluation of items.
Table 3.
Physical Activity Items
| Item Stems In the past 7 days, … |
Response categories |
|---|---|
| how many days were you so physically active that you breathed hard? | No days; 1 day; 2–3 days; 4–5 days; 6 – 7 days |
| how many days did you exercise so much that you breathed hard? | No days; 1 day; 2–3 days; 4–5 days; 6 – 7 days |
| how many days did you walk fast, bike, or swim for 10 minutes or more? | No days; 1 day; 2–3 days; 4–5 days; 6 – 7 days |
| how many days did you walk fast, bike, or swim for 30 minutes or more? | No days; 1 day; 2–3 days; 4–5 days; 6 – 7 days |
| how many days did you walk fast, bike, or swim for 60 minutes or more? | No days; 1 day; 2–3 days; 4–5 days; 6 – 7 days |
| how many days did you exercise really hard for 10 minutes or more? | No days; 1 day; 2–3 days; 4–5 days; 6 – 7 days |
| how many days did you exercise really hard for 30 minutes or more? | No days; 1 day; 2–3 days; 4–5 days; 6 – 7 days |
| how many days did you exercise really hard for 60 minutes or more? | No days; 1 day; 2–3 days; 4–5 days; 6 – 7 days |
| how many days did you play sports for 10 minutes or more? | No days; 1 day; 2–3 days; 4–5 days; 6 – 7 days |
| how many days did you play a physically active game for 10 minutes or more? | No days; 1 day; 2–3 days; 4–5 days; 6 – 7 days |
| how many days did you play sports for 30 minutes or more? | No days; 1 day; 2–3 days; 4–5 days; 6 – 7 days |
| how many days did you play a physically active game for 30 minutes or more? | No days; 1 day; 2–3 days; 4–5 days; 6 – 7 days |
| how many days did you play sports for 60 minutes or more? | No days; 1 day; 2–3 days; 4–5 days; 6 – 7 days |
| how many days did you play a physically active game for 60 minutes or more? | No days; 1 day; 2–3 days; 4–5 days; 6 – 7 days |
| how many days did you walk for at least 10 minutes without stopping? | No days; 1 day; 2–3 days; 4–5 days; 6 – 7 days |
| how many days did you walk for at least 30 minutes without stopping? | No days; 1 day; 2–3 days; 4–5 days; 6 – 7 days |
| how many days were you so physically active that you sweated? | No days; 1 day; 2–3 days; 4–5 days; 6 – 7 days |
| how many days did you exercise or play so hard that your body got tired? | No days; 1 day; 2–3 days; 4–5 days; 6 – 7 days |
| how many days did you exercise or play so hard that your muscles burned? | No days; 1 day; 2–3 days; 4–5 days; 6 – 7 days |
| how many days did you exercise or play so hard that you felt tired? | No days; 1 day; 2–3 days; 4–5 days; 6 – 7 days |
| how many days did you exercise or play so hard that you felt very tired? | No days; 1 day; 2–3 days; 4–5 days; 6 – 7 days |
| how many days were you physically active for 10 minutes or more? | No days; 1 day; 2–3 days; 4–5 days; 6 – 7 days |
| how many days were you physically active for 30 minutes or more? | No days; 1 day; 2–3 days; 4–5 days; 6 – 7 days |
| how many days were you physically active for 60 minutes or more? | No days; 1 day; 2–3 days; 4–5 days; 6 – 7 days |
| how physically active were you in the morning? | Not at all, A little bit, Somewhat, Quite a bit, Very much |
| how physically active were you during school recess? | Did not have recess/ breaks; Not at all A little bit, Somewhat, Quite a bit, Very much |
| how physically active were you during your school lunch period? | Not at all, A little bit, Somewhat, Quite a bit, Very much |
| how many days did you take gym or physical education (PE) class? | No days; 1 day; 2–3 days; 4–5 days; 6 – 7 days |
| how physically active were you during gym or physical education (PE) class? | Did not take gym/PE class; Not at all, A little bit, Somewhat, Quite a bit, Very much |
| how physically active were you? | Not at all, A little bit, Somewhat, Quite a bit, Very much |
| how physically active were you during the afternoon? | Not at all, A little bit, Somewhat, Quite a bit, Very much |
| how physically active were you during the evening? | Not at all, A little bit, Somewhat, Quite a bit, Very much |
| how physically active were you during your free time at home? | Not at all, A little bit, Somewhat, Quite a bit, Very much |
| how physically active were you with your friends? | Not at all, A little bit, Somewhat, Quite a bit, Very much |
| how physically active were you with your family? | Not at all, A little bit, Somewhat, Quite a bit, Very much |
| how many days did you climb a flight of stairs without stopping to rest? | No days; 1 day; 2–3 days; 4–5 days; 6 – 7 days |
| what was the longest time you walked without stopping? | None Less than 15 minutes; 15–29 minutes; 30–59 minutes; 60 minutes or more |
| what was the longest time you ran? | None Less than 15 minutes; 15–29 minutes; 30–59 minutes; 60 minutes or more |
| what was the longest distance you walked? | Across a room; About a block; A few blocks; About a mile; More than a mile |
| what was the longest distance you ran? | About a block; A few blocks; About a mile; About 2 miles; 3 or more miles |
| how many days did you walk for 10 minutes or more? | No days; 1 day; 2–3 days; 4–5 days; 6 – 7 days |
| how many days did you walk for 30 minutes or more? | No days; 1 day; 2–3 days; 4–5 days; 6 – 7 days |
| how many days did you walk for 60 minutes or more? | No days; 1 day; 2–3 days; 4–5 days; 6 – 7 days |
| how many days did you run for 10 minutes or more? | No days; 1 day; 2–3 days; 4–5 days; 6 – 7 days |
| how many days did you run for 30 minutes or more? | No days; 1 day; 2–3 days; 4–5 days; 6 – 7 days |
| how many days did you run for 60 minutes or more? | No days; 1 day; 2–3 days; 4–5 days; 6 – 7 days |
| how many days did you walk fast for 30 minutes without stopping? | No days; 1 day; 2–3 days; 4–5 days; 6 – 7 days |
| how many days did you run fast for 30 minutes or more without stopping? | No days; 1 day; 2–3 days; 4–5 days; 6 – 7 days |
| how many days did you run a mile or more? | No days; 1 day; 2–3 days; 4–5 days; 6 – 7 days |
| how many days did you run 3 miles or more? | No days; 1 day; 2–3 days; 4–5 days; 6 – 7 days |
| how many days did you play soccer, football, rugby, hockey or lacrosse? | No days; 1 day; 2–3 days; 4–5 days; 6 – 7 days |
| how many days did you play basketball, volleyball, or handball? | No days; 1 day; 2–3 days; 4–5 days; 6 – 7 days |
| how many days did you play tennis, squash, or racquetball? | No days; 1 day; 2–3 days; 4–5 days; 6 – 7 days |
| how many days did you play baseball, softball, cricket, or kickball? | No days; 1 day; 2–3 days; 4–5 days; 6 – 7 days |
| how many days did you bowl, play ping pong or shoot pool? | No days; 1 day; 2–3 days; 4–5 days; 6 – 7 days |
| how many days did you roller-blade, roller-skate or skate-board? | No days; 1 day; 2–3 days; 4–5 days; 6 – 7 days |
| how many days did you swim or play water sports? | No days; 1 day; 2–3 days; 4–5 days; 6 – 7 days |
| how many days did you ride a bike for 10 minutes or more? | No days; 1 day; 2–3 days; 4–5 days; 6 – 7 days |
| how many days did you ride a bike for 30 minutes or more? | No days; 1 day; 2–3 days; 4–5 days; 6 – 7 days |
| how many days did you ride a bike for 60 minutes or more? | No days; 1 day; 2–3 days; 4–5 days; 6 – 7 days |
| how many days did you practice with or play on a sports team? | No days; 1 day; 2–3 days; 4–5 days; 6 – 7 days |
| how many days did you hike, hunt, or horseback ride? | No days; 1 day; 2–3 days; 4–5 days; 6 – 7 days |
| how many days did you sail, row, or canoe? | No days; 1 day; 2–3 days; 4–5 days; 6 – 7 days |
| how many days did you take a dance class? | No days; 1 day; 2–3 days; 4–5 days; 6 – 7 days |
| how many days did you dance for 10 minutes or more? | No days; 1 day; 2–3 days; 4–5 days; 6 – 7 days |
| how many days did you dance for 30 minutes or more? | No days; 1 day; 2–3 days; 4–5 days; 6 – 7 days |
| how many days did you dance for 60 minutes or more? | No days; 1 day; 2–3 days; 4–5 days; 6 – 7 days |
| how many days did you jump rope? | No days; 1 day; 2–3 days; 4–5 days; 6 – 7 days |
| how many days did you take an exercise class? | No days; 1 day; 2–3 days; 4–5 days; 6 – 7 days |
| how many days did you garden or do yard work? | No days; 1 day; 2–3 days; 4–5 days; 6 – 7 days |
| how many days did you walk up steep hills without resting? | No days; 1 day; 2–3 days; 4–5 days; 6 – 7 days |
| how many days did you climb a flight of stairs without resting? | No days; 1 day; 2–3 days; 4–5 days; 6 – 7 days |
| how physically active were you outside your home? | Not at all, A little bit, Somewhat, Quite a bit, Very much |
| how physically active were you inside your home? | Not at all, A little bit, Somewhat, Quite a bit, Very much |
| how many days did you do chores around the house? | None Less than 15 minutes; 15–29 minutes; 30–59 minutes; 60 minutes or more |
| how many days did you play catch or Frisbee? | None Less than 15 minutes; 15–29 minutes; 30–59 minutes; 60 minutes or more |
| how physically active were you on a school day? | Not at all, A little bit, Somewhat, Quite a bit, Very much |
| on a usual day, how physically active were you? | Not at all, A little bit, Somewhat, Quite a bit, Very much |
| what is the longest time you stood without sitting down? | None Less than 15 minutes; 15–29 minutes; 30–59 minutes; 60 minutes or more |
Table 4.
Sedentary Behaviors Items
| Item Stems In the past 7 days, … |
Response categories |
|---|---|
| how much time did you play video or computer games on a weekday? | 1) none, 2) less than 1 hour, 3) 1 to 2 hours, 4) 3 to 4 hours, 5) 5 or more hours |
| how much time did you play video or computer games on a weekend day? | 1) none, 2) less than 1 hour, 3) 1 to 2 hours, 4) 3 to 4 hours, 5) 5 or more hours |
| how much time did you play video or computer games on a usual weekend morning? | 1) none, 2) less than 1 hour, 3) 1 to 2 hours, 4) 3 to 4 hours, 5) 5 or more hours |
| how much time did you play video or computer games on a usual weekday morning? | 1) none, 2) less than 1 hour, 3) 1 to 2 hours, 4) 3 to 4 hours, 5) 5 or more hours |
| how much time did you play video or computer games after school? | 1) none, 2) less than 1 hour, 3) 1 to 2 hours, 4) 3 to 4 hours, 5) 5 or more hours |
| how much time did you play video or computer games on a usual night? | 1) none, 2) less than 1 hour, 3) 1 to 2 hours, 4) 3 to 4 hours, 5) 5 or more hours |
| how much time did you spend on a computer on a usual weekday? | 1) none, 2) less than 1 hour, 3) 1 to 2 hours, 4) 3 to 4 hours, 5) 5 or more hours |
| how much time did you spend on a computer on a usual weekend day? | 1) none, 2) less than 1 hour, 3) 1 to 2 hours, 4) 3 to 4 hours, 5) 5 or more hours |
| how much time did you spend on a computer on a usual morning? | 1) none, 2) less than 1 hour, 3) 1 to 2 hours, 4) 3 to 4 hours, 5) 5 or more hours |
| how much time did you spend on a computer on a usual afternoon? | 1) none, 2) less than 1 hour, 3) 1 to 2 hours, 4) 3 to 4 hours, 5) 5 or more hours |
| how much time did you spend on a computer on a usual weekday? | 1) none, 2) less than 1 hour, 3) 1 to 2 hours, 4) 3 to 4 hours, 5) 5 or more hours |
| how much TV did you watch on a usual weekday? | 1) none, 2) less than 1 hour, 3) 1 to 2 hours, 4) 3 to 4 hours, 5) 5 or more hours |
| how much TV did you watch on a usual weekend day? | 1) none, 2) less than 1 hour, 3) 1 to 2 hours, 4) 3 to 4 hours, 5) 5 or more hours |
| how much TV did you watch on a usual morning? | 1) none, 2) less than 1 hour, 3) 1 to 2 hours, 4) 3 to 4 hours, 5) 5 or more hours |
| how much TV did you watch on a usual afternoon? | 1) none, 2) less than 1 hour, 3) 1 to 2 hours, 4) 3 to 4 hours, 5) 5 or more hours |
| how much TV did you watch on a usual night? | 1) none, 2) less than 1 hour, 3) 1 to 2 hours, 4) 3 to 4 hours, 5) 5 or more hours |
| on a usual day, how much time did you spend sitting with your friends or family? | 1) none, 2) less than 1 hour, 3) 1 to 2 hours, 4) 3 to 4 hours, 5) 5 or more hours |
| on a usual day, how much time did you spend sitting in a bus, subway, or car? | 1) none, 2) less than 1 hour, 3) 1 to 2 hours, 4) 3 to 4 hours, 5) 5 or more hours |
| on a usual day, how much time did you spend texting? | 1) none, 2) less than 1 hour, 3) 1 to 2 hours, 4) 3 to 4 hours, 5) 5 or more hours |
| on a usual day, how much time did you spend talking on the phone? | 1) none, 2) less than 1 hour, 3) 1 to 2 hours, 4) 3 to 4 hours, 5) 5 or more hours |
| on a usual day, how much time did you spend doing homework? | 1) none, 2) less than 1 hour, 3) 1 to 2 hours, 4) 3 to 4 hours, 5) 5 or more hours |
Table 5.
Strengthening Items
| Item Stems In the past 7 days, … |
Response Categories |
|---|---|
| how many days did you do sit-ups, push-ups, or pull-ups? | No days; 1 day; 2–3 days; 4–5 days; 6 – 7 days |
| on a usual day, how much time did you exercise to strengthen your muscles? | None; Less than 15 minutes; 15–29 minutes; 30–59 minutes; 60 minutes or more |
| how many days did you exercise to strengthen your muscles? | No days; 1 day; 2–3 days; 4–5 days; 6 – 7 days |
| how many days did you play games or do exercises where you jumped? | No days; 1 day; 2–3 days; 4–5 days; 6 – 7 days |
| how many days did you lift weights? | No days; 1 day; 2–3 days; 4–5 days; 6 – 7 days |
| how many days did you lift weights for 20 minutes or more? | No days; 1 day; 2–3 days; 4–5 days; 6 – 7 days |
| how many days did you exercise to strengthen your stomach muscles? | No days; 1 day; 2–3 days; 4–5 days; 6 – 7 days |
| how many days did you exercise to strengthen your back muscles? | No days; 1 day; 2–3 days; 4–5 days; 6 – 7 days |
| how many days were you strong enough to lift heavy things off the floor? | No days; 1 day; 2–3 days; 4–5 days; 6 – 7 days |
| how many days were you strong enough to lift heavy things over your head? | No days; 1 day; 2–3 days; 4–5 days; 6 – 7 days |
| how many days were you strong enough to carry a heavy back pack? | No days; 1 day; 2–3 days; 4–5 days; 6 – 7 days |
| how many days were you strong enough to carry heavy things with your hands? | No days; 1 day; 2–3 days; 4–5 days; 6 – 7 days |
| how many days were you strong enough to go up and down stairs? | No days; 1 day; 2–3 days; 4–5 days; 6 – 7 days |
| how many days were you strong enough to reach above your head to get heavy things? | No days; 1 day; 2–3 days; 4–5 days; 6 – 7 days |
| how many days were you strong enough to open a heavy door? | No days; 1 day; 2–3 days; 4–5 days; 6 – 7 days |
| how many days were you strong enough to hold a full plate or tray of food? | No days; 1 day; 2–3 days; 4–5 days; 6 – 7 days |
| how many days were you strong enough to get up from the toilet without help? | No days; 1 day; 2–3 days; 4–5 days; 6 – 7 days |
| how many days were you strong enough to get out of bed and stand without help? | No days; 1 day; 2–3 days; 4–5 days; 6 – 7 days |
| how many days were you strong enough to jump up and down? | No days; 1 day; 2–3 days; 4–5 days; 6 – 7 days |
| how many days were you strong enough to pour a drink from a full pitcher or carton? | No days; 1 day; 2–3 days; 4–5 days; 6 – 7 days |
| how many days were you strong enough to open a jar by yourself? | No days; 1 day; 2–3 days; 4–5 days; 6 – 7 days |
| how many days were you strong enough to carry your school books? | No days; 1 day; 2–3 days; 4–5 days; 6 – 7 days |
| how many days were you strong enough to get down on the floor? | No days; 1 day; 2–3 days; 4–5 days; 6 – 7 days |
| how many days did you do wrestling, gymnastics or cheerleading? | No days; 1 day; 2–3 days; 4–5 days; 6 – 7 days |
| how many days did you do squats, lunges, toe-raises or step-ups? | No days; 1 day; 2–3 days; 4–5 days; 6 – 7 days |
Discussion
The application of qualitative methods used in this study generated 129 unique PA items: PA (80 items), SB (23 items), and Strength (26 items). Items were worded consistent with the way children described activities, and included items that represented their non-adult like grouping of daily activities by including social and physical environment cues. Not surprisingly, the resultant item pools included a range of item concepts that are reflected in other PA measures such as specific activities, duration of activities as well as time of day or week of activity. However we also included more items reflecting social contexts, and body symptoms experienced with activity as children commonly mentioned these aspects in their recall of their activities. However, in comparison to existing child report instruments, our item pools were thoroughly cognitively tested and underwent rigorous comprehensibility reviews, which included in-depth comprehensibility, cultural and translatability reviews. Whether this approach confers an advantage in terms of measurement precision will be tested in future research that examines performance of the items in large population samples and compares the measures with other self-report and objective monitors.
Our objectives for the qualitative portion of our item bank development were met. More specifically, the items are both theoretically grounded for each of the activity domain concepts and supported by our expert interviews, are developmentally appropriate for children, both in terms of conceptualization as well as a breadth of item types to engage different aspects of possible child cognitive heuristics. Our items also underwent extensive comprehensibility assessment, both in terms of readability as well as cross-cultural applicability and translatability.
Our methods have several limitations. The child interviews and cognitive interviews were drawn primarily from a large urban population (Philadelphia, Pennsylvania region), and may not adequately reflect regional differences in conceptualization or phrasing of concepts. In addition, this sample of convenience had a higher proportion of African Americans than are present in the general US population. This limitation will be balanced with subsequent item pool development. Given the complexity and breadth of possible activities and behaviors, our initial items pools are relatively limited in scope. In particular, the inclusion of activities appropriate for young children (5 – 8 years old) and the number and breadth of items included in our SB and Strength banks provide a limited item pool which will need expansion.
Future Directions
The application of such qualitative approaches to better incorporate children’s conceptualization of activity behaviors and to better understand how children organize and recall their daily activities could significantly advance pediatric self-report activity instruments. Our next step will be to administer the items to a large sample and evaluate them using item response theory (IRT) methods. This will make it possible to introduce new items in the future and subsequently co-calibrate them with the existing items, providing an incremental means to introduce new item concepts and improve measurement.10,11 The calibrated item banks and related instruments including computer-adaptive tests produced by this mixed-method approach are expected to measure the target health concept with high precision and efficiency (i.e., low respondent burden) within a meaningful and understandable framework for the respondents. The resulting measures will be evaluated to ensure that they are responsive to changes in children’s general health and physical activity-related outcome.
Conclusion
This qualitative research has resulted in a theoretically grounded and child-informed conceptual framework of pediatric activity along with item pools that are comprehensible to children. The inclusion of items in a single instrument that reflect the identified child-centered activity constructs related to the time of day, social context, purpose, and person-experienced response to activity advances the measurement of pediatric activity. Our work provides strong support for the content validity of the new child self-report measures of PA, SB, and Strength activities, and supports the development of instruments that will provide valid and standardized endpoints for application and comparison across clinical studies assessing the relative effectiveness of interventions directed towards changing PA and SB patterns.
The PROMIS approach to developing person-reported outcomes relies on mixed method that links initial qualitative development with quantitative methods that employ item response theory (IRT) methods for item calibration.9,10 The application of IRT requires a theoretically grounded conceptualization and definition of the health attribute being measured. Empirically, IRT requires an underlying ordering of items, from low to high levels of the health attribute. For PA behaviors, the relative amount of energy expended could provide the underlying hierarchical framework necessary for IRT-based assessment. Thus, higher levels of or “scores” for a PA item bank would reflect performance of a higher amount of energy expending behaviors. In addition to addressing the need for a measurement hierarchy, the inclusion of items with varied contextual characteristics (e.g. location, and with whom one is active with) would enhance recall and also is often of interest to researchers or interventionists. These 2 considerations were used to support the development of our PA behavior item concepts – that the item banks could support scoring the necessary IRT hierarchical framework and could be used to determine some aspects (energy expenditure), but should be composed of sets of items that also cover the breadth of contexts of interest.
Acknowledgments
Grant Support: This work was funded by NIH/NIAMS as U01 AR057956 (PI – Forrest, CB): Pediatric PROMIS: Advancing the Measurement and Conceptualization of Child Health. PROMIS® was funded with cooperative agreements from the National Institutes of Health (NIH) Common Fund Initiative (Northwestern University, PI: David Cella, PhD, U54AR057951, U01AR052177; Northwestern University, PI: Richard C. Gershon, PhD, U54AR057943; American Institutes for Research, PI: Susan (San) D. Keller, PhD, U54AR057926; State University of New York, Stony Brook, PIs: Joan E. Broderick, PhD and Arthur A. Stone, PhD, U01AR057948, U01AR052170; University of Washington, Seattle, PIs: Heidi M. Crane, MD, MPH, Paul K. Crane, MD, MPH, and Donald L. Patrick, PhD, U01AR057954; University of Washington, Seattle, PI: Dagmar Amtmann, PhD, U01AR052171; University of North Carolina, Chapel Hill, PI: Harry A. Guess, MD, PhD (deceased), Darren A. DeWalt, MD, MPH, U01AR052181; Children’s Hospital of Philadelphia, PI: Christopher B. Forrest, MD, PhD, U01AR057956; Stanford University, PI: James F. Fries, MD, U01AR052158; Boston University, PIs: Alan Jette, PT, PhD, Stephen M. Haley, PhD (deceased), and David Scott Tulsky, PhD (University of Michigan, Ann Arbor), U01AR057929; University of California, Los Angeles, PIs: Dinesh Khanna, MD (University of Michigan, Ann Arbor) and Brennan Spiegel, MD, MSHS, U01AR057936; University of Pittsburgh, PI: Paul A. Pilkonis, PhD, U01AR052155; Georgetown University, PIs: Carol. M. Moinpour, PhD (Fred Hutchinson Cancer Research Center, Seattle) and Arnold L. Potosky, PhD, U01AR057971; Children’s Hospital Medical Center, Cincinnati, PI: Esi M. Morgan DeWitt, MD, MSCE, U01AR057940; University of Maryland, Baltimore, PI: Lisa M. Shulman, MD, U01AR057967; and Duke University, PI: Kevin P. Weinfurt, PhD, U01AR052186). NIH Science Officers on this project have included Deborah Ader, PhD, Vanessa Ameen, MD (deceased), Susan Czajkowski, PhD, Basil Eldadah, MD, PhD, Lawrence Fine, MD, DrPH, Lawrence Fox, MD, PhD, Lynne Haverkos, MD, MPH, Thomas Hilton, PhD, Laura Lee Johnson, PhD, Michael Kozak, PhD, Peter Lyster, PhD, Donald Mattison, MD, Claudia Moy, PhD, Louis Quatrano, PhD, Bryce Reeve, PhD, William Riley, PhD, Peter Scheidt, MD, Ashley Wilder Smith, PhD, MPH, Susana Serrate-Sztein, MD, William Phillip Tonkins, DrPH, Ellen Werner, PhD, Tisha Wiley, PhD, and James Witter, MD, PhD. The contents of this article uses data developed under PROMIS. These contents do not necessarily represent an endorsement by the US Federal Government or PROMIS. See www.nihpromis.org for additional information on the PROMIS® initiative.
Footnotes
Conflicts of Interest: The author’s declare no conflicts of interest.
References
- 1.Tucker CA, Bevans KB, Teneralli RE, Smith AW, Bowles HR, Forrest CB. Self-Reported Pediatric Measures of Physical Activity, Sedentary Behavior and Strength Impact for PROMIS®: Conceptual Framework. Pediatr Phys Ther. 2014 Oct;26(3) doi: 10.1097/PEP.0000000000000073. xxx-xxx. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Haley SM, Fragala-Pinkham MA, Dumas HM, et al. Evaluation of an item bank for a computerized adaptive test of activity in children with cerebral palsy. Phys Ther. 2009 Jun;89(6):589–600. doi: 10.2522/ptj.20090007. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Forrest CB, Bevans KB, Tucker C, et al. The Patient reported outcome measurement information system (PROMIS) for children and youth: Application to pediatric psychology. J Pediatr Psychol. 2012 Jul;37(6):614–621. doi: 10.1093/jpepsy/jss038. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Cella D, Yount S, Rothrock N, Gershon, et al. The Patient-Reported Outcomes Measurement Information System (PROMIS): progress of an NIH Roadmap cooperative group during its first two years. Medical Care. 2007;45:S3–S11. doi: 10.1097/01.mlr.0000258615.42478.55. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5. [accessed 11/05/2013]; http://www.nihpromis.org/Documents/PROMISStandards_Vers2.0_Final.pdf.
- 6.Klare G. A second look at the validity of readability formulas. Journal of Reading Behavior. 1976;8:129–152. [Google Scholar]
- 7.Wild D, Eremenco S, Mear I, Martin M, Houchin C, Gawlicki M. Multinational trials-recommendations on the translations required, approaches to using the same language in different countries, and the approaches to support pooling the data: the ISPOR Patient-Reported Outcomes Translation and Linguistic Validation Good. Value in Health. 2009;12:430–440. doi: 10.1111/j.1524-4733.2008.00471.x. [DOI] [PubMed] [Google Scholar]
- 8.Willis GB. Cognitive Interviewing: A Tool for Improving Questionnaire Design. Thousand Oaks, CA: Sage Publications; 2005. [Google Scholar]
- 9.Fortune-Greeley AK, Flynn KE, Jeffery DD, et al. Using cognitive interviews to evaluate items for measuring sexual functioning across cancer populations: improvements and remaining challenges. Quality of Life Research. 2009;18(8):1085–1093. doi: 10.1007/s11136-009-9523-x. (PMCID: PMC2759179) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Bjorner JB, Chang CH, Thissen D, Reeve BB. Developing tailored instruments: item banking and computerized adaptive assessment. Qual Life Res. 2007;16(Suppl. 1):95–108. doi: 10.1007/s11136-007-9168-6. [DOI] [PubMed] [Google Scholar]
- 11.Cella D, Gershon R, Lai J-S, Choi SW. The future of outcomes measurement: item banking, tailored short-forms, and computerized adaptive assessment. Qual Life Res. 2007;16:133–141. doi: 10.1007/s11136-007-9204-6. [DOI] [PubMed] [Google Scholar]