Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2018 May 4.
Published in final edited form as: J Nurs Meas. 2013;21(3):360–377. doi: 10.1891/1061-3749.21.3.360

Development and Evaluation of an Instrument to Measure Reasoning About Managing Asthma in Older School-Age Children and Adolescents

Eileen Kintner 1, Gwendolyn Cook 2, Lakisha Hull 3, Linda Meeder 4
PMCID: PMC5935131  NIHMSID: NIHMS937225  PMID: 24620511

Abstract

Background and Purpose

This article describes development and evaluation of a 4-scenario instrument designed to measure reasoning abilities for managing asthma symptoms.

Methods

Existing literature informed creation of a reasoning template and the reasoning scenarios. Think-aloud interviews appraised scenario functionality. Experts assessed content validity. Data from 2 groups of students with asthma aged 9–15 years (N = 132 and 307) were used to evaluate psychometric properties.

Results

The scenarios were deemed functional, content was relevant, and readability was age/grade appropriate. Cronbach’s alpha was 0.79. Exploratory and confirmatory factor analyses indicated a single component solution and revealed a good fit. Concurrent validity was established using correlations with asthma knowledge.

Conclusion

The instrument could be used to assess reasoning about symptom management and to evaluate educational interventions.

Keywords: decision-making, problem-solving, health education, health counseling, instrument development, content validity

Reasoning about asthma was identified as an essential part of the adolescent process of coming to accept asthma as a chronic condition (Kintner, 1997). Reasoning was deemed necessary to draw conclusions about asthma that would allow for coming to terms with, or acceptance of, one’s condition. Logical reasoning for managing asthma symptoms is important for subsequent use of effective asthma health management behaviors that impact asthma severity, use of health care services, and quality of life outcomes (Kintner et al., 2012; Kintner & Sikorskii, 2009). The concept was defined as reflective, introspective thinking through which situations are examined, options are considered, choices are made, and cause and effect relationships are associated (Kintner, 1997). An instrument to operationalize the concept was needed to further explore reasoning about managing asthma in older school-age children and early adolescents.

The purpose of this article is to describe development and evaluation of the Reasoning About Asthma (RAA) scenarios, an instrument designed to measure logical reasoning abilities for managing exacerbation of symptoms during acute episodes. The acceptance of asthma model (Kintner, 1997, 2004, 2007) and life span development perspective (Lerner, 1978; Santrock, 2010; Sugarman, 1986) guided development and evaluation of the instrument in four phases.

BACKGROUND AND CONCEPTUAL FRAMEWORK

Instruments used to assess self-efficacy (Bandura, 1986) related to decision-making and problem-solving abilities (Bransford & Stein, 1984; Heermann & Wills, 1992; Murdock et al., 2010; Pulgaron, Salamon, Patterson, & Barakat, 2010; Rhee, Hollen, Belyea, & Sutherland, 2008) have been unsuccessful in capturing significant differences when used to evaluate the effectiveness interventions (Heermann & Wills, 1992; Murdock et al., 2010; Pulgaron et al., 2010; Rhee et al., 2008). Although the instruments are useful, we realized that an objective measure of reasoning would be the best way to evaluate what students might actually do when experiencing acute exacerbation of asthma symptoms.

McQuaid, Howard, Kopel, Rosenblum, and Bibace (2002) investigated reasoning about the causal mechanism of asthma episodes in students with persistent symptoms believing that self management required foundational understanding of the causes and consequences of exacerbation of symptoms. Findings revealed levels of cognitive complexity ranged from the most rudimentary conception with no clear differentiation of cause and effect to the most sophisticated levels of cognitive complexity with integration of multiple causes and effects. Narayan and Corcoran-Perry (1997) offered a line of reasoning representation for how decisions are made based on assumptions consistent with information-processing theory. Information-processing theory is useful in understanding outcomes of reasoning as well as knowledge and cognitive processes that underlie reasoning. The line of reasoning representation describes how individuals with a range of experience use their knowledge to make decisions and then relate knowledge embedded within cognitive processes to outcomes. This representation was used to guide development of the RAA.

Demographic Considerations

Students aged 9–14 years experience increased morbidity and mortality over all other age groups across the life span as do females over males, Black over non-Hispanic White American groups, and families of a lower socioeconomic group over middle and upper groups (Akinbami, Moorman, & Liu, 2011). Therefore, influences of demographic characteristics, which may influence reasoning about managing symptoms, must be considered when examining reliability and validity of the RAA.

Asthma Knowledge Considerations

Student and parent factual knowledge about asthma and more education and experience of students and parents have been associated with significant increases in levels of reasoning about (a) causes and consequences of acute asthma episodes (McQuaid, Kopel, Klein, & Fritz, 2003) and (b) management of asthma symptoms (Kintner, 2004, 2007). Reasoning about symptom management was hypothesized to be positively associated with student and parent asthma knowledge.

PHASE 1, PROCEDURES FOR INSTRUMENT DEVELOPMENT

Line of Reasoning Template

Based on the work of Narayan and Corcoran-Perry (1997), a line of reasoning template for self-management of acute asthma episodes was proposed. Potential triggering cues were identified from existing literature (National Institutes of Health [NIH], 2007) to include a diagnosis of asthma, signs or symptoms, potential stimuli or triggers, prescribed treatments, and people in the area. Domain concepts allowed classification of triggering cues to determine episode severity, actual stimuli, medication options, self-management techniques, and supportive resources. An internal self-talk dialogue through which options and choices are weighed results in a preliminary summary statement of the experience. This intermediate conclusion leads to action. Actions might include avoiding or removing stimuli, using controller or reliever medications, using appropriate self-management techniques, and asking for assistance from others. Based on the effectiveness of the intermediate actions, conclusions may be drawn that contribute to management of future asthma episodes. The line of reasoning template was used to devise a blueprint for constructing scenarios that could be used to appraise reasoning abilities for managing symptoms.

Reasoning About Asthma Scenarios

In the interest of parsimonious data collection relative to ease of use and time and cost effectiveness, four scenarios reflective of varying degrees of symptom severity were invented: “grilling in the park,” “running the mile,” “playing in the snow,” and “severe respiratory infection.” The only commonality across scenarios was that parents were not present; the students needed to share how they would respond without consulting their parents. According to the blueprint, two scenarios were familiar and two were unfamiliar (Kintner & Sikorskii, 2009). Two scenarios were simple, whereas two were complex. Each season of the year was represented (i.e., fall, winter, spring, and summer). In two scenarios, the student was taking controller medications as prescribed. In one scenario, the student had stopped or missed taking controller medications for some reason. In the final scenario, the student was taking medications (i.e., antibiotic) in addition to asthma medications. Symptoms (i.e., dry, moist, productive, and nonproductive cough; breathlessness; wheezing; and retractions) varied across scenarios to reflect three levels of severity (i.e., mild, moderate, and severe). Stimuli or triggers (i.e., irritants, allergies, exercise, infection, dust, smoke, and cold air) also varied across scenarios. For each scenario, people other than parents (i.e., friends, neighbors, school personnel, family members, classmates, and health care professionals) were in the area. Each scenario was written to require different self-management techniques (i.e., avoidance, sipping warm fluids, purse-lip or diaphragmatic breathing, and mind–body relaxation). After devising the four scenarios, we needed to determine how the students would respond to each situation.

Think-Aloud Interviews

The think-aloud interviews (Phase 1), cross-sectional pilot study (Phase 3), and longitudinal intervention studies (Phase 4) were conducted in accordance with the Declaration of Helsinki and compliance with the Health Insurance Portability and Accountability Act and Family Educational Rights and Privacy Act. The interviews and studies were approved by three university institutional review boards and two school districts. Written informed consent was obtained from a parent and assent from the student for the interviews. Written informed consent was obtained from a parent for themselves and the student and assent from the student for the cross-sectional and longitudinal studies.

Think-aloud interviews were conducted in 2000 to explore how fifth-grade students use asthma knowledge and personal experience to determine what they would do to manage acute exacerbation of asthma symptoms. Nurses were asked to identify fifth-grade students diagnosed with asthma who the nurses believed might be willing and able to share their thinking about managing their chronic condition. The nurses contacted the parents and offered them fliers about the interviews. Names and contact information of families interested in learning more about the interviews were provided to the principal investigator (PI). Students were recruited from a potential list of interviewees who were currently under a planned medical treatment regime and who had a moderate to severe disease history based on health care provider report. Purposive sampling was used to obtain equal numbers of males and females of different race, culture, ethnic, and socioeconomic backgrounds. Sample size was determined by data saturation when responses were repeating what was already found in previous interviews and no new information was being added to the findings.

Interviews were scheduled after potential participants were contacted by telephone and their verbal consent was obtained. A letter identifying the nature of the study and expectations of the students was mailed along with consent and assent forms to the parents of students with asthma. Interview appointments were scheduled at a convenient time and location for everyone. Seven interviews were conducted in students’ homes, and one interview was conducted in a meeting room at a local hospital. Although within sight and/or sound of the interview, parents were not part of the interview process. All interviews were completed in less than 1 hr.

Interviews were conducted using think-aloud techniques (Davison, Vogel, & Coffman, 1997). The interviewer began by explaining expectations for thinking aloud. Students were informed that the interviewer was going to read four different scenarios that could happen to fifth-grade students with asthma. Students were asked to respond to each of the scenarios by stating all of their thoughts aloud as the thoughts came to mind. The interviewer and students engaged in a series of practice think-aloud exercises in preparation for the actual scenarios. For example, the interviewer posed, “When you arrive home from school, you realize that you are hungry. What would you do?” The audio-recorded interview began when both the interviewer and student were comfortable with the expectations and students demonstrated they were able to think aloud. As each scenario was read, students were asked to verbalize what they would do.

Efforts were taken to gain rich data by establishing trust and rapport and by providing a relaxed environment. Interviewing skills used to facilitate thinking aloud included active listening and expressing nonverbal interest. After responding to all four scenarios, the students were asked to share how they felt about the exercise. Personalized thank you notes were sent following the interviews acknowledging the richness of each student’s contribution.

Data processing began with verbatim transcription of the interviews including pauses, intonations, exclamations, and areas of emphasis. Following transcription, two investigators listened to the tapes while reading the transcripts for accuracy and understanding of the overall flow. Analysis consisted of isolating how students responded when presented with the scenarios.

Trustworthiness (Guba & Lincoln, 1989) of this data was maintained through use of persistent observation. Anecdotal notes provided descriptions of the time, place, and context in which data were collected and analyzed. Two outside reviewers were able to evaluate the process, decisions, and interpretations made.

PHASE 2, DESCRIPTION, ADMINISTRATION, AND SCORING OF THE RAA

Insights about how fifth-grade students might use knowledge and personal experience to determine what they would do to manage asthma symptoms were used to operationalize the reasoning about asthma scenarios. The self-report survey completed by the students was entitled “What I Would Do.” All scenarios were written at an older school-age student’s (Grades 4–6) comprehension level. In addition to the four scenarios used for the think-aloud study, a fifth scenario entitled “visiting friends with pets” was created to serve as an example.

Questions and response options were written and formatted to fit one page. Triggering cues and domain concepts, intermediate summary and actions, and the conclusions were reflected in eight questions and response options. The questions and response options reflected recognizing symptoms, classifying severity, identifying stimuli, selecting helpers, considering medication use, using management techniques, and summarizing experiential learning. Students were directed to read each scenario and then respond to each question by circling the most appropriate response options. For Items 5 and 8, students needed to write their responses in complete sentences using their own words. Response options were preselected for each question of the exemplar scenario to demonstrate how to complete the survey.

We initially conceptualized examining responses to seven discrete items across four scenarios, specifically symptom recognition, severity classification, stimuli identification, helper selection, medication consideration, management technique use, and summary of experiential learning (Kintner & Sikorskii, 2009). However, we realized that rather than focusing on discrete knowledge-based parts across scenarios, we needed to examine how students applied their knowledge and experience within each scenario (Kintner et al., 2012). Using an iterative process that included revisiting data collected during our initial qualitative study (Kintner, 1997), data from the think-aloud interviews, and item analysis of a series of quantitative studies collected more than 5 years (N > 500 students; Kintner, 2004, 2007; Kintner et al., 2012; Kintner & Sikorskii, 2009), we determined that by analyzing discrete parts within each scenario rather than across all four scenarios, we were better able to capture each student’s pattern of overall reasoning ability.

Scoring of the instrument captured logical reasoning abilities for each student within each of the four scenarios or items (Kintner et al., 2012). Individual templates for each scenario were used for determining logical reasoning abilities. Predetermined scores were provided by scenario for each response option to Questions 1–4 and 6–8. Subjects received 0 points for less logical responses, 1 point for more logical responses, and 2 points for most logical responses. Mean scores for responses to Questions 1–4 and 6–8 of each scenario were computed. Question 5 offers the opportunity for students to summarize the scenario using their own words to inform completion of Questions 6–8 and is therefore not included in the scoring. Whereas foundational knowledge in reasoning about asthma management is reflected in responses to Questions 1–4, higher levels of cognitive complexity are reflected in responses to Questions 6–8. Then, a composite mean score rounded to the third decimal point for each scenario was computed to form the four-item scale. Finally, a grand mean score for the four items or scenarios was computed. Grand mean scores were considered to be continuous and had the potential to range from 0.000 to 2.000, with higher scores reflecting increased logical reasoning ability.

METHODS

Phase 3, Preliminary Evaluation

Expert Review

Face validity was assessed by (a) two master’s-prepared certified elementary schoolteachers and (b) one master’s-prepared family nurse practitioner who was also a certified school nurse. Content validity was evaluated by senior nurse researchers specializing in decision making and problem solving related to management of chronic conditions. A standardized form was used to evaluate the scale (Kintner, 1996).

Readability, Comprehension, and Appropriateness

A cross-sectional pilot study was conducted in 2002–2003 to determine the RAA’s readability and age and grade appropriateness. A paper-and-pencil, self-report survey design was used. The convenience sample consisted of English-literate students aged 9–15 years with mild intermittent to severe persistent asthma and their parents. Flyers were offered to families through physicians’ offices and schools. Families interested in learning about the study were directed to contact the PI. Questionnaire packets containing a cover letter, consent and assent forms, and two questionnaire booklets along with a prepaid and pre-addressed return envelope were mailed to families agreeing to participate. In addition to the RAA, students completed four other unrelated instruments. Parents completed four unrelated instruments and the General Health History Survey (GHHS; Kintner, 2004, 2007; Kintner et al., 2012; Kintner & Sikorskii, 2009) for caregivers of students with asthma. This is a 39-item survey used to collect demographic and asthma-related information. Families were called 1 week after the packets were mailed to ask if they had any questions about the study or completion of the instruments. Of the 178 packets mailed, 132 (74%) were returned. Families were offered a monetary cash award of $10 for the return of completed surveys. Statistical Package for Social Sciences (SPSS) 19.0 for Windows (IBM, 2011) was used for data entry, management, and analysis. Mean scores were computed for the instrument. Data were analyzed using descriptive and parametric statistics.

Phase 4, Psychometric Evaluation

Data of three intervention studies conducted from 2005 to 2013 were used for psychometric evaluation of the RAA. Recruitment, enrollment, and descriptions of the samples and data collection methods for intervention studies were reported elsewhere (Kintner et al., 2012; Kintner & Sikorskii, 2009). Data analysis methods and the sensitivity of other instruments being used were considered when determining adequate sample size for the three studies. A minimum sample size of 214 students was required based on equations provided by Kim (2005), the degrees of freedom, and 80% power for rejection of the proposed confirmatory factor model using a root mean square error of approximation (RMSEA) of .05.

Data were collected using audio-linked data entry systems loaded on password-protected and encrypted laptop computers during scheduled home visits. Two evaluators assisted parent and student dyads in completion of their surveys at two time points. Time 1 data were collected prior to participating in an asthma education program, and Time 2 data were collected 1 month following participation in a program. Students completed the RAA and three other unrelated instruments. Both students and parents completed the Knowledge of Asthma Survey (KAS; Kintner, 1996). Parents completed the GHHS (Kintner, 1996, 1997) and three other unrelated instruments. Families were awarded $30 for their time and effort at each time point.

The Knowledge of Asthma Survey

This is an 18-item matching and multiple-choice survey completed by students with asthma and their parents (Kintner, 1996; Kintner & Sikorskii, 2009). Correct responses are summed. Scores have the potential to range from 0 to 20.

General Health History Survey

This is a 39-item self-report survey designed to collect demographic and disease-related information from parents of students diagnosed with asthma (Kintner et al., 2012; Kintner, 1996, 1997, 2004, 2007; Kintner & Sikorskii, 2009). Parent education and occupation items and a family annual income item were used to compute the Nam-Powers Socioeconomic Index Scores (NP-SEIS; Nam & Powers, 1983). Scores for the NP-SEIS have the potential to range from 0 to 99.

Mean scores were computed for the RAA. Summed scores were computed for KAS and NP-SEIS. Descriptive statistics were computed for all variables. Internal consistency reliability of the RAA was evaluated using item-to-total correlations and Cronbach’s alpha correlation coefficient. Reliability assessments were conducted for the entire sample as well as subgroups of the sample based on sex/gender, race/ethnicity, age, grade in schools, and socioeconomic status at Time 1 and Time 2. Construct validity for the entire sample was assessed using exploratory factor analysis at Time 1 and Time 2 and confirmatory factor analysis at Time 2. Concurrent validity was considered using Pearson’s r correlation coefficient to examine associations between student’s reasoning about symptom management and parent and student knowledge of asthma at Time 1 and Time 2. SPSS 19.0 for Windows (IBM SPSS, 2011) was used for determining internal consistency reliability and concurrent and construct validity based on exploratory factor analysis. EQS 6.1 for Windows (Bentler & Wu, 2006) was used for confirmatory factor analysis.

RESULTS

Phase 1, Think-Aloud Interviews

The sample consisted of eight English-speaking, fifth-grade students who followed a planned medical treatment regime for their asthma and who were diagnosed with moderate (n = 5) to severe persistent (n = 3) asthma based on health care provider report. All students were prescribed controller and reliever medications, including inhaled corticosteroids as well as long- and short-acting bronchodilators. Length of time from diagnosis to the date of the interview ranged from less than 1 year to 9 years (M = 5.19, SD = 3.59). Although all of the students resided in the same county, they lived in different school districts and attended different schools.

Students were unique and consistent in their approach to responding to each of the four scenarios. Response patterns were influenced by a wide range of factors, varied from one student to another, and ranged from the most rudimentary conception with no clear differentiation to the most sophisticated levels with hierarchical integration. Students relied primarily on personal experience when responding.

Phase 3, Preliminary Evaluation

Expert Review

Reviewers (N = 5) reported that the scale appeared sound and relevant. The purpose and aims were deemed logically consistent with the final product. The directions were clear and appropriate. The questions were considered grammatically correct, conveying a single thought, clear in meaning, appropriate for response choice, and free of excess wording. Response options were judged to be appropriate for the instrument and arranged in a logical order. Content was deemed logically consistent and highly relevant with the instrument’s theoretical underpinnings.

Readability, Comprehension, and Appropriateness

The convenience for the mailed data collection sample consisted of 132 students diagnosed with asthma aged 9–15 years (M = 11.98, SD = 1.62) enrolled in Grades 3–9 (M = 6.52, SD = 1.59) who were able to read and understand English. Completion rate for all of the reasoning about asthma scenarios was 93.2% (n = 123). When writing in complete sentences and using their own words, more than 85% (n = 117) of the students summarized all of the scenarios (Question 5) and shared experiential learning conclusions (Question 8) for all scenarios. Demographic or asthma-related information revealed no identifiable patterns in students who chose not to complete the survey.

Descriptive statistics for the composite scenario scores and the overall grand mean are presented in Table 1. Scores functioned as predicted before students participated in any formal asthma health education program. For running the mile scenario, mean scores were moderately left or negatively skewed and more peaked indicating that, as predicted, exercise-induced asthma symptoms were more familiar to students.

TABLE 1.

Descriptive Statistics for the Composite RAA Scenario Items and the Grand Mean Scores of Students Who Participated in the Preliminary (Mailed) Evaluation Study

Scenario N Range M SD Skewness Kurtosis
1. Grilling in the park 132 0.474–1.873 1.410 0.303 −0.718 −0.029
2. Running the mile 123 0.651–1.858 1.518 0.206 −0.920 1.545
3. Playing in the snow 123 0.483–1.826 1.306 0.253 −0.247 0.055
4. Respiratory infection 125 0.488–1.865 1.364 0.323 −0.637 −0.389
RAA grand mean 132 0.805–1.752 1.396 0.194 −0.656 0.201
Open in a new tab

Note. RRA = Reasoning About Asthma.

Students were grouped by sex/gender, race, age, grade, and socioeconomic status for comparison. Cross-group comparisons for the composite scenario scores and the grand mean scores are presented in Tables 2–4. For the composite scenario scores, no significant differences were found between Black and non-Hispanic White students for the grilling in the park and running the mile scenarios; however, significant differences were indicated between Black and non-Hispanic White students for the playing in the snow (p = .012) and severe respiratory infection scenarios (p = .003). For the grand mean scores, no significant differences in scores were found based on sex/gender, age group, or grade in school; however, significant differences in grand mean scores were found between students of Black and non-Hispanic White origins (p = .014). Although statistically significant differences were initially indicated between the lowest and highest socioeconomic groups (p = .013), when using the harmonic mean for a sample of 28 to account for unequal group sizes, post hoc analysis using Tukey, Duncan, and Scheffé failed to reach significance at the p < .05 level.

TABLE 2.

Cross-Group Comparisons of RAA Scores Between Males and Females Who Participated in the Preliminary (Mailed) Evaluation Study

Scores Males
(n = 71, 54%)		 Females
(n = 61, 46%)		 t df p
M SD M SD
1. Grilling in the park 1.414 0.303 1.407 0.304 0.132 130 .896
2. Running the mile 1.495 0.222 1.544 0.187 −1.310 121 .193
3. Playing in the snow 1.276 0.228 1.339 0.276 −1.384 121 .169
4. Respiratory infection 1.330 0.299 1.403 0.347 −1.258 123 .211
RAA grand mean 1.379 0.183 1.415 0.207 −1.055 130 .294
Open in a new tab

Note. RAA = Reasoning About Asthma.

TABLE 4.

Cross-Group Comparisons of RAA Grand Mean Scores Based on Age Group, Grade in School, and Socioeconomic Status of Students Who Participated in the Preliminary (Mailed) Evaluation Study

Groupings N M SD Sum of
Squares		 df Mean
Square		 F p
Age group
  9–10 years 29 1.317 0.199 Between 0.312 4 .078 2.145 .079
  11 years 21 1.362 0.243 Within 4.623 127 .036
  12 years 24 1.440 0.164 Total 4.936 131
  13 years 37 1.430 0.182
  14–15 years 21 1.427 0.162
  Total 132 1.396 0.194
Grade in school
  Third and fourth 16 1.301 0.246 Between 0.340 4 .085 2.346 .058
  Fifth 24 1.374 0.198 Within 4.596 127 .036
  Sixth 18 1.343 0.221 Total 4.936 131
  Seventh 34 1.420 0.170
  Eighth and ninth 40 1.449 0.161
  Total 132 1.396 0.194
Socioeconomic status
  Lower 0–49 points 34 1.309* 0.233 Between 0.420 3 .140 3.748 .013*
  Low middle 50–69 points 30 1.411 0.170 Within 4.180 122 .037
  Upper middle 70–89 points 29 1.421 0.182 Total 4.599 115
  Upper 90–99 points 23 1.473* 0.169
  Total 116 1.396 0.200
Open in a new tab
*

p value significance <.05; however, the group sizes were unequal. When using the harmonic mean sample size of 28, post hoc p values using Tukey, Duncan, and Scheffe were not significant.

Phase 4, Psychometric Evaluation

Internal Consistency Reliability

The combined sample for the evaluator-monitored data collection consisted of 307 students diagnosed with asthma aged 9–13 years (M = 10.3, SD = 0.9) and their parents. Item and scale summary of the RAA for the entire sample at Time 1 and Time 2 are presented in Table 5. Item-to-total correlations (Time 1, r = .51–.63), alpha if item deleted (Time 1, 0.68–0.74), and Cronbach’s standardized alpha correlation coefficient for internal consistency for the entire sample at Time 1 (α = 0.77) and Time 2 (α = 0.79) for this new instrument provided strong evidence of internal consistency reliability. Scale and item summaries citing the number of subjects, minimum and maximum scores, means, standard deviations, item-to-total correlations, and alpha if item deleted for subgroups based on sex/gender, race/ethnicity, age, grade in school, and socioeconomic status at Time 1 and Time 2 are presented in Tables 6–7.

TABLE 5.

RAA Item and Scale Summary, Internal Consistency Reliability, and Exploratory Factor Analysis at Time 1 and Time 2 of Students Who Participated in the Psychometric (Monitored) Evaluation Study

N Range M SD Internal Consistency
Reliability		 Exploratory Factor Analysis:
Principal Components
Item-to-Total
Correlation		 Scale Alpha
if Item
Deleted		 KMO
Sampling
Adequacy		 Rescaled
Factor
Loading		 Rescaled
Amount of
Variance
Explained
Preprogram
  Grilling in the park 295 0.389–1.894 1.105 0.290 .576 0.699 .806
  Running the mile 295 0.681–1.770 1.236 0.222 .632 0.678 .754
  Playing in the snow 295 0.500–1.757 1.039 0.232 .571 0.704 .712
  Respiratory infection 295 0.577–1.846 1.125 0.302 .506 0.744 .785
  Reasoning about asthma 295 0.500–1.568 1.122 0.205 .506–.632 0.773 .709 .712–.806 58.524
1 month postprogram
  Grilling in the park 280 0.000–1.929 1.299 0.326 .616 0.713 .861
  Running the mile 280 0.637–1.882 1.398 0.240 .679 0.688 .797
  Playing in the snow 280 0.598–1.864 1.221 0.251 .548 0.744 .688
  Respiratory infection 280 0.611–1.935 1.410 0.299 .531 0.755 .751
  Reasoning about asthma 280 0.000–1.825 1.326 0.232 .531–.679 0.788 .770 .688–.861 60.323
Open in a new tab

Note. KMO = Kaiser-Meyer-Olkin.

TABLE 6.

RAA Scenario Score Ranges, Means, and Standard Deviations and Internal Consistency Reliability by Sex/Gender, Age, Grade in School, Race/Ethnicity, and Socioeconomic Status at Time 1 of Students Who Participated in the Psychometric (Monitored) Evaluation Study

N Range M SD Internal Consistency
Reliability
Item-to-Total
Correlation		 Standardized
Alpha
Sex/gender
  Male 163 0.500–1.553 1.111 0.200 .505–.616 0.761
  Female 132 0.726–1.568 1.136 0.211 .506–.648 0.786
  Total 295
Age in years
  9–10 187 0.726–1.553 1.125 0.190 .462–.613 0.749
  11 82 0.500–1.568 1.129 0.217 .580–.651 0.797
  12–13 26 0.669–1.558 1.083 0.264 .575–.741 0.841
  Total 295
Grade in school
  Fourth 81 0.726–1.553 1.119 0.198 .496–.652 0.789
  Fifth 163 0.500–1.501 1.115 0.193 .451–.555 0.726
  Sixth 43 0.750–1.568 1.116 0.233 .565–.758 0.835
  Seventh 8 0.669–1.548 1.101 0.328 .680–.959 0.931
  Total 295
Race/ethnicity
  non-Hispanic Black 109 0.669–1.557 1.074 0.203 .558–.681 0.800
  non-Hispanic White 73 0.500–1.568 1.157 0.220 .516–.668 0.773
  Mixed Black/White 58 0.799–1.483 1.127 0.183 .418–.550 0.713
  Hispanic/Latino 28 0.785–1.498 1.150 0.209 .492–.627 0.775
  Total 268
Socioeconomic status
  Lower (0–49) 157 0.500–1.568 1.096 0.203 .476–.631 0.769
  Low middle (50–69) 104 0.669–1.558 1.142 0.197 .484–.587 0.753
  Upper middle (70–89) 24 0.785–1.557 1.212 0.220 .537–.704 0.816
  Upper (90–99) 9 0.797–1.462 1.119 0.234 .314–.888 0.782
  Total 294
Open in a new tab

Note. RAA = Reasoning About Asthma.

TABLE 7.

RAA Scenario Score Ranges, Means, and Standard Deviations and Internal Consistency Reliability by Sex/Gender, Age, Grade in School, Race/Ethnicity, and Socioeconomic Status at Time 2 of Students Who Participated in the Psychometric (Monitored) Evaluation Study

N Range M SD Internal Consistency
Reliability
Item-to-Total
Correlation		 Standardized
Alpha
Sex/gender
  Male 157 0.824–1.802 1.320 0.217 .484–.671 0.780
  Female 123 0.000–1.825 1.334 0.050 .476–.687 0.796
  Total 280
Age in years
  9–10 182 0.732–1.684 1.319 0.212 .498–.676 0.774
  11 75 0.000–1.783 1.326 0.271 .538–.678 0.799
  12–13 23 0.928–1.825 1.381 0.241 .565–.818 0.842
  Total 280
Grade in school
  Fourth 78 0.732–1.681 1.307 0.209 .453–.699 0.766
  Fifth 152 0.000–1.692 1.303 0.237 .528–.654 0.775
  Sixth 42 0.896–1.783 1.411 0.219 .544–.704 0.807
  Seventh 8 1.031–1.825 1.499 0.265 .813–.972 0.950
  Total 280
Race/ethnicity
  non-Hispanic Black 102 0.824–1.718 1.280 0.222 .488–.690 0.788
  non-Hispanic White 68 0.000–1.825 1.341 0.286 .599–.677 0.825
  Mixed Black/White 59 0.757–1.802 1.356 0.203 .548–.640 0.780
  Hispanic/Latino 9 0.835–1.642 1.326 0.204 .405–.712 0.801
  Total 238
Socioeconomic status
  Lower 145 0.732–1.668 1.291 0.228 .534–.701 0.799
  Low middle 101 0.000–1.825 1.374 0.231 .416–.605 0.724
  Upper middle 25 0.923–1.783 1.369 0.247 .639–.749 0.850
  Total 271
Open in a new tab

Note. RAA = Reasoning About Asthma.

Construct Validity

Exploratory factor analysis using principal components extraction method from the covariance matric with no rotation indicated a single component solution. Table 5 presents an overall scale summary for the combined sample at Time 1 and Time 2. With strong component loadings of .69–.86 on a single factor, the amount of variance explained at Time 2 was 60%. The Kaiser-Meyer-Olkin (KMO) measure was greater than .70, indicating an adequate sample size. Statistically significant correlations (p < .05) between student reasoning about asthma management and parent and student asthma knowledge at Time 1 and Time 2 are presented in Table 8. Confirmatory factor analysis of data at Time 2 terminated normally in four iterations with parameter estimates appearing in order and no special problems encountered during optimization (see Figure 1). A non-significant chi-square, the comparative fit index (CFI = 0.99), and the root mean square error of approximation (RMSEA = .06 with a 90% confidence interval between .00 and .15) indicated a good fit between the data and the model.

TABLE 8.

Pearson r Correlation Coefficients and Level of Significance Between Reasoning Items, Reasoning About Asthma, and Parent and Student Knowledge of Asthma at Time 1 and Time 2 of Students Who Participated in the Psychometric (Monitored) Evaluation Study

1 2 3 4 5 6 7
1. Grilling in the park — .63** .40** .39** .79** .11 .08
2. Running the mile .62** — .50** .37** .79** .16** .04
3. Playing in the snow .44** .52** — .49** .76** .06 .13*
4. Severe respiratory infection .45** .46** .41** — .76** .12* .16*
5. Reasoning about asthma score .83** .82** .73** .75** — .14* .13*
6. Parent knowledge of asthma score .15* .27** .05 .15* .20** — .08
7. Student knowledge of asthma score .28** .29** .22** .33** .31** .14* —
Open in a new tab

Note. Intercorrelations for participants at Time 1 or baseline (n = 304) are presented above the diagonal, and intercorrelations for participants (n = 283) at Time 2 1 month following participation in an education program are presented below the diagonal.

*

p < .05.

**

p < .01

Figure 1.

Figure 1

Open in a new tab

Confirmatory factor model for the RAA. The confirmatory factor model for the RAA scenarios indicates goodness of fit between the data and the specified model.

Note. e = error.

DISCUSSION

This article described development and evaluation of a four-scenario instrument designed to measure logical reasoning ability for managing exacerbation of asthma symptoms during acute episodes. Existing literature informed development of the reasoning template that was useful in laying the foundation for creating the asthma scenarios. Think-aloud interviews revealed how fifth-grade students managed symptoms using knowledge gained through personal experiences. Each student had a pattern of response across various scenarios that provided a more complete picture of his/her ability to manage asthma symptoms. This revelation influenced the scoring method for the instrument. We determined that by examining seven discrete parts of knowledge within the four scenarios, we were able to capture students’ overall logical reasoning ability.

Face and content validity of the RAA were confirmed by expert reviewers. The paper-and-pencil, cross-sectional study determined readability/comprehension and age/grade appropriateness. The scoring method offered a continuous variable with normal distribution curves for use in comparing scores based on demographic characteristics.

Psychometric evaluation of the RAA for students with asthma demonstrated internal consistency reliability and construct and concurrent validity for the combined sample and subsamples. Internal consistency reliability was good. Corrected item-to-total correlations for the combined sample and subgroups were more than adequate. Cronbach’s standardized alpha correlation coefficient was higher for older students than younger students, females than males, and upper socioeconomic than lower groups; although not to the degree that would create barriers for use. Reliability estimates were higher for Black than non-Hispanic White Americans before participating in an education program and higher for non-Hispanic White than Black Americans following participation in a program. Exploratory and confirmatory factor analyses provided strong support for the extent to which the instrument actually reflects the construct. Support for relationships between reasoning about asthma management and parent and student asthma knowledge confirmed concurrent validity.

The RAA offers a more objective alternative to perceived self-perceptions of decision-making and problem-solving ability that have been used in the past (Murdock et al., 2010; Pulgaron et al., 2010; Rhee et al., 2008). For example, positive expectation about the ability to solve problems functioned as a liability in highly stressed situations for students who self-reported the highest level of problem-solving efficacy (Murdock et al., 2010). The RAA could be used to assess the association between subjective and objective reasoning ability. Considered to be a more objective measure, the RAA might be better able to detect differences between groups or changes over time when evaluating interventions.

Findings of this study are consistent with cognitive development and complexity and complement the work of McQuaid et al. (2003) who investigated reasoning about the causal mechanism of asthma episodes. The RAA captured logical reasoning abilities across a spectrum of complexity from less logical to more and most logical. Logical reasoning abilities related to managing symptoms appear to be related to age of the student and grade in school, with higher scores for older students than younger students and for higher versus lower grade levels.

We acknowledge inherent limitation of paper-and-pencil, self-report surveys mailed to subjects who participated in the pilot study. We were unable to guarantee that students completed their surveys without the assistance of their parents. Using trained personnel to monitor and support student completion of the survey for the intervention studies was beneficial in safeguarding that students completed their surveys without assistance. Data collectors reported that students appeared to take pleasure in using their reasoning abilities to complete this survey and benefited from thinking about what they would do and what they could learn. We also acknowledge oversampling of Black Americans and students of single-parent households and lower socioeconomic groups. Additional research is needed to further explore reliability and validity of the RAA for more diverse groups including students of Hispanic or Latino backgrounds.

Clinicians and researchers are exploring the roles of decision making and problem solving (Adams, Smith, & Ruffin, 2001; Burkhart & Ward, 2003; McQuaid et al., 2003; Wade, Holden, Lynn, Mitchell, & Ewart, 2000; Wade et al., 1997; Wroe, 2002) and reasoning and information processing (Burkhart & Ward, 2003; Comino, Zwar, & Harris, 2002; Jaing, Sepulveda, & Casillas, 2001; Rich, Patashnick, & Chalfen, 2002; Ringsberg, Lepp, & Finnström, 2002; Wade et al., 2000) in asthma management. Findings of this report confirmed face and content validity, readability and comprehension, age and grade appropriateness, and reliability and construct validity of the RAA for students with asthma. The instrument could be used as a more objective measure in clinical settings to assess logical reasoning about managing exacerbations of asthma symptoms and in research settings to evaluate interventions designed to foster logical reasoning in older school-age children and adolescents with asthma.

TABLE 3.

Cross-Group Comparisons of RAA Scores Between Non-Hispanic Black and White Students Who Participated in the Preliminary (Mailed) Evaluation Study

Scenarios Non-Hispanic
Black
(n = 38, 29%)		 Non-Hispanic
White
(n = 80, 61%)		 t df p
M SD M SD
1. Grilling in the park 1.344 0.344 1.431 0.262 −1.523 116 .131
2. Running the mile 1.496 0.259 1.524 0.188 −0.648 108 .518
3. Playing in the snow 1.210 0.299 1.342 0.222 −2.544 108 .012*
4. Respiratory infection 1.239 0.326 1.435 0.278 −3.109 60a .003*
RAA grand mean 1.326 0.224 1.428 0.158 −2.531 55a .014*
Open in a new tab
a

Equal variances are not assumed.

*

p value significance <.05.

Acknowledgments

This research was funded in part by grants from the National Institutes of Health (Institutional National Research Service Award, Postdoctoral Nursing Research Fellowship, National Institute of Nursing Research, T32 NR07073-06 Health Promotion Risk Prevention, R21 NR009517 Staying Health—Asthma Responsible and Prepared™, and RO1 RN010544 Comparison of Asthma Programs for Schools), the University of Michigan Office of the Vice President for Research, the Michigan Department of Education State Teen Health Center Grant, the University of Michigan School of Nursing Center for Adolescent Health Research Joy Baxt Endowment Funds, and the Ed and Molly Smith Centennial Fellowship at the University of Texas at Austin School of Nursing.

We acknowledge study participants for their contribution in development of the instrument, school personnel involved in the recruitment of subjects, and the parent and student dyads for their time and effort in completing the surveys. Contact the first author to request an electronic document containing the reasoning templates, scenario scripts, illustrations, survey items, response options, and scoring guidelines.

Contributor Information

Eileen Kintner, The University of Texas at Austin.

Gwendolyn Cook, The University of Texas at Austin.

Lakisha Hull, City of Los Angeles.

Linda Meeder, Michigan Department of Community Health.

References

  1. Adams RJ, Smith BJ, Ruffin RE. Patient preferences for autonomy in decision making in asthma management. Thorax. 2001;56(2):126–132. doi: 10.1136/thorax.56.2.126. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Akinbami LJ, Moorman JE, Liu X. Asthma prevalence, health care use, and mortality: United States, 2005–2009. National Health Statistics Reports. 2011;(32):1–14. [PubMed] [Google Scholar]
  3. Bandura A. Social foundations of thought and action: A social cognitive theory. Englewood Cliffs, NJ: Wadsworth Publishing; 1986. [Google Scholar]
  4. Bentler PM, Wu EJC. EQS 6.1 for Windows. Encino, CA: Multivariate Software; 2006. [Google Scholar]
  5. Bransford JD, Stein BS. The ideal problem solver. New York, NY: W. H. Freeman; 1984. [Google Scholar]
  6. Burkhart PV, Ward HJ. Children’s self-reports of characteristics of their asthma episodes. Journal of Asthma. 2003;40(8):909–916. doi: 10.1081/jas-120023583. [DOI] [PubMed] [Google Scholar]
  7. Comino E, Zwar N, Harris M. Management of children after presenting to hospital with acute asthma but not requiring admission. Australian Family Physician. 2002;31(5):494–496. [PubMed] [Google Scholar]
  8. Davison GC, Vogel RS, Coffman SG. Think-aloud approaches to cognitive assessment and the articulated thoughts in simulated situations paradigm. Journal of Consulting and Clinical Psychology. 1997;65(6):950–958. doi: 10.1037//0022-006x.65.6.950. [DOI] [PubMed] [Google Scholar]
  9. Guba EG, Lincoln YS. Fourth generation evaluation. Newbury Park, CA: Sage; 1989. [Google Scholar]
  10. Heermann JA, Wills LM. Effect of problem-solving instruction and health locus of control on the management of childhood asthma. Children’s Health Care. 1992;21(2):76–83. [Google Scholar]
  11. IBM. IBM SPSS 19.0 for Windows. Armonk, NY: Author; 2011. [Google Scholar]
  12. Jaing JT, Sepulveda JA, Casillas AM. Novel computer-based assessment of asthma strategies in inner-city children. Annals of Allergy, Asthma & Immunology. 2001;87(3):230–237. doi: 10.1016/S1081-1206(10)62232-6. [DOI] [PubMed] [Google Scholar]
  13. Kim KH. The relation among fit indexes, power, and sample size in structural equation modeling. Structural Equation Modeling. 2005;12:368–390. [Google Scholar]
  14. Kintner EK. Doctoral dissertation. 1996. Testing of the school-aged child and adolescent acceptance of asthma model. Document Number: UMI Order PUZ9720695, Accession Number: 2000013828. [Google Scholar]
  15. Kintner EK. Adolescent process of coming to accept asthma: A phenomenological study. Journal of Asthma. 1997;34(6):547–561. doi: 10.3109/02770909709055399. [DOI] [PubMed] [Google Scholar]
  16. Kintner EK. Lack of relationship between acceptance and knowledge of asthma in school-age children and early adolescents. Journal for Specialists in Pediatric Nursing: JSPN. 2004;9(1):5–14. doi: 10.1111/j.1088-145x.2004.00005.x. [DOI] [PubMed] [Google Scholar]
  17. Kintner EK. Testing the acceptance of asthma model with children and adolescents. Western Journal of Nursing Research. 2007;29(4):410–431. doi: 10.1177/0193945907299657. [DOI] [PubMed] [Google Scholar]
  18. Kintner E, Cook G, Allen A, Meeder L, Bumpus J, Lewis K. Feasibility and benefits of a school-based academic and counseling program for older school-age students with asthma. Research in Nursing and Health. 2012;35(5):507–517. doi: 10.1002/nur.21490. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Kintner EK, Sikorskii A. Randomized clinical trial of a school-based academic and counseling program for older school-age students. Nursing Research. 2009;58(5):321–331. doi: 10.1097/NNR.0b013e3181b4b60e. [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Lerner RM. Nature, nurture, and dynamic interactionism. Human Development. 1978;21:1–20. [Google Scholar]
  21. McQuaid EL, Howard K, Kopel SJ, Rosenblum K, Bibace R. Developmental concepts of asthma: Reasoning about illness and strategies for prevention. Applied Developmental Psychology. 2002;23:179–194. [Google Scholar]
  22. McQuaid EL, Kopel SJ, Klein RB, Fritz GK. Medication adherence in pediatric asthma: Reasoning, responsibility, and behavior. Journal of Pediatric Psychology. 2003;28(5):323–333. doi: 10.1093/jpepsy/jsg022. [DOI] [PubMed] [Google Scholar]
  23. Murdock KK, Greene C, Adams SK, Hartmann W, Bittinger S, Will K. The puzzle of problem-solving efficacy: Understanding anxiety among urban children coping with asthma-related and life stress. Anxiety, Stress, and Coping. 2010;23(4):383–398. doi: 10.1080/10615800903118049. http://dx.doi.org/10.1080/10615800903118049. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Nam CB, Powers MG. The socioeconomic approach to status measurement. Houston, TX: Cap & Gown Press; 1983. [Google Scholar]
  25. Narayan SM, Corcoran-Perry S. Line of reasoning as a representation of nurses’ clinical decision making. Research in Nursing & Health. 1997;20(4):353–364. doi: 10.1002/(sici)1098-240x(199708)20:4<353::aid-nur8>3.0.co;2-f. [DOI] [PubMed] [Google Scholar]
  26. National Institutes of Health. National Heart, Lung, and Blood Institute: Expert panel report 3—Guidelines for the diagnosis and management of asthma. Washington, DC: U.S. Department of Health and Human Services; 2007. Retrieved from http://www.nhlbi.nih.gov/guidelines/asthma/asthgdln.htm. [Google Scholar]
  27. Pulgaron ER, Salamon KS, Patterson CA, Barakat LP. A problem-solving intervention for children with persistent asthma: A pilot of a randomized trial at a pediatric summer camp. Journal of Asthma. 2010;47(9):1031–1039. doi: 10.1080/02770903.2010.514633. http://dx.doi.org/10.1080/02770903.2010.514633. [DOI] [PubMed] [Google Scholar]
  28. Rhee H, Hollen PJ, Belyea MJ, Sutherland MA. Decision-making program for rural adolescents with asthma: A pilot study. Journal of Pediatric Nursing. 2008;23(6):439–450. doi: 10.1016/j.pedn.2008.01.079. http://dx.doi.org/10.1016/j.pedn.2008.01.079. [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Rich M, Patashnick J, Chalfen R. Visual illness narratives of asthma: Explanatory models and health-related behavior. American Journal of Health Behavior. 2002;26(6):442–453. doi: 10.5993/ajhb.26.6.5. [DOI] [PubMed] [Google Scholar]
  30. Ringsberg KC, Lepp M, Finnström B. Experiences by patients with asthma-like symptoms of a problem-based learning health education programme. Family Practice. 2002;19(3):290–293. doi: 10.1093/fampra/19.3.290. [DOI] [PubMed] [Google Scholar]
  31. Santrock JW. Adolescence. 13. Boston, MA: McGraw-Hill; 2010. [Google Scholar]
  32. Sugarman L. Life-span development: Concepts, theories, and interventions. New York, NY: Routledge; 1986. [Google Scholar]
  33. Wade SL, Holden G, Lynn H, Mitchell H, Ewart C. Cognitive-behavioral predictors of asthma morbidity in inner-city children. Journal of Developmental and Behavioral Pediatrics: JDBP. 2000;21(5):340–346. doi: 10.1097/00004703-200010000-00004. [DOI] [PubMed] [Google Scholar]
  34. Wade S, Weil C, Holden G, Mitchell H, Evans R, III, Kruszon-Moran D, Wedner HJ. Psychosocial characteristics of inner-city children with asthma: A description of the NCICAS psychosocial protocol. National Cooperative Inner-City Asthma Study. Pediatric Pulmonology. 1997;24(4):263–276. doi: 10.1002/(sici)1099-0496(199710)24:4<263::aid-ppul5>3.0.co;2-l. [DOI] [PubMed] [Google Scholar]
  35. Wroe AL. Intentional and unintentional nonadherence: A study of decision making. Journal of Behavioral Medicine. 2002;25(4):355–372. doi: 10.1023/a:1015866415552. [DOI] [PubMed] [Google Scholar]

RESOURCES