Table 1.
Appraisal Criteria as well as the Grading System for Each Criterion as it Applies to Pediatric Audiology
| Characteristic | Description | Grade criteria |
|---|---|---|
| Conceptual clarity | Tool covers relevant domains intended to be measured (e.g., detection, localization, speech understanding) | A = Completely covered |
| B = Adequately covered | ||
| C = Inadequately covered | ||
| Norms and standard values | Large scale normative data for infants and children with normal hearing and PCHI. Experimental data collected using the tool is also considered given the lack of large scale norms available | Published data are available from: |
| A = A large number infants and children with normal hearing and with PCHI who wear hearing aids | ||
| B = A large number of infants and children with normal hearing | ||
| C = Experimental data using the tool with infants and children with normal hearing and PCHI who wear hearing aids | ||
| Measurement model | There should not be ceiling or floor effects in measurement, particularly when used to measure the abilities of children with hearing loss | A = No issues |
| B = Few or marginal evidence of skewing | ||
| C = Substantial skewing | ||
| Item/instrument bias | The tool, and items within it, must not show evidence of bias when used with children who have PCHI. Bias-free tools have been evaluated on population subgroups and/or have had the response scale of the tool evaluated with Rasch analysis | A = Tool/items have been reviewed by parents of children with PCHI and acceptability is published OR Rasch analysis is good |
| B = Adequate face validity to support low bias OR factor analysis is good/Rasch analysis shows some issues | ||
| C = Bias is evident or tested OR inadequate statistical analysis | ||
| Respondent burden | The tool should be brief and clear enough for the caregiver to complete. The terminology used should not be offensive to those with hearing loss or deafness | The tool is: |
| A = Brief (≤ 15 min) and has high acceptability for caregiver | ||
| B = Either appropriately longer or some reported problems of acceptability | ||
| C = Lengthy and acceptability is problematic | ||
| Administrative burden | The tool should be easy to administer, score, and interpret | The tool is: |
| A = Scored by hand and the resulting metric is relevant and interpretable for the clinician and caregiver | ||
| B = Scored by a computer and interpretation is obscure | ||
| C = Costly and complex scoring; interpretation by another professional required | ||
| Reliability | The tool should give consistent results, within itself, and across time and testers | Internal consistency coefficient alpha: A ≥ .80; B < .80, >.70; C < .70 |
| Retest intraclass correlation coefficient: A ≥ .75; B > .40, < .75; C ≤.40 | ||
| Discriminant validity | The scores should differ for two subgroups of the population who would be expected to have different scores (e.g., normal hearing vs. hearing impaired children, on some items related to hearing) | A = Strong, expected direction, supported by clinical evidence |
| B = Moderate or conflicting evidence | ||
| C = Weak or based solely on statistical evidence | ||
| Convergent (criterion-related) validity | The tool should have been validated against a gold-standard measure, and/or the subscale structure of the tool has been statistically evaluated | A = Correlation of ≥ .60; confirmed factor structure |
| B = Correlations of > .30, <.60; few problems with factor structure | ||
| C = Correlation of ≤ .30; weak or not confirmed factor structure | ||
| Ecological validity* | The tool evaluates the child’s responses within the context of specific, realistic environments and assesses the child as an active participant | A = Specific, realistic environments assessed |
| B = Some situations are applicable and realistic for the child | ||
| C = Environments are unrealistic and nonspecific | ||
| Responsiveness | The scores on this tool have been shown to change, in the expected direction, when important changes are made to hearing status, hearing aid intervention, or therapy | Criteria for change are: |
| A = Strong, supported by patient and clinical evidence | ||
| B = Moderate or conflicting evidence | ||
| C = Weak or based solely on statistical evidence | ||
| Alternate/accessible forms | The tool has been experimentally evaluated for use with different administration formats (e.g., paper and pencil vs. computer-assisted vs. interview-format administration) | A = Appropriate or varied modes are available and have been tested |
| B = Some accommodations or testing among caregiver of children with PCHI | ||
| C = No accommodations or mode information for special groups | ||
| Culture/language adaptations | The tool has been adapted and reevaluated for use with different languages and/or cultures (e.g., translations, use within deaf culture, with those who are deaf/blind) | A = Evidence of testing and applicability for cultural subgroups and interpretations |
| B = Evidence of translations or testing with subgroups; some problems | ||
| C = No evidence of testing or applicability to groups |
Source: Adapted from Andresen (2000).
*
Not from Andresen (2000) criteria