Table 1.

Delphi Panel Review

Key steps in the review process Step 1. Panelists independently reviewed and rated each measure on four criteria and provided written comments Step 2. We analyzed ratings, synthesized comments, and provided summary findings to panelists Step 3. Panel met by teleconference to discuss measures for which ratings did not show consensus among panelists and measures with strong ratings for all criteria except feasibility Step 4. Panelists independently rerated each measure on four criteria and provided written comments Evaluation criteria used in Delphi Panel Review Usefulness: The measure can be used to monitor and inform quality improvement efforts aimed at improving organizational health literacy Meaningfulness: The measure assesses a component of organizational health literacy that is meaningful to key stakeholders (e.g., patients, clinicians, administrators) Face validity: The measure appears to capture the construct it is designed to assess Feasibility: The measure can be computed with accuracy and implemented in a timely manner, without undue burden Classifying the degree of consensus among panelists Consensus: ≤2 ratings deviated from the median score by ≥1.5 points Lack of consensus: ≥3 ratings occurred in each tail of the rating scale (i.e., ≥3 ratings of 1 or 2 and ≥3 ratings of 4 or 5) Inconclusive: Ratings did not meet the criteria for consensus or lack of consensus