Table 1.

Proposed Delphi Flow Process for Treatment Adaptation

Delphi Procedure	Example	Action/Outcome/Deliverable
Step 1. Identify Research Question/ Problem Statement
• Need to modify content/context of treatment	Adapt “enhanced” version of PST-PC:	Target deliverable:
	• Incorporate compensatory cognitive skills training	• Adapted treatment manual
	• Tailor to veterans with history of concussion
	• Feasible for delivery in integrated primary care
Step 2. Identify & Select SMEs
• Specify method (non-random preferred)	Purposeful sampling	7 SME candidates contacted
• Specify expertise needed	Three essential content domains:	All agreed to participate:
	• Concussion rehabilitation	• 2 neuropsychologists
	• Integrated primary care	• 2 psychiatrists, 1 psychologist
	• Brief problem-solving interventions	• 2 psychologists a
Step 3. Establish Data Collection & Analytic Procedures
• Specify design (quantitative/qualitative, mixed)	Mixed-method (sequential, convergent) Delphi study	Qualitative interview + survey
• Specify analytic paradigm	Rapid analysis (qualitative), descriptive statistics (quantitative)	-
• Operationally define consensus	≥ 80% Agreement (content is “acceptable as-is”)	-
• Establish communication medium	Remote, anonymous to one another but not study team	1:1 Email, telephone calls
• Create stimulus materials	Materials vary by the goal of each round:	Study team created:
	• Prepare and orient SMEs to role	• Introductory materials (study overview/ rationale, role induction), content primers (clinical practice guideline excerpts, original PST-PC manual, outline of proposed content changes)
	• Collect data (acceptability, feasibility, integrity)	Study team created:
	• Interview	• Round 1: 11-item semi structured interview
	• Surveys	• Round 1: 7-item professional background survey;
		Round 2: 58-item survey & 5 open-ended questions; Round 3: 5-item survey
	• Data summary sheets	Study team created:
	• Qualitative	• Bulleted list of critical feedback (rapid analysis)
	• Quantitative	• Summary of forced-choice ratings (descriptives)
Step 4. Collect & Analyze Data
• Multi-round feedback loop	Planned 3-4 rounds:	3 Rounds needed:
	• Interview, Survey x 2, Teleconference	• Interview, Survey x 2
• Gather/ consolidate actionable data (varies by round)	R1	R1
	• Outgoing to SMEs: Introductory materials (study overview/ rationale, role induction), content primers (clinical practice guideline excerpts, original PST-PC manual, outline of proposed content changes), professional background survey	• Distributed materials to 7 SMEs
	• Incoming from SMEs: Professional background survey, interview feedback	• Interviewed 7 SMEs
		• Summarized background data
		• Created R1 feedback summary
	• Interim edits	• Drafted treatment manual
		• Created 58-item R2 survey
	R2	R2
	• Outgoing to SMEs: R1 feedback summary, first draft of treatment manual, R2 survey	• Distributed materials to 7 SMEs
	• Incoming from SMEs: R2 survey with open text feedback	• Received surveys from 5 SMEs
		• Created R2 feedback summary:
		• 91.4% of items reached consensus
		• 8.6% required edits to content (e.g., additions, clarifications)
	• Interim edits	• Revised treatment manual
		• Created 5-item R3 survey
	R3	R3
	• Outgoing to SMEs: R2 feedback summary, second draft of treatment manual, R3 survey	• Distributed materials
	• Incoming from SMEs: R3 survey	• Received surveys from 5 SMEs
		• Created R3 feedback summary:
		• 100% consensus
	• Close feedback loop	• Distributed R3 feedback summary
		• Finalized treatment manual
Step 5. Decision-Making
• Final assessment (e.g., further study v. implement)	Evaluate sufficiency/ quality of product	Study team determined sufficient evidence exists to move forward with additional study.
• Next steps	Prepare and use final deliverable	Adapted treatment manual is now being used in a clinical trial.

Note:

^aPsychologists in this category were also considered integrated primary care experts. PST-PC = Problem-Solving Training in Primary Care; R1 = Round 1; R2 = Round 2; R3 = Round 3; SME = subject matter expert.