Table 4.
HDHK process evaluation procedures
| Process Evaluation Question | Data Sources | Tools/Procedures | Timing of Data Collection | Data Analysis or Synthesis | Reporting | |
|---|---|---|---|---|---|---|
| Fidelity | To what extent was the intervention implemented consistently with the underlying theory and philosophy? # | Facilitators HDHK research staff |
* Independent self-reported checklist from Lead and Co Facilitator regarding the amount of pre-prepared content delivered and any non-delivered content * Observation checklist of delivered content from independent observer to validate facilitator self-report checklist |
* At the conclusion of each session (Checklists will be faxed to research staff post-session) * At least two observations per seven session program |
* Calculated score based on percentage of intended content included * Percentage difference score of researcher versus facilitator average of intended content included |
* Formative - informal feedback to staff prior to the next intervention session * Summative - reported after the program is complete |
| Dose delivered | To what extent were all intervention components implemented? | Facilitators HDHK research staff |
* Independent self-reported checklist from Lead and Co Facilitator regarding the amount of pre-prepared content delivered and any non-delivered content (theoretical and practical) * Observation checklist of delivered content from independent observer to validate facilitator self-report checklist |
* At the conclusion of each session (Checklists will be faxed to research staff post-delivery) * At least two observations per seven session program |
* Calculated score based on percentage of intended content included * Percentage difference score of researcher versus facilitator average of intended content included |
* Formative - informal feedback to staff prior to the next intervention session * Summative - reported after the program is complete |
| Dose received | How did participants react to the activities presented during the intervention? | Participants | * Participant evaluation of each of the attended sessions (enjoyment, amount learnt, ease of understanding content, relevance to self, likelihood of strategy use, likes and suggested changes) * Post-program evaluation (likes, dislikes, comments, attitudes before, attitudes after, program activities, facilitators, resources) |
* Post-session * Post-program |
* Quantitative analysis of session scores; Qualitative analysis of likes and suggestion themes per session and across the program | * Summative - reported after the program is complete |
| To what extent did participants engage in recommended follow-up behavior? | Participants | * Father's log book and green slip homework activities; Kid's log book | * Father's log book (post-program); Kids log book (during and post program); Father's green slip homework (weekly) | * Level of follow-up activity completed included as a factor during quantitative analysis | * Summative - reported after the program is complete | |
| Reach | What proportion of fathers attended greater than 70% of sessions (5 of 7)? | Facilitators | * Session roll for attendance | * Recorded at each session | * Level of attendance included as a mediator during quantitative analysis | * Summative - reported after the program is complete |
| Recruitment | What recruitment activities were used to attract organizations and participants? What recruitment activities attracted participants to the program? |
HDHK research staff HDHK research staff |
* Marketing spread sheet maintained by research staff (target market, activity, date, response); Recruitment/screening spread sheet recording the date of first contact by a potential participant * Identification of the most influential marketing method by the potential participant and any other marketing they had been exposed to |
* Each time recruitment activities were undertaken * Recorded at initial phone screening of potential participant |
* Narrative description of procedures; graphing of marketing methods and participant response dates to identify successful trends * Narrative description of influential recruitment strategies; Calculate percentages of most influential strategies leading to participant contact with the research team |
* Formative - reported weekly to identify responses to marketing methods used; Summative - reported after the program is complete to inform next cohort * Formative - reported weekly to identify responses to marketing methods used; Summative - reported after the program is complete to inform next cohort |
| Context | What barriers to participation were experienced by the participants? | Facilitators using participant information | * Focus groups as part of the program to identify barriers to implementing the information/activities covered during the program | * Session 6 of 7 (facilitators to record the barriers identified by participants) | * Themes identified through qualitative analysis | * Summative - reported after the program is complete |
| What aspects of the program did and didn't appeal to the participants? | Participants | * Post-program evaluation (likes, dislikes, comments, attitudes before, attitudes after, program activities, facilitators, resources) | * Post-program | * Themes identified through qualitative analysis and statistical analysis of questionnaire | * Summative - reported after the program is complete | |
# Underlying theory and philosophy is embedded in the content given to facilitators to deliver the participants. If all content is delivered as per the program package, consistence of theory and philosophy is said to have been achieved.