Skip to main content
. 2019 Jan 28;2019(1):CD011651. doi: 10.1002/14651858.CD011651.pub2

Joseph 2010.

Methods Included as process evaluation
Intervention study design: parallel‐group randomised controlled trial
Unit of allocation: child
Process evaluation methods: survey based, including multi‐variate analyses of outcomes
Participants Setting: 6 high schools in Detroit, Michigan, USA
Age of children: mean age, 15.3 years
Child characteristics (BME/SES): 97% of students were African American; 52% were eligible for federal school lunch programmes
Asthma status: asthmatic only; severity unclear
Intervention recipients: children only
Interventions School type: high schools
Intervention description: tailored computer programme (Puff City): the web‐based programme focuses on 3 core behaviours, namely, controller medication adherence, rescue inhaler availability, and smoking cessation/reduction, and consists of 4 consecutive educational computer sessions that make use of both normative ("compared with other students") and ipsative ("compared with your last session") feedback. Messages are voiced over to accommodate low literacy. Participant‐specific information necessary for tailoring is obtained at baseline and during the 4 sessions.
Control description: students randomised to the control group were directed to existing generic asthma websites
Theoretical framework: not 1 single framework was named, but theories around content were mentioned
Outcomes Core processes evaluated (child level): attrition, dosage, adherence
Notes Process evaluation category: stand‐alone and integrated (2 papers)
Breadth and depth: breadth and depth
Voice of children given prominence: featured but not sufficiently
Note: study is not included as an outcome evaluation because the comparison group received asthma education (this study evaluated the added impact of providing tailored messaging)
Funding source: National Heart, Lung, and Blood Institute, National Institutes of Health
Risk of bias
Bias Authors' judgement Support for judgement
Random sequence generation (selection bias) Unclear risk N/A
Allocation concealment (selection bias) Unclear risk N/A
Blinding of participants and personnel (performance bias) 
 All outcomes Unclear risk N/A
Blinding of outcome assessment (detection bias) 
 All outcomes Unclear risk N/A
Incomplete outcome data (attrition bias) 
 All outcomes Unclear risk N/A
Selective reporting (reporting bias) Unclear risk N/A
Other bias Unclear risk N/A
Transparent and clearly stated aims Low risk Each paper includes clearly stated aims
Explicit theories underpinning and/or literature review Low risk Not a single theory, but some aspects of learning are grounded in pedagogical techniques
Transparent and clearly stated methods and tools Low risk All methods were clearly stated
Selective reporting Unclear risk Data were apparently collected at different time points, but only follow‐up data were presented. Otherwise no evidence indicates that data collected were not presented
Harmful effects Unclear risk Some consideration of negative factors in the design ‐ e.g. access to the referral co‐ordinator. However, whether the analysis fully accounts for this is unclear
Population and sample described well Unclear risk Characteristics of the population were generally described well ‐ but ambiguities between papers that ostensibly describe the same study population surround the numbers involved
Continuous evaluation Low risk Data were collected at different time points, but only follow‐up data were presented
Evaluation participation equity and sampling High risk High level of non‐participation is a matter of concern
Design and methods overall approach Unclear risk Evaluation took into account multiple time points (all data were not necessarily presented): however data from multiple stakeholders were not collected
Tools and methods of data collection reliable/credible Low risk Tools and methods used were reliable
Tools and methods of data analysis reliable/credible Low risk Analysis was straightforward, in part because of the relatively straightforward research design
Performance bias/neutrality/credibility/conformability Low risk One of the papers explicitly addressed negative cases
Reliability of findings and recommendations Unclear risk Taken together, the 3 papers associated with this study present an accurate description of the processes undertaken, but numerous children did not participate and no clear explanation for this was provided
Transferability of findings Unclear risk Study authors did not address transferability, but the data are rich enough for exploration of contextual factors, etc. No explanation is provided as to why so many children did not participate
Overall risk of bias of process evaluation Unclear risk This study was well conducted in most respects, but it is unclear why so many children failed to participate