Table 4.

Summary of studies on administration mode

Author (date); Country; Study Design; Measurement properties evaluated	Objective	Sample size (N); Setting; Age; Population	Main findings	QualSyst Score (%)
Bender (2007) [68]; USA; Quantitative; Prospective; Reliability	To test the effect of reporting mode on accuracy of inhaled cortico-steroid adherence reporting in children with asthma and their parents under conditions similar to those of an asthma clinical trial	N = 104; Outpatients; 8–18 years; Asthma	All methods led to over-reporting compared to electronic device on asthma pump. More than half of children over- reported adherence by > 25% Discrepancy was greatest in computer interview condition	77
Castarlenas (2015) [69] Spain; Quantitative; Cross-sectional; Acceptability, construct validity	To examine the agreement between verbally and electronically administered NRS-11b (eNRS) for pain	N = 191; School; 12–18 years; Healthy	Bland Altman LOA fell outside the a priori limit for 95%. LOA at 80% fell inside the maximum limit established a priori. K-coefficients ranged from 0.786–0.912 indicating almost perfect agreement. 83% preferred the eNRS	77
Eaton (2010) [89]; USA; Quantitative; Cross-sectional; Construct validity, feasibility	To examine whether paper and pencil surveys and web surveys yield equivalent risk behaviour prevalence estimates when using the Youth Risk Behaviour Survey	N = 5227; School; Unclear; Healthy	Prevalence estimates from paper and pencil and web-based surveys were generally equivalent. Questionnaire mode was only significantly (p < 0.05) associated with 7 of 74 risk behaviours	82
Fouladi (2006) [70]; USA; Quantitative; Cross-sectional; Construct validity, feasibility	To examine systematic differences in the responses of 4th, 5th, and 6th graders to measures of stress, coping, and humour among three modes of assessment: paper-and-pencil questionnaires, computer-assisted self-interviewing (CASI), or a combination of paper-and-pencil and CASI. Scales used – feel bad scale, school agers coping strategies inventory, the multi-dimensional sense of humour scale	N = 1245; School; 9–12 years; General	CASI means and medians were higher (p < 0.002) and correlations between CASI measures tended to be lower than those obtained with paper and pencil and mixed modes. CASI variances were lower	65
Geerdink (2009) [71]; Netherlands; Quantitative; Cross-sectional; Acceptability, construct validity, feasibility	To develop a reliable and user-friendly digital child health assessment questionnaire (CHAQ) to complete systematically at the outpatient paediatric rheumatology clinic	N = 51; Outpatients; Unclear; Juvenile arthritis	Correlation between the digital and paper versions was high (r = 0.974). No statistically significantly differences in median outcome were found in visual analogue scale (VAS) pain (25.6 vs 25.9 mm) and VAS well-being (20.1 vs 19.5 mm). Although the mean time (5.06 min) to complete the digital CHAQ was significantly longer than the mean time (3.75 min) to complete the paper form, the majority of patients (75%) preferred the digital version. User-friendliness received maximum positive score	59
Jensen (2010) [72]; Denmark; Quantitative; Prospective; Acceptability	To examine the assessments and priorities by children and adolescents of health care in a paediatric outpatient clinic, to examine the influence of the time factor on assessments and priorities by children and adolescents of health care, and to determine their preferred method of evaluation	N = 346; Outpatients; 11–17 years; Range of diagnoses	50.1% of children and adolescents preferred to complete an electronic questionnaire to a paper one. They did not want to receive questionnaires by email	45
Jones (2010) [73]; New Zealand; Quantitative; Prospective; Acceptability, construct validity, reliability	To investigate the reliability and validity of a computerised anxiety assessment (smiley faces program revised (SFP-R)) and to explore children’s preferences for the method of anxiety assessment	N = 206; School; 5–13 years; Healthy	The online SFP-R demonstrated good reliability (⍺ = 0.75) and strong convergent validity with the modified children’s dental anxiety scale (r = 0.75). Test–retest reliability r = 0.67. Children preferred the computerised assessment to pen and paper methods	54
Knight (2007) [74]; USA; Quantitative; Cross-sectional; Acceptability	To determine adolescents’ preferences for method of substance abuse screening	N = 2133; Outpatients; 12–18 years; General medicine	Paper was the preferred method (mean rank (MR) = 2.92, 95%CI 2.87–2.96) vs. computer (MR = 2.38, 2.33–2.43), nurse (MR = 2.43, 2.39–2.47), and doctor (MR = 2.30, 2.25–2.35). Participants stated they were more likely to be honest with paper followed by computer, rather than responding to questions administered by a doctor or nurse. Those reporting on the computer were significantly more likely to report drug and alcohol use	67
Lloyd (2011) [75]; UK; Quantitative; Cross-sectional; Construct validity, feasibility, Reliability,	To examine the psychometric properties of an Internet version of a children and young persons’ quality of life measure (Kid’s Life and Times) originally designed as a paper questionnaire	N = 3440; School; 10–11 years; Healthy	Exploratory principal component analysis supported 5 components, in line with the paper version. Items loaded on to the expected components. Internal consistency was similar to that reported for the paper version (⍺ all > 0.76). Domain scores were similar to those reported in the literature for the paper version. Non-response was lower with the online version (1% vs 1.72–3.83%)	72
Magnus (2016) [90]; USA; Quantitative; Cross-sectional; Construct validity	To test the equivalence of scores obtained with the PROMIS paediatric depressive symptoms, fatigue and mobility measures across computer and telephone administration	N = 377; Home; 8–17 years; Healthy	There were high correlations between the two modes of administration (0.71–0.94), although fatigue scores were affected by mode of administration, but the differences in scores were sufficiently small that they would not affect overall interpretation of results	77
Mangunkusumo (2005) [76]; Netherlands; Quantitative; Cross-sectional; Acceptability, construct validity	To assess whether scores of an internet administered adolescent health questionnaire (KIVPA) are equivalent to those obtained via paper and pencil. To compare adolescents’ evaluation of administration modes	N = 565; School; 13–17 years; Healthy	Internet questionnaire generally resulted in equal scores to pen and paper mode. Adolescents in the internet one-item mode group more frequently reported satisfaction with appearance compared with the Internet multiple items mode (p ≤ .01). The internet group had more adolescents reporting that they had a sufficient number of friends compared to the paper mode (p ≤ .01)	77
Mangunkusumo (2006) [77]; Netherlands; Quantitative; Cross-sectional; Construct validity, feasibility	To compare the feasibility, presence of score differences and subjective evaluations by children between Internet and identical paper questionnaires (International study of asthma and allergies in childhood questionnaire)	N = 249; School; 10–12 years; Healthy	There were similar mean scores between administration modes. ICC 0.64–0.9. One third of items showed moderate agreement between modes (kappa 0.43–0.6). The remaining items had very good agreement (kappa 0.61–0.95). There were fewer missing data with the internet version	82
Mauz (2018) [78]; Germany; Cross-sectional; Acceptability, construct validity, feasibility	To determine whether prevalence rates or mean values of self-reported health indicators for children and adolescents age 11–17 years differ between self-administered paper-based questionnaires and self-administered web-based questionnaires (German Health Interview and Examination Survey for Children and Adolescents)	N = 1194; Home; 11–17 years; Healthy	Most questions showed mode equivalence except for alcohol consumption. Higher levels of consumption were reported online (p < 0.001). Male adolescents preferred the online mode. Those choosing the web-based response format were more likely to have higher household income and higher educational attainment (actual data not reported)	71
McCabe (2005) [79]; USA; Quantitative; Cross-sectional; Construct validity, feasibility	To examine the feasibility and mode effects of using a web form vs paper form survey to collect alcohol and tobacco data from 3rd and 4th grade students	N = 323; School; Not specified (3/4 grade); Healthy	There were minimal differences between survey modes. (future alcohol use and lifetime alcohol use showed significant difference, p < 0.05))	55
Moskowitz (2004) [80]; USA; Quantitative; Cross-sectional; Construct validity, feasibility	To assess the effect of telephone audio computer-assisted self-interviewing (A-CASI) and computer-assisted telephone interviewing (T-ACASI), on self-reports of smoking behaviour and smoking susceptibility among adolescents 12–17 years of age (adapted from Youth Attitudes and Practices Survey)	N = 2444; Home; 12–17 years; Healthy	Adjusted estimates of current smoking were higher in the self-administered T-ACASI (8.3% vs 4.5%). The commitment not to smoke among those who had never smoked was also higher in the T-ACASI (45% vs 34.9%). Parental presence was negatively associated with smoking. T-ACASI survey had more missing data than CATI	77
Nitikman (2017) [81]; Canada; Quantitative; Prospective; Construct validity, feasibility, reliability	To validate and test the reliability of using the Internet as a method of administering health-related quality of life questionnaires in a paediatric spine population (Scoliosis Research Society 30 (SRS-30) and Pediatric Outcomes Data Collection Instrument (PODCI))	N = 96; Outpatients; 11–18 years; Scoliosis	There was no significant difference in scores between methods of administration at the 2 time points (p = 0.206). Patients expressed a preference for the internet option (84%)	63
Raat (2007) [82]; Netherlands; Quantitative; Cross-sectional; Construct validity, feasibility, reliability	To evaluate the indicators of feasibility, reliability and validity of the Child Health Questionnaire-Child Form (CHQ-CF). To compare the results in those of those who complete the standard paper version compared to an internet version	N = 933; School; 13–17 years; Healthy	The internet version resulted in fewer missing answers. All scales clearly discriminated between adolescents with no, a few or many self-reported chronic conditions. The paper administration resulted in statistically significant, higher scores on 4 of 10 CHQ-CF scales compared with the internet administration (P < 0.05), but Cohen’s effect sizes d were ≤ 0.21. Mode of administration interacted significantly with age (P < 0.05) on four CHQ-CF scales, but Cohen’s effect sizes for these differences were also ≤ 0.21	96
Raat (2007) [83]; Netherlands; Quantitative; Cross-sectional; Construct validity, feasibility	To compare the results from written and internet questionnaires about respiratory symptoms to find out if both forms yielded the same responses (International Study of Asthma and Allergies in Childhood (ISAAC) questionnaire)	N = 933; School; 13–17 years; Healthy	The Internet version showed fewer missing answers not statistically significant). The respiratory items did not show statistically significant score differences between the Internet and written modes of administration. Both approaches yielded equal results	96
Robles (2015) [84]; Spain; Quantitative; Cross sectional; Construct validity, feasibility,	To develop web-based Spanish and Catalan versions of the EQ-5D-Y, and to compare scores and psychometric properties with the paper version	N = 715; School; 8–18 years; Healthy	Both formats of EQ-5D-Y showed low percentages of missing values (n = 2, and 4 to 9 for web and paper versions respectively), and a high ceiling effect by dimension (range from 79 to 96%). Percent agreement for EQ-5D-Y dimensions on the web and paper versions was acceptable (range 89% to 97%), and k ranged from 0.55 (0.48–0.61, usual activities dimension) to 0.75 (0.68–0.82, mobility dimension). Mean score difference on the VAS was 0.07, and the ICC for VAS scores on the two formats was 0.84 (0.82–0.86). Both formats showed acceptable ability to discriminate according to self-perceived health, reporting chronic conditions, and mental health status	83
Sun (2015) [91]; Canada; Quantitative; Longitudinal; Acceptability, construct validity, feasibility	To evaluate agreement between electronic (called Panda) and paper versions of the faces pain scale revised (FPS-R) and colour analogue scale (CAS)	N = 62; Hospital; 4–18 years; Surgical	Panda scores correlated strongly with original scores at T0 and T30 (r > 0.93 for FPS-R; r > 0.87 for CAS). Most participants expressed a preference for the iPod Panda version (76–81%)	67
Trapl (2013) [85]; USA; Quantitative; Cross sectional; Acceptability, feasibility	To examine the impact of 3 data collection modes (paper, PDA, audiPDA (APDA)) on the number of questions answered, data quality, and student preference	N = 275; School; Not specified (7th grade); Healthy	APDA respondents completed significantly more questions compared to paper and PDA (p < 0.001). PDA and APDA had significantly fewer missing data than did paper (p < 0.001). No differences were found for student evaluation	63
Varni (2009) [86]; USA; Quantitative; Cross-sectional; Construct validity	To implement the multigroup confirmatory factor analysis (CFA) method for invariance testing across mode of administration for children’s self-reported health-related quality of life (in person, mail and telephone) using PedsQLTM 4.0 Generic Core Scales	N = 3741; Home or clinic; 5–18 years; Chronic illness	Strong factorial invariance across the mode of administration groups was demonstrated based on stability of the Comparative Fit Index (CFI) between the models, and several additional indices of practical fit including the Root Mean Squared Error of Approximation (RMSEA), the Non-Normed Fit Index (NNFI), and the Parsimony Normed Fit Index (PNFI). Children across the three modes of administration groups interpreted items on the PedsQLTM 4.0 Generic Core Scales in a similar manner	75
Wood (2011) [87]; France; Quantitative; Cross-sectional; Acceptability, construct validity, feasibility	To compare concordance and preference for electronic and paper versions of the faces pain scale revised, and to determine whether the electronic version can be used by children 4 years and over	N = 234; Hospital; 4–12 years; Inpatients	Overall weighted kappa = 0.846 and Spearman’s correlation between scores on the 2 versions was 0.91. The mean difference between scores was neither clinically nor statistically significant. 83.2% chose the same face on both versions. The PDA was preferred by 87.4% of participants	88
Young (2009) [88]; Canada; Quantitative; Prospective; Construct validity, feasibility	To test the impact of web administration on well-established measures of children’s physical function and quality of life using the ASK and PedsQL measures	N = 91 time 1 N = 69 time 2; Hospital; 8–14 years; Chronic illness	Both measures were highly reliable in web and paper format. Inter-method ICC = 0.98 for ASK and 0.64 for PedsQL compared to ICC of 0.99 and 0.94 respectively for paper formats. The web ASK seems to be valid compared to paper format. Consistency in administration mode may be more important when using the PedsQL	88