Table 2.
Results of the studies included in the systematic review.
| Study | Electronic delivery method | ePROMa, outcome, and results | |||
|
|
Web/PCb | Device |
|
|
|
| [46] | Web | Tablet | DLQIc | ||
|
|
|
|
|
Preference | 76% prefer electronic |
|
|
|
|
|
Completion time | Electronic took 9 s longer than pencil and paper (P=.008), older participants took longer (r2=.257, P=.012) |
|
|
|
|
|
Agreement | ICCd=.98, CI 0.97-0.99 |
| [20] | Web | – | EORTCe, QLQ-C30f | ||
|
|
|
|
|
Completion rate | 92% completed the first measurement, 74% completed the 6-month measurement, 82% completed ≥4 of 7 sessions |
|
|
|
|
|
Satisfaction and other outcomes | 92% found it easy to use, 85% continued using it, 85% recommended it |
| [45] | Web | PC | SAQg, SF-36h | ||
|
|
|
|
|
Preference | 82% preferred electronic, there was no effect on preference with age, sex, race, computer use, education, visual impairment, or reading level |
|
|
|
|
|
Completion rate | No differences in the completion rate |
|
|
|
|
|
Completion time | SAQ completion time: 5.53 min electronic, 4.78 min paper (P<.05); SF-36 completion time: 6.76 min electronic, 5.44 min paper (P<.05); the log-on procedure was not significantly different |
|
|
|
|
|
Agreement between electronic and paper | For the 5 SAQ domains r=0.84-0.93; for the 8 SF-36 subscales: r=0.54-0.75 |
| [31] | Web | PC | – | ||
|
|
|
|
|
Preference | 83.6% preferred pencil and paper, 14.4% preferred internet |
|
|
|
|
|
Data completion | Unanswered questions: 9.3% pencil and paper, 4.9% internet (==t =14.85, P=.01) |
|
|
|
|
|
Data missing | Internet answers were more detailed than pencil and paper answers in 4 of 5 questions (P<.05) |
| [41] | Web | Tablet | DASHi | ||
|
|
|
|
|
Data completion | 24% of questions were unscorable with pencil and paper, compared with 2% for electronic (P<.001); electronic was more likely to be scorable (ORj=13.5, P<.001) |
|
|
|
|
|
Data missing | Mean (SD) of 2.6 (4.4) with pencil and paper vs 0.1 (0.8) with electronic (P<.001), electronic format had an inverse relationship with omitted questions (beta=–0.358, P<.001) |
| [23] | Web | – | PAM-13k, MacNewl, FQm, EORTC, QLQ-C30, HADSn | ||
|
|
|
|
|
Demographic factors | Preferred electronic over paper: younger age (P=.008), married/cohabitating (P=.004), internet available (P<.001), educated (P=.092) |
|
|
|
|
|
Preference | 77.8% prefer web-based forms |
|
|
|
|
|
Completion time | Web-based, ~9.5 min; paper-based, ~24 min |
|
|
|
|
|
Data completion | Inadequate responses did not exist for the web version due to the system design |
|
|
|
|
|
Data missing | Fewer total data points missing on paper-based forms than on web-based forms (P<.001) |
| [49] | Web | Tablet | PASEo, BARSEp, PSQIq | ||
|
|
|
|
|
Demographic factors | Factors affecting preference of electronic vs paper: daily computer use, perceived ease of use, reported anxiety while completing the digital questionnaire (all P<.05) |
|
|
|
|
|
Preference | Electronic preferred over pencil and paper (z=4.96, SE 3.428, P<.001) |
| [28] | PC | Tablet | EORTC, QLQ-30 | ||
|
|
|
|
|
Completion rate | Completion rate 43%-58% from 2005-2010, <20% since 2011 (ePROr) |
|
|
|
|
|
Adherence and compliance | Pencil and paper associated with non-completion (OR=2.72, P<.001) and poor adherence (OR=2.23, P<.001), male sex associated with poor adherence (OR=1.69, P=.010) |
| [47] | PC | PC | RAQoLs | ||
|
|
|
|
|
Satisfaction | Electronic > P-P (P=.003) |
|
|
|
|
|
Preference | 64% prefer electronic |
|
|
|
|
|
Completion time | Pencil and paper, 6 min; electronic, 5 min P=.194 |
|
|
|
|
|
Agreement between electronic and paper | ICC=.982 |
| [26] | Web | Tablet | EORTC, QLQ-C30 | ||
|
|
|
|
|
Attitude/ willingness |
92.3% of those exposed to both electronic and paper vs 59% of those exposed only to paper (P=.001) were willing; patients exposed only to paper more likely to report barriers: data privacy (P=.003), technical knowledge (P=.02), discomfort using technology (P=.02), no internet (P=.05) |
| [34] | Web | Tablet | – | ||
|
|
|
|
|
Adherence | Patient adherence: 76.8% for pain monitoring, 50.4% for medication monitoring, and 100% for education |
|
|
|
|
|
Satisfaction | Limited effort, comfortable, education session appreciated, added value with self-management, medication overview with reminders was supportive |
|
|
|
|
|
Experience | Measured using a Likert scale, mean (SD): learnability, 4.8 (0.4); usability, 4.8 (0.5); desirability, 4.6 (0.4); and would recommend app, 4.8 (0.4) |
| [43] | PC | PDAt | LCSSu | ||
|
|
|
|
|
Satisfaction | 98% of patients reported it acceptable and easy to use, 80% learned it in <3 minutes, 100% of nurses and 86% of physicians said it’s easy to use |
|
|
|
|
|
Completion time | Electronic, 2.2 min; pencil and paper, 3-5 min |
|
|
|
|
|
Agreement between electronic and paper | Pearson r=0.92, ICC=.92, Lin's CCCv=.92 |
| [29] | Web | iPhone, iPad, or iPod | Pain VASw | ||
|
|
|
|
|
Completion rate, adherence, compliance | Compliance decreases over time, >35 years old had increased compliance (P<.05), compliance greater with iPad than iPhone (P<.0025), technical difficulties decreased compliance (P<.0025), |
|
|
|
|
|
Demographic factors | Information technology comfort level had no impact on adherence |
|
|
|
|
|
Agreement | iPhone, ICC=.99 (95% CI 0.92-1.00); iPad, ICC=.97 (95% CI 0.88-0.99) |
| [25] | Web | – | MSIPx, MSQoL-54y, MFIS-5z, LMSQoLaa | ||
|
|
|
|
|
Other symptom insights | 46% have greater insights into symptoms; 18% feel better able to handle symptoms; 65.4% feel it’s important for other health care professionals to have access; advantages include availability, overview of symptoms, gain insights, forced to reflect, look back on history; disadvantages include it’s tiring, lot of work, complicated, repeated questions, grammatical errors, no space for free text, monthly completion, login problems, not used friendly, data aren't used by physician |
| [51] | Web | PDA | AQLQbb, ACQcc, RQLQdd | ||
|
|
|
|
|
Agreement between electronic and paper | AQLQ (P=.009), ACQ (P=.12), RQLQ (P=.05) |
| [48] | Web | Tablet | WOMACee, FJS-12ff | ||
|
|
|
|
|
Completion time | WOMAC: pencil and paper 170 s, electronic 117 s (P<.001); FJS-23: pencil and paper 22 s, electronic 37 s (P<.001) |
| [32] | Web | – | SF-36 | ||
|
|
|
|
|
Preference | THRgg 81.8% preferred pencil and paper (CI 78.8-84.7), TKRhh 86.8% preferred pencil and paper (CI 83.1-89.8) |
|
|
|
|
|
Demographic factors | Preferred electronic over paper: younger age (P<.001), male sex (P<.001), higher education level (P<.001), higher BMI (P=.004) |
| [50] | Web | PC | SF-36, MFI-20ii, HADS | ||
|
|
|
|
|
Completion rate | 73.2% with pencil and paper vs 17.9% with internet: difference of 55.3 (48.3-62.3); after a reminder: 76.5% with pencil and paper vs 64.2% with internet: difference 12.2 (4.5-20) |
|
|
|
|
|
Preference | 55.4% prefer pencil and paper |
|
|
|
|
|
Data completion, missing data | 63.4% data completion with pencil and paper vs 97.8% with internet (P<.001): difference 34.5 (26.6-42.3) |
| [21] | Web | Smartphone | MMAS-8jj | ||
|
|
|
|
|
Demographic factors | Preferred electronic over paper: younger age (P=.002), live in the city (P<.001), higher education level/stable employment (P<.001), more seizures (P=.01), lower medication adherence and own a smartphone (P=.001) |
|
|
|
|
|
Attitude/willingness | 65.5% would use it if it was free, 72.3% if it was easy to operate, 59% think it decreases medical visits and related costs, 71.7% say privacy must be protected |
| [44] | Web | PC | KIVPAkk | ||
|
|
|
|
|
Preference | Mean (SD) pleasantness: 2.7 (0.9) for pencil and paper vs 3.0 (0.8) for internet (P<.01); mean (SD) difficulty: 3.6 (0.7) for pencil and paper vs 3.9 (0.7) for internet (P<.01) |
| [27] | PC | Tablet | CSGAll | ||
|
|
|
|
|
Feasibility in older patients | ≥50% unable complete without assistance (reason: computer illiteracy) |
| [39] | Web | PC | CHQ-CFmm | ||
|
|
|
|
|
Data completion, missing data | 0.54% with paper vs 0.04% with internet (P<.01) |
| [33] | Web | PC and tablet | EQ-5Dnn, PHQ-9oo | ||
|
|
|
|
|
Satisfaction | 92.3% found it easy to use, 87.6% thought it time appropriate, 77.3% saw a perceived benefit |
|
|
|
|
|
Other factors affecting perception of benefit | Provider review (OR 6.56, P<.001) |
| [42] | Web | Tablet | FFbHp, BASDAIqq, SF-36 | ||
|
|
|
|
|
Experience | Older age requires more support |
|
|
|
|
|
Preference | 62.1% prefer electronic, especially those of younger age and with increased computer knowledge (P<.01) |
|
|
|
|
|
Data completion | Significantly greater with electronic |
|
|
|
|
|
Agreement between electronic and paper | r=0.87-0.98; P>.05 |
| [40] | PC | PC | VAS GH, VAS Pain, VAS PGArr, ROADss, TJCtt | ||
|
|
|
|
|
Preference | 86% prefer electronic |
|
|
|
|
|
Completion time | Electronic 7.3 min, pencil and paper 7.9 min (P=.006); older age requires greater time for both (electronic: P=.02, pencil and paper: P=.005) |
|
|
|
|
|
Agreement between electronic and paper | No difference between methods and high correlation (all P>.05, CCC>.849) |
| [37] | PC | Tablet | BASDAI, BASFIuu, NRSvv | ||
|
|
|
|
|
Preference | 83.4% prefer the tablet |
|
|
|
|
|
Completion time | Tablet 5.1 min, paper 7.9 min (P=.04) |
|
|
|
|
|
Agreement | ICC>0.9 (P<.0001) |
| [30] | Web | Tablet | KDQOL-36ww, ESASxx | ||
|
|
|
|
|
Logistics | Internet/cellular access, link to electronic health records |
|
|
|
|
|
Infection control | Hand sanitizer, stylus |
|
|
|
|
|
Financials | Financial support necessary? |
|
|
|
|
|
Design | Minimalistic, large font, black writing on white background, no distracting graphics, adapted to population |
| [22] | Web | – | Symptom self-management tool for PLWHyy | ||
|
|
|
|
|
Symptoms diminish with targeted strategies | Decreased frequency (effect size=.37) and intensity (effect size=–8.41) over time for all symptoms except diarrhea |
| [35] | Web | Tablet | EQ-5D, ODIzz, NDI1, HOOS2, KOOS3, QuickDASH4 | ||
|
|
|
|
|
Completion rate | No differences in unanswered questions (P>.05) |
|
|
|
|
|
Preference | Satisfaction similar; however, 41.4% prefer the tablet (P<.001); total 60.38% |
|
|
|
|
|
Data completion | No difference in completion rate (P=.208) |
|
|
|
|
|
Completion time | No difference in the completion time (P>.05) |
| [38] | Web | Tablet | PSS5, FFI6, ODI | ||
|
|
|
|
|
Preference | 68% prefer electronic |
|
|
|
|
|
Data completion | Pencil and paper 14 times greater completion (PSS, P=.008), 260 times greater completion (FFI, P<.001), 11 times greater completion (ODI, P<.001) |
|
|
|
|
|
Agreement between electronic and paper | Differences in patient-reported outcomes scores not significant (P>.05) |
| [36] | Web | PC | Nutrinet Sante | ||
|
|
|
|
|
Preference | 92.2% prefer web; web considered more acceptable (P=.002) and with fewer barriers (P=.03) |
|
|
|
|
|
Data completion | No data missing in web |
|
|
|
|
|
Completion time | No significant differences in completion time |
|
|
|
|
|
Cost | For a cohort of 500,000 subjects: paper €4,965,833 (€9.94/subject); web-based tool €150,000 (€0.3/subject) |
|
|
|
|
|
Agreement between electronic and paper | Agreement ICC=.86-1.00 qualitative variables; ICC=.69-1.00 for 18 qualitative variables (height, weight, hip circumference, waist circumference were all different) |
| [24] | Web | Tablet | EORTC | ||
|
|
|
|
|
Preference | 65.98% prefer electronic |
|
|
|
|
|
Habits and attitudes | 64.4% of the clinic ePROM group and 91.1% of the home ePROM group found it useful and adequate for QOL; 82.2% would appreciate discussing results with a physician |
|
|
|
|
|
Feasibility and suggestions | Perceived benefits included that it was always available, feeling well cared at home, and low cost; the disadvantages included that it was too impersonal and technical issues; suggestions included adjustable font size |
aePROM: electronic patient-reported outcome measure.
bPC: personal computer.
cDermatology Life Quality Index.
dICC: interclass correlation coefficient.
eEORTC: EORTC: European Organization for the Research and Treatment of Cancer.
fQLQ-C30: Quality of Life Questionnaire Core 30.
gSAQ: Seattle Angina Questionnaire.
hSF-36: Short Form-36.
iDASH: Disabilities of the Arm, Shoulder, and Hand.
jOR: odds ratio.
kPAM-13: Patient Activation Measure short form.
lMacNew: MacNew Heart Disease Health-related Quality of Life questionnaire.
mFQ: Fatigue Questionnaire.
nHADS: Hospital Anxiety and Depression Scale.
oPASE: Physical Activity Scale for the Elderly.
pBARSE: Barriers Self-Efficacy Scale.
qPSQI: Pittsburgh Sleep Quality Index.
rePRO: electronic patient-reported outcome.
sRAQol: Rheumatoid Arthritis Quality of Life Questionnaire.
tPDA: personal digital assistant.
uLCSS: Lung Cancer Symptom Scale.
vCCC: concordance correlation coefficient.
wVAS: visual analogue scale.
xMSIP: Multiple Sclerosis Impact Profile.
yMSQoL-54: Multiple Sclerosis Quality of Life-54.
zMFIS-5: Modified Fatigue Impact Scale-5.
aaLMSQoL: Leeds Multiple Sclerosis Quality of Life.
bbAQLQ: Asthma Quality of Life Questionnaire.
ccACQ: Asthma Control Questionnaire.
ddRQLQ: Rhinoconjunctivitis Quality of Life Questionnaire.
eeWOMAC: Western Ontario and McMaster Universities.
ffFJS: Forgotten Joint Score.
ggTHR: total hip replacement.
hhTKR: total knee replacement.
iiMFI-20: Multidimensional Fatigue Inventory.
jjMMAS-8: Morisky Medication Adherence Scale.
kkKIVPA: Korte Indicatieve Vragenlijst voor Psychosociale Problematiek bij Adolescenten.
llCSGA: Cancer-Specific Geriatric Assessment.
mmCHQ-CF: Child Health Questionnaire-Child Form.
nnEQ-5D: European Quality of Life-5 Dimensions (General Health).
ooPHQ-9: Patient Health Questionnaire-9.
ppFFbH: Hannover Functional Ability Questionnaire.
qqBASDAI: Bath Ankylosing Spondylitis Disease Activity Index.
rrPGA: Patient Global Disease Activity.
ssROAD: Recent-Onset Arthritis Disability Index.
ttTJC: tender joint count.
uuBASFI: Bath Ankylosing Spondylitis Functional Index.
vvNRS: numeric rating scale.
wwKDQOL-36: Kidney Disease Quality of Life Instrument.
xxESAS: Edmonton Symptom Assessment System.
yyPLWH: people living with HIV/AIDS.
zzODI: Oswestry Disability Index.
1NDI: Neck Disability Index.
2HOOS: Hip Disability and Osteoarthritis Outcomes Score.
3KOOS: Knee Injury and Osteoarthritis Outcomes Score.
4QuickDASH: abbreviated version of Disabilities of the Arm, Shoulder, and Hand.
5PSS: Perceived Stress Scale.
6FFI: Foot Function Index.
7None mentioned in particular.