Table 4.

Summary of refinements made to ePRO symptom-report and clinical algorithms during ePRO system development

Iterative changes made to ePRO symptom-report
Area	Issue identified	Changes made
Selection of relevant symptom-report items	Participants reported issues with comprehension and interpretation of some items and response categories.	Initial long list of 95 items from 7 validated EORTC symptom-reports refined to 25 items in accordance with patient interview data.
Selection of relevant symptom-report items	Consultation with clinicians indicated that shortlist of 25 symptom-report items did not cover all symptoms patients are likely to experience following hospital discharge.	Addition of five items, including items about wound problems, feeding tubes, other side effects and any contact with healthcare professionals.
Development of clinical algorithms: Participant symptom reporting	Patients could report ‘Very much’ for symptoms that could generate Level 2 or Level 3 actions, but weekly telephone follow-up interviews revealed that symptoms were to be expected depending on context (e.g. shortness of breath after physical activity versus at rest)).	Addition of sub-items and two branching logic questions if potentially concerning symptoms reported: (i) Branching question (“Is this a current issue?”) added to 8 items to determine if reported symptom is a current or resolved problem. (ii) Branching question specific to symptom added to 9 items to clarify severity (e.g. if patient reported ‘quite a bit’ or ‘very much’ shortness of breath, an additional branching question asked, “Were you short of breath while sitting down or resting?”.
Development of clinical algorithms: Symptom severity thresholds	Stakeholder meetings and discussion with clinicians indicated that severity of some symptoms is expected to vary during recovery (e.g. high levels of pain are expected during week 1 post-discharge but would be concerning at week 6.	Symptom severity thresholds were adapted for 11 items (e.g. pain, nausea & vomiting, appetite loss) to account for expected variation during recovery between weeks 3–8 post discharge. For example, high levels of pain would generate a maximum of a Level 2 action during weeks 1–2 post-discharge and generate a Level 3 action in week 3.
Development of clinical algorithms: Combined item scoring	Patients responses to 1 of the 2 jaundice items (itching) could generate an inappropriate jaundice alert due to the combined scoring of these items, even though itching may be due to allergies, dry skin etc.	Following discussion with clinicians, the scoring for itching and jaundice was split to prevent false positive jaundice alerts. A branching question was also added to the itching item to determine if this was due to a known cause, such as an allergy or dry skin.
Development of patient self-management advice	Patients reported uncertainty about whether their symptoms were expected or concerning and found this troubling.	Following analysis of telephone-based clinical consultations, reassurance relating to expected symptoms was identified and incorporated into Level 1 patient self-management advice.