Abstract
Aims
Previous generations of home monitoring systems have had limited usability. We aimed to develop and evaluate a user-centred and adaptive system for health monitoring and self-management support in patients with heart failure.
Methods and results
Patients with heart failure were recruited from three UK centres and provided with Internet-enabled tablet computers that were wirelessly linked with sensor devices for blood pressure, heart rate, and weight monitoring. Patient observations, interviews, and concurrent analyses of the automatically collected data from their monitoring devices were used to increase the usability of the system. Of the 52 participants (median age 77 years, median follow-up 6 months [interquartile range, IQR, 3.6–9.2]), 24 (46%) had no, or very limited prior, experience with digital technologies. It took participants about 1.5 min to complete the daily monitoring tasks, and the rate of failed attempts in completing tasks was <5%. After 45 weeks of observation, participants still used the system on 4.5 days per week (confidence interval 3.2–5.7 days). Of the 46 patients who could complete the final survey, 93% considered the monitoring system as easy to use and 38% asked to keep the system for self-management support after the study was completed.
Conclusion
We developed a user-centred home monitoring system that enabled a wide range of heart failure patients, with differing degrees of IT literacy, to monitor their health status regularly. Despite no active medical intervention, patients felt that they benefited from the reassurance and sense of connectivity that the monitoring system provided.
Keywords: Chronic disease management, Heart failure, Digital health, Mobile health, Patient-centred care, Mixed-method research
Introduction
The provision of evidence-based care to heart failure patients is a major challenge to health systems worldwide.1 Traditional clinician-centred and poorly integrated models of care are not well suited for the management of complex chronic conditions, such as heart failure.2 Although the needs and risks of heart failure patients often change over time,3 patients tend to spend ∼98% of their time outside hospitals with no possibility of information exchange with their healthcare providers.4 While this may imply that more intensive and proactive monitoring is needed, from clinicians perspective, even the existing recommendations for gradual drug titration and intermittent safety checks often pose a significant challenge.5
Innovative models of care delivery that make better use of digital technologies may help fill the gaps in healthcare provision and could provide a sustainable and more affordable complement to the prevailing labour-intensive models of heart failure care.2,6 Such models of care could not only help to improve the delivery of evidence-based care, but they also have the potential to make healthcare more personalized by offering an efficient way of capturing patients' informed preferences. However, despite the intuitive appeal of such systems, the evidence for their effectiveness and efficiency is inconsistent.7–9
One of the key limitations of previous generations of non-invasive, home monitoring systems has been the insufficient engagement of users with the technology, resulting in low adoption rates.10 In many technology-focused studies, the need to tailor monitoring systems to user's capacity and preferences has often not been given sufficient priority when the product was being developed.11 Without adequate user engagement with the system, however, any potential benefits from such innovations are likely to be missed or significantly diluted.
We report the findings from the Seamless User-centred Proactive Provision Of Risk-stratified Treatment for Heart Failure (SUPPORT-HF) study that aimed to develop and evaluate the usability of a simple and adaptive system for remote health monitoring and non-pharmacological self-management in heart failure.
Methods
Co-design of the health monitoring and self-management support system
The SUPPORT-HF system and our iterative approach in improving it have been described elsewhere.12 In brief, the system included an Android-based tablet computer with a touch screen that connected wirelessly via Bluetooth to a blood pressure (BP) and heart rate monitor and an electronic weighing scale. The application included additional features that allowed participants to review their personal readings via a graphical display, access educational material (such as video clips about heart failure and drug management), and communicate with the study team (clinicians, administrators, engineers, and a social scientist) by pressing a ‘contact me’ button that triggered an email and text messages to the study team. The study team could also send text messages to participants to comment on their health status and in-built alerts were issued, recommending patients to contact their own doctor or nurse if their weight had increased by >2 kg over 4 days.
We adopted an iterative and patient participatory approach informed by action research13 and agile software development14 principles to develop the remote home monitoring system. To better understand patients' need and capacity for home monitoring, we first arranged a co-design workshop with 15 heart failure patients and their caregivers. This co-design workshop helped the study team to narrow the range of options for hardware selection and software design. After this initial development and testing phase, we involved 52 participants and their caregivers over a follow-up period averaging 6 months in adaptation and testing of the system. Qualitative and quantitative methods were employed to understand and tackle usability issues. More specifically, the tablet computer recorded all participant interactions (in addition to the health monitoring data) and securely transmitted these through 3G/4G Internet to a web-server hosted within the National Health Service network infrastructure. These quantitative data about user interactions with the system were combined with information from ethnographic observations and interviews to inform changes to the system. During the course of the study, several changes were made, which related to functional features (e.g. data visualization and instructions for health status measurement) and non-functional features (filtering algorithms for elimination of measurement errors and data synchronization mechanisms resistant to intermittent mobile network disconnection). To minimize the burden of software updates to participants, we created a private application distribution channel via the Google play store, which automatically updated the Android application for all participants.
Study design
SUPPORT-HF was a non-interventional cohort study, which aimed to iteratively develop and evaluate the usability of a remote health monitoring and non-pharmacological self-management support system for heart failure patients with varying degrees of physical and cognitive functioning. In this study, we report the final summative evaluation of the system by focusing on the quantitative findings.
Setting
Participants were identified from two hospitals (from acute and ambulatory settings) and one community heart failure service provider in South Central England. Participant recruitment, follow-up, and evaluation took place in each patient's home environment.
Participants
Consenting adults with a confirmed diagnosis of heart failure (with or without reduced left ventricular systolic function) were eligible for recruitment into the study, provided they were able to read and understand English. There was no clinical exclusion criterion and, where appropriate, participants' caregivers were invited to take part in the study. By employing such broad eligibility criteria, we aimed to recruit a suitably diverse patient population with health and demographic characteristics that are typical for heart failure patients in the UK. Inclusion of patients with high rates of physical disabilities and varying independence was particularly encouraged to gain insights into different views on the study procedures and the usability of the system. Overall, 58 heart failure patients consented to take part in the study. For technical and health-related issues, six patients could not contribute quantitative data to the usability evaluation and are not included in this report. However, interviews with some of these participants and their relatives will be included in a separate qualitative report. These six patients did not differ materially in their clinical characteristics from the other 52 patients who contributed data to this report (summary characteristics of all 58 patients and reasons for missing usability information from these participants are provided in Supplementary material online, Table 1 and Figure 1).
Study procedures
After informed consent was obtained, participants were shown the steps they needed to follow to answer questions about their health and wellbeing and to measure their BP, heart rate, and weight. Over the course of the study, we evaluated different iterations of questionnaires for capturing patients' symptoms and finally adopted a self-assessed New York Heart Association class questionnaire.15 Study participants were asked to complete the symptoms diary and physiological measurements on a daily basis if possible, but the recommendations were not prescriptive so that the study team could learn from participants' preferences on timing and frequency of measurements. In addition, some participants were offered the opportunity to reduce the frequency of measurements during the course of the study if their recordings were stable. Furthermore, participants could choose not to complete certain aspects of the monitoring system depending on their preferences or wellbeing (for example, weight measurement could be skipped if they are unable to stand, or completion of the diary could be hidden if the patient had severe visual impairment and could not be assisted by a caregiver). Finally, other sections of the application that allowed them to learn more about heart failure, to review their own readings, and to request contact were demonstrated to the patient.
Patients were informed that their participation in the study was complementary to routine health care provision, and that research staff would not assume any responsibility for medical management. There were no home visits for clinical assessment after the initial recruitment, but participants were provided with contact details of the study staff for any questions and comments that they had in relation to the use of the equipment and were asked for permission to be visited by a social scientist for observations and interviews.
Study outcomes and outcome measurement
The study's primary outcome measure was system usability, defined as ‘the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency, and satisfaction in a specified context of use’.16,17 In this study, we specified three usability indicators: (i) successful completion of home monitoring tasks, which would involve the three steps of task initiation, completion on the first attempt, and timely data transfer to central servers; (ii) the ease with which participants updated their physiological measures using the monitoring devices provided; and (iii) participants' attitudes towards the usefulness and accessibility of the system, including the frequency of system use. A mixed-method approach was used to evaluate system usability. Qualitative findings concerning user expectations and experiences are subject to a separate report. Quantitative information was obtained from a participant questionnaire (informed by Technology Acceptance Models18) and the usage logs of the monitoring system, which allowed precise estimation of timing, errors, and duration of user interactions. We calculated the duration of tasks, defined as the time interval from the participant being shown a message to perform a certain task to the time when the data were captured and displayed by the tablet computer to the user. We calculated failure rates in completion of individual tasks by dividing the total number of initiated but not completed measurements by the total number of measurement attempts per patient per week.
Secondary outcomes included clinical events and changes to the validated Minnesota Living With Heart Failure (MLWHF) score and its physical and emotional subdomains. Questionnaires were programmed into the application and were self-administrated in the second week of study participation and then every 3 months.
Statistical analyses
We used descriptive statistics to summarize the survey findings. For estimation of changes in usability indicators and quality-of-life scores over time, we used generalized estimating equations, with robust standard errors, to take account of the interdependency of measures at the participant level. We used Poisson distribution for all frequency measurements and normal distribution for all measurements of duration and quality-of-life scores. In survival analyses, early withdrawal from the study can potentially bias the effect estimates and, in our case, could lead to more positive assessment of usability. To reduce such risks, censoring dates for early withdrawals were extended to the end of the planned 6-month follow-up (or study completion date, whichever came first), with all usage data assumed to be zero between the withdrawal and censoring dates. For all other participants, study end date was defined as the date of final assessment, date of death, or a maximum of 45 weeks after the first patient had entered the study. All analyses were undertaken using Stata version 13.1.
Study findings are reported in accordance with the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) recommendations.19 The study was approved by the NHS Health Research Authority (NRES Committee South Central, Oxford, reference: B13/SC/0125).
Results
Between June 2013 and May 2014, 58 patients were recruited into the study, of which 52 contributed data to the current usability evaluation. The clinical and socio-demographic characteristics of study participants are presented in Table 1. The median age of participants was 77 years, which is consistent with the median age of 78 years for hospitalized heart failure patients in the UK in 2012.20 About half of participants (46%) included in the study had no or very limited prior experience with digital technologies such as smart phones and touch-screen tablet computers.
Table 1.
Baseline clinical and socio-demographic characteristics
| Age in years, median (IQR) | 77 (65–83) |
| Women, n (%) | 18 (35) |
| Diabetes mellitus, n (%) | 21 (40) |
| Chronic obstructive lung disease, n (%) | 10 (19) |
| Moderate or severe valvular disease, n (%) | 24 (46) |
| Left ventricular ejection fraction, median % (IQR) | 34 (28–50) |
| Level of competency in use of digital technologies | |
| Very limited or none, n (%) | 24 (46) |
| Competent, n (%) | 22 (42) |
| Expert, n (%) | 6 (12) |
| Self-assessed severity of symptoms | |
| NYHA class 1, n (%) | 9 (18) |
| NYHA class 2, n (%) | 22 (43) |
| NYHA class 3, n (%) | 14 (27) |
| NYHA class 4, n (%) | 6 (12) |
| MLWHF score, median (IQR) | 35 (18–57) |
| EQ5D score, median (IQR) | 2 (1–4) |
| Systolic blood pressure (mmHg), median (IQR) | 118 (104–136) |
| Diastolic blood pressure (mmHg), median (IQR) | 73 (65–82) |
| Heart rate (b.p.m.), median (IQR) | 75 (67–86) |
| Weight (kg), median (IQR) | 77 (67–90) |
| Creatinine (mmol/L), mean (SD) | 109 (47) |
| Urea (mmol/L), mean (SD) | 10 (4.8) |
| Haemoglobin (g/dL), median (IQR) | 13 (12–14) |
SD, standard deviation; IQR, interquartile range; NYHA, New York Heart Association; MLWHF, Minnesota Living With Heart Failure questionnaire; EQ5D, European quality-of-life score; b.p.m., beats per minute.
During the median observation period of 6 months (interquartile range [IQR] 3.6–9.2 months), three patients (6%) died (one due to severe COPD and right-sided heart failure, one due to acute myocardial infarction, and one due to cancer complications). One patient (2%) underwent cardiac transplantation and 15 patients (29%) had at least one unscheduled admission to the hospital, of which 5 were due to cardiorenal problems. Three other patients (6%) had an elective heart failure-related admission. The median MLWHF score was 35 (IQR 18–57) and did not change significantly over time (P = 0.93). Its physical and emotional subdomains did not show any significant trend over time either (P = 0.79 and 0.70, respectively).
Usability assessment
During the second week of study participation, 97% of attempted BP measurements and 97% of attempted weight measurements were completed successfully with display of results to the user and onward transfer of the data to central servers. The rate of successful first attempts remained high with 95 and 99.6% for BP and weight measurement, respectively, at the end of the follow-up period (P = 0.46 and 0.05, for change in BP and weight measurement, respectively).
Durations of the measurements for completion of the diary, BP/heart rate measurement, and weight measurement are shown in Figure 1. The time spent completing the symptom diary declined from 18 s (robust confidence intervals [CI] 13–25 s) in the second week of study participation to 3.9 s at the end of the study (CI 2.8–4.9 s; P for trend <0.001), possibly as a result of learning as well as simplification of the questionnaire. In the second week of the study, it took participants 32 s (CI 29–35 s) to measure their weight and 65 s (CI 58–71 s) to measure BP and heart rate. These durations did not change materially over time. By the end of study, it took participants about 1.5 min (96 s) to complete all three tasks.
Figure 1.
Duration of self-monitoring activities by week of study. Mean duration and 95% confidence intervals for (A) diary measurement, (B) blood pressure measurement, and (C) weight measurement over the study duration.
The frequency of system usage declined modestly over time due to fatigue (because no active personalized intervention was provided and/or clinical findings did not vary much) and medical advice (because patients were clinically stable; Figure 2). Nonetheless, at the end of the 45-week observation period, participants still used the system on 4.5 days per week (CI 3.2–5.7 days). These high adherence rates were broadly consistent with the survey of participants' attitudes towards usefulness and accessibility of the system (Figure 3). 83% of participants reported that they were using the system fully independently and the remainder required some assistance from their caregivers. 93% of participants felt that the system was easy or very easy to use. Despite the fact that no actual medical intervention was offered in this study, 78% of participants agreed or strongly agreed to the statement that ‘using the system helps me manage my health better’. All participants indicated that they would recommend the system to others, should it become more widely available. At the final visit, when the home monitoring system was to be retrieved, 20 participants (38%) asked whether they could keep it for self-management support after study completion.
Figure 2.
Frequency of system usage by the week of study.
Figure 3.
Participants' attitudes towards the usefulness and accessibility of the system at final evaluation. (The survey was conducted to assess patients' experiences and attitudes when using the latest iteration of the home monitoring system). Consequently, those who have died during the course of study (three participants) and those who stopped using the system before the final evaluation (three participants) were not included in the assessment.
Discussion
Through co-design with patients and their caregivers, we were able to develop a reliable and easy-to-use system for heart failure home monitoring with high user satisfaction. The monitoring system allowed participants to measure their health status in <2min, access educational material about heart failure, and wirelessly send and receive information about their health status and usage behaviour. Despite the high average age of participants with little prior experience with using digital devices, at the end of the study most (83%) reported using the system alone without getting any additional help. Although this study did not intend to follow-up participants over longer term, nor did it include any medical intervention or adherence management advice, a large proportion of participants valued the sense of connectivity that the monitoring system offered. Participants continued to use the system on 4.5 days per week up to the end of the study and over a third asked to keep the system for self-management after the study completion.
One of the major barriers to the widespread use of digital technologies for management of chronic diseases such as heart failure has been their limited usability and inadequate incorporation of user feedback into system design and evaluation.11,21 In SUPPORT-HF, we were able to work with patients and their caregivers from the outset to co-design the system and develop a better understanding of the needs and capacity of high- as well as low-risk heart failure patients.
Through automated collection of user interactions with the system and their combination with ethnographic observations and interviews, we were able to monitor and learn users' behaviour, and modify the system accordingly. This approach allowed us to develop and refine several system features that enabled the elderly participants with limited technology literacy to use the system with confidence. These features for example included a simple touch button to request contact and remote automatic updates of the software without the need for any steps taken by the patients.
Although the high rate of patient engagement and their sense of empowerment by the technology are the first necessary requirements for successful monitoring and management of patients with chronic conditions, these will not be sufficient for keeping the promise of digital health. So far, little has been done to integrate the home monitoring data with other sources of health information such as electronic health records. The convergence of information from different sources combined with high user engagement rates makes more accurate prediction of risk and treatment personalization possible. In addition, the changing needs of patients require continuous adaptation of the system for personalized feedback and communication with the patients. Future studies, including the SUPPORT-HF 2 trial (ISRCTN86212709), will test the hypothesis that integrated and IT supported models of care delivery can lead to better medical outcomes with greater system efficiency.
In conclusion, we developed an adaptive and user-friendly home monitoring system that enabled a wide range of heart failure patients, even those with no prior competency in using digital technologies, to monitor their health status regularly. Despite no active medical intervention, patients benefited from the reassurance and sense of connectivity that the monitoring system provided. How to make use of this reliable connection between patients and healthcare providers for more effective and efficient healthcare delivery requires further interventional studies.
Supplementary material
Funding
K.R. is funded by an NIHR Career Development Fellowship grant and supported by the NIHR Oxford Biomedical Research Centre. C.A.E. is funded by the Rhodes Trust. SUPPORT-HF was funded by the NIHR Oxford BRC programme, and by an NIHR Career Development Fellowship to K.R.
Conflict of interest: none declared.
Supplementary Material
Acknowledgements
We thank our study participants for their support of our research. Educational material within the described intervention was extracted from heartfailurematters.org and was reproduced with kind permission from the European Society of Cardiology and the British Heart Foundation. We are also grateful to the Health Experiences Research Group, Department of Primary Care Health Sciences, University of Oxford for sharing interview extracts with us from their study of heart failure, which are also available on the www.healthtalkonline.org website.
Appendix
SUPPORT-HF investigators
Study management committee
Kazem Rahimi (George Institute for Global Health, University of Oxford), Carmelo Velardo (Institute of Biomedical Engineering, University of Oxford), Andreas Triantafyllidis (Institute of Biomedical Engineering, University of Oxford), Nathalie Conrad (George Institute for Global Health, University of Oxford), Syed Ahmar Shah (Institute of Biomedical Engineering, University of Oxford), Tracey Chantler (George Institute for Global Health, University of Oxford), Hamid Mohseni (George Institute for Global Health, University of Oxford), Emma Stoppani (George Institute for Global Health, University of Oxford), Francesca Moore (George Institute for Global Health, University of Oxford), Chris Paton (George Institute for Global Health, University of Oxford), and Lionel Tarassenko (Institute of Biomedical Engineering, University of Oxford).
Steering committee
John Cleland (University College London, London); Felicity Emptage (patient representative); Tracey Chantler (George Institute for Global Health, University of Oxford); Andrew Farmer (Nuffield Department of Primary Care Health Sciences, University of Oxford); Raymond Fitzpatrick (Department of Public Health, University of Oxford); Richard Hobbs (Nuffield Department of Primary Care Health Sciences, University of Oxford); Stephen MacMahon (George Institute for Global Health, University of Oxford); Alan Perkins (patient representative), Kazem Rahimi (George Institute for Global Health, University of Oxford); Lionel Tarassenko (Institute of Biomedical Engineering, University of Oxford).
Other investigators and collaborators
Paul Altmann (Oxford University Hospitals NHS Trust); Badri Chandrasekaran (Great Western Hospitals NHS Foundation Trust, Swindon); Connor A. Emdin (George Institute for Global Health); Johanna Ernst (George Institute for Global Health and Institute of Biomedical Engineering); Paul Foley (Great Western Hospitals NHS Foundation Trust); Fred Hersch (George Institute for Global Health); Gholamreza Salimi-Khorshidi (George Institute for Global Health); Joanne Noble (Oxford Health NHS Foundation Trust); Mark Woodward (George Institute for Global Health).
Contributor Information
SUPPORT-HF Investigators:
Kazem Rahimi, Carmelo Velardo, Andreas Triantafyllidis, Nathalie Conrad, Syed Ahmar Shah, Tracey Chantler, Hamid Mohseni, Emma Stoppani, Francesca Moore, Chris Paton, Lionel Tarassenko, John Cleland, Felicity Emptage, Tracey Chantler, Andrew Farmer, Raymond Fitzpatrick, Richard Hobbs, Stephen MacMahon, Alan Perkins, Kazem Rahimi, Lionel Tarassenko, Paul Altmann, Badri Chandrasekaran, Connor A Emdin, Johanna Ernst, Paul Foley, Fred Hersch, Gholamreza Salimi-Khorshidi, Joanne Noble, and Mark Woodward
References
- 1. Callender T, Woodward M, Roth G, Farzadfar F, Lemarie J-C, Gicquel S, Atherton J, Rahimzadeh S, Ghaziani M, Shaikh M, Bennett D, Patel A, Lam CSP, Sliwa K, Barretto A, Siswanto BB, Diaz A, Herpin D, Krum H, Eliasz T, Forbes A, Kiszely A, Khosla R, Petrinic T, Praveen D, Shrivastava R, Xin D, MacMahon S, McMurray J, Rahimi K. Heart failure care in low- and middle-income countries: a systematic review and meta-analysis. PLoS Med 2014;11:e1001699. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2. Rahimi K, Patel A, MacMahon S. Two decades of research on innovative models of care delivery for patients with heart failure: the end or just the beginning? Arch Iran Med 2012;15:439–445. [PubMed] [Google Scholar]
- 3. Rahimi K, Bennett D, Conrad N, Williams TM, Basu J, Dwight J, Woodward M, Patel A, McMurray J, MacMahon S. Risk prediction in patients with heart failure: a systematic review and analysis. JACC Heart Fail 2014;2:440–446. [DOI] [PubMed] [Google Scholar]
- 4. Ariti CA, Cleland JGF, Pocock SJ, Pfeffer MA. Congestive heart failure days alive and out of hospital and the patient journey in patients with heart failure: insights from the Candesartan in Heart failure: Assessment of Reduction in Mortality and morbidity (CHARM) program. Am Heart J 2011;162:900–906. [DOI] [PubMed] [Google Scholar]
- 5. Hancock HC, Close H, Fuat A, Murphy JJ, Hungin APS, Mason JM. Barriers to accurate diagnosis and effective management of heart failure have not changed in the past 10 years: a qualitative study and national survey. BMJ Open 2014;4:e003866. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6. Christensen CM, Grossman JH, Hwang J. The Innovator's prescription: a disruptive solution for health care. New York: McGraw Hill, 2009. [Google Scholar]
- 7. Anker SD, Koehler F, Abraham WT. Telemedicine and remote management of patients with heart failure. Lancet 2011;378:731–739. [DOI] [PubMed] [Google Scholar]
- 8. Desai AS, Stevenson LW. Connecting the circle from home to heart-failure disease management. N Engl J Med 2011;363:2364–2367. [DOI] [PubMed] [Google Scholar]
- 9. Pandor A, Thokala P, Gomersall T, Baalbaki H, Stevens JW, Wang J, Wong R, Brennan A, Fitzgerald P. Home telemonitoring or structured telephone support programmes after recent discharge in patients with heart failure: systematic review and economic evaluation. Health Technol Assess 2013;17:1–207, v–vi. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10. Chaudhry SI, Mattera JA, Curtis JP, Spertus JA, Herrin J, Lin Z, Phillips CO, Hodshon BV, Cooper LS, Krumholz HM. Telemonitoring in patients with heart failure. N Engl J Med 2010;363:2301–2309. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11. Catwell L, Sheikh A. Evaluating eHealth Interventions: the need for continuous systemic evaluation. PLoS Med 2009;6:e1000126. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12. Triantafyllidis A, Velardo C, Chantler T, Shah SA, Paton C, Khorshidi R, Tarassenko L, Rahimi K. A personalised mobile-based home monitoring system for heart failure: the SUPPORT-HF study. Int J Med Inform 2015; 10.1016/j.ijmedinf.2015.05.003. [DOI] [PubMed] [Google Scholar]
- 13. Munn-Giddings C, Winter R. A handbook for action research in health and social care. London: Routledge; 2013. [Google Scholar]
- 14. Meso P, Jain R. Agile software development: adaptive systems principles and best practices. Inf Syst Manag 2006;23:19–30. [Google Scholar]
- 15. Holland R, Rechel B, Stepien K, Harvey I, Brooksby I. Patients’ self-assessed functional status in heart failure by New York Heart Association class: a prognostic predictor of hospitalizations, quality of life and death. J Card Fail 2010;16:150–156. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16. Ergonomic requirements for office work with visual display terminals (VDTs). ISO International Standards; 1998;9214–9241. [Google Scholar]
- 17. Daniels J, Fels S, Kushniruk A, Lim J, Ansermino JM. A framework for evaluating usability of clinical monitoring technology. J Clin Monit Comput 2007;21:323–330. [DOI] [PubMed] [Google Scholar]
- 18. Davis FD, Bagozzi RP, Warshaw PR. User acceptance of computer technology: a comparison of two theoretical models. Manage Sci 1989;35:982–1003. [Google Scholar]
- 19. Vandenbroucke JP, von Elm E, Altman DG, Gøtzsche PC, Mulrow CD, Pocock SJ, Poole C, Schlesselman JJ, Egger M. Strengthening the Reporting of Observational Studies in Epidemiology (STROBE): explanation and elaboration. PLoS Med 2007;4:e297. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20. National Heart Failure Audit: April 2011–March 2012. London: National Institute for Cardiovascular Outcomes Research (NICOR), the Institute of Cardiovascular Science, University College London; 2013.
- 21. Lilford RJ, Foster J, Pringle M. Evaluating eHealth: how to make evaluation more methodologically robust. PLoS Med 2009;6:e1000186. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.