An analysis of the development and implementation of a smartphone application for the delivery of antimicrobial prescribing policy: lessons learnt

Abstract

Objectives

Smartphone usage amongst clinicians is widespread. Yet smartphones are not widely used for the dissemination of policy or as clinical decision support systems. We report here on the development, adoption and implementation process of the Imperial Antimicrobial Prescribing Application across five teaching hospitals in London.

Methods

Doctors and clinical pharmacists were recruited to this study, which employed a mixed methods in-depth case-study design with focus groups, structured pre- and post-intervention survey questionnaires and live data on application uptake. The primary outcome measure was uptake of the application by doctors and its acceptability. The development and implementation processes were also mapped.

Results

The application was downloaded by 40% (376) of junior doctors with smartphones (primary target user group) within the first month and by 100% within 12 months. There was an average of 1900 individual access sessions per month, compared with 221 hits on the Intranet version of the policy. Clinicians (71%) reported that using the application improved their antibiotic knowledge.

Conclusions

Clinicians rapidly adopted the mobile application for antimicrobial prescribing at the point of care, enabling the policy to reach a much wider audience in comparison with paper- and desktop-based versions of the policy. Organizations seeking to optimize antimicrobial prescribing should consider utilizing mobile technology to deliver point-of-care decision support. The process revealed a series of barriers, which will need to be addressed at individual and organizational levels to ensure safe and high-quality delivery of local policy at the point of care.

Introduction

Evidence-based medicine has driven the use of formal policies in clinical decision making, with international and national guidelines for clinical practice being used to develop these policies. A wealth of literature reports on adherence of healthcare professionals to local clinical policies as a marker for patient safety and treatment efficacy.^1,2 One area where the impact of local policy and guidelines is well documented is in the treatment of infection.¹ The provision of local policy, guidelines and clinical decision-making tools to prescribers has been shown to be an effective means of optimizing antimicrobial prescribing.^1,2 In efforts to improve clinical practice and adherence in this field, healthcare institutions are constantly developing innovative ways to widen access to local policy and guidelines, including the use of pocket guides, posters, the Internet and electronic prescribing tools.¹

Timely access to medical information is a key factor in ensuring effective and safe medical practice.³ Technology has the potential of improving accessibility to data and resources.⁴ The use of smartphone technology by healthcare professionals can help improve standards of care, while the ubiquitous use of smartphones has also influenced the way in which evidence-based medicine is communicated to healthcare professionals.⁵ There is now an ever-increasing choice of commercial medical applications available for download on smartphones. Amongst these, a number are dedicated to the diagnosis and treatment of infectious diseases.⁶ The convenient ‘pocketable’, ‘anywhere, anytime’ access to medical information via smartphones is an attractive prospect for healthcare professionals increasingly at ease with mobile technology.^7,8 In light of the proliferation of mobile technology in healthcare, the US FDA and the European Commission have issued interim guidelines, initiating the development of standards and definitions for the use of mobile technology in healthcare.^9,10

Given the high rates of adoption of smartphones in the medical community^5,11 and the increasing number of medical software applications being developed,^6,12 the use of smartphones as a point of access to local policy resonates with the broader strategy of utilizing electronic health (‘eHealth’) and mobile communications to contribute towards high-quality and safe healthcare.^13–15 An understanding of the potential risks and benefits of mainstreaming mobile health technology interventions in the field of antimicrobial prescribing is critical, given the increasing use by clinicians and high levels of interest by healthcare organizations and policy makers. Yet, to date, there has been no in-depth systematic risk assessment. The ubiquitous use of mobile technology, especially smartphones in clinical settings, offers an ideal platform for the dissemination of point-of-care medical information to clinicians. Despite this, there exists a level of scepticism by healthcare professionals towards the use of innovative technologies, and this has a direct impact on their adoption and consistent use.¹⁶ This is the result of a series of barriers and lack of information and knowledge that is required to use an innovation properly at both the individual and organizational levels.^17,18

At Imperial College Healthcare NHS Trust (ICHT), which consists of five teaching hospitals at three sites in West London, the antimicrobial prescribing policy is available to clinicians in many formats, including a poster, pocket guide and on the Trust's Intranet. In this study, we report on the development of the policy into a free smartphone application known as the ‘IAPP’ (Imperial Antibiotic Prescribing Policy) and its uptake and acceptability amongst clinicians. Furthermore, we map quantitatively and qualitatively the adoption process and discuss the identified barriers and potential unwanted consequences at the individual and organizational levels of implementing a mobile application for antimicrobial prescribing, and provide potential solutions to addressing them.

Methods

Design

The development of the IAPP was clinician-led and supported by the organizational hierarchy using resources from the academic research unit based at Imperial College, London. The IAPP was developed in iterative stages. We employed an in-depth case-study design, which involved both qualitative and quantitative methods of inquiry that informed each other and enabled triangulation of the findings.¹⁹ The selected design was appropriate to our study, which aimed to explore the technology journey in-depth and systematically search for potential risks and benefits to individuals and organizations during the implementation process. Ethical approval was obtained from the Imperial Joint Research Office, who confirmed that this work did not require submission to the Multi Centre Research Ethics Committee.

Data were collected from observational notes and the minutes taken during three focus group meetings in the early stages of development of the application, structured pre- and post-intervention (1 year after introduction) survey questionnaires (see Supplementary data, available at JAC Online) and real-time quantitative data on downloads and use of the application by clinicians over a period of 1 year. The pre-intervention survey of clinicians was used to assess: (i) the local prevalence of smartphone use amongst clinicians; (ii) clinicians' awareness and usage of the existing antibiotic policy; and (iii) their experience of using external clinical applications on their smartphones. The results of the pre-intervention questionnaire were used to inform the development of the IAPP. A software company was employed to develop the user interface of the application to our specifications. The focus groups comprised both consultant and junior doctors and pharmacists.

To garner opinions and feedback from users, the post-intervention structured questionnaire was designed and disseminated 1 month and, again, 1 year after the launch of the IAPP. Post-intervention data collection allowed sufficient time for the early adoption and diffusion of the product by target users. The method of dissemination was the same as for the pre-intervention questionnaire. Doctors attending training sessions at the postgraduate centres of the three hospital sites were asked to complete the questionnaire during two consecutive weeks. Pharmacists were e-mailed the questionnaire with a covering message explaining its purpose. A reminder e-mail was sent after 2 weeks.

Setting and time period

The study was conducted in 2011–12 over a period of 16 months, from April 2011 (inception) to July 2012 (1 year post-intervention assessment). ICHT, which consists of five teaching hospitals at three sites in West London, was purposefully selected as the research setting. The development process began in April 2011 and lasted ∼4 months. The launch date of the application was in the first week of August 2011, to coincide with the intake of newly qualified doctors. This provided an opportunity to maximize on the uptake and diffusion of the application amongst junior medical staff, who were the primary target group of the intervention.

Participants and sampling

Each of the postgraduate centres on the three sites holds two training sessions per week, with an average attendance of 30 doctors at each. Therefore, in total, the number of doctors the questionnaires could have been distributed to is 360. There are 90 clinical pharmacists across the sites and the questionnaires were e-mailed to all of them. Three focus group meetings were held over the course of the study. Focus group members included all the relevant stakeholders involved in the development and adoption of the application. At least one consultant, one junior doctor and one specialist antibiotic pharmacist participated in every focus group session.

Primary outcome measure

The primary outcome measure was the uptake of the application (IAPP) by doctors across the hospitals studied over a period of 12 months. The entire process of development and implementation was mapped to provide a detailed measure of the steps and the unwanted consequences, to ensure patient safety and quality of care was assured.

Data analysis

Data from the focus groups and the open-ended questions in the survey questionnaires were analysed using the framework approach.²⁰ Emerging themes were categorized according to the perceived risks and benefits associated with the development and implementation of the innovative application. Live data on access to the IAPP were collected via Flurry Analytics (Portable Pixels, Camden, London, UK; http://portablepixels.com/), an online analytics interface, which collated, analysed and displayed usage data in spreadsheets.

Results

Pre-intervention survey results

Seventy-one doctors and 16 clinical pharmacists responded to the structured pre-intervention questionnaires. The doctors comprised registrars and junior-level trainees. The response rate was 20% (71/360) for doctors and 18% (16/90) for pharmacists. Eighty-two percent of doctors (58/71) and 75% of pharmacists (12/16) reported using a smartphone at work. Android and iPhone devices made up 90% of the smartphones used, with 76% of the respondents stating that they would use a smartphone version of the app if it was made available.²¹ All of the respondents were aware of the policy. Half of the doctors reported using commercially available applications to inform their practice.

Mapping the development and implementation process

Development

The development process included stakeholder input from information technology (IT), application designers and the organization's communication team. Additional functionalities, e.g. creatinine clearance, weight-related dosing in obesity and therapeutic drug monitoring functions, were embedded as decision support tools into the final product. All clinical calculators had to be validated by the clinical pharmacy team.

During the development process, the IAPP was adapted to the local context. Due to parts of the hospital sites having very poor Wi-Fi connectivity and limited Internet access, it was decided that the app should be ‘native’ and work directly from the device it is downloaded on. Due to the application containing contact information for individual staff, it was decided not to host it on the iTunes store and a separate external uniform resource locator (‘URL’) was created from where the app could be downloaded. The IAPP was username and password protected and was disseminated to all staff as described below. The final IAPP product comprised a mobile evolved version of the policy (Figure 1a and b) with additional functionality, including therapeutic drug monitoring and clinical calculators, such as those for creatinine clearance and ideal and obese body weight dosing (Figure 1c and d).

Figure 1.

Snapshots of the final IAPP product. Further colour examples of the app are available from www1.imperial.ac.uk/medicine/about/institutes/cipm/centre_outputs/antimicrobialstewardship.

The development of the app for iPhone and Android devices cost £5000 initially, with an additional annual cost of £1500–2500 for updates. Each additional update to the policy cost between £400 and £800 during 2011–12. The convenience of updating the mobile version of the policy means that updates occur more frequently than the traditional annual update of the paper version. There were two updates to the policy in 2011–12. The design and printing of the pocket and poster versions of the policy cost £2000–3000 annually, with an initial charge of £3000 for design.

Implementation and adoption

The ICHT employed a multimodal strategy to disseminate the IAPP using four main communication channels: (i) teaching sessions on pharmacy for junior doctors in the postgraduate centres at all three sites; (ii) e-mails sent to all new doctors in the Trust; (iii) the Intranet homepage that appears each time the Internet browser is opened on a Trust computer; and (iv) the Trust's newsletter. In each case, potential users were provided with ‘on the spot’ information on how to download the application on their smartphones.

The IAPP was launched as a freely available application to all hospital staff on 1 August 2011. Within the first month of release, 40% (376) of all new junior doctors had downloaded the IAPP onto their smartphones (iPhone/Android). The adoption rate remained steady (Figure 2) throughout the 12 months. One year after the launch of the application (August 2012), there were 990 unique users, equivalent to 100% of the primary target users (junior doctors with smartphones). The average number of monthly users was 250–300. The monthly average number of individual sessions on the app is 1900 (Figure 3), compared with 221 sessions per month on the Intranet version of the policy. On average, ∼100 individual sessions were recorded per day. As part of their regular teaching sessions for junior doctors, the specialist antibiotic pharmacists provided instructions for downloading the IAPP. These sessions are indicated in Figure 3 and correspond to a rise in the number of downloads.

Figure 2.

Monthly adoption rate of the application by staff over a 12 month period.

Figure 3.

Using smartphone technology to track the use of policy in real time over its first month of release.

Updating the policy

Following a series of changes to the paper policy, the smartphone app was updated in June 2012 and again in September 2012, and a pop-up message was sent to all users to update their mobile devices. During the first update, it was identified that many people did not remember their password or username and, to make downloading easier, the password and username were removed. A more serious problem identified was poor access to Wi-Fi and the Internet, which disrupted the download process of the application. Many staff had outdated software on their mobile phones and were not able to successfully download the updated application to their mobile devices. In anticipation of the former, the application was created to work offline as a native app, but the problems of staff not having updated their device software had not been anticipated. Furthermore, making the app ‘native’ as opposed to ‘web based’ meant that the decision to download the latest version was with the end-user, as a web-based app would automatically recognize the latest version of the software from the web provider and be automatically updated, provided there was sufficient Wi-Fi access. Each update to the application needs to be updated across both iPhone and Android devices.

Post-intervention survey results

The post-intervention questionnaire provided feedback and usage statistics. In total, 81 questionnaires were completed. This represents a 23% response rate (81/360). The most appealing features of the IAPP reported by clinical users were its usability, accessibility at point of care and transportability. Clinical decision support features, such as patient group-specific prescribing advice, e.g. elderly/frail and penicillin allergy, were also identified as features most appealing to the end-user. Of those who provided their opinion, 71% (42/59) considered that the IAPP added to their knowledge base regarding antimicrobial prescribing and 81% (48/59) stated that using the app helped them adhere to the policy. Clinicians were asked for their views on whether they would feel uncomfortable using the IAPP on their smartphones in front of patients and 20% (12/60) noted that they found it difficult to use the app on their smartphones in this situation. The main reasons provided were that it would be considered ‘unprofessional’ by patients and because of a perception that using their personal smartphones during work was against policy.

Discussion

In this study, we found a high level of smartphone usage amongst junior doctors, who were adept at using smartphone-based clinical applications. This indicates that the methods of communication used by healthcare organizations are lagging behind the needs of their technologically advanced employees. A large proportion of the target audience (40% of junior medical staff within 1 month) adopted the IAPP. The ease of adoption and use highlights the user-friendly platform offered by current smartphones. Furthermore, it indicates that healthcare organizations need to be in line with their staff when it comes to the use of the right medium for communication and sharing of information. Organizations aspiring to a sustainable channel of engagement with their staff need to shift from personal computer- and paper-based media to mobile technologies. To make full use of the potential of mobile technology in medical practice, the IT infrastructure of healthcare organizations needs to be brought up to date.

The analytic functions of smartphone applications allow for unprecedented tracking of usage of policy and information in real time, which has the potential to provide unique insights into the impact of multimodal interventions on the usage and application of information in healthcare; in this instance, we were able to demonstrate the impact of training sessions on uptake and usage of policy (Figure 3). Most of the peaks in the use of policy in the first month are attributed to teaching sessions held with junior doctors to inform them about the policy. The demand for smartphone applications aimed at healthcare professionals is rapidly growing.^6,12 The IAPP aimed to meet this demand and provide a mobile tailor-made, point-of-care reference platform developed with user involvement for the dissemination of local policy and guidelines targeting prescribing. In the case of antimicrobial prescribing, providing wider access to local policy is imperative to efforts to optimize antimicrobial prescribing, contain antimicrobial resistance and decrease the incidence of healthcare-acquired infections. Overall, we found that providing the antimicrobial policy as a smartphone application allowed us to make it available to a wider audience of users, who reported that using the application helped them adhere to policy and improved their antimicrobial knowledge.

The diffusion of mobile technology amongst healthcare staff and the increasing use of this technology to provide medical information has led to questions about the impact of using mobile devices in healthcare settings and the potential for disruptions in care,²² with calls for better regulation.²³ Like all new systems introduced in healthcare, a robust risk assessment is required to ensure that the unintended consequences of introducing mobile systems for the provision of ‘on the go’ medical information are identified and addressed. We identified several barriers, both organizational and individual (Table 1), that need to be recognized to ensure the safe and effective use of mobile technology to deliver point-of-care decision support in acute care.

Table 1.

Barriers to adopting smartphone technology to access an antimicrobial decision support tool

Organizational barriers	Individual barriers	Unintended consequences of not addressing barriers
the requirement for web-based applications to enable automatic update of data and information	individual end-user's need for prompts to update native applications—provision of choice to end-users may not be appropriate	application users may choose not to update native applications and not have access to up-to-date policy to inform the choices they make for patients
organizational commitment to providing mobile health (‘mHealth’) technology across all available platforms, e.g. iPhone, Android, BlackBerry, electronic tablets	knowledge of the mobile technology being used and how to update software and upload applications correctly	risk of compromising standardization in access to and use of organizational policies
the organizational culture and policy on the use of mobile devices at work	patient and staff preferences and beliefs and attitudes about using mobile devices in clinical settings	potential for poor uptake of the technology and misinformed beliefs about the purpose of using personal devices during healthcare staff and patient consultations

Organizational barriers

Policies in healthcare are constantly reviewed and updated in light of emerging evidence. For a seamless process, applications need to be web based so that they are automatically updated. This requires and relies on the availability of network connections and Wi-Fi in all areas where clinical work is being conducted. At present, the IT infrastructure of most hospitals in the UK is not able to assure this high level of access. To address this, IAPP was developed as a native application. This type of application relies on individual users to update their applications and the software on their telephones. When the policy was updated 1year on, several users had not selected to update their application to the latest version. From the perspective of patient safety and organizational coherence, it is essential that the most up-to-date version of the policy is used. Some individuals reported inability to download the application and, when explored further, it was found that the barrier to adoption wasn't the application, but rather end-user unfamiliarity with the device they were using and not being aware of the need for regular software updates on their device. For the safe and effective use of mobile devices, the availability of revisions and updates should not be reliant on action being taken by the end-user. Rather, mobile applications capable of being automatically updated need to be developed. To achieve this, organizations can for instance use ‘cloud technology’, whereby the servers housing the applications are hosted on the Internet and accessed via mobile applications. Such systems will require robust IT support at the organizational level, to ensure they are accessible in clinical areas where they are of most use as a point of reference.

Another potential negative influence is the risk of compromising standardization in the access and use of policies and guidelines by staff. Updating the app more frequently and rapidly than the paper copy means that only healthcare professionals with smartphones are able to access the up-to-date version of the policy. Retrieving or replacing out-of-date pocket-guide and printed versions of the policy is not a feasible option. This leads to problems in the uniformity of practice and can pose a risk to patient safety if significant changes are made to the mobile policy, for instance changing the way narrow therapeutic agents are dosed and monitored. The rapidity and ease of changing the mobile version in comparison with the paper version of the policy runs the risk of two different recommendations being endorsed by the organization, leading to confusion for staff. However, the appealing feature is that using mobile technology means that any changes made to policy can be rapidly communicated with those who are able to access it on their mobile devices.

Individual barriers

Healthcare professionals who participated in the post-intervention survey expressed reluctance to using their smartphone devices in the presence of patients. This is understandable, but it need not be a deterrent as prescribers often refer to a variety of sources of information as part of their practice, e.g. web-based guidelines, the British National Formulary and pocket guides. With the decreasing use of white coats in UK hospitals, doctors are less likely to be able to carry weighty references, such as national formularies, in their pockets and smartphones are a practical alternative source of mobile information. The objection to using smartphones in the presence of patients stems from the social connotations associated with using mobile devices. Building on this work and developing further mobile applications for patients as well as for clinicians can be one way of addressing the perceived negative connotations associated with using smartphones in the work place.

Concerns about the use of mobile devices in healthcare settings and their potential as a reservoir of infection have been reported.^24–27 Clothing, skin and equipment in the clinical environment may become transiently contaminated and serve as a reservoir from which organisms can be transferred to patients.^28,29 Phones carried on the person fall into this category. The key to protecting patients is effective hand hygiene using alcohol hand gel between contact with the potentially contaminated surface, the smartphone, in this case, and the patient.³⁰ This is a key point that needs to be remembered when weighing the risks and benefits of using smartphones in the clinical setting.

Study limitations

The development and evaluation process reported here was conducted in real time, following a pragmatic approach alongside the demands of service delivery across a multisite university teaching hospital. Some of the authors were system developers of the application under study; therefore, the possibility for unintended bias in the assessment needs to be acknowledged.

Conclusions

Mobile technology, in particular the smartphone platform, offers point-of-care access to clinical references and resources and complements the more traditional platforms of policy dissemination. In this study, we have identified the rapid uptake and use of policy by healthcare staff when presented in mobile format. The process identified key barriers that require addressing in order to implement mobile technology successfully. Addressing the IT infrastructure may pave the way for the capability to support paperless systems, which will address the identified barriers. The future of smartphone application development in healthcare will depend as much on identifying opportunities for the use of such technologies, as on strategic action and vision by policy makers and managers at healthcare organizations, to integrate electronic and mobile health systems.

Planned future studies into the potential of smartphone technology as a platform for clinical decision making will include an evaluation of patient outcomes and clinical quality improvement in addition to its evaluation as a platform to support training for staff and as a means of communication with patients towards more integrated care.

Funding

This work is supported by the National Institute for Health Research Biomedical Research Centre Funding Scheme at Imperial College (funding number not applicable), the National Centre for Infection Prevention and Management (CIPM) funded by the United Kingdom Clinical Research Council (UKCRC G0800777) and the Showcase Hospitals Fund.

Transparency declarations

None to declare.

Supplementary data

The structured pre- and post-intervention (1 year after introduction) survey questionnaires are available as Supplementary data at JAC Online (http://jac.oxfordjournals.org/).