RadRemind!—an Automated Paging System for Reminding Radiology Residents of Their Roles and Responsibilities

Abstract

Tasks which are scheduled at irregular intervals afford greater scheduling flexibility but are also more difficult to remember. Difficulty remembering clinical tasks not only create potential inefficiencies in patient care but were also identified as a source of anxiety among our residents. We implemented RadRemind! an automated system of pager notifications in order to improve patient care and reduce residents’ anxiety. RadRemind! utilizes only an external shared APACHE, MySQL, PHP server. A set of CRON jobs activate a PHP script which pulls information from our resident schedule as JSON data. It then identifies the appropriate residents to notify and then sends a cURL request to a web-based paging service to trigger notifications to residents’ pagers. Each activation of the script was logged in an SQL database. An anonymous survey assessing multiple aspects of efficacy was sent to residents after 3 months of implementation. Seventeen of 29 residents responded to the survey. Residents reported a mean of 2 successful reminders (defined as responsibilities which had been otherwise forgotten prior to the page notification) in the last month which was found to be statistically significant via one-sample t test (t = 3.3, p < 0.01). Paired t test showed a statistically significant (t = 2.9, p = 0.01) decrease of 2 points in reported anxiety. Repeated measures analysis of variance found a statistically significant variation in reported utility (F(3,16) = 15.9, p < 0.01)) by type of reminder such that reminders for interventional radiology call were found to be more useful than reminders for other tasks. Use of an automated paging system is an effective method of reminding residents of irregularly scheduled responsibilities and is associated with reduced scheduling related anxiety.

Supplementary Information

The online version contains supplementary material available at 10.1007/s10278-021-00437-2.

Background

Tasks which are scheduled at irregular intervals afford greater scheduling flexibility than regularly scheduled tasks (e.g., every third day); however, irregularly scheduled tasks are also more difficult to remember. For radiology residents, difficulty keeping track of assigned tasks creates potential inefficiency in patient care as well as formation of negative psychological patterns including scheduling-related anxiety.

Automated reminders have been shown to improve patient compliance in the setting of mammographic screening [1] and vaccinations [2]. In the residency setting, use of automated reminders improves residents’ compliance with ambulatory care protocols [3], faculty evaluations [4], and procedure documentation [5].

We thus implemented RadRemind! (an automated system of scheduled pager notifications) in order to gently remind residents of when they are scheduled for a particular task and investigate whether this reduced their sense of anxiety.

Intervention

Context

In our residency, 29 residents rotate at 2 primary sites, 1 secondary site, and 2 external locations. At each primary site, daily resident responsibilities include study protocoling, covering evening calls, and overnight interventional radiology call based on a schedule with irregular intervals. Each week, there are 1 to 2 resident run didactic lectures in addition to weekly tumor board/multidisciplinary conferences at each primary site. The pool of available residents for each task depends on numerous factors including rotation assignment, competing responsibilities, and attendance at conferences (e.g., RSNA or AIRP).

A schedule which assigns each role to a resident is published prior to the beginning of the month and is also edited in real-time as needed. Like irregularly scheduled intervals, real-time reassignment also improves scheduling flexibility; however, it also creates situations in which one might be assigned to a task which they were not aware of if they had not checked the schedule since the time the change was made.

As the protocoling responsibility begins in the morning, short call begins in the evening, IR call extends overnight, and assignments may change throughout the day, there are multiple points during the day when one might check the schedule in order to verify whether there is an imminent responsibility to fill. Fear of failure to timely check or even successfully checking but failure to remember whether one had checked was identified as a source of anxiety among residents.

Detailed Implementation

Our day-to-day schedule is hosted on month-specific Google spreadsheets. Spreadsheet data was retrieved as JSON [6] by a PHP script which took a specified role of interest (e.g., protocoling, short call, etc.) as an argument and identified the resident/associated beeper number assigned to the specified role and day. Our institution’s pager notification system allows paging through a CGI form with POST parameters which were utilized via cURL. The script was activated at specified times as CRON jobs which triggered notifications to the assigned resident(s) via pager. Options for e-mail and text message notification were also created though not used due to poor reception in the hospital. A logging function was added to analyze overall uptime. A schematic of the process is depicted in Fig. 1 and code samples are included in the supplemental materials.

Fig. 1.

Schematic diagram of RadRemind! Scheduling information on Google Spreadsheets is accessed by a PHP script which interfaces with a web-based paging service and logs paging notifications at specified times

The hardware and network requirements of the implementation were minimal requiring only a server running Apache, PHP, and MySQL. Use of a shared server was inexpensive and non-prohibitive as the LAMP server configuration (Linux, Apache, MySQL, PHP) is the most ubiquitous shared hosting configuration.

Timing of Notifications

The greatest volume of feedback and alterations to the implementation concerned timing of notifications with respect to specific tasks. Two general classes of notifications were found to vary in optimal timing characteristics: notifications sent while the resident was present at work and notifications while the resident was off duty. Notifications sent during the work day were selected to be in temporal proximity to the task while not conflicting with other tasks (e.g., notification for protocoling 10 min prior to shift at the end of morning lecture) while notifications to off-duty residents were selected to occur at nonobtrusive times (e.g., notification for weekend IR call at 4PM the prior day). The timing of notifications with respect to scheduled tasks are shown in Table 1.

Table 1.

Timing of notifications varied based on specific task. Weekday (a, left) and weekend (b, right) schedules varied in their optimal notification times

Survey Data

An anonymous survey was sent to all residents with the following items:

1. Prior to the automated messaging system, how much anxiety did you experience regarding supervision/call? (1–10 scale)

2. After use of the automated system, how much anxiety do you experience regarding supervision/call? (1–10 scale)

3. During the last month (excluding vacation/VA), how many times were you on call? (free number)

4. During the last month (excluding vacation/VA), how many times were you on protocoling? (free number)

5. How useful have the pages been for protocoling? (1–10 scale)

6. How useful have the pages been for call? (1–10 scale)

7. How useful have the pages been for resident conference? (1–10 scale)

8. How useful have the pages been for IR call? (1–10 scale)

9. Within the last month, how many times has the automated paging system reminded you of a responsibility that you were not aware/had otherwise forgotten of prior to being paged? (free number response)

10. Any suggestion for changes or new features? (free text response)

Statistical analysis was performed in Octave.

Outcomes

Seventeen of 29 residents responded to the survey. Sixteen respondents had received notifications for IR call and 13 had received notifications for resident conference by the time of the survey. Descriptive statistics for selected measures are shown in Table 2.

Table 2.

Mean, standard deviation (SD), and number of respondents for selected numerical measures assessed via anonymous survey

Survey results
Measure	Mean (± SD)	Number of respondents
Pre-intervention anxiety	4.6 (2.6)	17
Post-intervention anxiety	2.6 (1.7)	17
Number of times on call	7.8 (3.2)	17
Number of times protocoling	3.8 (2.9)	17
Usefulness for call	7.8 (3.3)	17
Usefulness for protocoling	7.8 (3.3)	17
Usefulness for resident conference	7.4 (3.5)	13
Usefulness for IR call	8.6(2.5)	16

Reported Anxiety

Respondents reported decreased anxiety from a mean pre-intervention anxiety rating of 4.6 to 2.6 post-intervention (Fig. 2) which was statistically significant via paired t test (t(16) = 2.9, p = 0.01).

Fig. 2.

Box-whisker plot showing pre-intervention, post-intervention, and change in residents’ reported anxiety. Paired t test showed a statistically significant decrease in anxiety (t(16) = 2.9, p = 0.01) with mean decrease of reported anxiety of 2.3 (CI 5–95: 0.5, 4.1)

Successful Reminders

Residents reported a mean of 2.2 successful reminders (defined as responsibilities which had been otherwise forgotten prior to the page notification; Fig. 3) in the last month which was statistically significant via one-sample t test (t(16) = 3.3, p < 0.01).

Fig. 3.

Histogram showing number of respondents reporting specific numbers of successful reminders. Residents reported a mean of 2 successful reminders (defined as responsibilities which had been otherwise forgotten prior to the page notification) in the last month which was found to be statistically significant via one-sample t test (t(16) = 3.3, p < 0.01)

Utility by Type of Reminder

Repeated measures analysis of variance (ANOVA) found a statistically significant variation in reported utility (F(3,16) = 15.9, p < 0.01) by type of reminder such that reminders for interventional radiology call were found to be more useful than reminders for other tasks.

Correlation Analysis

Correlation analysis was performed to assess relationship between change in anxiety, reported utility, and number of successful reminders or number of times on task within the last month. Participants who had not received notifications for IR call or resident conference were removed from dataset and 13 total responses were analyzed. Strong correlations were found between respondents’ assessment of the utilities of reminders for call, protocoling, and IR call. However, no other significant correlations were found (Table 3).

Table 3.

Correlation table for analysis to assess correlation between change in anxiety, reported utility of notification, and selected numerical measures. Strong correlations were found between respondents’ assessment of the utilities of reminders for call, protocoling, and IR call. However, no other significant correlations were found (*p < 0.01)

		1	2	3	4	5	6	7	8
1	Change in anxiety	1
2	Number of successful reminders	− 0.52	1
3	Usefulness for call	− 0.36	0.41	1
4	Usefulness for protocoling	− 0.40	0.41	*0.98	1
5	Usefulness for resident conference	− 0.32	0.43	0.27	0.23	1
6	Usefulness for IR call	− 0.35	0.42	*0.95	*0.94	0.28	1
7	Number of times on call	− 0.29	− 0.29	− 0.06	− 0.03	− 0.36	− 0.02	1
8	Number of times on protocoling	0.01	− 0.16	0.11	0.17	− 0.29	0.14	0.33	1

Discussion

Successful Reminders and Reduced Scheduling Anxiety

Use of an automated paging system was an effective method of reminding residents of responsibilities that might otherwise have been forgotten. Furthermore, the presence of an automated notification system led to a reduction in reported scheduling-related anxiety which was not correlated with the number of the times residents were successfully reminded. Thus, presence of the system itself was associated with reduced anxiety.

Utility by Reminder Type

Reminders for IR call were found to be more useful than reminders for other types of call. IR call is unique in that it is home call and that emergent interventions should be able to be arranged within 30 min of case acceptance. It is possible that reminders for IR call were found to be more useful due to preventing residents from engaging in activities that would potentially prohibit adequately timely response.

Selection Limitation

Resident response was limited and self-selected. Additionally, a number of respondents had not received notifications for specific types of reminders (e.g., first year residents who have not had IR rotation or had not been assigned a resident lecture). It is possible that residents who felt strongly enough to reply to the survey were also those who experienced an impact in anxiety; however, informal feedback from a few residents who identified themselves as non-responders indicated that many were satisfied with the intervention but felt that there were too many departmental surveys at the time to respond to and they forgot.

Uptime

During the project, numerous external factors affected service uptime. Migration of the external host to new DNS servers, server-wide upgrade of PHP version, and changes to php configuration caused a failure to send 10 notifications throughout 3 months of log monitoring (failure rate of 1%). The benefit of a shared hosting server includes inexpensiveness and relatively minimal server upkeep; however, there is limited control over certain factors which may affect overall uptime.

Notification via Pager

Throughout duration of deployment, there was discussion regarding whether notification via paging was the optimal method of notification. The decision to utilize paging rather than cell phone or e-mail notification was site-specific and based on comparatively improved network coverage compared with SMS and Internet which were unreliable within the hospitals. One survey respondent suggested redundant notification by E-mail as well as paging; however, when suggested, a few expressed concerns regarding inbox clutter. Optimal method of notification is an ongoing consideration and influenced by multiple factors specific to the context of implementation.

Conclusion

Use of an automated paging system is an effective method of reminding residents of irregularly scheduled responsibilities and reduces residents’ scheduling-related anxiety.

Attestation

On behalf of all authors listed, I attest that this work is the original work of all authors listed, that all authors have seen and approved this work and that this article has neither been published nor is under consideration elsewhere.

Supplementary Information

Below is the link to the electronic supplementary material.

Funding

This project has not been funded.

Declarations

Conflict of Interest

The authors declare that they have no conflict of interest.