Rapid creation of a website to produce educational and clinical support resources for global use during and beyond the COVID-19 pandemic

Abstract

It became clear at the onset of the pandemic that radiography could play an important role in diagnosing and staging COVID-19. The key modality would be mobile chest radiography. However, at the onset of the pandemic, no literature existed to indicate whether or not chest X-ray imaging could be used effectively to diagnose or exclude COVID-19. This article explains how a website was created, at speed, during the initial phase of the COVID-19 pandemic. Containing holistic information, the website helped enable rapid redeployment of radiographers onto the frontline where chest X-ray imaging was needed. It aimed to help train radiographers take (and interpret) chest radiographs in high-risk areas that contained large numbers of COVID-19 patients. Within one year, the website had been used in 157 countries. This article documents the approach taken to create the website and suggestions are made about how, in the future, a rapid approach could be achieved to create other websites – should an international crisis occur again.

This paper also outlines how stakeholders and content authors from across the world were brought together and supported to create the website. It goes on to explain the leadership style that was adopted to create the website and why that style was selected. An explanation is offered about the project management approach and how its ingredients relate to a published model. Aside from simply providing a historical account of how the website was created, we hope the narrative offers food for thought on how to respond rapidly during an international crisis to formulate and implement a unified international-level solution which addresses an urgent need.

Introduction

By April 2020 the COVID-19 pandemic had gripped the world and many societies were instructed to stay at home. Social and professional mobility were severely restricted to minimise the spread of the disease and preserve life. In many countries, healthcare services were refocused to diagnosing and managing patients with COVID-19 symptoms, this included healthcare professionals being redeployed to frontline or supporting work.¹ Intensive therapy units bore the brunt of the workload and numerous wards and other facilities had to be converted into makeshift intensive therapy units to cope with the demand.² Staff redeployment meant normal services, such as cancer treatment, were either curtailed or massively reduced.

Radiography played an essential role in the diagnosis of COVID-19, and chest X-ray imaging became the primary diagnostic imaging tool. CT was known to have value in diagnosing COVID-19, but its limitations meant it was as not widely used as conventional chest radiography. The limitations included not all centres having a CT scanner, the limited throughput capacity of CT balanced against the vast numbers of people needing imaging, the large amount of time it would take to clean the CT unit and CT room between patients and the requirement to undertake chest imaging on intensive therapy units on patients who were too sick to be imaged in the radiography department.

Within the UK, by early April 2020, it became clear that radiographer staff redeployment into taking (and interpreting) mobile chest radiographs in ‘high-risk environments’ had to occur rapidly. A lack of suitable, easy to access and freely available training information to facilitate this transition meant that individual hospitals would have to create their own materials, and this would have two detrimental consequences. First, the time needed to create the materials would be high, but the training need was urgent and immediate. The delay incurred during the development of training materials would have been detrimental to patient care and during that time staff would not have been fully prepared for the clinical demands. Second, training material development activities would have needed to have been replicated across hospitals within the UK and worldwide as no national or international effort was being coordinated to address the problem. The replication of training materials development activities would have incurred tens of thousands of hours from radiography professionals, and others, from across the world. This replication of development activity would have detracted radiographers from fully engaging in patient care activities as was urgently needed.

The need to create, at speed, training and clinical support materials for taking and interpreting chest X-ray radiography became the basis of the rationale to create a singular free to access web-based resource;³ The facilitating factors leading to its creation have already been documented into an open access book.⁴ Within a year of creation, the web resource had been used in 157 countries. It went on to be nominated and shortlisted for two major awards, The Times Higher Educational Award for UK and Ireland (international category) and The Smarter Working Lives Award.

We led the development of the website and because of this the Editors of this Special Issue asked us to write an article about its creation, and to include some thoughts about ‘during and after the pandemic’. The Editors felt, by outlining the process in creating the website this article may have value in the future if other disasters were to occur and a unified international response was necessary. Due to the purpose of the article it does not have a standard layout; the article simply documents how the website was created and some reflective thoughts are included at various points. The article is divided into several subsections, which aim to group together relevant points.

Rationale for creating the website and our personal context

We proposed a need existed for training and clinical support information and that the need was at an international level, not simply local or national levels. A free to access and evidence-based web-based resource was therefore suggested by us.

We retired at the same time on 31st August 2020 and until then we were full time university academics. One of us was teaching focused, being a Senior Lecturer who had been programme leader for a bachelor's degree in diagnostic radiography for many years. The other was [full] Professor, research focused and had been Research Dean of a large multi professional school for many years, and his personal research considered aspects of diagnostic radiography. As the pandemic took hold, we were both on a trajectory to retirement, with less than six months left to work. Both of us were passionate about wanting to do something helpful during the pandemic and our contribution ended up being instigating and then leading the creation of the chest radiography website resources to image or exclude COVID-19. Knowing there would be a substantial personal and professional time investment to achieve this, we gained approval from our line managers and the support of our wife/partner. Due to us being on a path to retirement at the time of leading the creation of the website, our academic workload had started to reduce. Should this not have been the case then consistent with other radiography academic staff our workload would have increased due to the COVID-19 pandemic. Such workload increases were necessary due to academic staff having to create new online learning materials because students could not attend class, as documented in the book ‘Reflections on the COVID-19 pandemic’.⁵ A reflective thought at this point ponders on whether the website would have been created if we had not been on a path to retirement, due to competing demands.

Establishing the need for the website

Through professional speculation, insight and consultation, it became clear that radiography could play an important role in diagnosing and staging COVID-19 and the key modality would be mobile chest radiography. However, at the onset of the pandemic, no literature existed to indicate whether or not chest X-ray imaging could be used to diagnose or exclude COVID-19. Confirmation of the use of chest X-ray imaging came from two clinical sources, a consultant radiographer, and a consultant radiologist as both confirmed that it was already being used in many hospitals to do so. Having been given this confirmation we moved forwards to create the website.

We commenced by examining the literature about taking and interpreting chest radiography; it became clear that a minimal amount was freely available and evidence-based and that nothing was available on how to interpret images with COVID-19 in mind. Similarly, nothing was available nationally or internationally about how to protect the radiographer from contracting COVID-19 when taking images during mobile radiography.

A learning point at this stage is that clinical practice isn't always evidence-based in a formal fashion, whereby rigorous research has been published and then it is used to guide practice. Clinical practice can evolve experientially, on the intuition and working experience of practitioners and much later it might become the focus of research activity to prove, formally, its efficacy. Thus, particularly in a time of crisis such as during the COVID-19, it is worthwhile examining published literature and people's new experiences and practices, to help identify actual problems and potential solutions.

Defining the project

Normally, before conducting a project, many key factors are considered in detail. Over a protracted period, several draft documents are produced and debated and eventually a project proposal is agreed ready for resourcing and subsequent implementation. The process of creating a detailed and well-made proposal is an art and can take many months. In the case of the COVID-19 chest X-ray website being proposed here, we did not create a formal project proposal, and the project commenced within 48 h of conception. The reason why we took this approach is the luxury of having time to reflect and think did not exist–the solution needed to be created and implemented as quickly as possible. We did not know how long the pandemic would last, so if we took too long to create the website the pandemic could be over and the work we were doing would have no value. Equally, if we took too long then many people could have died through lack of information about using chest X-ray, interpreting it and the protective measures needed by radiographers to minimise the chance of them contracting COVID-19.

Against this backdrop we decided some components of the website should go live within 5 weeks of commencement. We managed to achieve this target but to do so we focused the development work into the more critical components first. Development of the website materials carried on until about week 10, at which point all components of the website were live. With hindsight, taking a phased approach to making the website live was the right thing to do as people started accessing it immediately. In fact, demand was so high that in the first day the International Society of Radiographers and Radiologic Technologists (ISRRT) eLearning platform crashed several times. In the future, should the need arise again, a phased approach should be considered to making a website live should be taken instead of waiting until the whole website has been created, as the additional delay in doing so could impact negatively in radiography practice, radiographer wellbeing and ultimately patient care and management.

We agreed that the website materials should be easy to access and freely available. The reason for this was attributed to us seeing the problem and our solution as being international and that not all countries or individuals would be able to pay an access fee. The most important factor was that radiographers should be able to access the information for free and to be allowed to use it how they wanted to. The website content was also copyright free. We feel, particularly for future international crises, having available free to access and copyright free information is essential and particularly valuable for those individuals who are at a personal or societal financial disadvantage.

Deciding what was ‘in’ and ‘out’ of scope was determined incrementally over time as the project moved forwards and it wasn't until about week 4 that we had a clear list of content that needed including. At the onset we did not know what needed including, other than the obvious–for example taking and interpreting chest radiographs and the use of personal protective clothing/equipment. The less obvious became clear as we listened to advice given to us by colleagues, examples of these topics included information about patient informatics (particularly in temporary/makeshift hospitals), coping with family life and isolation because of COVID-19 and audit of professional practice.

At the onset we had an overall aim but not a clear picture of the detail. For future reference, there is an important point to learn about this–it is essential to be able to work with high levels of uncertainty and to be able to make operational and strategic decisions based only on having some of the necessary information available to do so. This point relates also to how the project was led/managed–from our perspective as project leads, a major limiting factor was we could not find anything helpful in grey or peer reviewed literature to give us clues on how to go about leading/managing such a project in ‘unprecedented times’ within the radiographic profession at an international level.

Resource and funding

We did not have months or years to develop a funding proposal or to construct paperwork to bid for funds. The project needed to commence immediately, and it was this that drove the resource and funding model we opted for. Normally, when a project is being scoped the resources to deliver it would be considered at the same time. Resource might arise from existing budget within an organisation when the organisation itself decides the work to be done is central to its mission and it is therefore willing to redeploy existing resource to achieve the proposed work. Where justification cannot be made to use existing organisational resources to conduct a new project, an application is made externally for funds to cover the costs. Whether the project is to be funded internally or externally, long periods of time are used to create documents and make the case for funding, which when approved allows finances to be released for the work to be completed. As a rule of thumb, internal funds are substantially easier to acquire than external funds–not least because internal monies are ringfenced locally. External funding sources are highly competitive to acquire, and the process of getting them can take anything between 1 and 3 years. Furthermore, for external funding applications, it is often the case that the funding application is rejected by the funding body and for certain funding bodies the success rates can be as low is 10%. Therefore, in this situation, a substantial amount of time would have been invested into the funding application and to no avail–literally it would be back to square 1 with a view to identify a different funding opportunity to apply to.

When the proposal to create the website for taking and interpreting chest radiographs was first proposed, early in April 2020, time was not a luxury. We therefore chose a benevolent approach and we accepted it would involve a high level of uncertainty, which meant the success of the project was also uncertain. The uncertainty arose because of the lack of clarity at the onset regarding lack of detail on what the project would include and thus a lack of detail on what resources would be needed. The benevolent approach involved us approaching individuals and organisations to give their time and resource for free and to do that with immediate effect and to place their trust in us that we have a sensible vision, which when achieved, would save lives.

A learning point for the future about how a cohesive international radiography response (e.g. rapid creation of a singular website) to a crisis could be resourced, is for the resource/funding model should be agreed well in advance of a crisis, anticipating that another crisis will occur. The model could include how national and international radiographer societies will collaborate and what practical steps they could take to identify and solve problems they all face. The model should include matters like resourcing and finance.

Stakeholders

Initial thoughts about developing the website considered the use of a low-cost website to host it and to ask a web developer to donate their time to upload content for free. We moved on quickly from that idea to ask International Society of Radiographers and Radiologic Technologists (ISRRT) to host it on their eLearning platform; they readily agreed and they provided a web-programmer's time free of charge to upload materials provided by us. Hosting the material on the ISRRT eLearning platform had additional benefits, for instance it gave professional credibility to the work and allowed an easy route to promote the materials internationally to relevant end users. European Federation of Radiographer Societies (EFRS) were approached to become involved, partly to add professional credibility to the work and partly because through them we could access people who could be approached to author content. The UK Society and College of Radiographers (SCoR) were informed about the proposed website as a matter of courtesy because both of us were members of that society. However, through SCoR, eLearning for Health (eLfH) were approached and became involved. Again, SCoR added professional credibility to the work, being the professional body for radiographers in the UK. eLfH, a UK government funded organisation, offered their web developer and eLearning platform to create and host learning materials for free. The outcome of this was the web resources became spread across two eLearning platforms (ISRRT and eLfH) with hypertext links between them. The final organisational stakeholder was the Italian radiography society, proposed by EFRS,–Federazione della Associazioni Scientifiche Radiologia (FASTeR). We involved them because at the time we were creating the web resources the Italians probably had the most experience in imaging COVID-19 using chest radiography and similarly they had extensive experience in using personal protective equipment (PPE). Alongside these organisational stakeholders were individual stakeholders. These mainly consisted of the content authors and others who helped to create content and facilitate the processes. For content authors we needed an appropriate mix of people with different knowledge domains and skills. We opted for a multi-professional approach, which included information technology specialists, occupational therapists, clinical radiographers, British consultant radiographers (who interpret images on the same basis as radiologists), virologists, eLearning specialists, etc.

Unusually this project did not have any patient or public involvement. In this regard it was at variance with contemporary practice. However, due to social and legal isolation requirements because of COVID-19 we felt this component would be hard to achieve and so we took a pragmatic decision and excluded it. A learning point here is in times of crisis it might be that pragmatic decisions have to be taken which do not sit well with contemporary policy; however, we believe such decisions can be justified as long as there is a clear rationale for doing so.

We believe the approach we took to create the website, regarding stakeholder involvement, would been different and substantially quicker if a model had existed on how national and international radiographer societies might collaborate in times of international crises.

Project management

Recognising the complexity and demanding nature of the project and the need to complete it very quickly and in difficult circumstances we opted for a relatively autocratic leadership style in which the two of us made almost immediate decisions on what would be done or not done and how it would be done. That said, we were open to suggestions and consulted regularly to seek people's opinions.

There was no project team, albeit the two of us worked together closely, overseeing everything from conception to completion while having frequent communication with stakeholders and others to ensure things moved forwards at a satisfactory rate and to a suitable standard. The project was managed by stages and goals were all short term, e.g. ‘something’ had to be completed by the end of today or by the end of tomorrow. Co-dependencies existed between aspects of the project, and in these cases we ensured that all elements progressed at similar rates while ensuring completion syntax was correct and that adequate consideration existed between co-dependencies so that when the elements were integrated the ‘whole’ worked smoothly. A project tracker was updated daily, or more frequently as needed, to ensure tasks were on schedule, completed and signed off for inclusion into the website. Overall, this approach fit well with our decision to make the website live using a phased approach.

Due to the detail of the project not being overly clear at the onset, particularly in terms of scope or content, the number of people involved with the work grew over time and as needs arose. The number of additional people reduced as time progressed and particularly towards project completion. It was not until the project was fully completed that we knew how many people had been involved with it.

Project management and leadership was onerous and time-consuming. Tasks included approaching organisations and individuals to ask them to become involved and to explain what was required from them, by when and to what standards. Beyond the initial approach we communicated regularly with stakeholders to indicate how the project was progressing and by week 4 we were able to predict which elements of the website could go live at the end of week 5 and therefore when people could start using it. An acknowledgement section at the end of this article identifies the individuals who were involved with the website creation and for the nominations for the two prestigious awards.

Nobody refused to help, and everybody delivered their part on time and to a standard required by the project. By way of example for a standard, all authors were asked to provide evidence-based content in written format using English language and within a specific timeframe (eg 5 days). Artwork and photography had to be copyright free, as did textual material. Links to YouTube videos were made and where suitable video material was not available within YouTube the authors became responsible for creating their own copyright free videos which were uploaded into YouTube. The project was broken down into multiple mini projects/work packages. Each of these could be a whole ‘chapter’, a ‘sub chapter’, the ‘creation of videos’ and so on.

Project leadership and management was an all-consuming responsibility, involving a need to have fingers on all pulses constantly and to chivvy things along when needed. Contact with stakeholders was via email, video calls and phone calls. During the life of the project, several hundred emails were exchanged, and a few hundred video/telephone discussions were held.

Remotely, as project leaders, we worked alongside each other and the two of us had a different focus. Ken Holmes focused on supporting writers directly and helping to produce content. Peter Hogg focused on leadership and management of the project as a whole. The overarching leadership role fit with Peter's professional profile as he had over 30 years of experience in this regard. A formal project management model was not adopted in totality, however the approach we took contained many of the elements described in PRINCE2.⁶ We had a high level of communication between each other, often needing 2 phone calls a day during the project life. Together we reviewed all written content and provided feedback to authors on improvements that were needed and together we edited the work into the final format before a web-programmer uploaded it into one of the eLearning platforms. A suggestion was made by Donna Newman, President of ISRRT, to have multiple choice questions (MCQs) at the end of each chapter/section so the reader evaluate themselves on the content. We realised creating MCQs was an onerous task for individual content authors, some of whom may have been unfamiliar with this, so together we created them all. Other stakeholders also made suggestions about what might be included and as best we could the inclusions were made. We had no formal mechanism for soliciting comments about the project during its life and suggestions were therefore made on an ad hoc basis and informal. Decision making was by achieved by us agreeing during one of our regular phone calls on whether to include/address a suggestion or not. This fairly autocratic approach worked well.

In terms of leadership, the difficult aspects were encountered nearing project completion, and these were mainly political in nature and easily solved. Managing expectations was not a problem at any point in the life of the project or afterwards as everybody was pleased to help and to be involved. Similarly, the professional organisations were appreciative of what work was being done and regularly proffered encouraging remarks.

Early in the project we agreed the website would be made live in a phased and incremental fashion, with the most important components being completed first. These components were the clinically focused aspects, including how to conduct and interpret chest radiographs and looking after self during the pandemic (e.g. use of PPE and how to cope with self-isolation). As indicated earlier, peer reviewing and editing was completed by both of us and once we had agreed a component was to an acceptable standard we handed it to a web-programmer for inclusion into the web site. Prior to it going live, the authors of each component were asked to check their part through and approve its final release. This latter step was to ensure we did not accidently introduce errors in the final editing phase or when uploading content into the eLearning platforms. Our approach to publication therefore had a good quality assurance element to it, with multiple checks and reviews, with a view to ensuring a high level of accuracy and comprehensibility of the final product.

Once all elements of the project were fully completed, we handed over the finished product to ISRRT/eLfH for them to manage as the work was housed on their eLearning platforms. There was no intention of conducting content updates in the future as the project was intended to serve a specific purpose in a particular period in time. Similarly, there was no intention to evaluate the project and that remains to be the case at the time of writing this article.

Marketing the final product and its value beyond the pandemic

Marketing the websites housed within the ISRRT and eLfH eLearning platforms commenced approximately 2 weeks before the initial phase of the work was made public at the end of week 5. Social media, including Facebook and Twitter, were used to promote the website. This was followed by formal statements about it being released by EFRS and ISRRT as well as national radiography societies releasing information about it too. News statements and an editorial⁷ were also published about it within peer reviewed journals. The marketing strategy was simple and clear – to highlight the purpose and value of the website to potential end users and to indicate how it could be accessed.

Having the learning resources on the ISRRT website (and eLfH website) will enable access anytime from anywhere in the world with the internet. Many of the learning resources within the website will be highly relevant moving forward as we continue to recover from the pandemic. The resources are copyright free and could be used as a model for rapid development of more holistically educational material. In future pandemics if the learning material can be identified a team of experts could use the platforms and content from the websites to produce learning packages. Content within the website may have wider transferability in areas such as radiographic procedure, infection control practices and moving and handling.

Approaching benefit-risk discussions from a position of stressing the need for and benefits of an examination using ionising radiation while considering the risk is a transferable process to all X-ray examinations. Moving forward, radiographer professional bodies could perhaps collaborate to identify educational material and produce the material which can be free to accessed globally. This could be available before the next pandemic preempting the need for rapid response and using a more research-based approach.

Problems and risks

There were several risks and problems and the first concerns us, the people who led the creation of the website. While we were on a trajectory to retirement we had not retired and in addition to developing the website our jobs continued alongside the project with the added complexity of teaching and meetings moving to an online format. A mitigation for this was, given the UK was ‘locked down’, there was little else to do other than work at that time so there were no competing distractions in the evenings or weekends.

With author input from around the globe, combined with a punishing time schedule, we had to be clear about when authors and others had to complete their tasks by stating a date and time when things needed to be completed by. When stating a time, we always used UK time as the reference point in order to minimise ambiguity and delay/deadline being missed.

A particularly difficult problem encountered early on concerned people needing to work together in small teams with individual members not being located together or even in the same time zone. Many of those who worked together on this project had never met before. Video conference facilities solved most of the problems, however not all could be solved in this way. For example, new videos needed creating and for these, people had to meet on site to create them. To do this, approval was given from employers for face-to-face meetings to occur and they had to comply with local/national guidance on minimising COVID-19 infection transmission.

From our point of view, we always had a concern at the back of our minds that the project might not come to fruition. However, we consoled ourselves by agreeing the worst that could happen, if this was the case, is we would only end up being embarrassed.

Lessons learnt and what might be done differently next time

New clinical techniques were included into the web site, not adhering to our request for an evidence-based narrative. However some of these went onto become the focus of research to establish their efficacy. We are unclear what the lesson learnt here might be, other than, as indicated earlier in this article, in difficult novel situations where time is pressured new practice might emerge based on professional intuition and experience and so latitude for this should be encouraged in similar situations in the future.

The website was created only in English due to the official language of ISRRT being English. Further consideration of language should be considered ahead of a future world crisis to determine whether English is adequate, or whether another language or languages are needed for websites such as this one.

During the time the website was being created we wondered whether radiographer societies, national and international, have disaster management plans and committees to help address international-level issues and responses. We are still not clear on this. If not, then perhaps at an international level, the international radiographer societies and national societies might consider creating a mechanism that allows them to work together quickly and easily should the need arise in the future.

After project handover to ISRRT and eLfH, we felt there was mileage in exploring with ISRRT on whether there is potential in creating further free to access learning materials and books as other professional and scientific bodies have done.⁸ On this basis we proposed the idea to ISRRT and have worked with them to produce a free to access book, which considers radiographers reflections.⁴ Evaluation of this by ISRRT will occur at some point and a decision will be made to move the idea forwards, or not.

When creating the website, time was not a luxury and the schedule was punishing. We did not consider producing the materials in other electronic formats (eg mobile phone app), but in the future others should reflect on this potential deficiency and decide whether or should be addressed in future world crises.

Finally, we believe there is value in somebody revisiting the website content with a view to updating it at some point, to ensure its accuracy and to insert references to support novel practices that were proposed within it at the time of writing it.

Summary

A website was created in the initial phase of the COVID-19 pandemic to address radiographer training needs to take and interpret chest radiographs in high-risk areas in which the radiographer was exposed to contracting the disease. The website was used in 157 countries and is still being used at the time this article was written. It addresses a range of important areas, allowing quick redeployment of staff for imaging those suspected of having the disease. With a global and immediate reach, using the internet to deliver, at speed, information and solutions to the radiography community is likely to have merit in future international crises. In the article we have outlined our experiences of initiating and leading a project to its completion in a time of crisis and, as highlighted, some of our experiences could have value to inform how things might be done differently in the future, not least by radiography societies having a cohesive approach across the world with an agreed method of working together in advance of the next international crisis.

Conflict of interest statement

None.