Using an electronic patient reported outcome measure (ePAQ-MPH) to assess potential harm for long-waiting patients in gynaecology

Abstract

Background

During the COVID-19 pandemic, outpatient waits for gynaecology appointments increased by 60% in the UK National Health Service (NHS). The aim of this study was to use the electronic Personal Assessment Questionnaire-Menstrual, Pain and Hormonal (ePAQ-MPH) electronic patient reported outcome measure (ePROM) to assess symptoms, impact and potential harm for patients waiting > 60 weeks for general gynaecology appointments at a teaching hospital.

Methods

1070 patients waiting > 60 weeks for a new appointment (range 60–72 weeks) were invited to complete ePAQ-MPH online to measure gynaecological symptoms and health-related quality-of-life (HRQoL). Patients could also call to cancel appointments no longer needed. Non-responders were telephoned weekly for three further weeks and asked to complete ePAQ-MPH. Patients scoring > 80/100 for ePAQ-MPH domains relating to HRQoL had their appointment escalated to be seen within six-eight weeks. Thematic content analysis was undertaken of free-text concerns recorded using ePAQ-MPH.

Results

526 patients completed ePAQ-MPH (49.2%), 169 of these scored greater than 80/100 for one or more HRQoL domains and were seen within 6–8 weeks. 103 patients (9.6%) requested to cancel their appointment. Reasons included problem resolving spontaneously (33%), problem treated by general practitioner (10%) and being seen by a private provider (28%). Commonly recorded free-text concerns related to wanting a diagnosis (n = 142), management of condition (n = 98) and pain management (n = 77).

Conclusions

ePAQ-MPH may be used effectively to prioritise patients waiting for an outpatient appointment in gynaecology. Wider use of ePROMs to support waiting list validation within the NHS and elsewhere should be considered.

Background

During the COVID-19 pandemic there was a sustained pause in routine gynaecological care, including pausing referrals into secondary care for ‘benign’ gynaecological conditions in the UK. This has resulted in a significant increase in the waiting time to be seen for routine gynaecology care in an outpatient clinic in the UK.

In January 2021, 460,000 patients were waiting for a new-patient appointment and it was estimated that there were a further 400,000 hidden referrals (additional patients who would have sought care had the pandemic not occurred) yet to join the waiting list [1]. By April 2022 it was estimated that this number had increased to 570,000 patients awaiting a new first appointment in gynaecology in the UK National Health Service [2].

There is significant concern about the risks of both physical and psychological harm to patients, who are now often waiting for more than one year for their gynaecology clinic appointment. The Royal College of Obstetricians and Gynaecologists report- ‘Left for too long: understanding the scale and impact of gynaecology waiting lists’ includes detailed and personal anecdotes from patients describing the physical and psychological impacts of their long wait to be seen, including the impact on fertility and late diagnosis of progressive conditions such as endometriosis and cancers. However, whilst these personal anecdotes are compelling and disturbing, there is a paucity of data and research (both quantitative and qualitative) on the impact and consequences of the prolonged wait [2, 3].

Waiting list prioritisation and validation to reduce harm has been suggested by both the Royal College of Obstetricians and Gynaecologists (RCOG) and NHS England as a process to enable harm reduction and prioritise care for those who need it most urgently, but clear methodology, processes and pathways for undertaking this have not been suggested [4]. A specific definition of what constitutes harm has also not been provided by these bodies, but this could include both psychological and physical harm caused directly as a result of the longer wait to be seen and may include pain, damage to organs, fertility, development of a cancer, depression, anxiety and psychological distress. Loss of income due to not being able to work, or not able to access education are also things that could cause harm in this group.

The use of electronic patient reported outcome measures (ePROMs) within some areas of gynaecology, notably urogynaecology and endometriosis, is well established [5, 6]. The electronic Personal Assessment Questionnaire Menstrual, Pain and Hormonal (ePAQ-MPH) is a valid and reliable electronic PROM developed to assess the frequency and impact of gynaecological problems including heavy menstrual bleeding, pelvic pain, hypo-oestrogenism, pre-menstrual syndrome and sexual dysfunction [7]. The objective of this study was to use the ePAQ-MPH ePROM to measure the symptoms and associated impact on health-related quality of life (HRQoL) for patients waiting for more than sixty weeks for a new outpatient appointment in the general gynaecology clinic at a teaching hospital, in order to evaluate and attempt to reduce potential harm caused by the long wait.

Methods

ePAQ-MPH is a web-based ePROM, in the form of an electronic questionnaire [7]. It is completed online remotely prior to clinic attendance and is used to support assessment and monitoring of symptoms and their impact on HRQoL.

The ePAQ-MPH ePROM comprises of four dimensions: menstrual symptoms, pelvic pain, gynaecological hormonal symptoms and sexual symptoms. ePAQ-MPH produces a report for each completion allowing the clinician to review the results (Fig. 1). Data from the ePROM can also be downloaded as an Excel file for statistical analysis.

Fig. 1.

ePAQ-MPH report highlights domain scores recorded for each of the 15 scored domains, impact scores shown adjacent in circle, patient details including BMI, age and parity and free-text concerns shown at top of report

Previous psychometric testing of ePAQ-MPH has grouped items into scored domains [7]. ePAQ-MPH contains fifteen scored domains (Fig. 1). Domain scores are calculated by dividing the sum of all item scores in that domain by the total possible item score and multiplying this by 100 to produce a scale ranging from 0 (best possible) to 100 (worst possible health status). Domain impact scores are produced in the same way and range between 0 (no impact on HRQoL) to 3 (worst impact on HRQoL). These are all visible on the ePAQ-MPH report (Fig. 1).

On the week commencing 14th March 2022, letters were sent to all patients referred on a routine pathway who had been waiting for longer than 60 weeks (range 60–72 weeks) for an outpatient appointment in the general gynaecology clinic at Norfolk and Norwich University Hospitals NHS Foundation Trust (n = 1070). This excluded patients with gynaecological oncology, urogynaecology, menopause, endometriosis, paediatric and adolescent gynaecology and fertility conditions. This project was registered as a service evaluation (project number 43447) and undertaken in conjunction with South Norfolk Clinical Commissioning Group (CCG). The sixty-week cut off was used as this was the current target in the NHS in Norfolk and Waveney for the maximum wait for an appointment.

The letter explained about the current long waits in the general gynaecology clinic at the Norfolk and Norwich University Hospitals and asked patients to complete the ePAQ-MPH PROM. Each letter contained details of the online link to the questionnaire and the patents unique voucher code to access this. The letter informed patients that the results would enable the hospital to ‘best support’ them whilst they were waiting to be seen in the general gynaecology clinic. A support phone line staffed by the patient support team at South Norfolk CCG was provided to support anyone having difficulty completing ePAQ-MPH online and for anyone who wished to cancel their appointment.

A team of administrators reviewed daily completions of ePAQ-MPH over the following four weeks. From two weeks after the letters were sent, all patients not completing the ePAQ-MPH ePROM online were contacted by telephone. The administration team attempted to contact all patient up to five times during the study period which continued for 28 days starting one week after the invitation letters were sent. We were aware that patients had been waiting a long time for their appointments and wanted to give them every opportunity to participate, which was why we set a high limit on the number of attempts to contact someone before they were excluded (n = 5).

For the purpose of this study a cut-off of 80/100 for domain scores for the Menstrual quality -of-life domain, Pelvic pain quality -of-life domain and Hormonal quality -of-life domain were used to expedite patients to an urgent appointment in the outpatient general gynaecology clinic (within six to eight weeks maximum). A score of 80/100 for HRQoL in these areas was used as a cut off as this indicated a significant frequency and impact of symptoms, such that a wait of more than 60 weeks to be seen was felt to be a significant risk of causing harm to the patient. The score of 80/100 indicates symptoms which are present ‘all of the time’ or ‘most of the time’, which was the reason that this cut off was selected. Data for the present analysis of ePAQ-MPH symptom scores and HRQoL impact were used anonymously from patients who answered ‘Yes’ to the final item of the questionnaire, which seeks consent to allow confidential use of their answers for approved research, audit and service evaluation.

To assess the concerns of patients affected by long wait, a thematic content analysis of data from the free-text component of ePAQ-MPH was undertaken to better understand the concerns of patients affected by the long wait and add an important patient reported qualitative component to the present study. ePAQ-MPH includes a free-text question which asks: ‘Considering the issues that currently concern you the most, what do you hope to achieve from any help, advice or treatment?’. Patients are invited to record up to three free-text responses, each of up to one hundred characters.

Free-text data from consenting patients for this item were imported into Microsoft Excel. Content analysis of free-text data was conducted [8]. The aim was to count and record the number of concerns reported by patients and compare these against the current ePAQ-MPH domain structure. A researcher (SJ) read and became thoroughly familiar with the comments recorded throughout the free-text data. These comments were then coded categorically according to the 15 domains of the ePAQ-MPH ePROM. Content not fitting into these categories, and therefore not assessed by ePAQ-MPH, were then coded separately. Free-text comments were then analysed coded independently by two members of the research team (SJ and TG) and any ambiguities were resolved by discussion.

Results

During the four-week study period, 526 patients completed ePAQ-MPH (49.2% of those invited), 169 of these 526 patients scored greater than 80/100 for one or more HRQoL domains and were expediated to be seen in the general gynaecology clinic within 6–8 weeks. A total of 103 patients (9.6%) contacted the helpline to request to cancel their appointment. Overall response rate (completing ePAQ-MPH or calling to request cancellation of appointment) was 58.8%.

A further 316 (29.5%) patients were contacted by the patient support team and either declined to participate or did not subsequently complete ePAQ-MPH following this contact. A final 145 patients (13.6%) did not complete ePAQ-MPH and could not be contacted by the patient support team. Attempts to contact all these patients by telephone were made at least five times during the study period.

The reasons that patients cancelled their appointment can be seen in Table 1. These included the problem resolving spontaneously (33%), problem treated effectively by GP (10%) and being seen by a private provider (20%).

Table 1.

Reasons for cancelling new appointment in gynaecology clinic, after wating x > 60 weeks (range 60–72 weeks)

Reason for cancellation	Number of patients (percentage)
Already been seen in the gynaecology clinic during the study period	n = 28 (27.2%)
Problem resolved spontaneously	n = 34 (33%)
Problem treated by GP	n = 11 (10.6%)
Seen by a private healthcare provider	n = 21 (20.4%)
Moved away to different geographical area	n = 9 (8.7%)

Twenty-eight patients invited to participate in the study had already had an appointment and been seen in the general gynaecology clinic. This was because the study contacted all long-waiting patients and some of these had their first appointment during the study period.

Of the 526 patients completing ePAQ-MPH, 447 (85%) consented to use of their confidential data for analysis. The average age was 40.5 (range 18–82). Mean average BMI was 28.3 (median 27, range 16–60) and 36 out of 447 (8%) did not provide their height/weight. Average parity was 1.4 (range 0–7). Asked about whether they were ‘currently trying for a pregnancy’ 82 (18%) answered ‘yes, definitely’, 49 (10%) ‘yes, possibly’, 30% (6.7%) ‘no, not really’ and 262 (59%) ‘no, definitely not’. Twenty (4.5%) patients had a previous hysterectomy.

Overall, the highest average domain score was in the pre-menstrual syndrome domain (48/100), followed by the menstrual quality of life domain (42/100), hormonal quality of life domain (41/100) and hypo-oestrogenism domain (40/100). The four domains relating to sexual function had the lower average domain scores than those in the menstrual, pelvic pain and hormonal dimensions of ePAQ-MPH (Table 2).

Table 2.

Average ePAQ-MPH domain scores recorded by consenting patients (n = 447)

ePAQ MPH Domain	Mean average domain score (0-100)
Menstrual dimension
Menstrual irregularity	36.4
Menstrual duration	25.8
Menstrual heaviness	38.6
Intermenstrual bleeding	22.0
Quality of life	42.11
Pelvic pain dimension
Dysmenorrhoea	39.8
Non-cyclical pain	30.1
Quality of life	40.2
Hormonal dimension
Hypo-oestrogenism	40.3
Pre-menstrual syndrome	48.0
Quality of life	41.9
Sexual dimension
Vaginal bleeding and menstruation and sex	26.0
Dyspareunia	26.7
Pelvic pain and sex	30.0
Hormonal symptoms and sex	29.5

For free-text concerns recorded using ePAQ-MPH, 76 patients recorded zero free-text concerns, 109 recorded one concern, 99 recorded two concerns, 136 recorded three concerns, 30 recorded four concerns and seven recorded five concerns. There were therefore 870 components were identified, of which 305 fitted into the existing domain structure of ePAQ-PF (Table 3), a further 565 individual concerns did not fit into the domain structure (Table 4). The ePAQ-MPH domains with the most recorded concerns were hypo-oestrogenism, non-menstrual pelvic pain and heavy menstrual bleeding.

Table 3.

Free text concerns coding into existing ePAQ-MPH domains

ePAQ-MPH Domain	Concerns relating to ePAQ-MPH Domain (n (%))
Irregular menstruation	19 (6.2%)
Duration of periods	10 (3.3%)
Heaviness of periods	39 (12.8%)
Bleeding in between periods	25 (8.2%)
Quality of life in relation to periods	18 (5.9%)
Painful periods	23 (7.5%)
Pelvic pain outside of periods	36 (11.8%)
Quality of life in relation to pelvic pain	13 (4.3%)
Hormonal symptoms- night sweats, hot flushes, brain fog etc.	49 (16%)
Pre-menstrual syndrome	8 (2.6%)
Quality of life in relation to hormonal symptoms	14 (4.6%)
Bleeding and menstruation impacting on sex life	4 (1.3%)
Painful sex	26 (8.5%)
Pelvic pain impacting on sex life	9 (3%))
Hormonal symptoms impacting on sex life	12 (3.9%)
Total	305

Table 4.

Free text concerns recorded using ePAQ-MPH not coding into existing ePAQ-MPH domain

Area of concern (non-domain)	Number of concerns (n (%))
Confirmation of a diagnosis/advice/feeling heard	142 (25%)
Specific concern about management of their current condition/effective management/medications	98 (17.3%)
Pain management	77 (13.6%)
Non- gynaecology related concern (other health areas, e.g., bowel bladder)	49 (8.7%)
Improved quality of life	38 (6.7%)
Reassurance/reduce anxiety	35 (6.2%)
Non-menstrual bleeding, postmenopausal and or postcoital bleeding	32 (5.7%)
Fertility concern	32 (5.7%)
Hysterectomy requested	19 (3.3%)
Overall sex life	16 (2.8%)
Contraception concern	14 (2.5%)
Vaginal discharge/smell	6 (1%)
Concern relating to cervical screening	6 (1%)
Prolonged waiting time	1 (0.2%)
Total	565

The most frequently recorded concerns which did not fit into the ePAQ-MPH domain structure were wanting confirmation of a diagnosis/advice and concerns about feeling heard in relation to this (n = 142), specific concerns about the management of their present condition (n = 98) and needing help with pain management (n = 77). Only one patient wrote a concern regarding the long wait to be seen (0.2%). A word cloud graphically illustrating the word most frequently used by women recording concerns about their condition can be seen in Fig. 2.

Fig. 2.

A word cloud, showing the most frequently recorded words in the free-text concerns captured by ePAQ-MPH in this study. The larger the word size, the greater the frequency of use

Discussion

The primary objective of this study was to reduce harm to patients waiting to be seen in the gynaecology clinic for greater than 60 weeks. The secondary objective was to assess the value of using an electronic PROM for this purpose. The main findings were that 169 patients (16%) were expedited to an urgent appointment (6–8 weeks) because they demonstrated significant quality of life impact in relation to their symptoms. The response rate to the electronic PROM (49%) was good and allowed for a detailed remote assessment of patients waiting to be seen. However, it is difficult to prove in this study, that harm has been reduced for this group as a direct result of using ePAQ-MPH to assess and evaluate their symptoms; but identifying those with the most significant symptoms on the waiting list and escalating them to an urgent appointment may have reduced harm in this group and achieved the aims of the study.

A study, commissioned by the RCOG in November and December 2021, surveyed 837 women waiting to be seen in the gynaecology clinic and showed that four fifths (80%) felt that their mental health had worsened during the wait. For a quarter of these with worsening mental health, pain was given as reason. Over 75% said that their ability to work or to function socially was impaired and 61% said they felt despair at the long wait and 63% felt ignored [1, 2]. The findings of our study (with ePAQ-MPH responses from 447 consenting patients) support these results, demonstrating significant impacts on quality of life (including physical and social activities) and concerns about feeling heard and pain management predominating. These findings are likely to be applicable to all health systems around the world, where restrictions on healthcare caused by the COVID-19 pandemic response have affected waiting times. It is likely that wait times are worse in publicly funded (rather than privately funded) healthcare systems worldwide and patients without private healthcare have been worse affected by long waits worldwide.

Taking urgent steps to both reduce delays and to evaluate the symptoms, impact and concerns of patients waiting to be seen is essential. Using a valid and reliable ePROM allows clinicians to remotely assess patients’ symptoms, their impact or HRQoL and concerns. These data can be used to prioritise how urgently patients are seen in clinic and to ensure they are seen within the right service for them.

In the present study we used the ePAQ-MPH ePROM to prioritise long waiting patients and reduce harm relating to this, but the best way to use a PROM in this context, would be at the point of a new referral and utilise the results to evaluate and prioritise the referral as appropriate at this stage.

Prior to the COVID-19 pandemic, many gynaecology departments did not directly triage their referrals and relied on local pathways to ensure that patents were directed into the appropriate service and had optimal conservative management in the community prior. It has been shown that consultant led triaging can instead reduce referrals substantially, deescalating to primary care and ensuring that local pathways are followed [9]. The number of patients being removed from the waiting list because their problem had resolved spontaneously or been treated by their GP in this study supports the value of robust triaging of referrals. Using PROMs alongside this to gain an important patient reported component would strengthen this further as PROMs allow patients an opportunity to report symptoms of an intimate and sensitive nature, which may be difficult to express in a face-to-face consultation [10, 11], which could aid evaluation and prioritisation of new referrals.

Using an electronic PROM over a paper-based tool presents a number of advantages including easy collation and access of the results and the opportunity to integrate these into the patient’s electronic medical record. Patients can complete the ePAQ-MPH ePROM using their smartphone, tablet or computer and there are mechanisms to invite patients to complete this via email and text messaging. In almost all other areas of life, such as banking, arranging insurance, booking holidays and requesting goods and services, an online website or App will be used. Mirroring this within healthcare, whilst ensuring equity of access is an important step forward. The Getting It Right First Time (GIRFT) report in maternity and gynaecology (September 2021) [12] cited a current lack of utilisation of patient reported outcomes within gynaecology and stated a ‘clear need’ for more PROMs in gynaecology to be developed and used. Whilst PROMs are very well established within urogynaecology [5], and endometriosis [6], their routine use within menstrual disorders, menopause, premenstrual syndrome and pelvic pain is not. ePAQ-MPH is one of the few tools developed and tested to evaluate these areas and its routine use could be widened as it addresses an unmet need [12].

The limitations of the present study include the use of an electronic PROM which excluded those without internet access. A paper-based questionnaire could have been used instead. To mitigate this, our patient support team helped patients complete their ePAQ-MPH via the telephone where possible. The range of ages (18–82) suggests that there was broad access to the ePROM via the internet. The act of not being able to complete ePROM also allows for screening for digital poverty, which can allow for patients to be offered tablets, data packages and education in IT through local council schemes in England. Digital poverty is often a marker of poorer access to care, and indeed other services and systems operating in society. Currently many tasks undertaken in our lives, such as shopping, banking, paying taxes, booking holidays and communicating with others is done digitally, via the internet. Therefore, using internet-based tools to support the delivery of healthcare and communication with healthcare providers is both logical and desirable for many service users. The key challenge with using an internet-based system is ensuring that support is provided to those who do not have internet access, either by supporting them to access the internet and use the system provided, or by providing an alternate paper based or telephone system. Non completion of a digital service, such as completing the electronic PROM in this study, should act as a trigger to provide additional assessments and support without using the internet. Care needs to be taken to ensure that pathways and processes in healthcare delivery provide alternatives for those in digital poverty.

A further limitation was that the letters were sent in English and the ePROM was not available in other languages. It was not possible to translate ePAQ-MPH for this study, but this is a project which is now planned. The use of artificial intelligence translation tools will be transformative in this area in the future.

A further limitation may be the selection of 80/100 as the cut off score for escalating patients to a sooner appointment. This cut off is arbitrary and not evidence based and may have excluded some patients at risk of harm. Significantly reducing waiting times for all patients should be a priority, but when waiting times are very long an enhanced triaging process using an ePROM can provide additional acuity and value in patient assessment, to try and prevent harm by prioritising those most likely to benefit from a sooner appointment. Collecting further qualitative data regarding harm and impact of the long wait on their care, likely using semi structured interviews, would have provided more specific data about harm due to the long wait for care, but was outside of the remit of this study. The strengths of the study include the good response rate (60% overall) which compares favourably to other studies [13] and the high quality of the data collected.

Conclusion

The present study presents a clear solution for assessment of patients on waiting lists in general gynaecology, utilising an electronic PROM (ePAQ-MPH) completed remotely by patients to aid triage and prioritisation of patients to reduce harm. Alternative ePROMs could be used in the same way to assess patients in other areas of gynaecology (urogynaecology, endometriosis, cancer, fertility). Further research is needed including data on patient experience and ensuring equity of access to ePROMs through translated versions for non-English speakers and support for those in digital poverty without internet access to maximise response rate and utility.

Author contributions

TG Gray: Project development, data collection and analysis, manuscript writingS Johnson: Data analysis, manuscript editingE Prosser-Snelling: Manuscript editingPD Simpson Project development, manuscript editing.

Funding

No funding was received for this study.

Data availability

The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.

Declarations

Ethics approval and consent to participate

This project was registered a service evaluation with the Norfolk and Norwich information governance department and further ethical review was not required after seeking advice from the Research and Development department, Norfolk and Norwich University Hospitals NHS Foundation Trust. Only data from patients providing informed consent via the electronic patient reported outcome measure (ePAQ-PF) was used in this study.

Consent for publication

Not applicable for this study.

Competing interests

The authors declare no competing interests.