Psychometric evaluation of a quality of recovery score for the postanesthesia care unit—A preliminary validation study

Ursula Kahl; Katrin Brodersen; Sarah Kaiser; Linda Krause; Regine Klinger; Lili Plümer; Christian Zöllner; Marlene Fischer

doi:10.1371/journal.pone.0289685

. 2023 Aug 15;18(8):e0289685. doi: 10.1371/journal.pone.0289685

Psychometric evaluation of a quality of recovery score for the postanesthesia care unit—A preliminary validation study

Ursula Kahl ¹, Katrin Brodersen ¹, Sarah Kaiser ¹, Linda Krause ², Regine Klinger ¹, Lili Plümer ¹, Christian Zöllner ¹, Marlene Fischer ^1,^3,^*

Editor: Seunghwa Lee⁴

PMCID: PMC10426991 PMID: 37582085

Abstract

Introduction

Patients’ perception of postoperative recovery is a key aspect of perioperative care. Self-reported quality of recovery (QoR) has evolved as a relevant endpoint in perioperative research. Several psychometric instruments have been introduced to assess self-reported recovery 24 hours after surgery. However, there is no questionnaire suitable for use in the postanesthesia care unit (PACU). We aimed to develop and psychometrically evaluate a QoR questionnaire for the PACU (QoR-PACU).

Methods

The QoR-PACU was developed in German language based on the 40-item QoR-40 questionnaire. Between March and November 2020, adult patients scheduled for elective urologic surgery completed the QoR-PACU preoperatively and during the PACU stay. We evaluated feasibility, validity, reliability, and responsiveness.

Results

We included 375 patients. After two piloting phases including 72 and 48 patients, respectively, we administered the final version of the QoR-PACU to 255 patients, with a completion rate of 96.5%. Patients completed the QoR-PACU at a median of 125.0 (83.0; 156.8) min after arrival in the PACU. Construct validity was good with postoperative QoR-PACU sum scores correlating with age (r = 0.23, 95% CI: 0.11 to 0.35, p < 0.001), length of PACU stay (r = -0.15, 95%CI: -0.27 to -0.03, p = 0.02), pain in the PACU (r = -0.48, 95% CI: -0.57 to -0.37, p < 0.001) and piritramide dose administered (r = -0.29, 95% CI: -0.40 to -0.17, p < 0.001). Cronbach’s alpha was 0.67 (95% CI: 0.61–0.73) with moderate test-retest reliability (ICC of 0.67, 95% CI: 0.38 to 0.83). Cohen’s effect size was 3.08 and the standardized response mean was 1.65 indicating adequate responsiveness.

Conclusion

The assessment of QoR in the early postoperative period is feasible. We found high acceptability, good validity, adequate responsiveness, and moderate reliability. Future studies should evaluate the psychometric properties of the QoR-PACU in more heterogeneous patient populations including female and gender-diverse patients with varying degress of perioperative risk.

Introduction

The improvement of postoperative recovery is a common aim of all disciplines involved in perioperative care [1–3]. Postoperative recovery after surgery and anesthesia has traditionally been assessed using objective parameters including but not limited to cardiovascular, pulmonary or infectious complications, pain or length of hospital stay [4–6]. In recent years patients’ perception of recovery after surgery has been increasingly recognized as a relevant outcome measure [1, 2, 7]. To allow for comparability across clinical studies, the Standardized Endpoints in Perioperative Medicine (StEP) initiative recommends six standardized outcome measures reflecting patient comfort: postoperative pain, nausea, time to gastrointestinal recovery, time to mobilization, sleep disturbance, and the assessment of postoperative quality of recovery (QoR) [8, 9]. In the same line, the introduction of patient-reported outcome assessments is recommended by the American Society for Enhanced Recovery and Perioperative Quality Initiative [10]. Various instruments have been developed to evaluate postoperative patient-reported recovery. Myles and colleagues developed the 40-item QoR-40 questionnaire that has been validated, translated, and used extensively [2, 11–14]. In 2013, the same research group developed the 15-item QoR-15 questionnaire which is a shorter version of the more extensive QoR-40 [7, 15–24]. Both instruments have been introduced to assess QoR one day after surgery [2, 7]. The importance of advanced recovery room care and the assessment of patient-centered outcomes early after surgery has recently been highlighted by an Australian feasibility study [25]. Yet, there is no instrument appropriate for application in the postanesthesia care unit (PACU).

The aim of this study was to develop a QoR questionnaire for the PACU (QoR-PACU) and to evaluate its feasibility, validity, reliability, responsiveness, and clinical acceptability in patients after general anesthesia for elective non-cardiac surgery. Therefore, we developed the German QoR-PACU and evaluated its psychometric properties in a cohort of patients scheduled for elective urologic surgery.

Materials and methods

Ethical approval and study registration

Ethical approval was obtained from the ethics committee at the Hamburg State Chamber of Physicians on February 11, 2020 (serial number PV7218). Each patient gave written informed consent before the initiation of study-related procedures. The study was registered at clinicaltrials.gov (NCT04528537).

Study design and participants

We performed a prospective observational cohort study at the Department of Anesthesiology of a tertiary care university hospital in Northern Germany. All study participants completed the QoR-PACU on the day before surgery to obtain baseline values. Patients were assessed postoperatively 120 minutes after arrival in the PACU allowing for a tolerance interval of ± 60 minutes. A subgroup of 19 patients underwent a second postoperative assessment after another 60 ± 30 minutes to evaluate test-rest reliability. Additionally, test-retest reliability was assessed in a different subset of 44 patients on the first postoperative day. Patients read and completed the questionnaire themselves. If necessary, patients were provided with glasses by the study team. All assessments were performed by three examiners (KB, SK, MF).

Patients were included, if they were 18 years or older and received general anesthesia for elective radical prostatectomy. We excluded patients scheduled for same-day surgery, ambulatory surgery or postoperative admission to the intensive care unit and patients without excellent German language skills.

Development and adaptation of the QoR-PACU

The aim of the study was to develop a questionnaire derived from the QoR-40 to assess the QoR in PACU. Three experienced anesthesiologists (MF, LN, CZ) independently selected 15 items each from the validated German translation of the QoR-40 [26], which they deemed to be of high clinical importance for recovery and self-perceived health status during the early postoperative period [26]. After thorough discussion, a consensus version containing 16 items was developed (version 1). Similar to the QoR-15 questionnaire, an 11-point numerical rating scale was used with a score from 0 (“none of the time”) to 10 (“all of the time”). For negative items, the scoring was reversed. A total score was calculated ranging from 0 to 160 points, with a higher score representing better recovery. We performed a pre-test of the QoR-PACU in a convenience sample of 10 patients to assess feasibility. After a successful pretest phase, we started a pilot testing of the QoR-PACU, aiming to optimize acceptability und feasibility of the questionnaire. During the first pilot testing, we administered the 16-item QoR-PACU (version 1) to 72 patients. However, we repeatedly noticed misunderstandings. One major issue was the 11-point response scale from 0 to 10, reflecting the frequency of positive or negative symptoms. A relevant number of patients was confused with simultaneous pain ratings, which are part of clinical practice in the PACU, and assessed intensity rather than frequency. Therefore, we reduced the 11-point scale to a 5-point scale from 0 to 4. For ease of understanding we linked each number with an adverb of frequency: 4 points = always, 3 points = most of the time, 2 points = occasionally, 1 point = rarely, 0 points = never, resulting in a total score from 0 to 64 points. For negative items, the scoring was reversed. During the second pilot testing, we administered the 16-item QoR-PACU (version 2) to another 48 patients. We noticed that feasibility of the questionnaire was hampered by the font size of the questionnaire. Some patients were not able to read the questionnaire themselves, so the study team read the questions aloud to them. In order to improve feasibility and comparability of the assessment, we enlarged the font size and provided patients with reading glasses. To ensure acceptability of the QoR-PACU, we interviewed patients and PACU staff and consequently reduced the QoR-PACU questionnaire to 13 items, resulting in a total score from 0 to 52 points. Four items were dropped for lack of importance as reported by patients and PACU staff: the distinction between severe and moderate pain, shivering, bad dreams, and the feeling of being alone. The item “nausea and vomiting” was separated into two items. This final version of the QoR-PACU was used in our study (version 3, S1 and S2 Tables).

Data collection

Medical history and demographic characteristics were collected during the preanesthesia visit. We recorded the following clinical data: age, gender, body mass index, Charlson Comorbidity Index (CCI), obstructive sleep apnea syndrome, medication, American Society of Anesthesiologists (ASA) physical status classification, education, and current profession. To preoperatively assess the risk for obstructive sleep apnea syndrome we used the STOP-Bang score that evaluates snoring, tiredness, observed apnea, high blood pressure, body mass index, age, neck circumference, and male gender with higher scores indicating a higher risk. We retrieved information about the duration of surgery, length of PACU stay, intra- and perioperative medication from anesthesia protocols. The Numeric Rating Scale (NRS) was used to assess pain intensity in the PACU.

Sample size

There is no consistent recommendation regarding sample size for the development and the evaluation of a questionnaire. A “rule of thumb” suggests at least 10 participants for each scale item [27, 28]. This would result in 160 participants for the 16-item version 1, or 130 participants for the 13-item version 3 of the questionnaire. We did not a priori decide on the exact number of items to be included in the questionnaire. Instead, we planned to conduct a pilot phase with at least 100 patients to optimize the development of the questionnaire. For the final analysis, we aimed to include at least 200 participants. Since we expected a large drop-out rate of about 20%, we opted for the overall inclusion of 375 patients.

Statistical analysis

Continuous variables are presented as median with 25^th and 75^th percentiles, or as mean ± SD. Categorical variables are given as absolute and relative numbers.

Validity was assessed using the postoperative QoR-PACU sum score. To assess construct validity, we compared postoperative QoR-PACU sum scores between categories of clinically relevant variables using a linear model and analysis of variance. Additionally, we analyzed correlations between postoperative QoR-PACU sum scores and clinically relevant continuous variables using Pearson’s product-moment correlation coefficients and corresponding 95% confidence intervals.

Reliability was analyzed using results of individual items of the postoperative QoR-PACU and sum scores of postoperative QoR-PACU of those who took the tests twice postoperatively. Reliability was assessed based on internal consistency using Cronbach’s alpha, split-half reliability and test-retest reliability using Pearson’s correlation coefficient. Pearson correlation was chosen over the intraclass correlation coefficient since it is reasonable to assume that the state of the patient changed within the one hour of time between both assessments [29].

Cronbach’s alpha was calculated between items of the postoperative QoR-PACU using the alpha function from the R-package “psych” version 2.1.9 [30]. An alpha coefficient of 0.70 and higher is considered to be an acceptable threshold for reliability [27]. To obtain split-half reliability the function splithalf.r from the “multicon” R-package in the version 1.6 was used on the items of the postoperative QoR-PACU results [31].

Responsiveness refers to the ability of the instrument to detect change over time [32]. Responsiveness was analyzed considering pre- and postoperative QoR-PACU sum scores and was expressed with Cohen’s effect size and standardized response mean. Cohen’s effect size is defined as mean difference between preoperative and postoperative QoR-PACU sum scores divided by the SD of the preoperative QoR-PACU sum scores. Standardized response mean was calculated as the mean difference between pre- and postoperative QoR-PACU sum scores divided by the SD of these differences.

The percentage of successfully completed pre- and postoperative QoR-PACU questionnaires was used to assess feasibility. To evaluate acceptability of the QoR-PACU, we involved patients and PACU staff and invited them to judge each item regarding importance and appropriateness.

All analyses were done on complete available cases, so no imputation of missing data was performed. P-values are presented as descriptive summary measures and do not represent results of confirmatory testing. No adjustment for multiplicity was performed. All analyses were performed with R Statistical Software, version 3.5.3 (R Foundation for Statistical Computing, Vienna, Austria).

Results

Demographic and clinical characteristics

Between March and November 2020, 255 patients were approached by the study team for the assessment with the QoR-PACU version 3. Nine patients were not available for the postoperative assessment in the PACU due to postponed surgeries. These patients were excluded from the analysis. A total of 246 patients completed the final version of the QoR-PACU questionnaire resulting in a completion rate of 96.5%. Fig 1 shows the flow of participants during the course of the study. Details on baseline demographic and clinical characteristics and perioperative variables related to surgery and anesthesia are presented in Table 1.

Table 1. Demographic and clinical characteristics.

Data are presented as median (25^th; 75^th percentile) or n (%). ASA: American Society of Anesthesiologists. PACU: postanaesthesia care unit.

	n = 246
Patient characteristics
Age (years)	64 (60; 69)
Body Mass Index (kg/m²)	26.5 (24.5; 28.9)
ASA physical status
II	214 (87)
III	32 (13)
Charlson Comorbidity Index	4 (4; 5)
Obstructive sleep apnea syndrome	17 (7.6)
Education
<9 years	2 (0.8)
9–10 years	28 (11.4)
10–12 years	95 (38.6)
12–13 years	16 (6.5)
University degree	105 (42.7)
Surgery
Duration of surgery (min)	153.0 (135.0; 175.8)
Surgical approach
Open retropubic radical prostatectomy	104 (42.3)
Robot-assisted radical prostatectomy	142 (57.7)
Anesthesia and perioperative medication
Duration of anesthesia (min)	223.5 (206.0; 247.8)
Premedication with Midazolam	8 (3.3)
Antiemetic prophylaxis
None	2 (0.8)
Dexamethasone	6 (2.4)
Ondansetron	3 (1.2)
Dexamethasone and Ondansetron	235 (95.5)
Anesthesia maintenance
Sevoflurane	241 (98.0)
Propofol	5 (2.0)
Sufentanil (cumulative; μg)	85.0 (70.0; 95.0)
Norepinehphrine (maximum dosage; μg/kg/min)	0.1 (0.07; 0.14)
Fluids
Crystalloids (ml)	2500 (2000; 3000)
Colloids (ml)	0 (0;0)
Postoperative care and medication
Length of PACU stay (min)	152.0 (118.3;196.5)
Piritramide (cumulative; mg)	3.75 (3.75;7.5)
Pethidine (cumulative; mg)	25.0 (25.0;25.0)
Discharge to
Normal ward	208 (84.6)
Scheduled overnight PACU stay	27 (11.0)
Unscheduled overnight PACU stay	11 (4.5)

Open in a new tab

QoR-PACU

Median sum scores of the four QoR-PACU assessments are presented in Table 2.

Table 2. QoR-PACU scores for each assessment.

Data are presented as median (25th; 75th percentile). QoR: Quality of recovery, PACU: postanesthesia care unit.

Assessment	n	Time	sum score
Preoperative	246	24 ± 6 hrs before surgery	50.0 (48.0; 51.0)
1st PACU	246	125.0 (83.0; 156.8) min after arrival in the PACU	42.0 (38.0; 44.0)
2nd PACU	19	189.0 min (148.8; 215.8) min after arrival in the PACU	45.0 (42.5; 47.0)
24 hours postop	44	24 ± 6 hrs after surgery	45.0 (42.8; 47.3)

Open in a new tab

Pre- and postoperative mean QoR-PACU scores for each item and the mean sum score are presented in Table 3. Fig 2 shows pre- and postoperative QoR-PACU scores.

Table 3. Mean QoR-PACU scores for each item and the mean QoR-PACU sum score.

Responsiveness is expressed with Cohen effect size (difference between preoperative and postoperative QoR-PACU scores, divided by the preoperative SD, 0.2 being considered small, 0.5 as medium, and 0.8 or greater as large [33]) and the standardized response mean (score difference divided by the SD of the score difference). Numbers are given as mean ± standard deviation (SD) unless otherwise indicated. QoR: Quality of recovery, PACU: postanesthesia care unit; CI: confidence interval.

QoR-PACU item	Preoperative	Postoperative	Mean change [95% CI]	% change from baseline	Cohen effect size	Standardised response mean
1. Pain	3.7 ± 0.6	2.1 ± 1.2	-1.6 [-1.8; -1.5]	43.0	2.6	1.2
2. Sore throat	3.8 ± 0.5	2.3 ± 1.2	-1.6 [-1.7; -1.4]	39.0	3.2	1.2
3. Dry mouth	3.7 ± 0.6	1.7 ± 1.1	-2.1 [-2.2; -1.9]	54.0	3.4	1.7
4. Not able to breathe easy	3.9 ± 0.4	3.6 ± 0.7	-0.3 [-0.4; -0.2]	7.7	0.7	0.4
5. Nausea	4.0 ± 0.2	3.6 ± 0.8	-0.3 [-0.4; -0.3]	10.0	1.9	0.5
6. Vomiting	4.0 ± 0.1	4.0 ± 0.2	0.0 [-0.1; 0.0]	0.0	0.0	0.0
7. Feeling too cold	3.9 ± 0.4	3.7 ± 0.7	-0.2 [-0.3; -0.1]	5.1	0.5	0.3
8. Feeling dizzy	4.0 ± 0.2	2.4 ± 1.2	-1.6 [-1.7; -1.4]	40.0	9.6	1.4
9. Feeling confused	4.0 ± 0.2	3.5 ± 0.9	-0.5 [-0.6; -0.4]	12.0	2.2	0.6
10. Feeling anxious	3.4 ± 0.9	3.7 ± 0.6	0.3 [0.2; 0.5]	-8.8	-0.4	-0.3
11. Able to understand instructions or advice	3.7 ± 0.5	3.7 ± 0.6	-0.1 [-0.1; 0.0]	0.0	0.0	0.0
12. Feeling comfortable	3.4 ± 0.6	3.2 ± 0.8	-0.2 [-0.3; -0.1]	5.9	0.3	0.2
13. Getting support from hospital doctors and nurses	3.8 ± 0.4	3.9 ± 0.4	0.0 [0.0; 0.1]	-2.6	-0.2	-0.2
Sum	49.0 ± 2.6	41.0 ± 5.0	-8.0 [-8.6; -7.4]	16.0	3.1	1.7

Open in a new tab

Fig 2 — Each item of the questionnaire is presented as a spoke. The 5-point numeric rating scale is presented on the axis with numbers from 0 to 4.

Validity, reliability, and responsiveness

Postoperative QoR-PACU sum scores did not differ across categories of clinically relevant variables: ASA physical status II (n = 214; 42 [38;45]) vs. ASA III (n = 32; 43 [40;44], p = 0.867). Low or intermediate risk for obstructive sleep apnea syndrome (n = 133; 41 [38;44]) vs. high risk or confirmed disease (n = 91; 42 [40;45], p = 0.134). The correlation between postoperative QoR-PACU sum scores and clinically relevant continuous variables is presented in Fig 3. Cronbach’s alpha was 0.67 (95% CI: 0.61 to 0.73), reflecting moderate internal consistency [34]. The average of all split-half correlations was 0.52. The average of all split-half reliabilities was 0.69 ± 0.08. Interitem correlations and correlations between the QoR-PACU sum score and each item are presented in Fig 4 and S3 Table. There was a positive correlation between the QoR-PACU sum score and the score at the second assessment approximately one hour later (r = 0.71, 95%CI: 0.37 to 0.88, p < 0.01) reflecting acceptable test-retest reliability. Cohen’s effect size and standardized response mean are presented in Table 3.

Fig 3 — Data are presented as Pearson’s product-moment correlation coefficients and corresponding 95% confidence intervals.

Discussion

The aim of this study was to establish a questionnaire to assess self-reported QoR during the recovery period after elective non-cardiac surgery. We developed the QoR-PACU based on the 40-item QoR-40 questionnaire. We tested the questionnaire in a cohort of 246 urological patients, 2 ± 1 hours after PACU admission. We found high acceptability and feasibility, good validity, adequate responsiveness, and moderate reliability.

A standardized tool for the assessment of patient-reported QoR in the PACU is urgently needed for both research and clinical purposes. Myles et al. emphasize that results of clinical research can only be considered valid if a reconfirmation is possible [8]. However, comparability and impact of clinical research is substantially diminished by different outcome definitions and the use of numerous instruments for psychometric assessment [8, 9, 35]. Therefore, it is important to standardize endpoints in clinical research.

For decades, the Aldrete scoring system has been widely used to determine, if a patient can be safely discharged from the PACU [20]. Items addressed by the Aldrete score are limited to physical aspects: activity, respiration, circulation, consciousness and color / oxygen saturation [21, 36]. However, the definition of adequate recovery may differ substantially between patients and caregivers. Including the patients’ perception of postoperative recovery immediately after surgery provides a basis for the optimization of recovery, which may result in better outcomes and might even have a beneficial effect on length of stay and healthcare costs [3].

We chose clinically relevant aspects of intra- and postoperative care to assess construct validity. Pain intensity, piritramide dose, and PACU length of stay negatively correlated with QoR in the PACU. Our results confirm that pain perception plays a substantial role in perioperative care with a major impact on patients’ perception of health status and recovery [37]. Despite homogeneity of surgical procedures, we found a large range of PACU length of stay. Shorter stay in the PACU correlated with better QoR, which confirms previous reports on the association between PACU length of stay and postoperative complications [38]. We assessed patients 120 ± 60 minutes after arrival in the PACU. This long period of recovery may have enhanced the understanding and completion of the questionnaire. However, in many anesthetic and surgical departments, it is common for patients to leave the PACU earlier [39]. Future studies should evaluate whether the QoR-PACU is also feasible earlier after arrival in the PACU.

Psychometric properties of the QoR-PACU revealed good validity and adequate responsiveness.

However, measures of reliability including internal consistency were moderate. The fact that the internal consistency of the QoR-PACU was not as high as expected is interesting, since all items of the QoR-PACU were derived from the QoR-40 which has been developed to evaluate the quality of recovery 24 hours after surgery and has been validated extensively. Items showing high validity and reliability 24 hours after surgery showed only sufficient internal consistency in the immediate postoperative period. Several factors may account for the difference in internal consistency. First and most importantly, the patients’ mental and physical condition changes rapidly during the early postoperative period. Of note, vigilance and pain perception are interconnected. When the effects of anesthesia and analgesia wear off, consciousness improves, and patients become more susceptible to postoperative pain. Therefore, it is reasonable to expect that self-perceived recovery in the early postoperative period will change substantially within a 30-minute time frame. Second, we noticed that the simultaneous application of measures of frequency, as used in the QoR-PACU, and measures of intensity, as used in the NRS, during the recovery period led to confusion with study participants. We tried to avoid this problem by reducing the 11-point scale to a 5-point scale linked with adverbs of frequency. Yet, difficulty in understanding measures of frequency might have influenced our results. Third, it is noteworthy that we observed a relatively small change from preoperative to PACU scores in our study population. The mean change from preoperative to PACU scores was <0.5 for 9 items, but not for the features pain, feeling confused, dry mouth, and sore throat. The majority of patients had a rather low perioperative risk as reflected by the ASA physical status. High perioperative risk has been found to be associated with poor recovery after colorectal cancer surgery [40]. Similarly, low ASA physical status might have contributed to the overall high QoR reported by participants of our study.

When evaluating responsiveness, it is commendable to compare the ability of the instrument to detect change to a gold standard [32, 41]. Unfortunately, in the present study this was not possible since there has not been any tool for the measurement of patient-reported recovery in the PACU to date. We followed the approach of Stark et al. and evaluated the ability of the QoR-PACU to detect change over time based on the Cohen’s effect size and the standardized response mean [7]. Additionally, the correlation of QoR-PACU sum scores with clinically important parameters, such as patient-reported pain intensity confirms the clinical relevance of the observed change in QoR-PACU sum scores.

We found the application of the QoR-PACU during the recovery period feasible with high response and completion rates. Sum scores were highest at baseline on the day before surgery and lowest during assessment in the PACU followed by an increase on the first postoperative day. The development of QoR-PACU sum scores over time indicates that the QoR-PACU adequately mirrors QoR despite moderate internal consistency.

This validation study was performed at the PACU of a prostate cancer clinic. All surgical procedures and perioperative care at our prostate cancer center are highly standardized. Although allowing for excellent comparability between participants, generalizability is limited. We included solely male patients scheduled for radical prostatectomy. Results from previous studies suggest that gender aspects have an impact on postoperative QoR and speed of recovery [2, 42–44]. Overall, female patients tend to have lower QoR and longer PACU stay [2, 42, 44]. Moreover, pain intensity, nausea, and vomiting after surgery are more frequently reported by female patients [42, 43]. Gender aspects may be of high importance in individualized perioperative care and postoperative recovery.

We developed the QoR-PACU in German language to allow for evaluation of the instrument in our German speaking patient population. It is a natural limitation of any questionnaire that it can only be used with patients who have excellent understanding of the respective language. To be useful for a broader international public, the QoR-PACU needs to be translated into English and subsequently to various other languages. Although there are multiple instruments for assessing recovery 24 hours after the intervention, we have developed the QoR-PACU from the QoR-40 [2]. We have explicitly decided to derive the QoR-PACU from the QoR questionnaires, since their use has been recommended by the StEP initiative [9].

Future studies should evaluate the psychometric properties of the QoR-PACU in a more heterogenous patient population, including female and gender diverse patients, as well as a greater variety of patient-related and procedure-related risk factors. This might reveal whether the issue of moderate internal consistency was primarily linked to the characteristics of the initial study cohort, or whether items have to be revised substantially to be suitable for patients in the PACU. For the modification of the QoR-PACU, it might be helpful to consider suggestions from patients and caregivers.

Conclusion

This preliminary validation study presents the development of a questionnaire to assess self-reported QoR after surgery in the PACU. We found high acceptability and feasibility, good validity, and adequate responsiveness. Against our hypothesis, we did not find high internal consistency. Based on these findings, the QoR-PACU should be modified. The modification process should also consider suggestions from healthcare professionals and patients. Future psychometric evaluation should include a more heterogeneous patient cohort including female and gender-diverse patients with varying degrees of perioperative risk.

Supporting information

S1 Table. QoR-PACU Version 3.0 preoperative.

(DOCX)

Click here for additional data file.^{(166.2KB, docx)}

S2 Table. QoR-PACU Version 3.0 postoperative.

(DOCX)

Click here for additional data file.^{(166.2KB, docx)}

S3 Table. Postoperative inter-item correlation.

Postoperative assessment: Inter-item correlations for the 13 items of the QoR-PACU score. Correlations are expressed as Pearson correlation coefficients.

(DOCX)

Click here for additional data file.^{(17.7KB, docx)}

Acknowledgments

The authors thank Lars Nawrath for his support in the process of selecting items of the QoR-40 questionnaire to develop the first version of the QoR-PACU.

All authors approved of the version to be published and agree to be accountable for all aspects of the work, thereby ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

Data Availability

Data cannot be shared publicly because of the German General Data Protection Regulation. Without patients’ consent the regulation does not allow for sharing data with researchers that are not part of the predefined research team. For further information please refer to the data protection officers of the City of Hamburg (Der Hamburgische Beauftragte für Datenschutz und Informationsfreiheit, Ludwig-Erhard-Str 22, 7. OG, 20459 Hamburg) and the University Medical Center Hamburg-Eppendorf (Datenschutzbeauftragter, Universitätsklinikum Hamburg-Eppendorf, Martinistr. 52, 20246 Hamburg, Tel.: 040/7410-56890, E-Mail: m.jaster@uke.de).

Funding Statement

Ursula Kahl was funded by the Clinician Scientist Program of the medical faculty of the University of Hamburg, during the conduct of the study. The University of Hamburg was not involved in any of the following: study design, conduct of the research, preparation of this manuscript, analysis and interpretation of data; writing of the report; decision to submit the article for publication.

References

1.Myles PS, Hunt JO, Nightingale CE, Fletcher H, Beh T, Tanil D, et al. Development and psychometric testing of a quality of recovery score after general anesthesia and surgery in adults. Anesth Analg. 1999;88:83–90. doi: 10.1097/00000539-199901000-00016 [DOI] [PubMed] [Google Scholar]
2.Myles PS, Weitkamp B, Jones K, Melick J, Hensen S. Validity and reliability of a postoperative quality of recovery score: the QoR-40. Br J Anaesth. 2000;84:11–5. doi: 10.1093/oxfordjournals.bja.a013366 [DOI] [PubMed] [Google Scholar]
3.Borrell-Vega J, Humeidan ML, Bergese SD. Defining quality of recovery—What is important to patients? Best Pract Res Clin Anaesthesiol. 2018;32:259–68. doi: 10.1016/j.bpa.2018.07.002 [DOI] [PubMed] [Google Scholar]
4.Myles PS. More than just morbidity and mortality—quality of recovery and long-term functional recovery after surgery. Anaesthesia. 2020;75 Suppl 1:e143–50. [DOI] [PubMed] [Google Scholar]
5.Shulman M, Myles P. Measuring perioperative outcome. Curr Opin Anaesthesiol. 2016;29:733–8. doi: 10.1097/ACO.0000000000000383 [DOI] [PubMed] [Google Scholar]
6.Neville A, Lee L, Antonescu I, Mayo NE, Vassiliou MC, Fried GM, et al. Systematic review of outcomes used to evaluate enhanced recovery after surgery. Br J Surg. 2014;101:159–70. doi: 10.1002/bjs.9324 [DOI] [PubMed] [Google Scholar]
7.Stark PA, Myles PS, Burke JA. Development and psychometric evaluation of a postoperative quality of recovery score: the QoR-15. Anesthesiology. 2013;118:1332–40. doi: 10.1097/ALN.0b013e318289b84b [DOI] [PubMed] [Google Scholar]
8.Myles PS, Grocott MPW, Boney O, Moonesinghe SR, COMPAC-StEP Group. Standardizing end points in perioperative trials: towards a core and extended outcome set. Br J Anaesth. 2016;116:586–9. [DOI] [PubMed] [Google Scholar]
9.Myles PS, Boney O, Botti M, Cyna AM, Gan TJ, Jensen MP, et al. Systematic review and consensus definitions for the Standardised Endpoints in Perioperative Medicine (StEP) initiative: patient comfort. Br J Anaesth. 2018;120:705–11. doi: 10.1016/j.bja.2017.12.037 [DOI] [PubMed] [Google Scholar]
10.Abola RE, Bennett-Guerrero E, Kent ML, Feldman LS, Fiore JF, Shaw AD, et al. American Society for Enhanced Recovery and Perioperative Quality Initiative Joint Consensus Statement on Patient-Reported Outcomes in an Enhanced Recovery Pathway. Anesth Analg. 2018;126:1874–82. doi: 10.1213/ANE.0000000000002758 [DOI] [PubMed] [Google Scholar]
11.Vignaud M, Morel C, Henault A, Futier E, Pereira B, Lambert C, et al. Variability and reliability of the French version of the Quality of Recovery-40 Questionnaire (QoR-40). Anaesth Crit Care Pain Med. 2021;40:100822. doi: 10.1016/j.accpm.2021.100822 [DOI] [PubMed] [Google Scholar]
12.Chen Y, Wang J, Liu S, Lai W, Liu J, Wang Z, et al. Development and Validation of the Chinese Version of the Quality of Recovery-40 Questionnaire. Ther Clin Risk Manag. 2020;16:1165–73. doi: 10.2147/TCRM.S281572 [DOI] [PMC free article] [PubMed] [Google Scholar]
13.Lee JH, Kim D, Seo D, Son J-S, Kim D-C. Validity and reliability of the Korean version of the Quality of Recovery-40 questionnaire. Korean J Anesthesiol. 2018;71:467–75. doi: 10.4097/kja.d.18.27188 [DOI] [PMC free article] [PubMed] [Google Scholar]
14.Terkawi AS, Myles PS, Riad W, Nassar SN, Mahmoud M, AlKahtani GJ, et al. Development and validation of Arabic version of the postoperative quality of recovery-40 questionnaire. Saudi J Anaesth. 2017;11:S40–52. doi: 10.4103/sja.SJA_77_17 [DOI] [PMC free article] [PubMed] [Google Scholar]
15.Kleif J, Edwards HM, Sort R, Vilandt J, Gögenur I. Translation and validation of the Danish version of the postoperative quality of recovery score QoR-15. Acta Anaesthesiol Scand. 2015;59:912–20. doi: 10.1111/aas.12525 [DOI] [PubMed] [Google Scholar]
16.Demumieux F, Ludes P-O, Diemunsch P, Bennett-Guerrero E, Lujic M, Lefebvre F, et al. Validation of the translated Quality of Recovery-15 questionnaire in a French-speaking population. Br J Anaesth. 2020;124:761–7. doi: 10.1016/j.bja.2020.03.011 [DOI] [PubMed] [Google Scholar]
17.Kim D, Kim JK, Yeo J. Translation and Validation of the Korean Version of the Postoperative Quality of Recovery Score QoR-15. BioMed Res Int. 2020;2020:3456234. doi: 10.1155/2020/3456234 [DOI] [PMC free article] [PubMed] [Google Scholar]
18.Kahl U, Gebhardt N, Brodersen K, Kainz E, Schirren L, Yu Y, et al. Validation of a translated Quality of Recovery-15 questionnaire in German patients undergoing elective noncardiac surgery. Br J Anaesth. 2021;127:e161–3. doi: 10.1016/j.bja.2021.07.026 [DOI] [PubMed] [Google Scholar]
19.Picconi E, Iacobucci T, Adducci E, Gualtieri E, Beccia G, Sollazzi L. Translation and validation of the Italian version of the postoperative quality of recovery score QoR-15. Minerva Anestesiol. 2020;86:787–9. doi: 10.23736/S0375-9393.20.14472-9 [DOI] [PubMed] [Google Scholar]
20.Nakatani S, Ida M, Tanaka Y, Okamoto N, Wang X, Nakatani H, et al. Translation and validation of the Japanese Version of the Quality of Recovery-15 Questionnaire. J Anesth. 2021;35:426–33. doi: 10.1007/s00540-021-02921-z [DOI] [PubMed] [Google Scholar]
21.Aldrete JA, Kroulik D. A postanesthetic recovery score. Anesth Analg. 1970;49:924–34. [PubMed] [Google Scholar]
22.Sá AC, Sousa G, Santos A, Santos C, Abelha FJ. Quality of Recovery after Anesthesia: Validation of the Portuguese Version of the ‘Quality of Recovery 15’ Questionnaire. Acta Med Port. 2015;28:567–74. doi: 10.20344/amp.6129 [DOI] [PubMed] [Google Scholar]
23.Miklić Bublić M, Miklić P, Barl P, Matas M, Sekulić A. CROATIAN VERSION OF THE QUALITY OF RECOVERY QUESTIONNAIRE (QoR-40): TRANSCULTURAL ADAPTATION AND VALIDATION. Acta Clin Croat. 2021;60:237–45. doi: 10.20471/acc.2021.60.02.09 [DOI] [PMC free article] [PubMed] [Google Scholar]
24.Yaghoobi S, Hamidfar M, Lawson DM, Fridlund B, Myles PS, Pakpour AH. Validity and reliability of the Iranian version of the quality of recovery-40 questionnaire. Anesthesiol Pain Med. 2015;5:e20350. [DOI] [PMC free article] [PubMed] [Google Scholar]
25.Ludbrook G, Lloyd C, Story D, Maddern G, Riedel B, Richardson I, et al. The effect of advanced recovery room care on postoperative outcomes in moderate-risk surgical patients: a multicentre feasibility study. Anaesthesia. 2021;76:480–8. doi: 10.1111/anae.15260 [DOI] [PubMed] [Google Scholar]
26.Plöger B. Validierung einer deutschen Übersetzung des Quality-of-Recovery-Scores-40 (QoR-40) als Maß der patientenzentrierten postoperativen Ergebnisqualität. [Marburg]: Philipps-Universität Marburg; 2008.
27.Boateng GO, Neilands TB, Frongillo EA, Melgar-Quiñonez HR, Young SL. Best Practices for Developing and Validating Scales for Health, Social, and Behavioral Research: A Primer. Front Public Health. 2018;6:149. [DOI] [PMC free article] [PubMed] [Google Scholar]
28.Nunnally J. Pyschometric Theory. 1978.
29.Vetter TR, Cubbin C. Psychometrics: Trust, but Verify. Anesth Analg. 2019;128:176–81. doi: 10.1213/ANE.0000000000003859 [DOI] [PMC free article] [PubMed] [Google Scholar]
30.Revelle W. psych: Procedures for Psychological, Psychometric, and Personality Research [Internet]. 2021 [cited 2022 May 1]. https://CRAN.R-project.org/package=psych
31.Sherman RA. multicon: Multivariate Constructs [Internet]. 2015 [cited 2022 May 1]. https://CRAN.R-project.org/package=multicon
32.Middel B, van Sonderen E. Statistical significant change versus relevant or important change in (quasi) experimental design: some conceptual and methodological problems in estimating magnitude of intervention-related change in health services research. Int J Integr Care. 2002;2:e15. doi: 10.5334/ijic.65 [DOI] [PMC free article] [PubMed] [Google Scholar]
33.Norman GR, Wyrwich KW, Patrick DL. The mathematical relationship among different forms of responsiveness coefficients. Qual Life Res. 2007;16:815–22. doi: 10.1007/s11136-007-9180-x [DOI] [PubMed] [Google Scholar]
34.Taherdoost H. Validity and Reliability of the Research Instrument; How to Test the Validation of a Questionnaire/Survey in a Research. SSRN Electron J [Internet]. 2016 [cited 2022 Jun 20]; https://www.ssrn.com/abstract=3205040
35.Bowyer A, Royse C. The importance of postoperative quality of recovery: influences, assessment, and clinical and prognostic implications. Can J Anaesth J Can Anesth. 2016;63:176–83. [DOI] [PubMed] [Google Scholar]
36.Aldrete JA. The post-anesthesia recovery score revisited. J Clin Anesth. 1995;7:89–91. doi: 10.1016/0952-8180(94)00001-k [DOI] [PubMed] [Google Scholar]
37.Nimmo SM, Foo ITH, Paterson HM. Enhanced recovery after surgery: Pain management. J Surg Oncol. 2017;116:583–91. doi: 10.1002/jso.24814 [DOI] [PubMed] [Google Scholar]
38.Mann-Farrar J, Egan E, Higgins A, Wysocki L, Vaux A, Arndell E, et al. Are Postoperative Clinical Outcomes Influenced by Length of Stay in the Postanesthesia Care Unit? J Perianesthesia Nurs Off J Am Soc PeriAnesthesia Nurses. 2019;34:386–93. [DOI] [PubMed] [Google Scholar]
39.Myles P, Leslie K, McNeil J, Forbes A, Chan M. Bispectral index monitoring to prevent awareness during anaesthesia: the B-Aware randomised controlled trial. The Lancet. 2004;363:1757–63. [DOI] [PubMed] [Google Scholar]
40.Jakobsson J, Idvall E, Kumlien C. Patient characteristics and surgery-related factors associated with patient-reported recovery at 1 and 6 months after colorectal cancer surgery. Eur J Cancer Care (Engl). 2017;26. [DOI] [PubMed] [Google Scholar]
41.Terwee CB, Bot SDM, de Boer MR, van der Windt DAWM, Knol DL, Dekker J, et al. Quality criteria were proposed for measurement properties of health status questionnaires. J Clin Epidemiol. 2007;60:34–42. doi: 10.1016/j.jclinepi.2006.03.012 [DOI] [PubMed] [Google Scholar]
42.Buchanan FF, Myles PS, Cicuttini F. Effect of patient sex on general anaesthesia and recovery. Br J Anaesth. 2011;106:832–9. doi: 10.1093/bja/aer094 [DOI] [PubMed] [Google Scholar]
43.Buchanan FF, Myles PS, Cicuttini F. Patient sex and its influence on general anaesthesia. Anaesth Intensive Care. 2009;37:207–18. doi: 10.1177/0310057X0903700201 [DOI] [PubMed] [Google Scholar]
44.Myles PS, McLeod AD, Hunt JO, Fletcher H. Sex differences in speed of emergence and quality of recovery after anaesthesia: cohort study. BMJ. 2001;322:710–1. doi: 10.1136/bmj.322.7288.710 [DOI] [PMC free article] [PubMed] [Google Scholar]

PLoS One. doi: 10.1371/journal.pone.0289685.r001

Decision Letter 0

Seunghwa Lee

8 Aug 2022

PONE-D-22-18272Psychometric evaluation of a quality of recovery score for the postanesthesia care unit – a prospective validation studyPLOS ONE

Dear Dr. Marlene Fischer,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Thank you for submitting interesting article. I believe our reviews will improve your paper.

Please submit your revised manuscript by Sep 22 2022 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.
A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.
An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Seung-Hwa Lee

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. In your Data Availability statement, you have not specified where the minimal data set underlying the results described in your manuscript can be found. PLOS defines a study's minimal data set as the underlying data used to reach the conclusions drawn in the manuscript and any additional data required to replicate the reported study findings in their entirety. All PLOS journals require that the minimal data set be made fully available. For more information about our data policy, please see http://journals.plos.org/plosone/s/data-availability.

Upon re-submitting your revised manuscript, please upload your study’s minimal underlying data set as either Supporting Information files or to a stable, public repository and include the relevant URLs, DOIs, or accession numbers within your revised cover letter. For a list of acceptable repositories, please see http://journals.plos.org/plosone/s/data-availability#loc-recommended-repositories. Any potentially identifying patient information must be fully anonymized.

Important: If there are ethical or legal restrictions to sharing your data publicly, please explain these restrictions in detail. Please see our guidelines for more information on what we consider unacceptable restrictions to publicly sharing data: http://journals.plos.org/plosone/s/data-availability#loc-unacceptable-data-access-restrictions. Note that it is not acceptable for the authors to be the sole named individuals responsible for ensuring data access.

We will update your Data Availability statement to reflect the information you provide in your cover letter.

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Partly

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: Yes

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: No

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: There is smaller sample size than calculated. It should be clarified. One more area is missing, the reasons that compelled the patients to fail to reach the follow up visits are to be evaulatied.

Otherwise, the article has merit to be accepted for publication.

Reviewer #2: Many thanks for sending this manuscript for the review. I have read it with great interest and you can find my suggestions for improvement below. In addition to those, my main concern is how a tool in German language could be of interest to wider international audience where English is the primary language. Another main concern is testing in PACU –120 minutes after PACU admission seems late as many PACU stays are much shorter for this surgical procedure.

Abstract:

Needs to be improved based on the comments below. Additionally add: refer to German tool, sample size is 255; others were in piloting stage of the tool development;

Introduction:

Overall, well described background. I suggest you improve the last sentence: "in patients after general anesthesia for elective non-cardiac surgery." – this is misleading as your tool was used only for patients undergoing urologic surgery and in German language only.

Material and methods:

- I wonder if quaternary (check spelling for this word please) hospital care is readily generalizable (see definitions of tertiary – quaternary care); your study was registered and there you mentioned tertiary care hospital (also the radical prostatectomy is a common procedure)

- This sentence is unclear, including chronologically – is this during the instrument development stage? How were the patients randomly selected? Or was this an opportunistic sample? Maybe this should be put in the next section? "We performed a pre-test of the QoR-PACU in a randomly selected cohort of 10 patients to assess feasibility."

- Please take care of writing this: 60 minutes ± 30 : should be 60 ± 30 minutes

- In the section: Development and adaptation of the QoR-PACU; I suggest you refer to pilot 1 and pilot 2 instead of early study period: "After a successful pretest phase, we administered the 16-item QoR-PACU (version 1) to 72 patients – the first pilot testing of the QoR-PACU." And similar where needed in this paragraph

- Sample Size: I suggest you use only 10 participants per item here and add that you aimed for at least 160 participants with full completion of the questionnaire version 3 (you did not reach 375 that you mention here). What I am confused about is several subdivision into subgroups for different testing... you should define sample sizes for all of these here or at least further develop figure 1 in results to show the numbers of patients doing reassessments

- I wonder if you chose the items from the QoR40 tool from English or valid German translation? Please clarify in the methods.

- Responsiveness definition: ability to detect clinical important change: what does this mean for your data? Add in table 2 which is important change?

- This is unclear: "The proportion of patients who successfully completed the QoR-PACU postoperatively was used to assess acceptability and feasibility." What proportion? How was this tested? Definitions?

Results:

- Please see above abour improving figure 1

- Table 2: i am ok with means here, however you should improve statistical part of the methods to reflect that as you say only medians with IQR will be reported; %change from baseline: be careful which should be negative – almost all? Also the paragraph before this table reports medians and then the table reports means – should be consistent; Also: the table title Responsiveness is not ok; please improve to XYZ and responsiveness.

- Figure 2: also add n of patients in brackets in the legend of the four colors; also improve figure 2 title

- Table 3: I suggest you remove this table completely as the construct validity in this context is not valid: your numbers of patients in each of the category are far from comparable in all clinically relevant variable as you say with the exception of N for ASA and OSAS (but I also disagree with the notion how your tool would be connected to mode of intubation for example). Also modify methods section appropriately

- Figure 3: I am fine with this one; please add what is presented in description: mean with 95% CI?? What statistical test was used for correlations – please also add this in the figure

Discussion:

- Some parts need to be modified to reflect my comments above including the first paragraph which needs to be significantly improved.

- How do you conclude this Is highly feasible and acceptable? Add the earliest 2 hours after PACU admission? Omit excellent recruitment this is ambiguous; leave high completion rates; also clarify in the methods section if patients alone completed the surveys or were the items read to them and explained?

- Your PACU times are interesting as in many countries patients mostly leave PACUs after around 60 minutes – please add to limitations how can your QoR instrument be used sooner;

- Use some citations when you refer to gender influences in the discussion

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: Yes: Shahjada Selim

Reviewer #2: No

**********

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

Attachment

Submitted filename: PONE-D-22-18272_reviewer.pdf

Click here for additional data file.^{(1.7MB, pdf)}

PLoS One. 2023 Aug 15;18(8):e0289685. doi: 10.1371/journal.pone.0289685.r002

Author response to Decision Letter 0

31 Oct 2022

Point by Point Response to the reviewers’ comments.

We thank the reviewers for taking the time to read and process our manuscript. We have integrated the reviewers’ suggestions and believe that this has substantially improved the manuscript.

Please find below the reviewers’ comments written upright in black and our responses written in blue italics.

Reviewer #1:

1. There is smaller sample size than calculated. It should be clarified.

We thank the reviewer for alerting us to this. According to the suggestions of both reviewer 1 and 2, we have amended the methods section for the following explanation:

“There is no consistent recommendation regarding sample size for the development and the evaluation of a questionnaire. A “rule of thumb” suggests at least 10 participants for each scale item (Boateng et al. 2018; Nunnally 1978). This would result in 160 participants for the 16-items version 1, or 130 participants for the 13-items version 3 of the questionnaire. We did not a priori decide on the exact number of items to be included in the questionnaire. Instead, we planned to conduct a pilot phase to optimize the development of the questionnaire. Therefore, we opted for the inclusion of 375 patients, allowing for an extensive pilot testing sample as well as a large drop-out rate.

See revised article with highlighted changes, p.7

2. One more area is missing, the reasons that compelled the patients to fail to reach the follow up visits are to be evaulatied.

We thank the reviewer for alerting us to this. Due to postponed surgeries, 9 patients could not be met in the PACU by the study team. We have included this information in the flow chart (Fig 1).

See revised Fig 1

Reviewer #2:

1. my main concern is how a tool in German language could be of interest to wider international audience where English is the primary language.

We agree with the reviewer that in order to be useful to an international public, the QoR-PACU needs to be translated to English and subsequently to various more languages. It is a natural limitation that any questionnaire can only be used with patients who have excellent understanding of the respective language the questionnaire was written in. We developed the QoR-PACU in German language because we wanted to validate and use the instrument in our German speaking patient population. We have added these thoughts to the discussion section of the manuscript.

See revised article with highlighted changes, p.19

2. Another main concern is testing in PACU –120 minutes after PACU admission seems late as many PACU stays are much shorter for this surgical procedure.

We agree with the reviewer that this topic merits further discussion.

In our study, the overall length of stay in our patient cohort was rather long with a median time of 152 minutes. Data on PACU length of stay varies widely across studies:

• A retrospective analysis of 17.047 patients revealed a median PACU length of stay of 117 (IQR 82, 165) minutes (Weissman, Scemama, and Weiss 2019).

• A study investigating the relationship of pain and postoperative nausea and vomiting with the PACU length of stay in 9.603 patients, showed a mean PACU length of stay of 3.2 ± 2.3 hours (Ganter et al. 2014).

• A study investigating whether BIS-guided anaesthesia reduced the incidence of awareness during surgery in adults, found a median PACU length of stay of 63 (40–95) min in 1.225 patients with BIS monitoring and 66 (40–100) min in 1.238 patients without BIS monitoring (Myles et al. 2004).

In order for the instrument to be suitable for patients with shorter PACU stays, it is necessary to re-evaluate the QoR-PACU earlier after arrival in the PACU.

We have included the following paragraph in the discussion:

“We assessed patients 120 ± 60 minutes after arrival in the PACU. This long period of recovery may have enhanced the understanding and completion of the questionnaire. However, in many anesthetic and surgical departments, it is common for patients to leave the PACU earlier (Myles et al. 2004). Future studies should evaluate whether the QoR-PACU is also feasible earlier after arrival in the PACU.”

See revised article with highlighted changes, p.17

Abstract:

Needs to be improved based on the comments below. Additionally add:

- refer to German tool,

We added this information to the abstract.

See revised article with highlighted changes, p.2

- sample size is 255, others were in piloting stage of the tool development;

We added the desired information: “We included 375 patients. After two piloting phases including 72 and 48 patients, respectively, we administered the final version of the QoR-PACU to 255 patients, with a completion rate of 96.5%.”

See revised article with highlighted changes, p.2

3. Introduction:

We have included the following sentence:” […] Therefore we developed the German QoR-PACU and evaluated its psychometric properties in a cohort of patients scheduled for elective urologic surgery.”

See revised article with highlighted changes, p.4

4. Material and methods:

I wonder if quaternary (check spelling for this word please) hospital care is readily generalizable (see definitions of tertiary – quaternary care); your study was registered and there you mentioned tertiary care hospital (also the radical prostatectomy is a common procedure)

We thank the reviewer for alerting us to this. Our institution includes departments which provide quaternary care. However, we agree that the procedure of prostatectomy (although performed at a highly specialized prostate cancer center) is better referred to as tertiary care. We have changed this term accordingly.

See revised article with highlighted changes, p.5

5. This sentence is unclear, including chronologically

– is this during the instrument development stage? How were the patients randomly selected? Or was this an opportunistic sample? Maybe this should be put in the next section? "We performed a pre-test of the QoR-PACU in a randomly selected cohort of 10 patients to assess feasibility."

We agree with the reviewer that this sentence belongs to the development section. We have changed this in the manuscript. Additionally, we changed the term “randomly selected cohort” to “convenience sample”.

See revised article with highlighted changes, p.6

6. Please take care of writing this: 60 minutes ± 30: should be 60 ± 30 minutes

We have changed this in the manuscript.

See revised article with highlighted changes, p.5

7. In the section: Development and adaptation of the QoR-PACU; I suggest you refer to pilot 1 and pilot 2 instead of early study period: "After a successful pretest phase, we administered the 16-item QoR-PACU (version 1) to 72 patients – the first pilot testing of the QoR-PACU." And similar where needed in this paragraph

We thank the reviewer for this suggestion. We have rephrased this in the manuscript.

See revised article with highlighted changes, p.6

8. Sample Size: I suggest you use only 10 participants per item here and add that you aimed for at least 160 participants with full completion of the questionnaire version 3 (you did not reach 375 that you mention here).

We are grateful for this pragmatic suggestion. We have changed the paragraph as follows to the following explanation:

See revised article with highlighted changes, p.7

9. What I am confused about is several subdivision into subgroups for different testing... you should define sample sizes for all of these here or at least further develop figure 1 in results to show the numbers of patients doing reassessments

Following the suggestion of the reviewer, we now present these two subsets of patients in figure 1.

See Fig1 and table 2

10. I wonder if you chose the items from the QoR40 tool from English or valid German translation? Please clarify in the methods.

We have selected the 15 items from the validated German translation of the QoR-40 (Plöger 2008). We have now specified this in the Development section.

See revised article with highlighted changes, p.6

11. Responsiveness definition: ability to detect clinical important change: what does this mean for your data?

12. Add in table 2 which is important change?

We thank the reviewer for alerting us to this critical issue. Indeed, the term “clinically important change” is problematic, as this would require the comparison with a gold standard (Middel et al., n.d.; Terwee et al. 2007). Regarding the QoR-PACU this is not possible, since there hasn’t been any tool for the measurement of patient-reported recovery in the PACU to date. The correlation of QoR-PACU sum scores with clinically important parameters, such as patient reported pain intensity (r= -0.48, 95%CI: -0.57 to -0.37, p <0.001) confirms the clinical relevance of the observed change in QoR-PACU sum scores. Since we did not analyse the correlation of single items with clinically important parameters, we rephrased the responsiveness definition as follows: “Responsiveness refers to the ability of the instrument to detect change over time.”

See revised article with highlighted changes, p.8

Additionally, we have added the following paragraph as a limitation:

“When evaluating responsiveness, it is commendable to compare the ability of the instrument to detect change to a gold standard (32,41). Unfortunately, in the present study this was not possible, since there hasn’t been any tool for the measurement of patient-reported recovery in the PACU to date. We followed the approach of Stark et al. and evaluated the ability of the QoR-PACU to detect change over time based on the Cohen’s effect size and the standardized response mean (7). Additionally, the correlation of QoR-PACU sum scores with clinically important parameters, such as patient-reported pain intensity confirms the clinical relevance of the observed change in QoR-PACU sum scores.”

See revised article with highlighted changes, p.18

To enable the appraisal of Cohen’s effect size, we’ve added the Cohen thresholds small (0.2), medium (0.5) and large (0.8) (Norman, Wyrwich, and Patrick 2007) to the legend of table 3.

See revised table 3

13. This is unclear: "The proportion of patients who successfully completed the QoR-PACU postoperatively was used to assess acceptability and feasibility." What proportion? How was this tested? Definitions?

We have clarified this terminology throughout the manuscript.

We have assessed acceptability of the instrument based on interviews with patients and PACU staff during the piloting and the testing phase. We did not quantify results of these interviews; however, we implemented their feedback in the development of version 3 of the QoR PACU.

We used the percentage of successfully completed pre- and postoperative QoR-PACU questionnaires to assess feasibility. The completion rate was 96.5%. We therefore concluded, the assessment of patient reported recovery in the PACU with the QoR PACU is highly feasible

See revised article with highlighted changes, p.6, p.9

14. Results:

Please see above abour improving figure 1

We have revised figure 1.

See revised Fig 1

15. Table 2:

- i am ok with means here, however you should improve statistical part of the methods to reflect that as you say only medians with IQR will be reported;

We have added this information in the statistical part of the methods.

See revised article with highlighted changes, p.8

- %Change from baseline: be careful which should be negative – almost all?

While the mean change is displayed as a negative or positive value, the % change from baseline is generally not displayed with an algebraic minus- or plus-sign. Please see for comparison Stark et al. (Stark, Myles, and Burke 2013).

- Also the paragraph before this table reports medians and then the table reports means – should be consistent;

We decided to report mainly medians, because we believe that it is the most appropriate way to report questionnaire results. However, the calculation of the standardised response mean is based on mean values, hence the inconsistency between the text and the table.

We have added the median sum scores of the four QoR-PACU assessments with the respective number of patients and time of assessment as Table 2.

See revised table 2

- Also: the table title Responsiveness is not ok; please improve to XYZ and responsiveness.

Following the approach of Stark et al. (Stark, Myles, and Burke 2013), we changed the title to “Change in mean QoR-PACU scores from preoperative to postoperative PACU assessment”.

See revised table 3

16. Figure 2: also add n of patients in brackets in the legend of the four colors; also improve figure 2 title

We included this information in the figure caption and changed the title to “Change in mean QoR-PACU scores”.

See revised Fig 2

17. Table 3: I suggest you remove this table completely as the construct validity in this context is not valid: your numbers of patients in each of the category are far from comparable in all clinically relevant variable as you say with the exception of N for ASA and OSAS (but I also disagree with the notion how your tool would be connected to mode of intubation for example). Also modify methods section appropriately

We thank the reviewer for this suggestion. We have deleted this table. We now present information on the two variables ASA and OSAS in the main text, since we do not want to omit negative results. Additionally, this information might be interesting for comparison with future studies.

See revised article with highlighted changes, p.13

18. Figure 3: I am fine with this one; please add what is presented in description: mean with 95% CI?? What statistical test was used for correlations – please also add this in the figure

We have included this information in the figure caption.

See revised Fig 3

19. Discussion:

Some parts need to be modified to reflect my comments above including the first paragraph which needs to be significantly improved.

- How do you conclude this Is highly feasible and acceptable?

Please see above (Reviewer 2, Point 13)

See revised article with highlighted changes, p.6, p.9

- Add the earliest 2 hours after PACU admission?

We added this information.

See revised article with highlighted changes, p.16

- Omit excellent recruitment this is ambiguous; leave high completion rates;

We changed the sentence as follows: “We found high acceptability and feasibility, good validity, adequate responsiveness, and moderate reliability.”

See revised article with highlighted changes, p.16

- also clarify in the methods section if patients alone completed the surveys or were the items read to them and explained?

We clarified this section as follows: “Patients read and completed the questionnaire themselves. If necessary, patients were provided with glasses by the study team.”

See revised article with highlighted changes, p.5

- Your PACU times are interesting as in many countries patients mostly leave PACUs after around 60 minutes – please add to limitations how can your QoR instrument be used sooner;

Please see above (Reviewer 2, Point 2).

See revised article with highlighted changes, p.17

- Use some citations when you refer to gender influences in the discussion

We have added the appropriate citations:

2. Myles PS, Weitkamp B, Jones K, Melick J, Hensen S. Validity and reliability of a postoperative quality of recovery score: the QoR-40. Br J Anaesth. 2000 Jan;84(1):11–5.

39. Buchanan FF, Myles PS, Cicuttini F. Effect of patient sex on general anaesthesia and recovery. Br J Anaesth. 2011 Jun;106(6):832–9.

40. Buchanan FF, Myles PS, Cicuttini F. Patient sex and its influence on general anaesthesia. Anaesth Intensive Care. 2009 Mar;37(2):207–18.

41. Myles PS, McLeod AD, Hunt JO, Fletcher H. Sex differences in speed of emergence and quality of recovery after anaesthesia: cohort study. BMJ. 2001 Mar 24;322(7288):710–1.

See revised article with highlighted changes, p.18-19

References

Boateng, Godfred O., Torsten B. Neilands, Edward A. Frongillo, Hugo R. Melgar-Quiñonez, and Sera L. Young. 2018. ‘Best Practices for Developing and Validating Scales for Health, Social, and Behavioral Research: A Primer’. Frontiers in Public Health 6 (June): 149. https://doi.org/10.3389/fpubh.2018.00149.

Ganter, Michael T, Stephan Blumenthal, Seraina Dübendorfer, Simone Brunnschweiler, Tim Hofer, Richard Klaghofer, Andreas Zollinger, and Christoph K Hofer. 2014. ‘The Length of Stay in the Post-Anaesthesia Care Unit Correlates with Pain Intensity, Nausea and Vomiting on Arrival’. Perioperative Medicine 3 (November): 10. https://doi.org/10.1186/s13741-014-0010-8.

Middel, Berrie, Hanna Kuipers-Upmeijer, Jelte Bouma, Michiel Staal, Dettie Oenema, Theo Postma, Sijmon Terpstra, and Roy Stewart. n.d. ‘EVect of Intrathecal Baclofen Delivered by an Implanted Programmable Pump on Health Related Quality of Life in Patients with Severe Spasticity’, 6.

Myles, PS, K Leslie, J McNeil, A Forbes, and MTV Chan. 2004. ‘Bispectral Index Monitoring to Prevent Awareness during Anaesthesia: The B-Aware Randomised Controlled Trial’. The Lancet 363 (9423): 1757–63. https://doi.org/10.1016/S0140-6736(04)16300-9.

Norman, G. R., Kathleen W. Wyrwich, and Donald L. Patrick. 2007. ‘The Mathematical Relationship among Different Forms of Responsiveness Coefficients’. Quality of Life Research 16 (5): 815–22. https://doi.org/10.1007/s11136-007-9180-x.

Nunnally, Jum C. 1978. Psychometric Theory. 2d ed. McGraw-Hill Series in Psychology. New York: McGraw-Hill.

Plöger, Birgit. 2008. ‘Validierung einer deutschen Übersetzung des Quality-of-Recovery-Scores-40 (QoR-40) als Maß der patientenzentrierten postoperativen Ergebnisqualität’. Marburg: Philipps-Universität Marburg.

Sivan, Manoj. 2009. ‘Interpreting Effect Size to Estimate Responsiveness of Outcome Measures’. Stroke 40 (12): e709–e709. https://doi.org/10.1161/STROKEAHA.109.566836.

Stark, Peter A., Paul S. Myles, and Justin A. Burke. 2013. ‘Development and Psychometric Evaluation of a Postoperative Quality of Recovery Score’. Anesthesiology 118 (6): 1332–40. https://doi.org/10.1097/ALN.0b013e318289b84b.

Terwee, Caroline B., Sandra D. M. Bot, Michael R. de Boer, Daniëlle A. W. M. van der Windt, Dirk L. Knol, Joost Dekker, Lex M. Bouter, and Henrica C. W. de Vet. 2007. ‘Quality Criteria Were Proposed for Measurement Properties of Health Status Questionnaires’. Journal of Clinical Epidemiology 60 (1): 34–42. https://doi.org/10.1016/j.jclinepi.2006.03.012.

Weissman, Charles, Jeremy Scemama, and Yoram G. Weiss. 2019. ‘The Ratio of PACU Length-of-Stay to Surgical Duration: Practical Observations’. Acta Anaesthesiologica Scandinavica 63 (9): 1143–51. https://doi.org/10.1111/aas.13421.

Attachment

Submitted filename: R1_Point-by-point_response_180922.docx

Click here for additional data file.^{(48.3KB, docx)}

PLoS One. doi: 10.1371/journal.pone.0289685.r003

Decision Letter 1

Seunghwa Lee

1 Jun 2023

PONE-D-22-18272R1Psychometric evaluation of a quality of recovery score for the postanesthesia care unit – a prospective validation studyPLOS ONE

Dear Dr. Marlene Fischer,

==============================I am especially sorry to late review. There's a lot of trouble to find reviewers.

==============================

Please submit your revised manuscript by 2023-07-01. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.
A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.
An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

We look forward to receiving your revised manuscript.

Kind regards,

Seunghwa Lee

Academic Editor

PLOS ONE

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #3: All comments have been addressed

Reviewer #4: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

Reviewer #3: Yes

Reviewer #4: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #3: Yes

Reviewer #4: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

Reviewer #3: Yes

Reviewer #4: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

Reviewer #3: Yes

Reviewer #4: Yes

**********

6. Review Comments to the Author

Reviewer #3: First of all, I would like to thank you for the opportunity to review the article entitled "Psychometric evaluation of a quality of recovery score for the postanesthesia care unit – a prospective validation study" which I found of great interest.

I agree with most of the comments made by the previous reviewers and the authors have tried to address them.

As one of the reviewers commented, it is true that the main limitation is that the instrument is in German and adapted to the context of the country, but since there are no instruments for assessing health outcomes in perioperative patients, I consider its publication to be relevant, allowing later cultural adaptation to other languages.

As a general comment, they mention that there are multiple instruments for assessing recovery 24 hours after the intervention, but instead of reviewing these instruments or generating a pool of items, they decide to use the QoL-40, and it would be convenient to justify this aspect.

Regarding the abstract, perhaps it would be good to include the time in which the patients answered the instrument before and after the intervention (which seems to be about 2h after the intervention).

In the methods section, it would be good to indicate how the subsamples were selected to assess test-retest reliability, as well as for the pilot studies and why this sample size.

In the description of the sample size calculation, they indicate that they have included 375 participants, but in reality, a portion were for the pilot study and 255 participants were actually used to measure metric properties.

In the table 3 where they indicate the change in each item, it would be good to include the content of the item to assess the reason for the effect size.

They have also calculated Cronbach's alpha for the total instrument. An important part of the validation of an instrument is to assess its factor structure and to check the unidimensionality. I do not know if the authors initiated any analysis in this regard although they did not mention it in the article.

As a limitation, it should be pointed out that all the patients underwent radical prostatectomy surgery. This obviously implies that women have not been included, but also that the mean age of the sample will be mostly older adults.

Another limitation is that the patients were included in the development of the instrument but only for the pilot study or the final interviews. It is highly recommended to include patients and professionals in all phases of the development process, for example, through focus groups.

Although the article presents some preliminary analyses for the validation of the instrument, I believe that if the authors are able to address some comments and to emphasize the limitations by explaining that it is a preliminary validation study, the article is of interest.

Thank you

Reviewer #4: (No Response)

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #3: Yes: Yolanda Pardo Cladellas

Reviewer #4: No

**********

PLoS One. 2023 Aug 15;18(8):e0289685. doi: 10.1371/journal.pone.0289685.r004

Author response to Decision Letter 1

3 Jul 2023

PONE-D-22-18272R1 – Revision 2

Reviewer's Responses to Questions

Reviewer #3:

1. As one of the reviewers commented, it is true that the main limitation is that the instrument is in German and adapted to the context of the country, but since there are no instruments for assessing health outcomes in perioperative patients, I consider its publication to be relevant, allowing later cultural adaptation to other languages.

With this study, we aimed to develop a new tool for the assessment of patient-reported quality of recovery (QoR) in the postanesthesia care unit (PACU). A questionnaire can only be used with patients who have excellent understanding of the respective language the questionnaire was written in. Therefore, to evaluate and to use the instrument in our German speaking patient population, we developed the QoR-PACU in German language.

We hope that international psychometricians and clinicians will find the QoR-PACU useful and will translate it into other languages.

We have added these considerations to the discussion section of the manuscript.

See revised article with highlighted changes, p.19

2. As a general comment, they mention that there are multiple instruments for assessing recovery 24 hours after the intervention, but instead of reviewing these instruments or generating a pool of items, they decide to use the QoL-40, and it would be convenient to justify this aspect.

We agree with the reviewer that using a larger pool of items to choose from, might have benefited the development of the QoR-PACU. Therefore, we have added this aspect as a limitation. Yet, we have explicitly decided to derive the QoR-PACU from the QoR questionnaires, since their use has been recommended by the StEP inititiative [1]. Future studies should include suggestions by anaesthesiologists, PACU nurses and patients.

Revised manuscript with highlighted changes, p.19

3. Regarding the abstract, perhaps it would be good to include the time in which the patients answered the instrument before and after the intervention (which seems to be about 2h after the intervention).

We have added this information to the abstract.

Revised manuscript with highlighted changes, p.2

4. In the methods section, it would be good to indicate how the subsamples were selected to assess test-retest reliability, as well as for the pilot studies and why this sample size.

Pilot study selection

The patients who were included in the pilot testing phase were selected consecutively in analogy to the patients included in the main analysis. All patients who met the inclusion criteria and did not meet exclusion criteria were approached by a member of the study team for participation in the study.

Pilot study sample size

We did not calculate sample sizes for the pilot study. Instead, the pilot studies were conducted in exactly the same way as the main study. The pilot phase was halted when we encountered problems that led us to modify the questionnaire.

The first pilot phase was halted after 72 patients, when we noticed misunderstandings regarding similarity between the 11-point response scale (measure of frequency) and the numeric rating scale used for assessment of pain (measure of intensity). The second pilot phase was halted after 48 patients, when we noticed that the font size of the questionnaire was too small for some patients.

Test-retest subgroup selection

To assess test-retest reliability, 19 patients completed the QoR-PACU twice, 60 ± 30 minutes after the first assessment. The two assessments were conducted by two different raters. On 5 consecutive days, two members of the study team were available for testing. During these days, all study participants underwent a second assessment and were included for test-retest reliability.

Test-Retest sample size

We aimed to analyze test-retest reliability in 20-25 patients [2–4]. As explained above, we performed retest on 5 consecutive days, enrolling a convenience sample of patients available on the respective days.

5. In the description of the sample size calculation, they indicate that they have included 375 participants, but in reality, a portion were for the pilot study and 255 participants were actually used to measure metric properties.

We thank the reviewer for alerting us to this misleading wording and we have rephrased the paragraph as follows:

“There is no consistent recommendation regarding sample size for the development and the evaluation of a questionnaire. A “rule of thumb” suggests at least 10 participants for each scale item [27,28]. This would result in 160 participants for the 16-item version 1, or 130 participants for the 13-item version 3 of the questionnaire. We did not a priori decide on the exact number of items to be included in the questionnaire. Instead, we planned to conduct a pilot phase with at least 100 patients to optimize the development of the questionnaire. For the final analysis, we aimed to include at least 200 participants. Since we expected a drop-out rate of 20%, we opted for the overall inclusion of 375 patients.”

Revised manuscript with highlighted changes, p.7

6. In the table 3 where they indicate the change in each item, it would be good to include the content of the item to assess the reason for the effect size.

Thank you for this advice. We have added the item description to Table 3, p. 13.

7. They have also calculated Cronbach's alpha for the total instrument. An important part of the validation of an instrument is to assess its factor structure and to check the unidimensionality. I do not know if the authors initiated any analysis in this regard although they did not mention it in the article.

Indeed, we have performed confirmatory and exploratory factor analyses. The exploratory factor analysis showed the presence of at least four factors (based on the p-value of the chi-squared test which was smaller than 5% for three factors and 0.112 for four factors). Those factors were not congruent with the questionnaire from which we adapted the items which had two dimensions, a physical and a mental dimension. Since our questionnaire is based on a validated questionnaire with two dimensions, we additionally performed confirmatory factor analysis, checking whether our adapted questionnaire shares the same dimensions. The confirmatory factor analysis showed a root mean squared error of approximation of 0.085 (90% CI: 0.07 – 0.10), which does not show a good fit. In the same line, the other indices show a rather unsatisfactory fit. Also, removing some of the items with a small loading did not improve the result. Since the results of the factor analyses do not add to the psychometric robustness of the QoR-PACU, but rather point to the necessity of further modifications of the instrument, we did not report these results but instead aim to continue improving the questionnaire.

If requested by the editor or the reviewer, we will be happy to include the results of the factor analyses in the manuscript or as supplementary information.

8. As a limitation, it should be pointed out that all the patients underwent radical prostatectomy surgery. This obviously implies that women have not been included, but also that the mean age of the sample will be mostly older adults.

We strongly agree with the reviewer. The fact that we included solely male patients of similar age and ethnicity, scheduled for the same surgical procedure, is one of the main limitations of our study.

We have mentioned this limitation in the manuscript:

“This validation study was performed at the PACU of a prostate cancer clinic. All surgical procedures and perioperative care at our prostate cancer center are highly standardized. Although allowing for excellent comparability between participants, generalizability is limited. We included solely male patients scheduled for radical prostatectomy. Results from previous studies suggest that gender aspects have an impact on postoperative QoR and speed of recovery [2,42–44]. Overall, female patients tend to have lower QoR and longer PACU stay [2,42,44]. Morevover, pain intensity, nausea, and vomiting after surgery are more frequently reported by female patients [42,43]. Gender aspects may be of high importance in individualized perioperative care and postoperative recovery.

[…]

We aim to conduct a follow-up study which will include a more heterogenous study cohort including female as well as gender-diverse patients, different age groups and a variety of surgical procedures.

See revised article with highlighted changes, p.18-19

9. Another limitation is that the patients were included in the development of the instrument but only for the pilot study or the final interviews. It is highly recommended to include patients and professionals in all phases of the development process, for example, through focus groups.

We agree with the reviewer, that it would be commendable to include patients in all phases of development, including the very early stages. We did not develop the QoR-PACU from scratch. Instead, we selected items from the QoR-40. At this stage of development, only professionals (experienced anesthesiologists) were involved.

For the follow-up study we will modify the QoR-PACU. The modification will comprise suggestions from anesthesiologists, PACU nurses and patients.

We have modified the conclusion as follows:

“This preliminary validation study presents the development of a questionnaire to assess self-reported QoR after surgery in the PACU. We found high acceptability and feasibility, good validity, and adequate responsiveness. Against our hypothesis, we did not find high internal consistency. Based on these findings, the QoR-PACU should be modified. The modification process should consider suggestions from healthcare professionals and patients. Future psychometric evaluation should include a more heterogeneous patient cohort including female and gender-diverse patients with varying degrees of perioperative risk.”

See revised article with highlighted changes, p.19-20

10. Although the article presents some preliminary analyses for the validation of the instrument, I believe that if the authors are able to address some comments and to emphasize the limitations by explaining that it is a preliminary validation study, the article is of interest.

We thank the reviewer for this appraisal of our study. We have specified the term “preliminary validation study” in the conclusion (see comment above). To address the nature of the study more precisely as expressed by the reviewer, we suggest to edit the title of our manuscript into “Psychometric evaluation of a quality of recovery score for the postanesthesia care unit – a preliminary validation study

See revised article with highlighted changes, p.19-20

References

1. Myles PS, Grocott MPW, Boney O, Moonesinghe SR, Group C-S. Standardizing end points in perioperative trials: towards a core and extended outcome set. Br J Anaesth. 2016;116: 586–589. doi:10.1093/bja/aew066

2. Demumieux F, Ludes P-O, Diemunsch P, Bennett-Guerrero E, Lujic M, Lefebvre F, et al. Validation of the translated Quality of Recovery-15 questionnaire in a French-speaking population. Br J Anaesth. 2020;124: 761–767. doi:10.1016/j.bja.2020.03.011

3. Lyckner S, Böregård I-L, Zetterlund E-L, Chew MS. Validation of the Swedish version of Quality of Recovery score -15: a multicentre, cohort study. Acta Anaesthesiol Scand. 2018;62: 893–902. doi:10.1111/aas.13086

4. Stark PA, Myles PS, Burke JA. Development and Psychometric Evaluation of a Postoperative Quality of Recovery Score. Anesthesiology. 2013;118: 1332–1340. doi:10.1097/aln.0b013e318289b84b

Attachment

Submitted filename: R2_response_to_reviewers_300623.docx

Click here for additional data file.^{(36.2KB, docx)}

PLoS One. doi: 10.1371/journal.pone.0289685.r005

Decision Letter 2

Seunghwa Lee

19 Jul 2023

PONE-D-22-18272R2Psychometric evaluation of a quality of recovery score for the postanesthesia care unit – a preliminary validation studyPLOS ONE

Dear Dr. Fischer,

Please submit your revised manuscript by Sep 02 2023 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.
A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.
An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

We look forward to receiving your revised manuscript.

Kind regards,

Seunghwa Lee

Academic Editor

PLOS ONE

Journal Requirements:

Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice.

Additional Editor Comments:

First of all, I would like to thank you for the opportunity to review the article entitled "Psychometric evaluation of a quality of recovery score for the postanesthesia care unit – a prospective validation study" which I found of great interest.

I agree with most of the comments made by the previous reviewers and the authors have tried to address them.

In the methods section, it would be good to indicate how the subsamples were selected to assess test-retest reliability, as well as for the pilot studies and why this sample size.

In the table 3 where they indicate the change in each item, it would be good to include the content of the item to assess the reason for the effect size.

Thank you

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

PLoS One. 2023 Aug 15;18(8):e0289685. doi: 10.1371/journal.pone.0289685.r006

Author response to Decision Letter 2

21 Jul 2023

Reviewer's Responses to Questions

Reviewer #3:

We hope that international psychometricians and clinicians will find the QoR-PACU useful and will translate it into other languages.

We have added these considerations to the discussion section of the manuscript.

See revised article with highlighted changes, p.19

Revised manuscript with highlighted changes, p.19

We have added this information to the abstract.

Revised manuscript with highlighted changes, p.2

4. In the methods section, it would be good to indicate how the subsamples were selected to assess test-retest reliability, as well as for the pilot studies and why this sample size.

Pilot study selection

Pilot study sample size

Test-retest subgroup selection

Test-Retest sample size

We thank the reviewer for alerting us to this misleading wording and we have rephrased the paragraph as follows:

“There is no consistent recommendation regarding sample size for the development and the evaluation of a questionnaire. A “rule of thumb” suggests at least 10 participants for each scale item [27,28]. This would result in 160 participants for the 16-item version 1, or 130 participants for the 13-item version 3 of the questionnaire. We did not a priori decide on the exact number of items to be included in the questionnaire. Instead, we planned to conduct a pilot phase with at least 100 patients to optimize the development of the questionnaire. For the final analysis, we aimed to include at least 200 participants. Since we expected a drop-out rate of 20%, we opted for the overall inclusion of 375 patients.”

Revised manuscript with highlighted changes, p.7

6. In the table 3 where they indicate the change in each item, it would be good to include the content of the item to assess the reason for the effect size.

Thank you for this advice. We have added the item description to Table 3, p. 13.

If requested by the editor or the reviewer, we will be happy to include the results of the factor analyses in the manuscript or as supplementary information.

We have mentioned this limitation in the manuscript:

[…]

We aim to conduct a follow-up study which will include a more heterogenous study cohort including female as well as gender-diverse patients, different age groups and a variety of surgical procedures.

See revised article with highlighted changes, p.18-19

For the follow-up study we will modify the QoR-PACU. The modification will comprise suggestions from anesthesiologists, PACU nurses and patients.

We have modified the conclusion as follows:

See revised article with highlighted changes, p.19-20

References

4. Stark PA, Myles PS, Burke JA. Development and Psychometric Evaluation of a Postoperative Quality of Recovery Score. Anesthesiology. 2013;118: 1332–1340. doi:10.1097/aln.0b013e318289b84b

Attachment

Submitted filename: R2_response_to_reviewers_300623.docx

Click here for additional data file.^{(36.2KB, docx)}

PLoS One. doi: 10.1371/journal.pone.0289685.r007

Decision Letter 3

Seunghwa Lee

25 Jul 2023

Psychometric evaluation of a quality of recovery score for the postanesthesia care unit – a preliminary validation study

PONE-D-22-18272R3

Dear Dr. Fischer,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Seunghwa Lee

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

Reviewer #3: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

Reviewer #3: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #3: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

Reviewer #3: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

Reviewer #3: Yes

**********

6. Review Comments to the Author

Reviewer #3: I believe that the authors have addressed most of the comments I made. I would only suggest that the factor analysis be part of the results, even if only as additional material, as I believe it may help future researchers interested in the validation of the instrument to assess the structure of the instrument. Thank you for the opportunity to carry out this review.

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #3: No

**********

PLoS One. doi: 10.1371/journal.pone.0289685.r008

Acceptance letter

Seunghwa Lee

2 Aug 2023

PONE-D-22-18272R3

Psychometric evaluation of a quality of recovery score for the postanesthesia care unit – a preliminary validation study

Dear Dr. Fischer:

I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department.

If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org.

If we can help with anything else, please email us at plosone@plos.org.

Thank you for submitting your work to PLOS ONE and supporting open access.

Kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Dr. Seunghwa Lee

Academic Editor

PLOS ONE

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

S1 Table. QoR-PACU Version 3.0 preoperative.

(DOCX)

Click here for additional data file.^{(166.2KB, docx)}

S2 Table. QoR-PACU Version 3.0 postoperative.

(DOCX)

Click here for additional data file.^{(166.2KB, docx)}

S3 Table. Postoperative inter-item correlation.

Postoperative assessment: Inter-item correlations for the 13 items of the QoR-PACU score. Correlations are expressed as Pearson correlation coefficients.

(DOCX)

Click here for additional data file.^{(17.7KB, docx)}

Attachment

Submitted filename: PONE-D-22-18272_reviewer.pdf

Click here for additional data file.^{(1.7MB, pdf)}

Attachment

Submitted filename: R1_Point-by-point_response_180922.docx

Click here for additional data file.^{(48.3KB, docx)}

Attachment

Submitted filename: R2_response_to_reviewers_300623.docx

Click here for additional data file.^{(36.2KB, docx)}

Attachment

Submitted filename: R2_response_to_reviewers_300623.docx

Click here for additional data file.^{(36.2KB, docx)}

Data Availability Statement

[pone.0289685.ref001] 1.Myles PS, Hunt JO, Nightingale CE, Fletcher H, Beh T, Tanil D, et al. Development and psychometric testing of a quality of recovery score after general anesthesia and surgery in adults. Anesth Analg. 1999;88:83–90. doi: 10.1097/00000539-199901000-00016 [DOI] [PubMed] [Google Scholar]

[pone.0289685.ref002] 2.Myles PS, Weitkamp B, Jones K, Melick J, Hensen S. Validity and reliability of a postoperative quality of recovery score: the QoR-40. Br J Anaesth. 2000;84:11–5. doi: 10.1093/oxfordjournals.bja.a013366 [DOI] [PubMed] [Google Scholar]

[pone.0289685.ref003] 3.Borrell-Vega J, Humeidan ML, Bergese SD. Defining quality of recovery—What is important to patients? Best Pract Res Clin Anaesthesiol. 2018;32:259–68. doi: 10.1016/j.bpa.2018.07.002 [DOI] [PubMed] [Google Scholar]

[pone.0289685.ref004] 4.Myles PS. More than just morbidity and mortality—quality of recovery and long-term functional recovery after surgery. Anaesthesia. 2020;75 Suppl 1:e143–50. [DOI] [PubMed] [Google Scholar]

[pone.0289685.ref005] 5.Shulman M, Myles P. Measuring perioperative outcome. Curr Opin Anaesthesiol. 2016;29:733–8. doi: 10.1097/ACO.0000000000000383 [DOI] [PubMed] [Google Scholar]

[pone.0289685.ref006] 6.Neville A, Lee L, Antonescu I, Mayo NE, Vassiliou MC, Fried GM, et al. Systematic review of outcomes used to evaluate enhanced recovery after surgery. Br J Surg. 2014;101:159–70. doi: 10.1002/bjs.9324 [DOI] [PubMed] [Google Scholar]

[pone.0289685.ref007] 7.Stark PA, Myles PS, Burke JA. Development and psychometric evaluation of a postoperative quality of recovery score: the QoR-15. Anesthesiology. 2013;118:1332–40. doi: 10.1097/ALN.0b013e318289b84b [DOI] [PubMed] [Google Scholar]

[pone.0289685.ref008] 8.Myles PS, Grocott MPW, Boney O, Moonesinghe SR, COMPAC-StEP Group. Standardizing end points in perioperative trials: towards a core and extended outcome set. Br J Anaesth. 2016;116:586–9. [DOI] [PubMed] [Google Scholar]

[pone.0289685.ref009] 9.Myles PS, Boney O, Botti M, Cyna AM, Gan TJ, Jensen MP, et al. Systematic review and consensus definitions for the Standardised Endpoints in Perioperative Medicine (StEP) initiative: patient comfort. Br J Anaesth. 2018;120:705–11. doi: 10.1016/j.bja.2017.12.037 [DOI] [PubMed] [Google Scholar]

[pone.0289685.ref010] 10.Abola RE, Bennett-Guerrero E, Kent ML, Feldman LS, Fiore JF, Shaw AD, et al. American Society for Enhanced Recovery and Perioperative Quality Initiative Joint Consensus Statement on Patient-Reported Outcomes in an Enhanced Recovery Pathway. Anesth Analg. 2018;126:1874–82. doi: 10.1213/ANE.0000000000002758 [DOI] [PubMed] [Google Scholar]

[pone.0289685.ref011] 11.Vignaud M, Morel C, Henault A, Futier E, Pereira B, Lambert C, et al. Variability and reliability of the French version of the Quality of Recovery-40 Questionnaire (QoR-40). Anaesth Crit Care Pain Med. 2021;40:100822. doi: 10.1016/j.accpm.2021.100822 [DOI] [PubMed] [Google Scholar]

[pone.0289685.ref012] 12.Chen Y, Wang J, Liu S, Lai W, Liu J, Wang Z, et al. Development and Validation of the Chinese Version of the Quality of Recovery-40 Questionnaire. Ther Clin Risk Manag. 2020;16:1165–73. doi: 10.2147/TCRM.S281572 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0289685.ref013] 13.Lee JH, Kim D, Seo D, Son J-S, Kim D-C. Validity and reliability of the Korean version of the Quality of Recovery-40 questionnaire. Korean J Anesthesiol. 2018;71:467–75. doi: 10.4097/kja.d.18.27188 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0289685.ref014] 14.Terkawi AS, Myles PS, Riad W, Nassar SN, Mahmoud M, AlKahtani GJ, et al. Development and validation of Arabic version of the postoperative quality of recovery-40 questionnaire. Saudi J Anaesth. 2017;11:S40–52. doi: 10.4103/sja.SJA_77_17 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0289685.ref015] 15.Kleif J, Edwards HM, Sort R, Vilandt J, Gögenur I. Translation and validation of the Danish version of the postoperative quality of recovery score QoR-15. Acta Anaesthesiol Scand. 2015;59:912–20. doi: 10.1111/aas.12525 [DOI] [PubMed] [Google Scholar]

[pone.0289685.ref016] 16.Demumieux F, Ludes P-O, Diemunsch P, Bennett-Guerrero E, Lujic M, Lefebvre F, et al. Validation of the translated Quality of Recovery-15 questionnaire in a French-speaking population. Br J Anaesth. 2020;124:761–7. doi: 10.1016/j.bja.2020.03.011 [DOI] [PubMed] [Google Scholar]

[pone.0289685.ref017] 17.Kim D, Kim JK, Yeo J. Translation and Validation of the Korean Version of the Postoperative Quality of Recovery Score QoR-15. BioMed Res Int. 2020;2020:3456234. doi: 10.1155/2020/3456234 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0289685.ref018] 18.Kahl U, Gebhardt N, Brodersen K, Kainz E, Schirren L, Yu Y, et al. Validation of a translated Quality of Recovery-15 questionnaire in German patients undergoing elective noncardiac surgery. Br J Anaesth. 2021;127:e161–3. doi: 10.1016/j.bja.2021.07.026 [DOI] [PubMed] [Google Scholar]

[pone.0289685.ref019] 19.Picconi E, Iacobucci T, Adducci E, Gualtieri E, Beccia G, Sollazzi L. Translation and validation of the Italian version of the postoperative quality of recovery score QoR-15. Minerva Anestesiol. 2020;86:787–9. doi: 10.23736/S0375-9393.20.14472-9 [DOI] [PubMed] [Google Scholar]

[pone.0289685.ref020] 20.Nakatani S, Ida M, Tanaka Y, Okamoto N, Wang X, Nakatani H, et al. Translation and validation of the Japanese Version of the Quality of Recovery-15 Questionnaire. J Anesth. 2021;35:426–33. doi: 10.1007/s00540-021-02921-z [DOI] [PubMed] [Google Scholar]

[pone.0289685.ref021] 21.Aldrete JA, Kroulik D. A postanesthetic recovery score. Anesth Analg. 1970;49:924–34. [PubMed] [Google Scholar]

[pone.0289685.ref022] 22.Sá AC, Sousa G, Santos A, Santos C, Abelha FJ. Quality of Recovery after Anesthesia: Validation of the Portuguese Version of the ‘Quality of Recovery 15’ Questionnaire. Acta Med Port. 2015;28:567–74. doi: 10.20344/amp.6129 [DOI] [PubMed] [Google Scholar]

[pone.0289685.ref023] 23.Miklić Bublić M, Miklić P, Barl P, Matas M, Sekulić A. CROATIAN VERSION OF THE QUALITY OF RECOVERY QUESTIONNAIRE (QoR-40): TRANSCULTURAL ADAPTATION AND VALIDATION. Acta Clin Croat. 2021;60:237–45. doi: 10.20471/acc.2021.60.02.09 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0289685.ref024] 24.Yaghoobi S, Hamidfar M, Lawson DM, Fridlund B, Myles PS, Pakpour AH. Validity and reliability of the Iranian version of the quality of recovery-40 questionnaire. Anesthesiol Pain Med. 2015;5:e20350. [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0289685.ref025] 25.Ludbrook G, Lloyd C, Story D, Maddern G, Riedel B, Richardson I, et al. The effect of advanced recovery room care on postoperative outcomes in moderate-risk surgical patients: a multicentre feasibility study. Anaesthesia. 2021;76:480–8. doi: 10.1111/anae.15260 [DOI] [PubMed] [Google Scholar]

[pone.0289685.ref026] 26.Plöger B. Validierung einer deutschen Übersetzung des Quality-of-Recovery-Scores-40 (QoR-40) als Maß der patientenzentrierten postoperativen Ergebnisqualität. [Marburg]: Philipps-Universität Marburg; 2008.

[pone.0289685.ref027] 27.Boateng GO, Neilands TB, Frongillo EA, Melgar-Quiñonez HR, Young SL. Best Practices for Developing and Validating Scales for Health, Social, and Behavioral Research: A Primer. Front Public Health. 2018;6:149. [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0289685.ref028] 28.Nunnally J. Pyschometric Theory. 1978.

[pone.0289685.ref029] 29.Vetter TR, Cubbin C. Psychometrics: Trust, but Verify. Anesth Analg. 2019;128:176–81. doi: 10.1213/ANE.0000000000003859 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0289685.ref030] 30.Revelle W. psych: Procedures for Psychological, Psychometric, and Personality Research [Internet]. 2021 [cited 2022 May 1]. https://CRAN.R-project.org/package=psych

[pone.0289685.ref031] 31.Sherman RA. multicon: Multivariate Constructs [Internet]. 2015 [cited 2022 May 1]. https://CRAN.R-project.org/package=multicon

[pone.0289685.ref032] 32.Middel B, van Sonderen E. Statistical significant change versus relevant or important change in (quasi) experimental design: some conceptual and methodological problems in estimating magnitude of intervention-related change in health services research. Int J Integr Care. 2002;2:e15. doi: 10.5334/ijic.65 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0289685.ref033] 33.Norman GR, Wyrwich KW, Patrick DL. The mathematical relationship among different forms of responsiveness coefficients. Qual Life Res. 2007;16:815–22. doi: 10.1007/s11136-007-9180-x [DOI] [PubMed] [Google Scholar]

[pone.0289685.ref034] 34.Taherdoost H. Validity and Reliability of the Research Instrument; How to Test the Validation of a Questionnaire/Survey in a Research. SSRN Electron J [Internet]. 2016 [cited 2022 Jun 20]; https://www.ssrn.com/abstract=3205040

[pone.0289685.ref035] 35.Bowyer A, Royse C. The importance of postoperative quality of recovery: influences, assessment, and clinical and prognostic implications. Can J Anaesth J Can Anesth. 2016;63:176–83. [DOI] [PubMed] [Google Scholar]

[pone.0289685.ref036] 36.Aldrete JA. The post-anesthesia recovery score revisited. J Clin Anesth. 1995;7:89–91. doi: 10.1016/0952-8180(94)00001-k [DOI] [PubMed] [Google Scholar]

[pone.0289685.ref037] 37.Nimmo SM, Foo ITH, Paterson HM. Enhanced recovery after surgery: Pain management. J Surg Oncol. 2017;116:583–91. doi: 10.1002/jso.24814 [DOI] [PubMed] [Google Scholar]

[pone.0289685.ref038] 38.Mann-Farrar J, Egan E, Higgins A, Wysocki L, Vaux A, Arndell E, et al. Are Postoperative Clinical Outcomes Influenced by Length of Stay in the Postanesthesia Care Unit? J Perianesthesia Nurs Off J Am Soc PeriAnesthesia Nurses. 2019;34:386–93. [DOI] [PubMed] [Google Scholar]

[pone.0289685.ref039] 39.Myles P, Leslie K, McNeil J, Forbes A, Chan M. Bispectral index monitoring to prevent awareness during anaesthesia: the B-Aware randomised controlled trial. The Lancet. 2004;363:1757–63. [DOI] [PubMed] [Google Scholar]

[pone.0289685.ref040] 40.Jakobsson J, Idvall E, Kumlien C. Patient characteristics and surgery-related factors associated with patient-reported recovery at 1 and 6 months after colorectal cancer surgery. Eur J Cancer Care (Engl). 2017;26. [DOI] [PubMed] [Google Scholar]

[pone.0289685.ref041] 41.Terwee CB, Bot SDM, de Boer MR, van der Windt DAWM, Knol DL, Dekker J, et al. Quality criteria were proposed for measurement properties of health status questionnaires. J Clin Epidemiol. 2007;60:34–42. doi: 10.1016/j.jclinepi.2006.03.012 [DOI] [PubMed] [Google Scholar]

[pone.0289685.ref042] 42.Buchanan FF, Myles PS, Cicuttini F. Effect of patient sex on general anaesthesia and recovery. Br J Anaesth. 2011;106:832–9. doi: 10.1093/bja/aer094 [DOI] [PubMed] [Google Scholar]

[pone.0289685.ref043] 43.Buchanan FF, Myles PS, Cicuttini F. Patient sex and its influence on general anaesthesia. Anaesth Intensive Care. 2009;37:207–18. doi: 10.1177/0310057X0903700201 [DOI] [PubMed] [Google Scholar]

[pone.0289685.ref044] 44.Myles PS, McLeod AD, Hunt JO, Fletcher H. Sex differences in speed of emergence and quality of recovery after anaesthesia: cohort study. BMJ. 2001;322:710–1. doi: 10.1136/bmj.322.7288.710 [DOI] [PMC free article] [PubMed] [Google Scholar]

PERMALINK

Psychometric evaluation of a quality of recovery score for the postanesthesia care unit—A preliminary validation study

Ursula Kahl

Katrin Brodersen

Sarah Kaiser

Linda Krause

Regine Klinger

Lili Plümer

Christian Zöllner

Marlene Fischer

Roles

Abstract

Introduction

Methods

Results

Conclusion

Introduction

Materials and methods

Ethical approval and study registration

Study design and participants

Development and adaptation of the QoR-PACU

Data collection

Sample size

Statistical analysis

Results

Demographic and clinical characteristics

Fig 1. The flow diagram shows patients included and excluded throughout the course of the study.

Table 1. Demographic and clinical characteristics.

QoR-PACU

Table 2. QoR-PACU scores for each assessment.

Table 3. Mean QoR-PACU scores for each item and the mean QoR-PACU sum score.

Fig 2. The radar chart–spider diagram shows mean scores of single items of the QoR-PACU preoperatively (dark green, n = 246), in the PACU (black, n = 246), at re-assessment in the PACU (blue, n = 19), and on the day after surgery (light green, n = 44).

Validity, reliability, and responsiveness

Fig 3. Correlation between postoperative QoR-PACU sum scores and clinically relevant continuous variables.

Fig 4. Correlations between each item and the sum score and between single items of the postoperative QoR-PACU using Spearman correlation coefficients.

Discussion

Conclusion

Supporting information

Acknowledgments

Data Availability

Funding Statement

References

Decision Letter 0

Seunghwa Lee

Roles

Author response to Decision Letter 0

Decision Letter 1

Seunghwa Lee

Roles

Author response to Decision Letter 1

Decision Letter 2

Seunghwa Lee

Roles

Author response to Decision Letter 2

Decision Letter 3

Seunghwa Lee

Roles

Acceptance letter

Seunghwa Lee

Roles

Associated Data

Supplementary Materials

Data Availability Statement

ACTIONS

PERMALINK

RESOURCES

Similar articles

Cited by other articles

Links to NCBI Databases