Development and validation of Daycare Anesthesia Satisfaction (DAS) questionnaire to assess patient’s satisfaction with daycare anesthesia

Abstract

Background and Aims

The patient’s satisfaction can be considered as a unique indicator of the quality of healthcare provided. The advantages of patient satisfaction surveys rely heavily on using standardized, psychometrically tested data collection approaches. There is a lack of a proper, psychometrically robust instrument to evaluate the patient’s perioperative satisfaction following all types of anesthesia in daycare facility. Hence, this study aimed to develop a Daycare Anesthesia Satisfaction (DAS) questionnaire to measure the patient’s satisfaction with the experience of daycare anesthesia services.

Material and Methods

A preliminary pool of questions was generated from research literature, expert consultations, and pilot tested on patients. The internal consistency and reliability of the preliminary questionnaire was evaluated by calculating Cronbach’s alpha, intraclass correlation coefficient (ICC), and feasibility with the formation of a final 27-item questionnaire. In the next step, the questionnaire was distributed to a larger group of patients in the phase 2 of postanesthesia care unit (PACU). The results were subjected to confirmatory factor analysis to determine the goodness of fit of the questions under each domain.

Results

The internal consistency of the preliminary questionnaire as measured by Cronbach’s alpha was 0.929. Intraclass correlation coefficient measured for test-retest reliability was 0.97 (95% confidence interval [CI]). Feasibility was confirmed, as 75% of the patients could fill the questionnaire within 15 min. In the second step of confirmatory factor analysis (CFA), questionnaire has been shown to have goodness of fit with Bentler’s comparative fit index (CFI) of 0.99 that is greater than the suggested cutoff of 0.90. The root mean square error of approximation (RMSEA) of 0.09 is also close to the suggested cutoff of 0.06.

Conclusion

This systematically developed and validated, 27-item DAS questionnaire can be tentatively recommended to be used to measure patient’s satisfaction with day care anesthesia services following all types of surgical procedures, under various types of anesthesia.

Introduction

The patient’s satisfaction is a unique indicator of the quality of healthcare provided. In modern medicine, the use of patient satisfaction as an outcome is more insightful than the use of morbidity and mortality data to improvise on the anesthetic care.[1] There is a shift in healthcare toward person-centered care (PCC), where the patient is encouraged to take an active part in the care process.[2] So measuring patient satisfaction becomes an important parameter of the continuous quality assessment and improvement in anesthesia services.[3]

There is no clear definition on how to define the concept of patient satisfaction in healthcare. In Donabedian’s quality measurement model, patient satisfaction is defined as patient-reported outcome measure, with patient reported experiences measuring the structures and processes of care. Patient satisfaction represents their attitudes toward aspects of care as well as patients’ emotions, feelings, and their perception of delivered healthcare services.[4,5,6,7] On the other hand, patient satisfaction can be defined as a degree of congruency between patient expectations of ideal care and their perceptions of real care received.

Basically, there are two approaches for evaluating patient satisfaction, qualitative and quantitative. The quantitative approach provides accurate methods to measure patient satisfaction. Standardized questionnaires (either self-reported or interviewer-administrated or by telephone) have been commonly used as assessment tools.[8] The advantage of a self-administered questionnaire over interview/investigator filled questionnaire is that, it is not influenced by investigator’s opinion. A standardized tool needs to be developed and refined to reflect positively on the main goals of patient-satisfaction survey.

The task of developing a new questionnaire or translating an existing questionnaire into a different language might be overwhelming. The greatest challenge perhaps is to come up with a questionnaire that is psychometrically sound and is efficient and effective for use in research and clinical settings. Due to the multidimensional and complex nature of satisfaction, questionnaire should use multiple items to understand patient’s perspective at specific events.[9]

A range of instruments has been used to evaluate different aspects of the patient’s experience with perioperative anesthetic care. The use of simple, single dimension questionnaire will give false high score and is inadequate to address the complexity of satisfaction.[10,11,12,13,14] A systematic review of questionnaires used for measuring patient satisfaction after ambulatory anesthesia has shown that only 2 of the 11 questionnaires evaluated were found to be psychometrically developed.[15] Of the two questionnaires, ISAS (Iowa Satisfaction with Anesthesia Scale) scale was designed only for monitored anesthesia care patients and EVAN-G questionnaire (Evaluation du Vecu de l’Anesthesie Generale) included both inpatients as well as ambulatory surgery patients.[16,17]

The aim of this study was to develop an instrument to measure the patient’s satisfaction with the experience of day care anesthesia services suitable for all surgical specialty patients, undergoing all types of anesthesia.

Material and Methods

This study had two main phases: the construction of a preliminary questionnaire, and then the process of psychometric evaluation to successively optimize the questionnaire. The Institutional Review Board (IRB) and Ethics Committee approval was obtained from the Institution (IRB Min No. 11739, dated 19/12/2018). For the development of a reliable validated questionnaire, a systematic stepwise approach was made.

Phase 1: Item and dimension generation

As the first step, the instruments that are used to measure patients’ experiences with ambulatory anesthesia and which are published in peer-reviewed journals were reviewed. Consultation was also done with senior, colleague anesthesiologists. From the above, as well as our understanding of the different aspects which needed to be covered for patient care in ambulatory anesthesia, the dimensions of care that emerged were physical comfort, emotional support, information provision, and involvement in care. The four different phases of anaesthetic care in daycare operation theatre, addressing the abovementioned dimensions, was decided and questions addressing the dimensions were generated.

Preoperative phase: This phase includes patient’s experience in the preoperative period in regard to information available after preanesthesia clinic visit (PAC), process of admission in day care (DC) ward and the reassessment by the anesthesiologist in the DC ward.

Intraoperative phase (physical aspect): This phase includes patient’s experience in the intraoperative phase in terms of how comfortable he/she was during transfer to OR, positioning on to the OR table, pain experiences during IV cannulation and anesthetic procedures.

Intraoperative phase (emotional aspect): This phase determines the emotional comfort and care during the intraoperative phase in regard to reassurance by anesthesiologist, privacy, and emotional support provided by the team involved.

Postoperative phase: This phase measures the postanesthetic experiences in terms of addressing any postoperative complications, information provided at discharge.

Questions generated were close ended and Likert-type format was used. The scale adopted to grade the responses were 1 = Unsatisfied, 2 = Not much satisfied, 3 = Just Satisfied, 4 = Very Much Satisfied, and 5 = Fully Satisfied. There was a total of 31 questions that were generated for the abovementioned dimensions. Since the concept of satisfaction is multidimensional, subjective, and individualized, the questionnaire was intended to be self-administered so that we obtain a true representation of the patient’s feelings.[13,18]

Review and revision of initial pool of questions

As the next step, the generated dimensions and the initial pool of questions were reviewed by experts in the Department of Anesthesia, to make sure they are grammatically correct, accurate in terms of content and construct. A cover letter and the questionnaire were included with the content validity survey explaining why experts were invited to participate, along with clear and concise instructions on how to rate each item. The number of experts chosen was as per recommendations.[19,20] To evaluate whether items were relevant, clear, and essential, experts were given a critical appraisal sheet with the following four inquiries: 1) the relevance of each question in the tool (how important the question is); 2) the clarity of each question (how clear the wording is); 3) the essentiality of each question (how necessary the question is); and 4) recommendations for improvement of each question. Six experts of the same field graded the questions of the questionnaire on a scale from 1 to 4 (a 4-point scale was used to avoid any neutral remarks).

Assessment of content validity: As a next step to the experts grading, the questionnaire was checked for content validity, which is a quantitative measure to check whether the instrument has an appropriate sample of items for the construct being measured. Both the item CVI (I-CVI) and scale CVI (S-CVI) was measured for the questions from the experts grading. All the questions which had CVI > 0.8 were included. The average expert CVI was 0.95 and scale CVI 0.821. Additional comments and recommendations by the experts were written on the hard copy of the questionnaire that was provided with the cover letter. Based on the above process, questions were added, modified, and removed, and finally a pretest questionnaire was formed with 27 questions under four dimensions.

Phase 2: Validation of the questionnaire

This phase included: 1) pilot study; 2) retesting of the final version of the questionnaire.

Pilot testing of the questionnaire

We pilot tested the questionnaire items on a small sample of 29 patients to check for any revision required. A total of 29 patients belonging to American Society of Anesthesiologists (ASA) grade 1 and 2, aged more than 18 years, and were literate were chosen randomly, irrespective of the surgical specialty, to participate in the pretest. They were administered the questionnaire in phase 2 of post anesthesia care unit (PACU) just prior to discharge. For retesting, another copy of the questionnaire was given to the participants in a sealed envelope that was filled by them on the following up visit after a period of 5–7 days. The filled questionnaire was assessed for internal consistency using Cronbach’s alpha and test-retest reliability by calculating intraclass correlation coefficient (ICC), and feasibility by measuring the percentage of questionnaires filled within 15 min. With the above process, the questionnaire was finalized to be retested.

Confirmatory Factor Analysis (CFA)

As a next step, we intended to assess if the questions generated under each phase or dimension had the goodness of fit under the chosen domain. This was done with the help of confirmatory factor analysis done on a larger sample of patients. A total of 228 patients, belonging to different surgical specialties, undergoing either general or regional anesthesia, were consecutively chosen and administered the questionnaire in phase 2 PACU. They were asked to fill up the questionnaire and drop the filled forms in a drop box before leaving DC ward.

CFA was performed on the sample of 220 participants who returned the filled questionnaire, to test whether the items in each domain fits the responses well. A Chi-square test for goodness of fit was used to assess model fit between the model and the sample. The root mean square error of approximation (RMSEA) was calculated, with a goal of 0.05 for good model fit.[21] The Bentler’s comparative fit index (CFI) was computed with a cut off point of 0.95 set for good model fit.[22,23] Cronbach’s alpha was used as an estimation of the reliability of the estimation of the underlying categories. Cronbach’s alpha between 0.7 and 0.95 was considered satisfactory.

Data were entered in Epidata software Version 3.1 and data analysis was done using SAS software Version 9.4 (SAS Institute Inc.)

DAS Questionnaire is attached in the appendix section.

Results

After the initial step of questionnaire validation by experts, the tool was pilot tested on 29 patients undergoing DC anesthesia, belonging to various surgical specialties. Out of 29 patients, only 25 filled the questionnaire during the first time and we included only those 25 patients for the re-test when they came for their follow-up appointment. The 25 responses were assessed for internal consistency, test-retest reliability, and feasibility. The internal consistency as measured by Cronbach’s alpha was 0.9294 in the first read and 0.9322 in second read. ICC measured for test retest reliability was 0.97 (95% CI). Feasibility was confirmed, as 75% of the patients could fill the questionnaire within 15 min. So, with the abovementioned process we finalized the preliminary questionnaire.

As a next step, CFA was done on 220 patients who filled and returned the questionnaire. The response rate was 96.5% as 220 out of 228 filled out the forms. The demographic details of the study group are given in Table 1. The patients were representative of various surgical specialties.

Table 1.

Descriptive statistics of the study population

Variables	n (n=220)	%
Age:Mean±SD	37.3±12.3
Sex
Male	150	68.2
Female	70	31.8
Surgical Specialty
ENT	23	10.5
Orthopedics	35	16.0
General surgery	59	26.9
Vascular	28	12.8
Endocrine	13	5.9
Spine	10	4.6
Hand surgery	6	2.3
Urology	46	21.0
Education Level
Primary	14	6.4
High School	52	23.6
Senior Higher Secondary	78	35.5
Graduate	63	28.6
Postgraduate	13	5.9
Previous Anesthesia Exposure
Yes	129	55
No	91	45
Previous Day Care Anesthesia Exposure
Yes	62	28.18
No	158	71.82
Type of Anesthesia
General	130	59.4
Regional	80	36.5
Monitored Anesthesia Care	10	4.1

The 27-item version of the questionnaire for patient satisfaction with day care anesthesia had four domains consisting of the preoperative phase (including PAC and DC admission ward), intraoperative phase—physical aspect, intraoperative—emotional aspect, postoperative phase including recovery and discharge. The questionnaire had goodness of fit with Bentler’s CFI of 0.99 that is greater than the suggested cutoff of 0.90. This implies that the overall fit is good. Similarly, the Chi-square P value of 0.04 is close to the suggested cutoff of zero. The RMSEA, 0.09 is also close to the suggested cutoff of 0.06. The overall Cronbach’s alpha, 0.84 implies that there is an overall consistency in the tool.

We also assessed the goodness of fit for each of the four domains which is given in Table 2. The domain of questions for postoperative phase (including recovery and discharge) has a greatest number with nine items and demonstrated good fit (Chi-square P value = 0.00, Bentler’s CFI = 0.88, Cronbach’s alpha = 0.87, and RMSEA = 0.12). The preoperative phase domain contained eight items and had a best fit with Chi-square P value of 0.00, Bentler’s CFI 0.92 and RMSEA 0.11. The Cronbach’s alpha for this domain is 0.86. The domain of questions for the physical aspect of intraoperative phase contains 5 questions having a best fit with Chi-square P value zero, Bentler’s CFI 0.95 and Cronbach’s alpha 0.80. The domain for the emotional aspect of the intraoperative phase also contains five questions. This domain had a reasonable fit with Chi-square P value zero, Bentler’s CFI 0.88, Cronbach’s alpha 0.82, and RMSEA 0.20

Table 2.

Confirmatory factor analysis to analyze domain-wise fit

	Model
Over all
Number of items	27
Chi-square P	0.04
RMSEA	0.09
Bentler’s CFI	0.99
Cronbach’s alpha	0.82
Factor 1: Preoperative phase (PAC and DC admission ward)
Number of items	8
Chi-square P	0.00
RMSEA	0.11
Bentler’s CFI	0.92
Cronbach’s alpha	0.86
Factor 2: Intraoperative phase (Physical aspect)
Number of items	5
Chi-square P	0.00
RMSEA	0.13
Bentler’s CFI	0.95
Cronbach’s alpha	0.80
Factor 3: Intraoperative (Emotional aspect)
Number of items	5
Chi-square P	0.00
RMSEA	0.20
Bentler’s CFI	0.88
Cronbach’s alpha	0.82
Factor 4 : Postoperative phase (Recovery and discharge)
Number of items	9
Chi-square P	0.00
RMSEA	0.12
Bentler’s CFI	0.88
Cronbach’s alpha	0.87

Discussion

In the past, patient was seen as a passive recipient of care which has changed and now there is focus on patient-centered care where patient’s experience and satisfaction are important aspects of qualitatively assessing the treatment rendered to them. Various tools are available to the measure satisfaction of patients undergoing surgery and anesthesia. Systematic review of the literature, for tools used to assess patient’s satisfaction following ambulatory surgery has found many articles that assessed patient satisfaction with anesthetic care but very few articles had used validated questionnaires to measure this outcome. So, we have undertaken the process of questionnaire development and process of validation as recommended in literature.[24]

Following the initial development of the question pool under various domains, we had obtained experts opinion to evaluate the content validity of the questionnaire. The content validity of the questionnaire should be evaluated when the initial form of the questionnaire is available, and we have undertaken this as the first step of our questionnaire development. The questionnaire has to be evaluated for content, construct, and criterion-related validity. Since content validity is a prerequisite for other the two types of validity, it receives the highest priority during instrument development. The item content validity (I-CVI) expresses the proportion of agreement on the relevancy of each item, which is between zero and one.[25] The scale CVI (SCVI) is defined as “the proportion of items on an instrument that achieved a rating of 3 or 4 by the content experts.”[26] The average content validity index of our questionnaire was 0.95 and scale CVI 0.821 which is more than the recommended value of >0.8. With the above process we had modified and removed few questions, to obtain the final questionnaire, which has 27 items under four domains: preoperative phase, intraoperative phase – physical aspect, intraoperative phase – emotional aspect, and postoperative phase.

As a next step, we pilot tested the questionnaire on a smaller sample of patients to check for confusion about any items, possible suggestions from respondents regarding improvement of questionnaire, the internal consistency, feasibility, and reliability.[27] The selected group of patients were also asked to fill up the same questionnaire in 5–7 days during their follow-up visit, to check for reliability and internal consistency. The internal consistency evaluated by Cronbach’s alpha was >0.9. The reliability of a questionnaire can be considered as the consistency of the survey when taken at two different time points. The correlation of the measures taken at the two separate time points was determined using an ICC, where 0.60 was considered marginal, 0.70 acceptable and anything over 0.80 considered high.[28] For our tool, the ICC was 0.97 (95% CI), which suggests the questionnaire to have reliability. The feasibility was confirmed as 75% of patients being able to fill up the questionnaire in less than 15 min.

The final step of the process of questionnaire validation was to check the content and construct validity on a larger sample of patients. The recommended sample varies as per guidelines and can range from 5:1 to 10:1 for the respondent-to-item ratio (i.e. 50 respondents for a 10-item questionnaire). This can be increased up to 30:1 also.[28,29,30,31] We have taken a sample of 228 patients undergoing all type of surgical procedures and anesthesia in daycare theaters. The response rate was 96.5% and confirmatory factor analysis was done on 220 patients, to check for construct and content validity. The CFA was done in this study as compared with exploratory factor analysis (EFA) because the domains were clearly established, and the questions were formulated under each domain. This analysis helps in determining the fit of the questions in each domain.[32,33,34] The overall Cronbach’s alpha of 0.82 implies that there is an overall consistency in the tool. Among the four domains, the preoperative, postoperative, and intraoperative physical domains have questions which has shown best fit in the selected domains. So, the theoretical construct validity of the questionnaire is satisfactory since the confirmatory factor analysis has shown good fit of the questions under each domain.

Limitations

We have not translated this questionnaire to various regional languages and checked for validation. Questionnaire being a self-administered one, was filled up only by the literate population. Considering developing countries with good proportion of illiterate population who might be excluded by the self-administered questionnaire, unless they have a literate relative to translate the exact questionnaire to them. Other limitation was the fact that, retesting should be done after 14 days (as suggested by biostatistician) for the study patient to forget the options chosen at the first attempt. The majority of patients presenting in our institute come from distant places and do not stay longer following the daycare surgeries. Hence, retesting was done at the follow-up visit itself (3–5 days) for patients’ convenience.

Conclusion

This systematically developed and validated, 27-items questionnaire can be tentatively recommended to be used to measure patient’s satisfaction with DC anesthesia services. It can be used to measure satisfaction following all types of surgical procedures, under various types of anesthesia.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.