Cross-Cultural Validation Into Filipino of the Consumer Assessment of Health-Care Providers and Systems Surgical Care Survey (S-CAHPS)

Abstract

Surgical experience is increasingly being recognized; however, Consumer Assessment of Health-Care Providers and Systems Surgical Care Survey (S-CAHPS), the tool for surgical experience, is available in English and Spanish only. To measure surgical experience among Filipinos, a culturally appropriate version should be validated. This study culturally adapted the S-CAHPS into Filipino. A five-step cross-cultural validation study was conducted. Language experts conducted forward translation, back translation, and panel reconciliation. Pretesting included content validation and pretesting of the Filipino S-CAHPS. Field testing involved 55 purposively selected postoperative patients who completed a 3-part survey from March to July 2018. The English S-CAHPS was reduced to 34 items to account cultural variations, yielding an item content validity index (I-CV) of 1.00. One-sample t test and Bland–Altman plots showed good linguistic equivalence. Correlation coefficients were ≥0.30, suggestive of good conceptual equivalence. Cronbach’s alpha values were 0.83 and 0.85 indicative of good reliability. The Filipino S-CAHPS showed acceptable psychometric properties. It is a valid and culturally appropriate instrument to measure surgical experience among Filipinos which can be utilized for quality improvement measures on both practice and policy levels.

Introduction

Patient experience is one of the increasingly recognized health concepts to promote patient-centered care. Defined as the sum of all interactions, shaped by an organization’s culture, that influence patient perceptions across the continuum of care (1,2), leaders have placed patient experience first on their list of organizational and patient care priorities (3). As a result of the confluence of several policy priorities, the direction of patient care has been steered toward the delivery of quality patient experience within the health-care systems (4). This shift in care priority started the development of a family of survey which measures patient experience in various health-care settings, known as the Consumer Assessment of Health-Care Providers and Systems (CAHPS).

Although the CAHPS questionnaires measure patient experience, these surveys were mostly designed for out-patient or in-patient medical care and did not encapsulate other areas of hospital service (5). One of the health-care settings that has received increasing attention in promoting quality patient experience is the surgical setting. Cognizant that the surgical patient experience is a unique phenomenon demarcated by preoperative decision-making, a brief surgical experience, and postoperative care (5), it is imperative to develop measures which encapsulate a surgical patient’s perioperative experience. Luckily in 2009, a version of the CAHPS, which measures surgical patient experience, was created by the CAHPS consortium, the American College of Surgeon (ACS), Surgical Quality Alliance, and different subspecialty groups. This questionnaire is now known as the Consumer Assessment of Health-Care Providers and Systems Surgical Care Survey or S-CAHPS (6 –8). The S-CAHPS is a 45-item questionnaire, and it is a standardized tool which asks patients to report their experience and satisfaction within a single episode of surgical care (9).

Currently, the S-CAHPS is publicly available in two languages only—English and Spanish (7,9). In the Philippines, although English is the second most commonly spoken language, several Filipinos may not be fluent in English. Likewise, noting that the developers of the S-CAHPS are from Western and European countries, countries which are culturally different from the Philippines, some items in the S-CAHPS may not be appropriate in the culture of the Filipino population. As a steppingstone to adequately measure surgical patient experience, the S-CAHPS should be administered in a language understandable to the target population and should fit their culture and norms. With an S-CAHPS version that is culturally sensitive, it can offer surgeons, nurses, and other health-care professionals the patient’s perspective about their surgical experience and can be utilized in modifying select aspects of their practices and policies which may be modified and improved.

This study translated, culturally adapted, and validated the S-CAHPS into Filipino. Specifically, this study determined the content equivalence, linguistic equivalence, conceptual equivalence, and internal consistency of the Filipino version of S-CAHPS.

Methods

Research Design

A cross-sectional, cross-cultural validation design was used to culturally adapt and translate the S-CAHPS to Filipino. The five steps of cross-cultural validation, as recommended by Beaton et al (10) and Polit and Yang (11), were employed in this study.

Step 1: forward translation

Forward translation is the translation of the items, response options, and instructions from the source language (English) to the target language (Filipino). A decentered or symmetric translation of the items, response options, and instructions was performed to allow modifications which will ensure contextual and cultural relevance in the Philippines. A team of 4 translators performed the forward translation. This team was composed of the investigators and 2 Filipino language experts. The authors, who are native speakers of Filipino and are fluent in English, translated and modified the S-CAHPS as deemed applicable. Then, we gave the preliminary Filipino S-CAHPS to 2 Filipino language experts who evaluated the accuracy and semantics of the translated questions.

Step 2: back translation

After forward translation, the Filipino S-CAHPS was back-translated, a process in which the synthesized and translated instrument is translated back to the source language (English). This step was conducted to ensure semantic equivalence (11). The back translation team was composed of 2 English language experts who back-translated the instrument independent from another (10,12).

Step 3: panel reconciliation

The third step was the reconciliation of the translated and back-translated versions through a consultative meeting. The Filipino and English language experts who did the first 2 steps of the translation processes were invited to resolve potential issues and discrepancies with the translated tools. During the consultative meeting, minor revisions were made in the prefinal tool (eg, use of tenses, use of prepositions, etc.).

Step 4: pretesting of prefinal tool

The fourth step is further divided into 2 substeps: (a) content validation for cultural relevance of the prefinal Filipino S-CAHPS and (b) pretest of the prefinal Filipino S-CAHPS to a small sample of respondents. The content validation substep was completed through the assistance of 7 content experts in health promotions, general surgery, general medical-surgical nursing, and perioperative nursing. Cultural relevance of the translated items was evaluated using a 4-point ordinal scale (13). After the content validation, the revised prefinal tool was pretested among 15 respondents. Upon completing the forms, respondents were invited for a cognitive interview using targeted verbal probes (11,14). These questions were formulated to identify any part of the Filipino S-CAHPS which were vague or needed to be revised.

Step 5: field testing of final tool

Field testing of the final Filipino S-CAHPS was conducted for 2 purposes: (a) to assess the extent at which the translated tool meets the quality standards for the original purpose of the tool and (b) to evaluate the equivalence of the translated and original tool (11). Since most Filipinos are fluent and/or conversant in both Filipino and English, the field testing of the questionnaire was done in a bilingual sample wherein both versions, with different item arrangement, were administered to the respondents.

Subject and Settings

This study included a total of 55 purposively selected surgical patients who met the following criteria: postoperative patients who were operated under general, regional, or a combination of these anesthetic strategies; were 18 to 59 years old; and were conversant in both Filipino and English. However, patients scheduled for neurological surgeries, for emergency surgeries, or have overt manifestations (eg, altered sensorium, etc) which preclude competent consent were excluded. This study was conducted in a tertiary private medical institution which provides general medical and surgical care to patients. This institution has a total of 812-bed capacity, 352 beds to private patients, and 460 beds for charity patients.

Data Collection and Ethical Considerations

After securing ethical clearance and institutional approval, we coordinated with the different department heads and nurse supervisors. Through the assistance of staff nurses in the units, we acquired a list of the postoperative patients in their respective areas. We initially screened these patients according to age, surgical procedure, and anesthetic plan. Afterward, we visited these potential respondents 24 hours after their surgery and invited them to participate in the study. We provided full disclosure about the study’s purpose, background, risk and benefits, expected participation, and rights of respondents before securing their written informed consent. After securing the consent, we provided the data collection forms which took the respondents approximately 15 to 20 minutes to complete. The completed forms were then assessed for completeness before we secured them in sealed and coded envelopes. All collected forms were then encoded in a password-protected hard drive which was kept in our research office. Data collection was conducted from March to July 2018.

Research Instruments

Three questionnaires were distributed among consenting respondents: the patient datasheet, the Filipino S-CAHPS, and the English S-CAHPS. The patient data sheet profiled the respondents’ sociodemographic characteristics and surgical data. The S-CAHPS by the ACS is 45-item questionnaire which assesses the surgical care experience of patients in the following areas: surgeon (1 item), before the surgery (13 items), during surgery (3 items), anesthesiology (8 items), after the surgery (9 items), overall care of surgeon (1 item), clerks and receptionists at the surgeon’s office (2 items), and about the patient (8 items). The S-CAHPS produces 6 composite measures of patient experience: information to help you prepare for surgery; how well surgeon communicates with patients before surgery; surgeon’s attentiveness on day of surgery; information to help you recover from surgery; how well surgeon communicates with patients after surgery; and helpful, courteous, and respectful staff at surgeon’s office. Both versions of the S-CAHPS were administered to the respondents.

Data Analysis

Statistical analyses were performed using Stata Statistical Software, Version 13, College Station, Texas: StataCorp LP. A P value ≤ .05 was considered significant. Descriptive statistics included mean, standard deviation, median, interquartile range (IQR), frequency, and percentage. Content equivalence was measured using item (I-CVI) and scale content validity indices (S-CVI/Average), with cutoff values of 0.78 and 0.90, respectively (13,15). Linguistic equivalence, on the other hand, was evaluated by calculating the mean difference of each item (Filipino minus English) and analyzing them using 1-sample t test. A mean difference of ±0.25 was considered acceptable (16,17). Bland–Altman plot and Lin’s concordance correlation coefficient were employed to assess the degree of agreement between the Filipino and English scores. To evaluate conceptual equivalence, Spearman’s rank-order correlation was utilized to determine the correlation between an item and its corresponding composite measure in both Filipino and English versions. A correlation coefficient of ≥0.30 denotes that an item is consistent with the content of the composite measure (17). Internal consistency was assessed using Cronbach’s alpha with a cutoff score ≥0.70 (11).

Results

Demographic and Surgical Profiles of Respondents

Most respondents were female (70.91%), married (72.73%), and completed a tertiary-level education (61.82%), with a mean age of 38.53 years old (standard deviation [SD] = 11.89). Most of the respondents also had hypertension (25.45%) as comorbidity. In terms of surgical profile, most respondents had an obstetric and gynecologic surgery (49.09%) under general anesthesia (60.00%). Most were ASA category II (67.27%) and never had any surgery before (IQR = 0-1) (Table 1).

Table 1.

Demographic and Surgical Profiles of the Respondents.^a

Characteristic	Frequency (f)	Percentage (%)	Mean (SD) or Median (IQR)
Age (years)			38.53 (11.89)
Sex
Male	16	29.09
Female	39	70.91
Marital status
Single	13	23.64
Married/live-in	40	72.73
Annulled	2	3.64
Widow/Widower	0	0.00
Educational attainment
No formal education	0	0.00
Primary education	3	5.45
Secondary education	18	32.73
Tertiary education	34	61.82
Postgraduate education	0	0.00
Comorbidities
None	14	25.45
Diabetes mellitus	10	18.18
Hypertension	14	25.45
Asthma	3	5.45
Polycystic ovarian cysts	6	10.91
Number of previous surgeries			0 (0-1)
Classification of surgery
Gastric and abdominal surgery	4	7.27
Obstetric and gynecologic surgery	27	49.09
EENT surgery	9	16.36
Orthopedic surgery	8	14.55
Urologic and nephrological surgery	4	7.27
Dermatological surgery	3	5.45
Anesthetic plan
General anesthesia	33	60.00
Regional anesthesia
Spinal anesthesia	15	27.27
Epidural anesthesia	1	1.82
Nerve block	0	0.00
Field block	0	0.00
Combined general and regional anesthesia	6	10.91
ASA rating
ASA category I	10	18.18
ASA category II	37	67.27
ASA category III	8	14.55

Abbreviations: IQR, interquartile range; SD, standard deviation.

^a N = 55.

Content Equivalence

Content equivalence was ascertained through a panel of content experts. After 3 rounds of validation, items 1 and 36 to 48 were omitted as these were not applicable in the Philippine health-care practice. Hence, only items 2 to 35 were used in the field testing. Moreover, some items were slightly modified to represent the contextual and cultural difference in the Philippines. The computed item-content validity index was 1.00 with a scale-content validity/average of 1.00.

Linguistic Equivalence

The mean (SD) scores and the mean difference (SD) scores (Filipino minus English) for each item of S-CAHPS are presented in Table 2. It can be noted that 1 item (item 14) was statistically different between the Filipino and English versions, with the English version having a statistically higher mean score. However, the mean difference was only −0.02 and was within the allowable difference of ±0.25. Similarly, examining the scores of the 5 composites, it can be noted that all were not statistically different (P > .05) between the Filipino and the English versions. The mean difference also ranged from −0.01 to 0.03.

Table 2.

Mean (SD) Scores and Mean Difference Scores for Each Item and Composite Measure of the Filipino and English Versions of the S-CHAPS.^a

Items and Composite Measures	Mean (SD) of Filipino Version	Mean (SD) of English Version	Mean Difference (SD) of Scoresb
Item number 2	3.22 (1.79)	3.13 (1.72)	0.09 (0.44)
Item number 3	2.94 (0.30)	2.96 (0.27)	–0.02 (0.13)
Item number 4	2.87 (0.39)	2.87 (0.39)	0.00 (0.00)
Item number 5	0.94 (0.23)	0.92 (0.26)	0.02 (0.13)
Item number 6	0.96 (0.19)	0.96 (0.19)	0.00 (0.00)
Item number 7	3.44 (1.01)	3.52 (0.89)	–0.08 (0.38)
Item number 8	2.38 (1.32)	2.4 (1.29)	–0.02 (0.56)
Item number 9	2.94 (0.23)	2.87 (0.39)	0.08 (0.32)
Item number 10	2.94 (0.23)	2.94 (0.23)	0.00 (0.00)
Item number 11	2.76 (0.41)	2.76 (0.41)	0.00 (0.00)
Item number 12	2.82 (0.50)	2.78 (0.59)	0.04 (0.19)
Item number 13	0.67 (0.46)	0.67 (0.46)	0.00 (0.00)
Item number 14	2.65 (0.57)	2.67 (0.58)	–0.02 (0.07)c
Item number 15	0.91 (0.29)	0.89 (0.31)	0.02 (0.13)
Item number 16	2.94 (0.23)	2.92 (0.26)	0.02 (0.24)
Item number 17	0.82 (0.39	0.82 (0.39)	0.00 (0.00)
Item number 18	0.98 (0.13)	1.00 (0.00)	–0.02 (0.13)
Item number 19	1.00 (0.00)	1.00 (0.00)	0.00 (0.00)
Item number 20	2.64 (0.73)	2.71 (0.69)	–0.07 (0.33)
Item number 21	0.78 (0.42)	0.78 (0.42)	0.00 (0.00)
Item number 22	2.90 (0.26)	2.90 (0.26)	0.00 (0.00)
Item number 23	1.00 (0.00)	1.00 (0.00)	0.00 (0.00)
Item number 24	2.85 (0.36)	2.85 (0.36)	0.00 (0.00)
Item number 25	9.44 (0.86)	9.45 (0.86)	–0.02 (0.13)
Item number 26	2.85 (0.52)	2.85 (0.49)	0.00 (0.19)
Item number 27	2.80 (0.56)	2.78 (0.57)	0.02 (0.13)
Item number 28	2.95 (0.30)	2.95 (0.23)	0.02 (0.19)
Item number 29	2.84 (0.46)	2.87 (0.34)	–0.04 (0.43)
Item number 30	0.67 (0.33)	0.87 (0.33)	0.00 (0.00)
Item number 31	2.86 (0.47)	2.85 (0.47)	0.01 (0.04)
Item number 32	2.90 (0.29)	2.90 (0.29)	0.00 (0.02)
Item number 33	2.78 (0.55)	2.77 (0.55)	0.01 (0.04)
Item number 34	2.96 (0.19)	2.96 (0.19)	0.00 (0.01)
Item number 35	9.64 (0.78)	9.64 (0.78)	0.00 (0.00)
Composite 1: information to help patient prepare before surgery	2.90 (0.32)	2.91 (0.30)	–0.01 (0.07)
Composite 2: surgeon’s communication with patient before surgery	2.87 (0.19)	2.84 (0.22)	0.03 (0.11)
Composite 3: surgeon’s attentiveness on the day of surgery	0.86 (0.31)	0.85 (0.31)	0.01 (0.07)
Composite 4: information to help patient recover from surgery	2.86 (0.34)	2.86 (0.31)	0.005 (0.15)
Composite 5: surgeon’s communication with patient after surgery	2.87 (0.24)	2.87 (0.24)	0.004 (0.03)

Abbreviations: S-CAHPS, Consumer Assessment of Health-Care Providers and Systems Surgical Care Survey; SD, standard deviation.

^a N = 55.^b Filipino version scores minus English version scores.

^c Significant at .05 using 1-sample t test.

The Bland–Altman plots (Figure 1) depict the agreement between the Filipino and the English scores of the 5 composite measures. The difference in mean scores was plotted against the mean scores of each composite for both Filipino and English versions. The mean difference (95% confidence interval [CI]) for the 5 composites were –0.01 (–0.14 to 0.12), 0.03 (–0.20 to 0.25), 0.01 (–0.12 to 0.14), –0.005 (−0.30 to 0.29), and 0.004 (–0.05 to 0.05), respectively. Overall, 98.18%, 94.55%, 98.18%, 92.73%, and 96.36% of scores lay within the 95% CI for the 5 composites, respectively. Lin’s concordance correlation coefficients for the 5 composites were 0.98, 0.84, 0.98, 0.90, and 0.99, respectively. Likewise, Bland–Altman plots of all 34 items revealed that 92.72% to 100.00% of points lay within their respective 95% CIs.

Figure 1.

Bland–Altman plot for composites 1, 2, 3, 4, and 5 of the Filipino and English versions of the S-CAHPS.

Conceptual Equivalence

Table 3. illustrates the results of the Spearman rank-order correlation for each item and their respective composite measure. In the Filipino version, the correlation coefficients ranged from 0.30 to 0.99, while in the English version, analogous figures ranged from 0.30 to 0.99. All correlation coefficients were statistically significant at 0.01, except for item 10 which was significant at 0.05 in both versions.

Table 3.

Spearman’s Rank-Order Correlation Coefficients for Each Item on the Filipino and English Versions of the S-CAHPS With Their Respective Subscale Score.^a

Items and Composite Measures	Spearman Rho for Filipino Version of S-CAHPS	Spearman Rho for English Version of S-CAHPS
Composite 1: information to help patient prepare before surgery
Item 3	0.70b	0.60b
Item 4	0.99b	0.99b
Composite 2: surgeon’s communication with patient before surgery
Item 9	0.39b	0.54b
Item 10	0.31c	0.30c
Item 11	0.73b	0.63b
Item 12	0.53b	0.56b
Composite 3: surgeon’s attentiveness on the day of surgery
Item 15	0.74b	0.76b
Item 17	0.99b	0.95b
Composite 4: information to help patient recover from surgery
Item 26	0.62b	0.68b
Item 27	0.75b	0.83b
Item 28	0.42b	0.44b
Item 29	0.73b	0.73b
Composite 5: surgeon’s communication with patient after surgery
Item 31	0.61b	0.61b
Item 32	0.63b	0.66b
Item 33	0.75b	0.78b
Item 34	0.47b	0.50b

Abbreviation: S-CAHPS, Consumer Assessment of Health-Care Providers and Systems Surgical Care Survey.

^a N = 55.^b Significant at .01.

^c Significant at .05.

Internal Consistency

The Cronbach’s alpha values for the Filipino and English versions were 0.83 and 0.85, respectively, which were not statistically different (χ² = 0.20, P = .656). These figures suggest good internal consistency for both versions of the instrument.

Discussion

This study translated, culturally adapted, and validated the S-CAHPS into the Filipino language. To the best of our knowledge, this study is the first attempt in translating and culturally adapting the S-CAHPS into the Filipino language. By and large, results showed that the Filipino S-CAHPS showed acceptable content equivalence, linguistic equivalence, conceptual equivalence, and internal consistency.

Results showed that after several iterations of content validation, the 34 remaining items showed good content and semantic equivalence, with 11 items being omitted from the original English S-CAHPS. These changes were made since most of these 11 items were not applicable in the Philippines. For example, item 1 (Our records show that the surgeon named below performed surgery on you on the date listed below…) was omitted since most respondents were visited during their first 24 hours of admission in the hospital and the name of the surgeon has been confirmed through their respective attending surgeon and the unit’s staff nurses. Similarly, items 36 and 37 were removed from the translated questionnaire as these are not standard practices in the Philippine health-care setting. Finally, items 38 to 45 were also omitted since these pertain to demographic characteristics. The decentering procedure, which gives equal importance to both English and Filipino languages, was essential to generate a Filipino S-CAHPS which is equivalent to its English version but considers the cultural diversity in the Philippines. Moreover, the rigorous procedure of back translation, a cornerstone approach in Brislin’s (18) model of translation, allows the identification and modification or elimination of potential vague and erroneous items (19).

On a separate note, this study demonstrated that the Filipino S-CAHPS had acceptable linguistic equivalence, that is, the translation of the instrument was an equivalent yet culturally acceptable translation of the English S-CAHPS, with only 1 item having a statistically significant mean difference score. Although there was a statistically different mean difference score for item 14, the mean difference was within the ±0.25 cut-off score denoting acceptable variation between the 2 versions (16,17). Moreover, results of the Bland–Altman plots, for all items and composite measures, support that there is no significant difference between the Filipino and English versions (20). Concordance scores also support that at least 84% to 99% of the scores in the Filipino and English versions were concordant (21). Nonetheless, the variation may be attributed to the rewording of the item, that is, the Filipino translation of the tool was trying to capture the intended meaning of the English version instead of creating a literal translation of words and phrases in the item. It should be noted that in the Filipino vocabulary, certain words and phrases do not have an exact translation, thus causing the variation.

It can also be gleaned from the results that the Filipino and English versions were comparable for each item with their respective composite measure, suggesting that there is a conceptual equivalence. This result means that the instruments were able to assess the same theoretical construct—surgical patient experience, in both cultures. It can be noted from the results that the difference in correlation coefficients between the Filipino and English versions did not exceed 0.10, except for items 3, 9, and 11, denoting more conceptual equivalence between the 2 versions (16,17). In most composite scores, the correlation coefficients were slightly higher in the English version than in the Filipino version except for composite measure 1.

Focusing on the internal consistency scores of the Filipino and English versions, the Cronbach’s alpha values were 0.83 and 0.85, respectively. These figures are comparable from each other (22), and it is suggestive that both versions have acceptable internal consistency (11) and were reliable in measuring surgical patient experience.

Despite these results, this study has its limitations. This study was conducted in a small number of surgical patients only. Results must also be analyzed with caution since self-report questionnaires were utilized; hence, over- and underestimations are possible. The study was also conducted within Metro Manila only, a highly urbanized metropolitan where health-care services may greatly vary compared to other institutions in the provinces of the country.

Conclusion

This study translated, culturally adapted, and validated the S-CAHPS into the Filipino language. To the best of our knowledge, this is the first study to use a rigorous cross-cultural adaptation of the S-CAHPS into a different language and culture. In general, results showed that the Filipino S-CAHPS showed acceptable psychometric properties.

The Filipino S-CAHPS, therefore, can be a valid and culturally appropriate instrument to measure surgical patient experience among the Filipino population. This Filipino version of the S-CAHPS allows non-English-speaking Filipinos to share their surgical experience to their health-care professionals which can provide impetus in developing appropriate interventions or policies which are geared toward improving their perioperative experience.

Author Biographies

Gian Carlo Torres was a PhD in Nursing student in the University of the Philippines - Manila. He is currently an Assistant Professor in the University of Santo Tomas, Manila, Philippines.

John Rey Macindo is an intensive care unit (ICU) staff nurse at the AMOSUP Seamen's Hospital. At the same time, he is a graduate student at the Faculty of Management and Development Studies, UP Open University.