Researching the Appropriateness of Care in the Complementary and Integrative Health Professions Part 3: Designing Instruments With Patient Input

Abstract

Objectives.

The purpose of this article is to describe how we designed patient survey instruments to ensure that patient data about preferences and experience could be included in appropriateness decisions. These actions were part of a project that examined the appropriateness of spinal manipulation and mobilization for chronic low back pain and chronic neck pain.

Methods.

We conducted focus groups, cognitive interviews, a literature review of measures in prior chiropractic and Complementary and Integrative Health (CIH) research and a pilot study to develop questionnaires of patient preferences, experiences, values, and beliefs.

Results.

Questionnaires were administered online to 2024 individuals from 125 chiropractic clinics. The survey included 3 long questionnaires and 5 shorter ones. All were administered online. The baseline items had 2 questionnaires that respondents could complete in different sittings. Respondents completed shorter biweekly follow-ups every 2 weeks, and a final questionnaire at 3 months. The 2 initial questionnaires had 81 and 140 items, the 5 biweekly follow-up questionnaires had 37 items each, and the endline questionnaire contained 121 items. Participants generally responded positively to the survey items, and 91% of the patients who completed a baseline questionnaire completed the endpoint survey 3 months later. We used “legacy” measures, and we also adapted measures and developed new measures for this study. Preliminary assessment of reliability and validity for a newly developed scale about coping behaviors indicates that the items work well together in a scale.

Conclusions:

This article documents the challenges and the efforts involved in designing data collection tools to facilitate the inclusion of patient data into appropriateness decisions.

INTRODUCTION

Appropriateness of care decisions have been based on the published literature on safety and efficacy and the judgments of experts, both clinical and scientific experts. What is missing is the voice of patients in this process. However, in an era of patient-centered care reflected in organizations such as the Patient-Centered Outcome Research Institute (PCORI),¹ inclusion of patient input should be considered essential. The Center of Excellence for Research in Complementary and Alternative Medicine (CERC) was established at RAND specifically to develop a method for studies on appropriateness that included patient input and costs.² While it is now self-evident that patient input should play a role in decisions that impact them, it is important to do that while at the same time ensuring the decisions are clinically appropriate and safe. In developing a method at RAND/UCLA to measure appropriateness (the RAND/UCLA Appropriateness Method),^3,4 considerable effort was made to make sure the decisions were evidence-based or based on clinical experience that could be agreed upon by a panel of experts. Hence, the patient component should be equally evidence-based, that is, based on actual data collected from patients.

The CERC national study collected data to assess patient beliefs and preferences, patient-reported outcomes, costs and resource allocation. These data were provided to the study’s expert panels so that expert panelists could take these findings into account when determining their ratings about the appropriateness of manipulation and mobilization for chronic low back and neck pain.^5,6

Although we describe experiences from a research study, our lessons learned may be applicable to Complementary and Integrative Health (CIH) providers as well. Complementary and Integrative Health providers and researchers both need rigorous patient measures to help them collect reliable and valid data that are relevant and not burdensome to patients. Our research team prioritized parsimony and survey items that were relevant to our respondents to improve participation rates and engagement with the study.

In this paper, we share lessons learned from our literature review, cognitive interviews, pilot study, and national study about 1) how to identify appropriate existing instruments to measure beliefs, preferences, and experiences with chronic pain and coping among CIH patients, 2) how to decide whether to modify a tool to better fit one’s study or clinical circumstance, 3) how to develop new measures and evaluate their reliability and validity, and 4) how to assemble multiple measures together into a single questionnaire.

THE PROBLEM

Researchers need rigorous methodologies and reliable and valid self-report measures to evaluate efficacy and effectiveness of therapies and to understand patients’ experiences and beliefs in chiropractic and other areas of CIH. Rigorous patient measures are also essential for clinicians. Clinicians may want information about patients’ perceptions of care, adherence to recommendations, and health-related quality of life.^7–10 Measures can be useful as a research tool and for patient care. The past half century has seen a gradual shift away from exclusive reliance on clinical and laboratory measures of illness or disease-specific outcome measures towards the development and utilization of comprehensive indices of patient health status, including patient self-report (what might be considered a more holistic approach to measurement).^11,12

There are various options when creating a patient survey:

• Using all or part of existing measures

• Revising items from an existing measure

• Creating new items

Many researchers and clinicians prefer to use existing measures that have already been evaluated and published, as this is often more efficient than designing a tool from scratch.^13,14 Using an existing tool enables them to compare results to other studies and other practitioners that used the same tool, and this provides a helpful point of comparison across research or patient subgroups. A challenge is that there are few existing measures of patient beliefs and coping that have been evaluated in chiropractic. It is important for CIH researchers and practitioners to know how to find measures, decide whether a measure will suite their needs, and understand other options if no existing measures are appropriate.

CIH researchers may opt to design their own data collection tools. This has the major advantage of enabling them to ask exactly the question they want, perhaps addressing a topic that no one else has attempted to rigorously study or measure. It gives them the opportunity to cover all their domains of interest, and to word items in a way that will make sense and be relevant to their target population.

Creating an instrument is a complicated and lengthy process. How can a researcher or clinician be sure s/he has identified all the relevant domains that a tool should capture? Which are the best response options to use and how will they affect analytic options later? How can one feel confident that their respondents will understand and respond to the items in the way that the study team intends? If researchers want to measure multiple constructs using multiple tools within the same questionnaire, how can they make sure that fatigue or confusion among respondents are not adversely affecting their responses? Lastly, how can researchers test a novel set of items to be confident that the items are reliable and valid? There are ways to address all these questions, but they require careful planning.

To summarize, our primary questions in this study were:

1. How do we choose instruments that are patient-centric and relevant to their experiences and that will capture their preferences and values?

2. How do we choose between, on the one hand, utilizing legacy measures^15–17 that have been widely used in previous studies so we can compare our study to previous work, and on the other hand, designing new instruments specific to this study?

3. How do we choose data collection instruments that are comprehensive but concise, reliable, valid, relevant, and non-burdensome?

THE SOLUTION

In the following sections are the steps we took and the solutions we arrived at in answering the three questions above. In the results section, we discuss what the outcomes were.

METHODS

This study was approved by RAND’S Institutional Review Board, referred to as the Human Subjects Protection Committee (HSPC). This study was registered as an observational study on ClinicalTrials.gov ID:

THE RAND CERC STUDY

The solution to our data collection instrument challenges was based on exploratory interviews, focus groups, literature review, cognitive interviews, a pilot study, a survey of 125 clinics and over 2000 patients, protocol documents, minutes and emails from 4 years of project activities, along with the experiences reported by our project staff who were involved in developing, testing and administering the questionnaire.

Exploratory Interviews

A researcher’s first step to developing an instrument is having a clear understanding of the constructs and domains she or he intends to measure. We conducted exploratory phone interviews with 40 chiropractic patients with low back or neck pain. Our exploratory interviews took 30–45 minutes using open-ended questions to guide patients to tell us the story of their pain condition. The responses were analyzed using pile sorting, which identified key domains in this area and laid them out in a framework.¹⁸ The purpose of the exploratory interviews was to determine what issues were relevant to patients and what we needed to measure to ensure their perspectives and concerns were captured. They helped to determine the constructs for which we needed to find or create questions.

Focus Groups

Based on qualitative methods used by members of the research team in previous studies,^19,20 we conducted focus groups and cognitive interviews prior to the main study. First, we conducted 6 focus groups (2 in Los Angeles, 2 in Chicago, and 2 in Boston) with patients age 18 and older to identify key aspects of experiences with chiropractic care. We included patients with a range of prior chiropractic experience and sociodemographic characteristics (age, gender, race/ethnicity). Participants were recruited with the help of a focus group vendor.

The focus groups were led by a senior Principal Investigator using a semi-structured guide and notes were taken by a research assistant. The groups began with open-ended questions inquiring about experiences with care and desired outcomes. Then a summary of the CAHPS and PROMIS measures was shown to participants to get their opinion about their relevance and importance in representing their own experience. The research team probed to identify important areas not captured by the existing measures. The groups were audio-taped, transcribed verbatim and analyzed using Atlas.ti software. Focus group themes were identified and compared to the hypothesized domains. We documented feedback about possible problems with existing items and identified new item content. New items were written as needed.

The purpose of the focus groups was to evaluate the applicability of standardized Patient Reported Outcomes (PROs) that assess patient experiences of care (e.g. Consumer Assessment of Healthcare Providers and Systems or CAHPS^®) and health-related quality of life (Patient-Reported Outcomes Measurement and Information System or PROMIS^®) for chiropractic patients who had experienced manipulation or mobilization or back pain. The focus groups contributed to the process of developing instruments because they evaluated whether the existing questions were perceived to be important by chiropractic patients and reflective of what is important about the care they receive and outcomes of care. The groups also explored possible gaps in the content of the existing measures that are important to patients. This information was used to determine if modifications to existing measures were needed and if there were gaps that require newitems. Revisions to existing items and new items were then drafted and subjected to cognitive interviews with chiropractic patients who have experienced back pain. The revised and new items were finalized based on cognitive interviews (see below).

Literature Review

While we collected exploratory and pilot data, we were also conducting a comprehensive search of the literature to identify existing tools (legacy instruments) that measured patient experiences, beliefs and preferences related to chiropractic care and/or chronic pain. We identified 49 existing tools (see list in Appendix A). Four researchers on our team with expertise in anthropology, psychology, sociology and public health read through the items. We included in our final set of measures some commonly-used outcomes such as the Oswestry Disability Index,^21,22 Neck Disability Index²³ and the PROMIS-29 v 2.0 profile measure²⁴ because those tools measured constructs relevant to patient experiences such as the impact of pain on a person’s everyday life. Two other tools that we selected for our questionnaire were the Survey of Pain Attitudes^25,26 and the Chronic Pain Self-Efficacy Scale.²⁷ We felt those tools addressed unique aspects of patient experiences and beliefs that were not already captured by other tools already identified.

Reviewing the 49 existing tools helped us learn about constructs and existing measures. The literature review and the exploratory data were used to create a framework. Our framework provided a clear list of constructs for which we needed measures, and it acted as a theoretical guide for selecting and developing our instruments. The literature review pertains to question 2, how do we choose between, on the one hand, utilizing legacy measures which have been widely used in previous studies so we can compare our study to previous work, and on the other hand, identify where designing new instruments specific to this study would be necessary. To that extent they help answer question 1, what is relevant.

Cognitive Interviews

We conducted face-to-face cognitive interviews to ensure the patient experience items we developed were understood by patients. We probed about item stem content and clarity of the response options. Each item was reviewed by at least 5 adult (18 and older) subjects. We limited our interview to 40 items per subject with a goal of a 60-minute interview. We tested 80 items overall, and we conducted 10 cognitive interviews in total to obtain 5 interviews per item. Half of the interviews were conducted on the west coast of the United States and the other half on the east coast. We conducted some cognitive interviews using intermittent probes and others using retrospective probes following completion of all items. The interviewer asked open-ended questions and probed about item stems, response options, and time frames. After completion of each cognitive interview, the items were discussed with the research assistant conducting the interviews. Any new items developed were documented and potential problems with the existing CAHPS and PROMIS measures noted. Where we identified problems with an existing item, we created a revised version of the item. For example, in the original version of the CAHPS Health Plan Study we inserted “chiropractor” as an exemplar in the instructions that refer to primary care providers. In the national study, we administered both the standard item (for comparability with existing studies) and the revised item. A total of about 20 additional items (revised items and new items) were included in the national sample following the cognitive interviews.

Cognitive interviews allowed us to determine if constructs we had identified could be understood by patients and communicated to the researchers. They also contributed to our understanding of respondent burden.

Pilot study

Once the items had been programmed into a web survey, the research team tested the web tools multiple times to ensure that they had been programmed correctly, that the skip patterns were correct, and that there were no other unforeseen issues with the web surveys. After the questionnaires were fully programmed, we conducted a pilot study with 89 low back or neck pain patients from 7 local chiropractic clinics. We gathered information at the end of an online questionnaire in our pilot study to assess patients’ reactions to our data collection tools and to identify potential problems with the length of the questionnaire or comprehensibility of the items. The items assessed participants’ experiences signing up for the study in their chiropractor’s office, their thoughts about completing the questionnaire in general and about the length of the questionnaire, and whether any of the items were confusing. We counted how many participants dropped out of the study at different phases of the pilot. The fact that drop out was low from one stage to the next indicated that the questionnaires were not overly burdensome, and that the incentives were sufficient. The pilot study contributed to all three of our questions.

RESULTS

Existing Tools

A drawback of using existing tools in their entirety was that some items were not relevant to all respondents. For instance, one of the 10 Oswestry Disability Index^21,22 items asks about the impact of pain on the respondent’s sex life, and the item does not include a “does not apply” option. Two pilot study respondents commented that because they were not sexually active, they did not know how to answer that question. One stated, “On the sex question, for example, I am single and celibate. It would have been better to put an “other” choice or left a comment box. Don’t just assume that everyone has a sex life.”

In some instances, we used subscales from an existing instrument rather than the entire instrument. For instance, the Chronic Pain Self-Efficacy Scale includes items that cover 3 distinct self-efficacy domains: pain management, coping and physical function.²⁷ We decided to use the items for only the pain management and coping domains because our questionnaire already included many items about physical function.

Selecting specific subscales of a larger tool gave us more control over which items we included and helped us avoid having a very long, repetitive questionnaire. A disadvantage of this approach is that if we wanted to compare our results with results from other published papers, we could only refer to studies that reported scores for each subscale. In the case of the Chronic Pain Self-Efficacy Scale, for instance, some authors have reported subscale scores,^28,29 but in other cases, especially if the tool is not the primary outcome measure, authors reported only the overall scale score.³⁰

Modifying Existing Instruments

In some instances, we took an existing item or set of items and revised the wording to better fit our study population. For example, we used the Credibility items from the Credibility/Expectancy Questionnaire,³¹ and one of those items was “At this point, how successful do you think this treatment will be in reducing your trauma symptoms?” Many CIH patients with chronic pain conditions reported seeking care from multiple types of providers, such as a chiropractor, primary care provider, and a massage therapist. We believed that if a questionnaire item referred generically to “this treatment,” and patients were completing the survey at home and not in a clinic, they may not know which treatment to report about. We changed the term to “your chiropractic treatment” to make this clear. Similarly, we thought the term “trauma symptoms” would be confusing because these patients may not have experienced anything that they or their providers would call traumatic. So, we changed that phrase to “your pain symptoms.” While these changes affected comparability with prior studies, we felt that relevance to our study population was more important.

In other cases, we used a combination of existing items and new items. For example, we used focus group input and the literature review to select Consumer Assessment of Healthcare Providers and Systems (CAHPS) Clinician & Group Survey 3.0 items relevant to chiropractic care (3 access to care items, 4 communication items, and 1 global rating of the provider item). We supplemented these items with 2 additional access to care items, 5 additional communication items, 1 global rating of office appearance item, 4 items assessing office assistants, 1 item on insurance coverage, and 3 items assessing perceived outcomes of care.

Practical Considerations with Using Existing Tools

There is strong support for the use of existing measures and measures from established item pools such as PROMIS,²⁴ rather than creating new measures. In our study, there were practical and logistical details to consider in using existing instruments. For instance, sometimes the exact language of a full instrument was easily accessible in published scholarly articles (e.g., Credibility/Expectancy Questionnaire³¹) or even on a website (e.g., the Center for Epidemiological Studies Depression Scale³²), but in other cases, the items were only available by request (e.g., Survey of Pain Attitudes^25,26). Some instruments required permission from the original authors, completion of a usage agreement, and/or payment to use. Hays and colleagues³³ provide a useful discussion on copyright restrictions of survey instruments. We strongly encourage researchers to contact the authors of any tool you intend to use to ask about usage requirements. We found that the instruments’ authors were sometimes valuable resources when we encountered questions later about how to score items, for example.

The articles that we identified in the initial search of the literature were useful resources once we had collected our data. We referred to them when we scored items and wanted to compare our scores with a reference population. (See Appendix A for a list of all tools reviewed.)

Creating new tools: An example about measure coping behaviors

When there were constructs that we wished to study, often because our exploratory phone interviews had revealed a richness and variability in the ways that patients talked about those constructs, and we could not find any instruments in the literature that adequately measured those domains, we created new items. One example of this from our study was measuring coping activities. In general, we followed established tenets for developing survey items.^34–36 Here we share our process of developing items to assess coping behaviors and quantitively assessing study participants’ responses to those items.

We understood from the exploratory interviews that people coped with pain in many ways beyond visits to healthcare providers. While some tools measured the degree to which people felt they could cope with pain in a general sense (e.g., Chronic Pain Self-Efficacy Scale²⁷), we aimed to measure the degree to which people coped with pain across the various domains of their life – for example, by changing or controlling their emotions and thoughts, by engaging in self-care, and by manipulating the environment around them. No existing tool captured this adequately. We identified broad domains of coping based on empirical data from our exploratory interviews and based on logic (e.g., we assumed that if patients were coping by modifying one part of their physical environment, like their home. they might also be modifying their work environment). We identified examples of coping behaviors in each domain from examples cited by respondents in the exploratory interviews and in the coping literature.^37,38 Figure 1 illustrates how our empirical findings, logical assumptions and literature review led to a set of 26 items assessing coping behaviors.

Figure 1.

Exploratory study findings, logical assumptions and information from the literature that informed CERC questionnaire items about coping behaviors

We created 3–4 items per domain measuring the frequency of coping activities. For most items, we applied a commonly-used 5-point response set: “never,” “rarely,” “sometimes,” “often,” and “always.” However, we used dichotomous yes/no response choices for 6 of the items (made large changes at home, made small changes at home, wore a lifting belt, and the 5 items about coping at work) because a frequency response did not make sense. For the item assessing the use of opioid medication, we referred to the Chronic Low Back Pain Taskforce Minimal Item set³⁹ for a list of example medications.

We cognitively tested these new items with a sample of 4 individuals. The items were also tested as part of our pilot survey with 89 patients. Table 1 shows the final items that were included to assess coping activities.

Table 1.

Chronic Pain Coping Behaviors Scale, Factor Loadings for Single Factor Solution

Item		Rotated factor loading
1	Meditated or used guided imagery	0.329
2	Ignored my pain*	0.063
3	Thought about what 1 need to do for my pain	0.336
4	Psychological counseling	0.524
5	Exercised (including yoga, walking, going to the gym, stretching, etc.)*	−0.035
6	Got injections/shots (including steroids, epidurals, cortisol, etc.)	0.482
7	Took over the counter pain medications	0.328
8	Took herbs, other supplements or vitamins specifically for pain	0.402
9	Took non-opioid prescription pain medications	0.557
10	Tookopioid prescription pain medications	0.533
11	Used hot pads/ice packs at home	0.542
12	Rested	0.458
13	Reduced the amount of time 1 spent with friends	0.730
14	Avoided social activities	0.718
15	Talked to someone who listened or gave me advice about my pain	0.500
16	Asked or received support from someone to help with my daily tasks	0.633
17	Received emotional support for my pain from family and friends	0.585
18	Did fun things with people to help get my mind off of the pain	0.278
19	Larger changes to your home to increase comfort and accessibility, like installing a ramp or getting a new chair or bed	0.414
20	Smaller changes like installing hand bars or buying different pillows	0.358
21	Wore a lifting belt, girdle or truss to prevent injury	0.286
Factor	Eigenvalue	Proportion of variance
Factor 1	4.597	63.1%

Note: Factor loadings come from a single factor solution based on a polychoric correlation matrix using responses from n=1808 respondents. Rotated (oblimin oblique) factor loadings are shown.

^*Items were not included in reliability analysis due to low factor loading

Analyses of Our New Measure of Chronic Pain Coping Behaviors

We analyzed data for the 21 chronic pain coping items as a preliminary assessment of their validity and reliability. Validity refers to the degree to which a measure actually captures the construct that the researcher intends to measure, while reliability refers to whether or not responses are consistent and stable.⁴⁰ We used responses from our national survey, which was conducted with patients from 125 chiropractic clinics in 6 cities across the US. The coping items were administered within our baseline online survey to n=2024 patients with chronic low back and/or chronic neck pain. After excluding 216 respondents who missed one or more of the 21 items, we had an analytic sample of n=1808 respondents.

To assess whether the items measured 1 or multiple dimensions of coping, we conducted an exploratory factor analysis using polychoric correlations to accommodate non-continuous variables. We examined the scree plot to of eigenvalues to identify the appropriate number of factors. After deciding to proceed with a single factor of coping, we identified 2 items that did not load highly onto that factor, and we proceeded with the other 19 items. We assessed internal consistency reliability of the 19 items using Cronbach’s coefficient alpha.⁴¹ Then, we assessed the fit of the single factor using confirmatory factor analysis.

Based on the results of the factor analysis, we proposed to keep 19 items in the Chronic Pain Coping Behaviors scale. In a confirmatory factor analysis using a probit structural equation model with a single latent variable (results not shown) we observed that all 19 items had statistically significant (p<0.05) factor loadings. Lastly, the alpha coefficient for the 19 items (Table 2) was 0.78, indicating acceptable reliability.

Table 2.

Chronic Pain Coping Approaches scale descriptive statistics and internal consistency (n=1808)

Descriptive statistics for sum score based on 19 items
Mean, SD	43.825
Range (out of possible 19–95)	Minimum= 20, Maximum=77
Alpha coefficient	0.780

This preliminary study suggests the items work well together in a scale. The team’s next step will be to conduct additional quantitative analyses focusing on the validity of these items, such as measuring the correlation between responses on these items to response on existing legacy measures.

Advantages of creating novel items

We created new items to address various other domains, including what respondents think it means for pain to be chronic. Creating novel items was helpful to our project because it allowed us to measure behaviors that no other existing instruments captured. Because we crafted these items specifically with this patient population in mind, they were more relevant to respondents. This was reflected in the responses to the study experience items in the pilot study which we asked respondents at the end of the survey. Multiple respondents commented on how the questions made sense to them and their conditions.

Bring It All Together: Weaving Tools and Items Into One Survey

Once the 3 parts of the Center (Patient Outcomes; Patient Preferences; Resource Allocation) had identified the measures that they wanted to include, we brought all the measures and items together into one survey, assessed them for appropriate ordering, length, and consistent wording, and then conducted a series of tests in preparation for our national study.

Ordering

First, we had to consider the order of the items. We spent considerable time putting the items into a thematically logical order. Items taken directly from an existing tool were kept in the same order as they appear in the original tool. We carefully implemented skip patterns to ensure that people were not asked questions that were irrelevant to them. Also, we avoided ordering items in a way that could create bias. For example, in one part of the survey, we presented a list of factors that could influence a person’s decision to get chiropractic care, and we asked respondents to rank each item in terms of importance. Because respondents may be more likely to endorse items that appear at the beginning of a list,⁴² we randomized the order of those items across all respondents.

Questionnaire Length

This was a 3-month longitudinal study, and it included 3 longer questionnaires and 5 shorter ones. All questionnaires were administered online. There were many measures that we wanted to administer at baseline, and in order to reduce burden on the respondents, we separated the baseline items into two questionnaires that respondents could complete in different sittings. Next, respondents completed shorter biweekly follow-up questionnaires every two weeks, and a final questionnaire at 3-months follow-up. For respondents who had both low back and neck pain, the 2 initial questionnaires had 81 and 140 items, the 5 biweekly follow-up questionnaires had 37 items each, and the endline questionnaire contained 121 items. The questionnaires were shorter for respondents who only reported having low back or neck pain (not both).

Consistency

We tried to be consistent across all our questionnaire items in terms of how items were worded and the response categories offered. One challenge was that some legacy items were written in first person (“I”) while others were written in third (“you”). Wherever possible, we modified items to make this consistent, but because we needed to include several existing measures using the original wording, it was not possible to make this consistent everywhere. Whenever possible, we also tried to use the same response categories, such as using the same 5-point response scale across multiple items rather than switching from a 5-point to a 7-point scale, whenever possible.

DISCUSSION

We noted earlier that this study set out to answer three main questions with regard to collecting data from chiropractic practices:

1. How do we choose instruments that are patient-centric and relevant to their experiences and that will capture their preferences and values?

2. How do we choose between, on the one hand, utilizing existing (legacy) measures which have been widely used in previous studies so we can compare our study to previous work, and on the other hand, design new instruments specific to this study?

3. How do we choose data collection instruments that are comprehensive but concise, reliable, valid, relevant, and non-burdensome?

As noted here, answering those three questions requires considerable effort and multi-method solutions; literature review, exploratory interviews, focus groups, cognitive interviews, pilot study, national survey. At first the questions seem quite simple, but they are woven into the broader research question of this Center. In this instance, the broader question is the appropriateness of chiropractic manipulation and mobilization in the treatment of chronic low back and neck pain. This is only the second study ever conducted in chiropractic to try to calculate a rate of appropriate care. Traditionally appropriate care was thought to be that which was efficacious and safe, and this was decided by research, researchers and clinicians.^4,43 However, the question is now being raised in an era of patient-centered care and patient-centered outcomes, or outcomes that are significant to the patient.⁴⁴ That raises a further question of what patient data can be collected within chiropractic treatment clinics that will allow researchers to answer that. In that broader context, the question then becomes how do we ensure that Evidence-Based Practice is truly Practice-Based Evidence. The significance of the study goes well beyond the three simple questions posed in this paper, but without solutions to those three, the broader questions cannot be answered,

This study has demonstrated that it is possible to collect a wealth of patient-centered data from chiropractic clinics, but the process is multi-faceted and quite demanding in terms of effort and resources. It can be difficult to find appropriate measurement tools to use with chiropractic/CIH patients. There may not be existing questionnaires that address a particular study’s constructs of interest, or existing tools may not apply to or have been validated with this population. Here we have shared our experiences with developing a questionnaire to assess multiple domains of patient experiences, beliefs and preferences about chiropractic care for chronic pain to demonstrate a set of approaches that researchers can use to identifying and creating tools.

We have also presented a detailed example showing how we used these methods to create a scale for measuring coping behaviors. We showed how the exploratory data collection and literature review findings, combined with logical assumptions, led us to identify key domains and key patient perspectives that we needed to capture if we wanted to understand what patients do from day to day to cope with their pain. From there, we created and tested 21 items for the general pain population and 5 additional items for people who are employed. Although the validity analysis is ongoing, we have presented preliminary findings using data from our national study to show that 19 of the 21 items general population items worked well together in a scale and had acceptable reliability. We believe these novel items are a useful contribution to the existing array of legacy measures related to coping with chronic pain, such as the Chronic Pain Self-Efficacy Scale.²⁷

The purpose of describing these methods is to encourage researchers and clinicians to consider the many possible approaches at their disposal for collecting information from patients. We are not suggesting that everyone should combine all these approaches the way we did, but rather that they should think carefully about which approach fits their needs best.

LIMITATIONS

The limitations are that this study was conducted only within the United States. While this should not affect the methods used in that many of the instruments selected were not developed in the US and have been used on other populations (and the literature review was not restricted to US articles), it does mean this study was only focused on the US.

While not a limitation for this study and the results reported here, the generous funding of this study by NCCIH would make it difficult for others to replicate the approach we have described at least in its totality. We were able to use a very comprehensive approach to develop the instruments (literature reviews, cognitive interviews, a pilot study, a national online survey, etc.). While this type of approach will ensure that the instruments developed have been rigorously tested, clearly this level of work would be beyond most research projects in CIH. At its peak, some 16 researchers were employed on this project. This level of funding for chiropractic research to date has not been replicated outside of the US.

CONCLUSION

It is important to collect valid data about patients’ experiences and beliefs for research and clinical care. In many instances, as with our study, the best approach may be to use existing measures for some constructs, to modify existing measures for other constructs, and to create entirely new measures for constructs where the existing measures are insufficient. In this paper, we have described how we used multiple qualitative methods as well as a review of the literature to identify constructs and then design questionnaires that were successfully administered as part of a national survey of chiropractic patients with chronic low back and neck pain. We have presented preliminary reliability and validity data for one of our novel measures, which addresses coping behaviors. We have also outlined suggestions for CIH researchers and providers who want to collect this sort of information from patients.

Appendix A.

Results of Initial Search for Tools about Patient Experiences, Beliefs and Preferences for Treatment for Chronic Pain

Tool
1. 100-point Modified Von Korff Pain and Disability Scales45
2. Acute Low Back Pain Screening Questionnaire46
3. Arthritis Self-Efficacy Scale47
4. Back Beliefs Questionnaire48
5. Bath Ankylosing Spondylitis Functional Index49
6. Beck Anxiety Inventory50
7. Beck Depression Inventory- II51
8. Borg’s rating of perceived exertion scale52
9. Bournemouth Questionnaire53
10. Center for Epidemiological Studies Depression Scale54
11. Chronic Pain Acceptance Questionnaire55
12. Chronic Pain Coping Inventory56
13. Chronic Pain Self-Efficacy Scale27 - Some subscales were used.
14. Coping Strategies Questionnaire57
15. Disabilities of the Arm, Shoulder and Hand Questionnaires58
16. General Health Questionnaire-2859
17. General Self-Efficacy Scale60
18. Global Perceived Effect Scale61
19. Global Rating of Change Scale62
20. Goal Pursuit Questionnaire63
21. Health Related Quality of Life Survey64
22. Injustice Experiences Questionnaire 65
23. MedRisk Instrument for Measuring Patient Satisfaction with Chiropractic Care66
24. Multidimensional Health Locus of Control Scale67
25. Neck Disability Index23 - Complete tool was used.
26. Northwick Park Neck Pain Questionnaire68
27. Oswestry Disability Index (ODI)21,22 - Complete tool was used.
28. Pain Catastrophizing Scale69-Some items were used.
29. Pain Disability Index70
30. Pain Self-Efficacy Questionnaire71
31. Patient Practitioner Orientation Scale72
32. Patient Satisfaction Scale73
33. Patient Specific Functional Scale74
34. Photograph Series of Daily Activities75
35. PROMIS-29 v 2.0 profile measure19-Complete tool was used.
36. Psychological Inflexibility in Pain Scale76
37. Quebec Pain Disability Scale77
38. Roter Interaction Analysis System78
39. Self-Efficacy Scale79
40. Sense of Coherence80
41. SF-6D81
42. Shoulder Pain and Disability Index82
43. Sport Injury Rehabilitation Adherence Scale83
44. Survey of Pain Attitude25,26 - Some domains were used.
45. Tampa Scale of Kinesiophobia84
46. Vanderbilt Pain Management Inventory85
47. Western Ontario and McMaster Universities Osteoarthritis Index86
48. Working Alliance Inventory87
49. ZUNG self-rating depression scale88

Note: While most of these tools did not get integrated in our questionnaire, nearly all of them provided us useful insight into the types of domains being assessed in the pain literature.