Validation of a general measure of treatment satisfaction, the Treatment Satisfaction Questionnaire for Medication (TSQM), using a national panel study of chronic disease

Abstract

Background

The objective of this study was to develop and psychometrically evaluate a general measure of patients' satisfaction with medication, the Treatment Satisfaction Questionnaire for Medication (TSQM).

Methods

The content and format of 55 initial questions were based on a formal conceptual framework, an extensive literature review, and the input from three patient focus groups. Patient interviews were used to select the most relevant questions for further evaluation (n = 31). The psychometric performance of items and resulting TSQM scales were examined using eight diverse patient groups (arthritis, asthma, major depression, type I diabetes, high cholesterol, hypertension, migraine, and psoriasis) recruited from a national longitudinal panel study of chronic illness (n = 567). Participants were then randomized to complete the test items using one of two alternate scaling methods (Visual Analogue vs. Likert-type).

Results

A factor analysis (principal component extraction with varimax rotation) of specific items revealed three factors (Eigenvalues > 1.7) explaining 75.6% of the total variance; namely Side effects (4 items, 28.4%, Cronbach's Alpha = .87), Effectiveness (3 items, 24.1%, Cronbach's Alpha = .85), and Convenience (3 items, 23.1%, Cronbach's Alpha = .87). A second factor analysis of more generally worded items yielded a Global Satisfaction scale (3 items, Eigenvalue = 2.3, 79.1%, Cronbach's Alpha = .85). The final four scales possessed good psychometric properties, with the Likert-type scaling method performing better than the VAS approach. Significant differences were found on the TSQM by the route of medication administration (oral, injectable, topical, inhalable), level of illness severity, and length of time on medication. Regression analyses using the TSQM scales accounted for 40–60% of variation in patients' ratings of their likelihood to persist with their current medication.

Conclusion

The TSQM is a psychometrically sound and valid measure of the major dimensions of patients' satisfaction with medication. Preliminary evidence suggests that the TSQM may also be a good predictor of patients' medication adherence across different types of medication and patient populations.

Background

This article reports on the development and testing of the Treatment Satisfaction Questionnaire for Medication (TSQM) and builds on the conceptual framework of Treatment Satisfaction (TS) which is featured in a companion article entitled: "The Development of a Conceptual Framework for Treatment Satisfaction." (a manuscript currently under review). Within this paper, we will begin by reviewing current literature that highlights the clinical importance of TS, as well as some of the measurement challenges facing researchers in this field. This is followed by description of a two-stage TSQM item generation process that included both patient focus groups and patient interviews. The results section presents the analyses used for TSQM scale identification and psychometric testing. These results were based on a large sample of patients enrolled in the NFO World Group's Chronic Ailment Panel (NFO-CAP). Finally, in the discussion section we focus on the psychometric characteristics of the TSQM, the comparative performance of two different methods for item scaling, and the potential uses of TS assessment in clinical settings.

Those advocating collaborative (patient-caregiver) models of health care delivery suggest that patient reported outcomes (PROs), and particularly measures of patient preference, ought to play a central role in the planning and delivery of medical care [1-3]. A subclass of PRO measures, patient satisfaction, has been used extensively to include patients' perceptions of care when evaluating the effectiveness of medical treatments and systems of healthcare delivery [4-7]. Patient satisfaction has been shown to affect patients' health-related decisions and treatment-related behaviors, which in turn, substantially impact the success of treatment outcomes [8,9]. For example, patients' satisfaction with the services they receive has been shown to predict treatment success, medical compliance, follow-through with treatment plans, and appropriate use of services [10-12]. In a similar way, patients' satisfaction with their medication predicts continuance of pharmaceutical treatment, correct medication use and compliance with medication regimens [13-16].

A variety of models have been used to describe how patients' satisfaction with medical treatment impacts their health-related decision-making [17-21]. Common to most models, it is proposed that patients' decisions to continue, alter, or discontinue medical treatments are influenced by a variety of characteristics, including; the desire to participate in treatment related decision-making [9,22] evaluation of actual and preferred health state [23-25] prior experiences with particular treatment choices [26] and real or anticipated beliefs regarding the effectiveness or harms of treatment [23,25,27]. The adverse decisional consequences of low TS on medication compliance is of particular concern to those treating patients with chronic disease conditions [12]. It has been estimated that up to one half of patients with chronic illness end up making medication-related decisions without seeking medical advice, becoming 'non-adherent' to such an extent that they compromise the effectiveness of treatment and strain broader systems of care [28]. In contrast, more acutely ill patients who perceive an immediate threat to their physical well-being may be more willing to tolerate short-term aggressive treatment regimens in hopes of restoring their former health.

In addition to its impact on treatment outcomes within the clinical setting, TS results have been incorporated into decisions regarding pharmaceutical formularies and cost-effectiveness evaluations of managed care organizations [29]. Some healthcare economists have suggested that in the near future planners within healthcare delivery systems and pharmaceutical industries will view assessment of treatment satisfaction as essential to their continued viability [30,31]. The interest of multiple stakeholder groups in TS has lead to important conceptual advances in this field and a proliferation of satisfaction measures [32,33]. Such measures can be roughly divided into those addressing patients' satisfaction with discrete aspects of medical treatments and those focusing on more systemic aspects of programmatic care [12,34-38]. Similarly, patients' satisfaction with their medication (TS-M) can be thought of as a very specific sub-dimension (or observational context) of TS which is a broader, super-ordinate class that encompasses patients' satisfaction with both medicinal and non-medicinal aspects of treatment. In turn, TS is a subset of patient satisfaction (PS) that broadly covers all aspects of medical treatments, interpersonal aspects of clinical care, and processes of treatment.

Measurement challenges

Unfortunately, across most illness conditions TS and PS research has been consistently hampered by serious measurement problems, including; distributional skew, ceiling effects, and missing response data [39-47]. Since ceiling effects and data skew reduce the power of statistical methods to detect meaningful group differences, numerous attempts have been made to resolve these problems including; the use of very extreme anchors, the use of non-neutral midpoints, and the expansion of the number of scale response options [48-50]. Nevertheless, systematic comparisons of these approaches have been sporadic and there remains a longstanding debate over the relative merits of such methods. Results from one of the few empirically-based comparisons by Ware and Hays [51], suggest that Likert-type scales might perform slightly better than Visual Analogue Scale (VAS) methods. Advocates of VAS methods contest this assertion and refer to the ease of use, brevity and condensed layout of VAS rating scales [52].

None of these scaling solutions, however, have been shown to wholly resolve the distributional problems associated with the cross-sectional measurement of TS and PS. Yet, there remains a persistent and largely unquestioned assumption that normal distributions of satisfaction scores can be obtained if only the construct were measured correctly. As a result, there are quite a few examples in the literature where patients' satisfaction ratings are suspect of social desirability or acquiescence responses bias [49,53]. There is a risk, however, of over generalizing an assumed respondent bias to all types of TS measures. For example, TS-M ratings may be less susceptible to social desirability bias compared to PS ratings of clinical care, as the latter is more likely to be influenced by patients' relationships with primary caregivers [54]. Moreover, if respondents tend to acquiesce and provide satisfied responses, it is more likely to occur when answering questions about less important or irrelevant aspects of care. Scales composed of large numbers of detailed and treatment-specific content typically contain a large number of items that are irrelevant to the experiences of a specific patient and thus are more susceptible to receiving a satisfied rating from respondents. In contrast, more generally worded questions are composed of items that allow respondents to interpret their meaning based on important aspects of their own experiences. Respondents are less likely to provide an acquiescent response to questions that are considered personally relevant.

An alternate mechanism may help explain the skewed distribution of TS-M ratings. Over time, clinical-selection may affect the composition of patient samples (sample drift) and result in a skewed cross-sectional distribution of satisfaction scores. It is hypothesized that such selection occurs over time as patients for whom a medication is working continue to take the medication, while those for whom it is not working, or for whom unpleasant side effects occur, seek alternative treatments. In general, one might expect sample drift to be greatest during the initiation of a new course of medication and, conversely, least when either a satisfactory medication has been found or when treatment alternatives have been exhausted. In the latter case, access to fewer treatment alternatives may be more likely among those with severe and persistent disease. Currently, it is unknown to what extent these various influences shape the observed distribution of satisfaction results in cross-sectional patient samples.

Rationale for current study

Although numerous disease-specific measures of patients' TS and TS-M have been reported in the literature [55-60] less attention has been paid to developing a more general measure of TS-M; one that would permit comparisons across medication types and patient conditions. Also, as addressed earlier, there is an unresolved controversy over the optimal method for scaling satisfaction items. Therefore, two central objectives have been identified for the current study:

• To develop a conceptually and psychometrically sound general measure of TS-M, capable of assessing patients' satisfaction with various medications designed to treat, control, or prevent a wide variety of medical conditions; and

• To examine the performance of such an instrument with respect to scaling alternatives so as to maximize the precision and validity of the final measure.

Methods & study design

Background item generation

The design of test items for the new instrument was based on a generalized conceptual framework of treatment satisfaction. The initial formation of the conceptual framework was grounded in a thorough review of the scientific literature that dealt with the core TS-M domains across a diversity of therapeutic areas. Subsequently, the draft conceptual framework was more fully elaborated using qualitative data from patient focus group interviews. Focus group participants (n = 30) were recruited to take part in one of three, two-hour sessions conducted in Los Angeles, Chicago, and Boston. Participants consisted of patients with at least one the following illness conditions: asthma, arthritis, cancer, cardiovascular disease, depression/anxiety, diabetes, infectious disease, migraine, and psoriasis. The focus group discussions were guided by a trained interviewer who, in accordance with established qualitative research procedures [61], focused on aspects of the treatment satisfaction framework, outlined in a discussion guide [62,63].

Over the course of the three focus group sessions, the discussion guide and conceptual framework on which it was based, were evolved through integration of the patients' perspectives from each preceding group. In this way the guide was iteratively refined to reflect the participants' perspectives. Once the framework was fully elaborated, the domains of TS-M included; (1) side effects, (2) symptom relief, (3) convenience, (4) effectiveness, (5) impact on daily life, and (6) tolerability/acceptability. Fifty-five draft TS-M items were designed to measure aspects of the conceptual framework and its domains. Further details of the qualitative methods and results can be found in a sister manuscript describing the development of the TS-M conceptual framework.

Initial item reduction and scaling (patient interviews)

In-depth patient interviews were conducted in order to reduce the 55-item pool by approximately half, leaving only those items that were most relevant across respondents. The interview sample consisted of 17 patients taking medication for the same conditions represented by focus group participants. During the 45–60 minute interviews, patients rated the importance or relevance of each item to their satisfaction with their medication using a 5-point scale (where 1 was most important and 5 was not important at all). These ratings were used to select items that were most relevant across all illness groups. When items were ranked equally, the conceptual framework was used to help assure adequate representation of theoretical dimensions in the framework. The final test item pool contained 31 items.

Two scaling methods, visual analogue scaling (VAS) and Likert-type scaling, were considered for use in the final instrument. In order to compare the relative performance of the two methods, two sets of TSQM items were created that differed only in terms of the rating scale used. For both sets, TSQM items were scaled using either a 5-point or 7-point scale. Five-point scales were used for unidimensional continua (e.g. extremely to not at all), while 7-point scales were used for bipolar continua (e.g., extremely positive to extremely negative). This provided roughly equivalent rating intervals across items. Non-neutral midpoints were used for 7-point scales, resulting in a greater range of positive response options than negative options for these items. This approach has been suggested elsewhere as a way of helping to address scale resolution problems associated with the upper end of skewed distributions [48].

Psychometric testing and refinement (national panel survey)

The remaining sections of this article describe the reliability and validity characteristics of the test items and scaling methods using a large sample of patients participating in the NFO – World Group's CAP. The NFO CAP consists of over 250,000 people suffering from one or more of over 60 chronic ailments and conditions. The panel is a representative sampling of one out of every 191 households in America, prescreened for more than 50 pieces of demographic information so as to represent the demographic characteristics of the population of the citizenry of the USA (for more information see: http://www.nfow.com).

Patients were recruited for this portion of the study that had the same illness conditions as represented within the focus groups and interviews (anxiety/depression, arthritis, asthma, cancer, cardiovascular disease, diabetes, infectious disease, migraine, and psoriasis). They were also required to be at least 18 years of age, able to read English, and able to complete a questionnaire on-line. The broad sampling provided a range of treatment intents (i.e., curative, preventive and symptom management) as well as routes of medication administration (i.e., injection, oral, topical, inhalation).

Invitations were sent electronically to 10,000 NFO panel members across the United States. Participants that accessed the study site via the Internet were assessed for eligibility, equally stratified by illness condition and gender, and then randomly assigned into 1 of the 2 scale conditions (VAS or Likert-type scaling methods). Since many participants had multiple illness conditions, and were on several medications at the same time, respondents were helped to clearly identify which particular medication and illness condition were the subject of study. A total of 6,713 individuals responded (a response rate of 67.2%), from this pool individuals were sequentially offered the opportunity to participate based on the availability of participant slots in each stratum. Five hundred and eighty seven individuals passed screening and were enrolled, of these, 567 provided complete data sets, with 287 respondents in the VAS arm and 280 in the Likert-type arm.

In addition to completing the test items, respondents were asked to provide information about the length of time they had been on their medication, the method of its administration, the frequency and severity of any side effects they might have experienced, and the likelihood that they would continue to take the medication given its current level of effectiveness and side effects. They were also asked about perceptions of their current state of health, the severity of their illness, and some basic socio-demographic information (e.g., age, gender, educational level, and ethnic background).

Results

Respondent characteristics

Respondents' ages ranged from 18 to 88 years, with a mean of 50.5 (SD 13.0), which did not differ significantly from the total NFO representative sampling (mean 48.8, SD 13.4). Thirty nine percent of respondents indicated that they had received four or more years of college education, and 60.1% stated that they were employed full-time. The educational proxy for socioeconomic status was roughly equivalent for the original NFO recruitment sample (31%). Table 1 presents the number of NFO respondents in each of the illness groups, the length of time on the current medication, the route of its administration, their rating of the severity of illness, and their rating of current health status. Approximately 70% of the sample reported on an oral medication, while the remaining 30% reported on medications that were used in a topical, inhalable, or injectable form. As expected given the randomization procedure, no significant differences were found between the scaling condition groups (Likert vs. VAS) by gender, age, educational level, employment status, ethnicity, mode of medication administration or length of time on medication.

Table 1.

TSQM Validation Survey: Respondent Characteristics (n = 567)

Illness Group	Major Route of Admin: Total (%)	Weeks on Medication Mean (SD)	Health Rating+ Mean (SD)	Illness Severity++ Mean (SD)
Migraine (n = 68)	Oral: 60 (88.2%)	57.2 (76.4)	2.8 (.8)	1.9 (.6)
Arthritis (n = 75)	Oral: 71 (94.7%)	38.2 (40.4)	2.9 (.9)	1.9 (.6)
High BP (n = 76)	Oral: 76 (100%)	52.1 (53.2)	2.5 (.8)	1.7 (.6)
Asthma (n = 72)	Inhaled: 62(86.1%)	92.4 (114.8)	2.9 (1.0)	1.8 (.7)
Diabetes (n = 63)	Injected: 53 (84.1%)	125.0 (115.4)	3.4 (.9)	2.2 (.5)
Psoriasis (n = 63)	Topical: 53 (84.1%)	49.7 (60.1)	2.9 (.9)	1.7 (.6)
High Cholesterol (n = 75)	Oral: 75 (100%)	31.0 (34.7)	2.8 (1.0)	2.0 (.6)
Depression (n = 75)	Oral: 75 (100%)	42.9 (42.7)	2.6 (.9)	2.1 (.5)

⁺1 = Excellent, 2 = Very Good, 3 = Good, 4 = Fair, 5 = Poor; ⁺⁺1 = Mild, 2 = Moderate, 3 = Severe

Construct dimensionality of the TSQM

Multi-step exploratory factor analyses (EFA) were employed to investigate the construct validity of the TSQM. Two separate EFAs were conducted, one using global TS-M items, and another using items that referred to more specific domains of medication experiences (e.g., Effectiveness, Side effects, Convenience) [64]. Such multi-step EFA procedures have been recommended by Gorsuch [65,66] and Russell [67] as a way to evaluate the structure and dimensionality of measures that include both global and specific item content. The global TS-M items are super ordinate conceptually and psychologically, and thus may be redundant and confounding measures of the construct. As the goal is to identify the underlying construct or factor, redundant and confounding variance should be minimized. The confounding of subordinate construct dimensionality by global items shows up as unwanted covariance, manifest as cross-loading of global items across the more specific factors. As a result, separate analyses of global and specific items provide scales with greater cohesion and homogeneity than when such a process is not followed.

A first EFA employed principal components extraction and a subsequent orthogonal varimax rotation of the more specific TS-M items. This resulted in a three-factor solution that accounted for 68.3% of the total variance. Items with the greatest loadings on these factors were then selected for inclusion in the final TSQM scales. The three factors in the final solution converged in five iterations, possessed Eigenvalues greater than 1.7 and explained 75.6% of the overall variance (see Table 2). These were labeled according to their item content: Side effects (SIDEF: 4 items, 28.4% of the variance), Effectiveness (EFFECT: 3 items, 24.1%), and Convenience (CONV: 3 items, 23.1%).

Table 2.

Loadings of Treatment Satisfaction with Medication Items (n = 567)

	Factor I	Factor II	Factor III
Side effects 1: Side effects interfere with physical function	.89	.10	.16
Side effects 2: Bothersomeness of side effects	.87	.09	.13
Side effects 3: Side effects interfere with mental function	.79	.05	.14
Side effects 4: Side effects impact overall satisfaction	.76	.19	.06
Effectiveness 1: Ability to prevent or treat the condition	.14	.90	.06
Effectiveness 2: Ability to relieve symptoms	.13	.88	.08
Effectiveness 3: Time it takes medication to start working	.09	.85	.09
Convenience 1: Convenience of administration	.16	.15	.88
Convenience 2: Ease/Difficulty of planning	.12	.11	.88
Convenience 3: Ease/Difficulty following schedule	.14	.06	.86

75.6% of Total Variance Explained; by Factor I (28.4%), Factor II (24.1%) and Factor III (23.1%)

A second EFA (principal component extraction and varimax rotation) was conducted using responses to five global satisfaction items, comprising a conceptually distinct second order factor of TS-M. Three items with the highest loadings were selected for final inclusion. The final solution was unidimensional (Eigenvalue = 2.3), with factor loadings between .86 and .90, which explained 79.1% of the total variance. The three items asked about were; 1) the confidence individuals had in the benefits of the medication, 2) their comparative evaluation of the benefits versus drawbacks of the medication, and 3) their overall satisfaction with the medication. The final instrument (see Table 3) consisted of 13 items that made up three specific scales (EFFECT, SIDEF, CONV) and one global satisfaction scale (GLOBAL). Scale scores were transformed into scores ranging from 0 to 100. The inter-correlations between scales shown in Table 4, suggest that the strongest specific-scale correlate of GLOBAL was EFFECT. It would be surprising if this were not the case, since medication is typically taken for its curative effects. SIDEF and CONV ratings were about equally correlated with results on the GLOBAL satisfaction scale.

Table 3.

Final Items for the Treatment Satisfaction Questionnaire for Medication (TSQM)⁺⁺

Item #	TSQM Item
1*	How satisfied or dissatisfied are you with the ability of the medication to prevent or treat your condition?
2*	How satisfied or dissatisfied are you with the way the medication relieves your symptoms?
3*	How satisfied or dissatisfied are you with the amount of time it takes the medication to start working?
4**	As a result of taking this medication, do you currently experience any side effects at all?
5	How bothersome are the side effects of the medication you take to treat your condition?
6	To what extent do the side effects interfere with your physical health and ability to function (i.e., strength, energy levels, etc.)?
7	To what extent do the side effects interfere with your mental function (i.e., ability to think clearly, stay awake, etc.)?
8	To what degree have medication side effects affected your overall satisfaction with the medication?
9	How easy or difficult is it to use the medication in its current form?
10	How easy or difficult is it to plan when you will use the medication each time?
11	How convenient or inconvenient is it to take the medication as instructed?
12	Overall, how confident are you that taking this medication is a good thing for you?
13	How certain are you that the good things about your medication outweigh the bad things?
14*	Taking all things into account, how satisfied or dissatisfied are you with this medication?

* These items are scaled on a seven point bipolar scale from 'Extremely Satisfied' to 'Extremely Dissatisfied'. **Item #4 is a dichotomous response option with a conditional skip to item #9. ⁺⁺Obtaining the TSQM: Electronic versions of the TSQM in multiple languages and scoring algorithms are available by contacting Quintiles, Inc. (415.633.3100/3243, FAX 415.633.3133, shoshana.colman@quintiles.com)

Table 4.

Interscale correlation matrices* for VAS/Likert-type methods

	Effectiveness (EFFECT)		Side effects (SIDEF)		Convenience (CONV)
	VAS**	Likert***	VAS**	Likert***	VAS**	Likert***
Effectiveness	1.00	1.00
Side effects	.23	.37	1.00	1.00
Convenience	.22	.36	.33	.35	1.00	1.00
Global Satisfaction	.60	.72	.36	.43	.41	.48

* Spearman correlations are significant at the .0001 level (2-tailed); **VAS sample (n = 287); ***Likert type sample (n = 280)

Scale characteristics and scaling comparisons

The performance of the two scaling methods was evaluated based on the strength of the factorial solution and the estimates of internal consistency of resulting TSQM scales. The factorial dimensionality and item loading order were the same using either scaling dataset. However, the strength of the factorial solution and Cronbach's Alpha coefficients were greater when using the Likert-type results compared to the VAS results. As expected, the score distributions resulting from both scaling methods were characterized by ceiling effects and skew that plague this class of PRO instrumentation (Table 5) [11,23,36,40]. Of note, the VAS scaling method had more problems with ceiling effects than the Likert-type scaling method, particularly on items making up GLOBAL. The Likert-type method tended to have higher skew statistics on two scales due to a more pronounced taper on the lower (dissatisfied) end of the scales.

Table 5.

The Distributional and Scale Characteristics of the TSQM

	Scale Mean (SD)		Cronbach's Alpha		% Scores at Scale Ceiling		Skewness**
TSQM Scales	VAS Method (n = 287)	Likert Method (n = 280)	VAS Method	Likert Method	VAS Method	Likert Method	VAS Method	Likert Method
Effectiveness	69.7 (21.8)	68.6 (20.4)	.87	.88	13.0%	8.9%	-.47	-.76
Side effects	84.3 (19.2)	83.7 (19.5)	.84	.88	44.3%	41.1%	-1.1	-1.2
Convenience	84.9 (19.7)	83.2 (18.7)	.86	.90	44.9%	36.8%	-1.3	-1.1
Global Satisfaction	78.0 (20.4)	71.1 (22.6)	.80	.86	25.1%	12.9%	-.81	-.97

** Skewness Standard Error VAS Method = .14, Likert-type Method = .15

Possible reasons for the distributional skew of SIDEF were explored further. The removal of respondents who reported rare or very infrequent side effects from the sample resulted in an essentially normal distribution (skew = -.13, <4% of scores at ceiling value). This suggested that respondents appropriately provided high satisfaction ratings in situations where the side effects of the medication were very infrequent. Thus, the skew and ceiling effects associated with this particular scale do not seem to be simply due to problems associated with an uninterpretable respondent bias.

Medication and illness characteristics associated with treatment satisfaction

No significant differences in mean TSQM scale scores were observed by gender or education level. Significant differences in satisfaction levels were found on all TSQM scales by route of medication administration (Figure 1). As documented elsewhere, individuals using injectables reported low satisfaction with convenience of use [68]. Also, despite low ratings on SIDEF and CONV by the injectable group, the GLOBAL and EFFECT ratings were highest – presumably due to the influence of insulin-dependence in our injectable sample. Also, consistent with other research, high GLOBAL and CONV ratings were associated with orally administered medications [68-70] although satisfaction with the effectiveness of oral medications was a bit lower than with the injectables. The topicals were associated with the highest levels of satisfaction with SIDEF and CONV, but the lowest levels of on the GLOBAL and EFFECT scales – likely due to their safety and ease of use, but relative ineffectiveness at treating the condition, which in this case was psoriasis [71]. Taken together, these observations provide some evidence for the criterion-related validity of the TSQM scales.

Figure 1.

Mean Medication Satisfaction Levels by Route of Administration Notes: Effectiveness by Route, F(3,552) = 11.98, p < .0001 Side Effects by Route, F(3,552) = 5.87, P < .001 Convenience by Route, F(3, 552) = 58.92, p < .0001 Global by Route, F(3, 552) = 4.89, p < .01

Consistent with earlier discussion of clinical drift, individuals on medication for less than two months reported significantly lower satisfaction with both the effectiveness and side effects of their medication than those on medications for a longer period (68.3, sd 18.8 vs. 74.4, sd 17.2, F(df) = 8.57(1), p = .004; 84.6, sd 16.4 vs. 88.4, sd 14.2; F(df) = 4.76(1), p = .03 respectively). This observation provides preliminary evidence that individuals who continue to experience either low effectiveness or troublesome side effects may be more likely to switch or discontinue their medication, and thus contribute to the changing distributional characteristics of cross-sectional satisfaction data over time. Illness conditions also appeared to influence satisfaction levels. Higher illness severity ratings were associated with lower levels of satisfaction on all TSQM scales, particularly SIDEF. Similarly, poorer appraisal of general health was associated with lower satisfaction scores on all scales, particularly EFFECT (Table 6). These findings are likely due to the inability of the current medication to cure or effectively manage the condition without intolerable side effects. The availability of, and access to, alternative treatment options may also have prevented 'clinical drift' and, as a result, influenced the distribution of patients' satisfaction scores.

Table 6.

Comparison of Satisfaction with Oral Medication by Patients' Ratings of Seriousness of Illness and Health Appraisal (n = 378)

	Seriousness of Illness
	Mild (n = 87) Mean (SD)		Moderate (n = 237) Mean (SD)	Severe (n = 54) Mean (SD)		F Value	p Value
Effectiveness	73.5 (18.9)		70.2 (19.2)	64.8 (25.5)		3.09	.05
Side effects	90.9 (15.1)		84.6 (18.7)	73.6 (24.0)		14.17	.000
Convenience	92.6 (11.5)		90.2 (14.2)	84.2 (19.5)		5.79	.003
Global	80.2 (19.4)		75.8 (18.8)	67.3 (28.2)		6.57	.002
	Appraisal of Health
	Excellent (n = 23) Mean (SD)	Very Good (n = 139) Mean (SD)	Good (n = 153) Mean (SD)	Fair (n = 48) Mean (SD)	Poor (n = 15) Mean (SD)	F Value	p Value
Effectiveness	76.6 (23.5)	75.9 (17.7)	67.5 (19.1)	61.6 (19.4)	63.3 (33.4)	7.03	.000
Side effects	91.8 (17.5)	88.3 (16.9)	81.7 (20.6)	81.4 (20.8)	75.8 (19.9)	4.08	.003
Convenience	92.8 (16.3)	92.8 (11.3)	87.9 (15.7)	88.5 (15.9)	83.3 (20.1)	3.21	.013
Global	81.1 (22.6)	82.4 (16.6)	71.9 (20.6)	68.6 (21.7)	64.8 (32.1)	8.20	.000

Determinants of overall satisfaction and medication persistence

A series of regression analyses were used to examine the specific aspects of TS-M that predicted GLOBAL TSQM satisfaction ratings. Table 7 presents the standardized beta coefficients and percent variance explained (Adjusted R²) within these statistical models. The three specific TSQM scales were all entered as independent variables and GLOBAL as the dependent variable for each different illness group. Across all groups, between 40–70% of the variance in GLOBAL ratings were explained by satisfaction ratings on the three specific TSQM scales. Moreover, EFFECT accounted for the most variance in GLOBAL, while the relative influence of the two other specific scales varied across illness groups.

Table 7.

Standardized Beta Regression and shared variance estimates of specific satisfaction scales Predicting global satisfaction ratings by illness group (n = 567)

	Effectiveness	Side effects	Convenience	Adjusted R Squared*
Migraine	.57***	.27**		.49
Arthritis	.53***	.34***		.52
Depression	.73***	.21**	.18**	.72
Asthma	.52***		.33***	.51
Diabetes	.57***	.20*	.25*	.67
Psoriasis	.54***	.26**	.23*	.60
Cholesterol	.48***		.36***	.43
Hypertension	.60***	.19*	.23**	.59

*Regression Model: Effectiveness + Side effects + Convenience = Global Satisfaction

In order to explore the effects of TS-M on patients' choice to either continue or discontinue using a medication, a composite variable was derived – "Likelihood to Discontinue" medication. This variable was computed by taking the respondents' ratings of their 'likelihood to continue on the current medication given its current level of effectiveness' and subtracting it from ratings of their 'likelihood to request a change in medication due to current side effects. The four TSQM scale scores were then entered as independent variables into a stepwise multiple regression analysis. The final model contained 3 of the 4 satisfaction scales; GLOBAL, SIDEF and EFFECT (Adjusted R² = .50, F(3,563) = 186.2, p < .0001). GLOBAL accounted for more variance than did the more specific TSQM scales and was the most significant independent variable accounting for medication non-persistence (standardized Beta coefficient = -.35, p < .0001). This scale was followed by SIDEF (Beta = -.22, p < .0001) and EFFECT (Beta = -.28, p < .0001). On its own, CONV was not found to be significantly correlated to respondents' ratings of Likelihood to Discontinue medication.

Results in Table 8 hint at the strength of the association between patients' TS-M and their expectations regarding future persistence with their medication regimen. Across six of the eight illness groups between 50 and 60% of the variation patients' expected persistence with medication was predicted by TSQM scores.

Table 8.

Standardized Beta Regression Coefficients and Shared Variance Estimates of Satisfaction Variables Predicting Ratings of Likelihood to Change Medication by Illness Group (n = 567)

	Global Satisfaction	Effectiveness	Side effects	Convenience	Adjusted R Squared*
Migraine	-.37***	-.32**	-.28**		.61
Arthritis	-.46***	-.23*	-.23*		.59
Depression	-.51***		-.37***		.56
Asthma	-.44***	-.29*			.42
Diabetes	-.37**	-.47***			.60
Psoriasis	-.31*	-.40***	-.23*		.56
Cholesterol		-.40***	-.49***		.55
Hypertension	-.36***		-.28**		.26

*Regression Model: Effectiveness + Side effects + Convenience + Global Satisfaction = Likelihood to Change Medication

Discussion

Sampling considerations

The response by two-thirds of the NFO panel invitees was much greater than that usually found for broad and less targeted Internet-based health surveys, which typically range between 20–30% [72-75]. Nevertheless, our samples should not be considered representative of the general US population, and only, at best, an approximate sampling of membership within each of the illness panels. Fortunately, concerns about sampling bias are rarely raised as a serious criticism in this type of psychometric study, since the main objective is to empirically examine the content and measurement dimensions that underpin a theoretical construct. So long as the constructs of measure remain fairly consistent across the illness populations, it is unlikely that a moderate degree of sampling and selection bias alters the item-item covariance structure used to identify the dimensionality of such constructs. Such bias becomes more problematic where determination and comparison of score levels is of interest, as is the case when estimating population parameters or testing group score differences.

A more serious threat to demonstration of the construct validity of the TSQM, however, is the interdependency between illness conditions, illness severity, and medication type. Such interdependencies preclude a clear delineation of their independent effects on TSQM results. This concern became most apparent during our examination of the effects of the route of medication administration on TSQM results. It was not possible to clearly separate the effects of medication route from illness group membership, a fact that we acknowledged when describing these results.

TSQM performance

Given the heterogeneity of the sample, the reliability and construct validity characteristics of the TSQM scales were surprisingly strong. The internal consistency estimates of the scales were high given the small number of items and their broad content coverage. The patterns of item loadings within the factor analyses provide clear evidence for the orthogonal dimensionality of the three specific TSQM scales and, by inference, discrete sub-dimensionality of the TS-M construct. In turn, when predicting GLOBAL ratings, the regression weights associated with specific TSQM scales differed across illness groups, reflecting the different importance of TS-M dimensions to overall satisfaction across illness/medication types. This finding provides further evidence that GLOBAL cannot be simplified to a generalizable summation of specific scales across all illness groups since the relative weighting of specific aspects of TS-M on GLOBAL scores appears to be influenced by both medication-specific and disease-specific experience across patient groups. Despite its non-uniform derivation, the importance of GLOBAL was manifest through its correlation with patients' perception of treatment persistence (likelihood to continue/discontinue medication), which was stronger than any specific dimension of TS-M, even the effectiveness dimension.

The results of scaling comparisons (VAS versus Likert-type method) support earlier work by Ware and Hays [51] that reported better predictive performance of a Likert-type scaling compared to VAS scaling of TS items. Despite our best efforts to assure both metric and sample equivalence between the two scaling conditions, better distributional characteristics and lower measurement error was associated with the Likert approach, particularly on GLOBAL. As a result, this scaling method was associated with a greater proportion of meaningful variance across a variety of parametric analyses. Moreover, a commonly cited advantage of VAS type scales is ease of completion, yet when asked, the patients in the two scaling conditions did not differ on the reported ease of questionnaire completion. As a result, the Likert-type scaling method was selected to scale the final version of the TSQM.

Atypical cross-sectional score distributions

As expected, our scaling efforts did not effectively correct the distributional problems. Indeed, some of the most consistent findings in the PS literature are the persistent distributional skew and ceiling effects associated with this type of data. One of major causes of such skew in the current study was item relevance. This was clearly demonstrated using SIDEF items, where satisfaction ratings were consistently high when problems with side effects were rare or non-existent. Approximately 50% of the sample reported rarely experiencing side effects and, when this was taken into account, the distribution of SIDEF satisfaction scores became essentially normal. A similar pattern of results might have occurred for CONV, however, information on the frequency of inconvenient medication-related events was not collected.

In addition to item relevance, we hypothesized that a portion of the skew and ceiling effects might be due to a continuous self- and clinical-selection process, leading to sample drift as over time, individuals who are less satisfied with either the effectiveness or side effects of their medication seek alternatives. Supporting this idea, respondents' length of time on medication was positively associated with mean differences on both EFFECT and SIDEF. Those on medications for more than two months expressed higher levels of satisfaction on the two dimensions of TS-M. Moreover, the distributions of scores had greater skew towards the more satisfied end of the continuum.

Acting against such a trend may be a lack of effective treatment alternatives for those with more serious conditions. Respondents who rated themselves as either in worse health, or as more ill, were less satisfied across all TSQM scales. One might hypothesize that patients with more severe conditions may have been willing to tolerate higher side effects in order to affect a cure. However, this does not easily explain the lower EFFECT scores also reported by persons with poor health ratings. It is most likely that less satisfaction with the effectiveness of treatment is associated with treatment resistant illness conditions and/or fewer effective treatment alternatives.

Our results suggest that non-normal distribution of cross-sectional satisfaction scores should not be quickly dismissed as an artifact of systematic respondent bias, but rather understood as the result of a complex interaction between clinical-selection, the availability and effectiveness of treatment alternatives, and respondents' health status over time. Unfortunately, the cross-sectional design of our current study does not permit a meaningful characterization of the cumulative effects of such characteristics on TSQM score distributions over time, and is only suggestive of a need for longitudinal research to more fully address this phenomenon.

Treatment satisfaction and the cost of care

From a disease-management perspective, it is likely that assessment of TS-M will become increasingly important in the future; in part due to the increasing prevalence of chronic disease in our aging population and the increasing number of patients being asked to persist with long courses of pharmaceutical treatments. With the exception of certain areas of medicine where patient compliance is particularly problematic, relatively little is known about the influence of patients' satisfaction on medication adherence behavior. It is particularly pressing, given rising costs of health care, to identified and address the causes of non-adherence; since such behavior increases the use of medical resources to manage treatment failure. While the health care costs resulting from non-compliance are fairly well characterized for many conditions, such costing studies less frequently include patient preference data. Longitudinal economic research is required to explore these important causal relationships.

The pharmaceutical industry may also be an interested stakeholder. Evidence of the growing importance of patients' satisfaction can be found at most levels of our health service delivery systems. For example, the Health and Human Services' Agency for Healthcare Research and Quality and the American Hospitals' Association has recently begun to require that patient satisfaction data be published by hospitals to aid patients in their selection of hospital services, and possibly informing financial reimbursement schedules [76]. Such developments may foreshadow the role of TS-M within the pharmaceutical sector, in that TS-M outcomes directly influence the degree of market success enjoyed by new therapeutic technologies and medications. For example, TS-M assessment may play an expanded role in formulary access decisions.

Relevance to the delivery of clinical care

Patients' dissatisfaction with treatment may act as an early warning of threats to the clinical effectiveness and efficiency of medical care. Patients who perceive their medication to be ineffective, laden with side effects, or very inconvenient to use are less likely to either fill prescriptions or take their medication as prescribed. This in turn can impact the effectiveness of treatment and may result in service inefficiencies associated with treatment failure. TSQM provides a unique opportunity to compare various medications used to treat a particular illness on the primary dimensions of treatment satisfaction. Routine assessment of patients' level of TS-M provides a way for clinicians to screen individuals whose current medication experiences may increase the risk of poor medication adherence. If collected from many patients, such information could foster a deeper consideration of patients' perspectives when evaluating the merits and drawbacks of various treatment alternatives.

As partial compensation for the potential drain that individualized assessments can place on already burdened clinical staff, the dimensionality of TSQM offers a set of convenient reference points to quickly focus patient-caregiver discussion on potential problem areas, thereby facilitating a corrective engagement process. For example, a better appreciation of patients' experiences with a particular medication's side effects or inconveniences may lead physicians to optimize dosing or review administration instructions with their patients. Alternately, if patients' dissatisfaction with the effectiveness of their medication does not seem to be clinically warranted, some patient education may be in order. This later point is particularly important when the therapeutic actions of a medication are not physically or mentally discernable by most patients (e.g., preventive treatments of hyperlipidemia or hypertension). In general, clinicians who show an active interest in patients' experiences are more likely to be seen as possessing good clinical skills and a genuine concern for patients' well-being [77].

Conclusion

Results from this initial validation study suggest that the TSQM is a psychometrically robust instrument, tapping the most important dimensions of patients' experiences with their medication. If carefully applied, the general nature of the instrument provides a way of evaluating and comparing patients' satisfaction with various types and forms of medications. Moreover, the TSQM may contribute to our understanding of patients' medication-related decisions and behaviors, thus proving TS-M to be both an important determinant and outcome of effective clinical care.

Authors' contributions

MJA, Principle Investigator, Project Director, Study Design & Planning, Psychometric Design & Analysis, Primary Authorship

AS, Study Coordinator, Second Authorship, Project Management, Literature Review, Qualitative Analysis, Data Management, Discussion Guide Design

SLH, Study Design & Planning, Contributing Author

SSC, Study Planning & Focus Group Facilitator, Design of Qualitative Methodologies, Discussion Guide Design

RNK, Literature Review, Contributing Author

MB, Initial Conceptual Framework Design, Contributing Author

CRR, Study Design & Planning, Contributing Author