Validity, reliability, and sensitivity-to-change properties of the psoriatic arthritis screening and evaluation questionnaire

Abstract

Psoriatic arthritis (PsA) is an inflammatory arthritis associated with irreversible joint damage in a subset of individuals. There is a need to screen early for this condition to prevent damage. To meet this need, we have developed the psoriatic arthritis screening and evaluation (PASE) questionnaire. The 15-item PASE questionnaire was administered to 190 individuals with either psoriasis or PsA. The PASE questionnaire was readministered to a subset of individuals with PsA in order to assess test–retest reliability and sensitivity-to-change. Receiver operator curves were constructed to optimize sensitivity and specificity for the diagnosis of PsA. Of the 190 participating in the study, 19.5% (37/191) participants were diagnosed with PsA. PASE total scores ranged from 15 to 74 (possible range, 15–75). The PsA group had a median Total score of 51 (25th and 75th percentile 44 and 57), and non-PsA group had a median total score of 34 (25th and 75th percentile 21 and 49) (p < 0.001). A PASE total score of 44 was able to distinguish PsA from non-PsA participants with 76% sensitivity and 76% specificity. Furthermore, 13 of the 15 items demonstrated significant test–retest reliability as assessed by Pearson correlation coefficient (r ≥ 0.5). PASE was sensitive-to-change with therapy; PASE scores were significantly lower for PsA individuals after systemic therapy (p < 0.034). The PASE questionnaire is a valid and reliable tool to screen for active PsA among individuals with psoriasis. PASE scores may be used as a marker of therapeutic response.

Introduction

Psoriatic arthritis (PsA) is an inflammatory arthritis associated with psoriasis that affects 11% of psoriasis patients [6, 19]. A delay in the diagnosis of PsA may result in an erosive arthropathy that permanently disables and deforms some individuals with this disease [7, 8, 14, 15]. An early diagnosis of PsA would prompt early use of disease modifying therapy and help to prevent this irreversible joint damage [1, 15]. Because psoriasis skin lesions usually precede the onset of joint symptoms by 10 years [4, 6, 11], and dermatologists manage 95% of psoriasis cases in the US [5], dermatologists are in an ideal position to screen individuals for PsA early.

In order to help dermatologists screen individuals with psoriasis for symptoms of inflammatory arthritis, we have developed the psoriatic arthritis screening and evaluation (PASE) questionnaire (Table 1) [13], a self-administered 15-item questionnaire. A previous pilot-study in a smaller population of individuals with psoriasis demonstrated that the PASE was able to distinguish PsA from non-PsA with 82% sensitivity. At least three other screening tools have been or are being developed to screen for PsA: the psoriatic arthritis questionnaire (PAQ) by Peloso et al. [2], the Toronto psoriatic arthritis screening tool (ToPAS) by Gladman et al. [9] and the psoriasis epidemiology project (PEST) (personal communication).

Table 1.

Psoriatic arthritis screening and evaluation (PASE) questionnaire; please circle or mark only one of the five choices on the following 15 questions. The answers to these questions will help us better understand your symptoms. This should take about 5–6 min to complete. Thank you for your time

Symptoms sub-scale	Strongly Disagree	Disagree	Neutral	Agree	Strongly Agree
1. I feel tired for most of the day	1	2	3	4	5
2. My joints hurt	1	2	3	4	5
3. My back hurts	1	2	3	4	5
4. My joints become swollen	1	2	3	4	5
5. My joints feel ‘hot’	1	2	3	4	5
6. Occasionally, an entire finger or toe becomes swollen, making it look like a ‘sausage’	1	2	3	4	5
7. I have noticed that the pain in my joints moves from one joint to another, e.g. my wrist will hurt for a few days then my knee will hurt and so on	1	2	3	4	5
SYMPTOM SCORE (Max 35)	Add scores for questions 1-7 and write in box A				A.
Function sub-scale	Strongly Disagree	Disagree	Neutral	Agree	Strongly Agree
8. I feel that my joint problems have affected my ability to work	1	2	3	4	5
9. My joint problems have affected my ability to care for myself, e.g. getting dressed or brushing my teeth	1	2	3	4	5
10. I have had trouble wearing rings on my fingers or my watch	1	2	3	4	5
11. I have had trouble getting into or out of a car	1	2	3	4	5
12. I am unable to be as active as I used to be	1	2	3	4	5
13. I feel stiff for more than 2 hours after waking up in the morning	1	2	3	4	5
14. The morning is the worst time of day for me	1	2	3	4	5
15. It takes me a few minutes to get moving to the best of my ability, any time of the day	1	2	3	4	5
FUNCTION SCORE (Max 40)	Add scores for question 8-15 and write in box B				B.
TOTAL PASE SCORE (Max 75)	Add scores in boxes A and B and write in box C				C.

Brigham and women's hospital has made no investigation and makes no representations or warranties, express or implied, as to the questionnaire. Brigham and women's hospital makes no representations or warranties of merchantability or fitness for any particular purpose or that the use of the questionnaire will not infringe any patents, copyrights, trademarks or other rights of any third party. in no event shall data generated by or conclusions drawn from use of the questionnaire be used for the provision of patient care. Brigham and women's hospital shall not be liable for any direct, indirect or consequential damages to licensee or any third party, or with respect to any claim by any third party, on account of or arising from use of the questionnaire

Although the initial pilot-study with the PASE tool was encouraging, there was a need to validate the tool in a larger population of participants. The purpose of this study is to provide a more comprehensive validation of the PASE questionnaire and evaluate other properties, such as test–retest reliability and sensitivity-to-change, after testing in a larger population of individuals with psoriasis.

Materials and methods

Study population

Institutional review board approval was obtained. Adults between the ages of 18–85 years with a diagnosis of psoriasis or PsA and who were able to read and understand English were eligible for the study. We did not exclude individuals with a concomitant diagnosis of other arthritides such as osteoarthritis (OA), gout, or rheumatoid arthritis. The gold standard for diagnosis of psoriasis was based on clinical evaluation by a board-certified dermatologist and the diagnosis of PsA and other arthritides was based on clinical evaluation by a board-certified rheumatologist and medical chart review. Specifically, the clinical diagnosis of PsA was determined using the Moll and Wright criteria, which include the presence of inflammatory arthritis (peripheral arthritis and/or sacroiliitis or spondylitis), psoriasis and (usual) absence of rheumatoid factor [16]. Rheumatologists assessed for the Moll and Wright criteria on the basis of the following: the patient's history and clinical exam, including tender and swollen joint count, the presence of dactylitis, and/or nail pitting, as well as history of morning stiffness, and review of radiographs when available [13]. A rheumatologist who employed the Moll and Wright criteria reviewed all cases to determine case from non-case. We were able to confirm that the majority of PsA cases were new. Existing cases of PsA had not received therapy. Recruitment took place at the Brigham and Women's Hospital dermatology clinic, arthritis clinic, and a dermatology–rheumatology combined clinic (Brigham and Women's Hospital, Boston, MA). A total of 194 individuals who were approached by study staff agreed to complete the PASE questionnaire. Sociodemographic data, psoriasis area and severity index (PASI) scores, and other clinical data were obtained from medical records and from the psoriasis and psoriatic arthritis follow-up study (PAFS), which is a hospital based registry study that collects detailed phenotypic data from individuals with psoriasis or PsA.

Questionnaire

A description of the instrument design and initial piloting of the PASE questionnaire has been published [13]. Briefly, an initial question pool was generated from a review of the literature, informal open patient interviews, and expert opinion from rheumatologists and dermatologists using a Delphi method. The initial development of the PASE included a stepwise approach including question generation, careful elimination of duplicate questions, pilot testing in a dermatology and rheumatology clinic, evaluation of reproducibility and reliability, and determination of the score indicative of rheumatology referral. PASE was designed for use in a busy clinical practice, for screening purposes, and for monitoring response to therapy. It was created using plain language for ease of readability and to reach as wide an adult audience as possible. The response scale is uniform throughout in an attempt to make it easy to understand and complete. The PASE questionnaire consists of 15 items divided into two subscales: a seven item Symptom scale and an eight item Function scale. Standardized response choices consist of five categories relating to agreement (strongly agree to strongly disagree). A response of “strongly disagree” is scored with one point, and “strongly agree” is scored with five points. The scale chosen and scoring assigned ensured that patients with a higher likelihood of PsA would score numerically higher than patients without PsA. A total score was calculated by summing responses to all 15 questions, and scores for function and symptom were calculated by summing the responses to corresponding questions. The total score ranged from a minimum of 15 points to a maximum of 75. Maximum symptom score was 35 and maximum function score was 40. Most individuals complete the PASE within 4–6 min and scoring takes no more than 1 min to complete. The PASE inquires about the subject's current health.

Analysis

The goal of this analysis was to evaluate further the ability of the total, function, and symptom scores in distinguishing PsA from non-PsA in a larger population. We also attempted to evaluate the PASE's test–retest reliability and sensitivity-to-change properties after treatment with systemic therapy. We used the following guidelines to score incorrectly completed questionnaires: if one item was left blank or unanswered, then we scored this item as zero and calculated the total score. Any participants with missing data on two or more questions were excluded from analysis.

Continuous measures were summarized by mean and standard deviation or by median and interquartile range where appropriate; categorical measures were summarized by frequency and percent. Wilcoxon rank sum tests were used to test for differences between the PsA and non-PsA groups for total, function, and symptom scores.

To determine the sensitivity and specificity of PASE to screen for PsA, we analyzed the responses of individuals who completed the PASE questionnaire before receiving systemic therapy. We chose to determine the sensitivity and specificity from this subset of participants because the PASE was designed to screen for active PsA rather than quiescent PsA secondary to disease remission. Receiver operator curves (ROC) were used to pick the best cut point for total PASE score to predict PsA.

To assess test–retest reliability we administered PASE twice (>2 weeks apart) to a subset of participants who had not received systemic therapy. Pearson correlation coefficients were used to assess the test–retest reliability of each PASE question. ICC statistic was also calculated to assess the stability of the overall PASE questionnaire.

To test sensitivity-to-change, we re-administered PASE to a subset of participants after initiation of systemic treatment for their PsA. We conducted a medical record review to confirm that a repeat clinical assessment by a board-certified rheumatologist was performed to measure therapeutic improvement. Paired t tests were calculated to determine whether participants' PASE scores differed before and after receiving systemic therapy. We also attempted to identify the domains in which patients reported the greatest improvement after receiving systemic therapy.

Results

Study staff administered PASE questionnaires to 194 participants in patient rooms while waiting to be seen by their physician. We excluded four responses from analysis due to missing data. Responses from the remaining 190 participants (Table 2) were included in this study. Of the 190 participants in our study, we have PASI scores on 63 (33%) participants. Of these 63 individuals, the PASI scores ranged from 0 to 47.6 with a median of 9.6 and mean of 12.8.

Table 2.

Characteristics of the study population

Description	Category	N	(%)
Gender
	Male	110	57.8
	Female	80	42.1
Race/ethnicity
	Caucasian	138	72.6
	African–American	8	4.2
	Hispanic	3	1.6
	Asian	1	0.5
	Multiracial	3	1.6
	Other	25	13.2
	Unknown	12	6.3
Income
	0-$20,000	39	20.5
	$20,000–$50,000	30	15.8
	$50,000–$100,000	36	19.0
	Over $100,000	22	11.6
	Unknown	63	33.2
PsA diagnosis
	Non-PsA	153	80.5
	PsA	37	19.5
Co-morbid conditions
	Rheumatoid arthritis	7	3.7
	Gout	13	6.8
	Osteoarthritis	29	15.3

PsA psoriatic arthritis

Validity

The scores of PsA patients significantly differed from the scores of patients without PsA (Table 3). In the PsA group, the median total score was significantly higher than the median Total score of the non-PsA group. Similarly, median function and symptom scores in the PsA group were significantly higher than the median function and symptom scores of the non-PsA group. In summary, all three scores were significantly higher for participants in the PsA group compared to the non-PsA group. We also compared the PASE scores of participants with OA (non-PsA) to the scores of the PsA group (non-OA) (Table 4). We excluded six participants with PsA from this analysis because they also had OA. Except for the functional score, the scores of participants with OA (non-PsA) were significantly lower than the scores of participants with PsA (non-OA).

Table 3.

Median PASE scores for PsA and non-PsA patients

Factor	Non-PSA		PSA		P*
	N	Median (25th, 75th percentiles)	N	Median (25th, 75th percentiles)
Functional score	153	17(11, 25)	37	26(22, 30)	<0.001
Symptom score	153	17(10, 24)	37	24(23, 27)	<0.001
Total score	153	34(21, 49)	37	51(44, 57)	<0.001

PASE psoriatic arthritis screening and evaluation tool; PsA psoriatic arthritis

^*Wilcoxon rank sum test

Table 4.

Median PASE scores for osteoarthritis and psoriatic arthritis patients

Factor	Non-PSA with OA		PSA (no OA)		P*
	N	Median (25th, 75th percentiles)	N	Median (25th, 75th percentiles)
Functional score	23	22.0 (16.0, 27.0)	31	26.0 (28.0, 32.0)	0.082
Symptom score	23	19.0 (14.0, 25.0)	31	24.0 (21.0, 29.0)	0.014
Total score	23	41.0 (30.0, 52.0)	31	53.0 (38.0, 58.0)	0.039

OA Osteoarthritis, PASE Psoriatic arthritis screening and evaluation tool, PsA psoriatic arthritis

^*Wilcoxon rank sum test

The total PASE scores ranged from 15 to 74 in this study, and a total PASE score of 44 was determined to be the optimal cut point for distinguishing PsA from non-PsA in all participants. At this cut point, the sensitivity was 76% [95% confidence interval (CI), 59–88%] and specificity 76% (95% CI, 68–82%); 76% of participants diagnosed as having PsA would be classified as having PsA by a PASE total score ≥44 (Fig. 1a). Our previous pilot-study found that a total PASE score of 47 was the optimal cut point for distinguishing PsA from non-PsA. In this study, a cut point of 47 gives a sensitivity of 70% (95% CI, 53–84%) and specificity 80% (95% CI, 73–86%). As mentioned in our previous pilot-study, PASE scores may be low in individuals with no active symptoms. Hence in a subanalysis, we excluded ten participants whose PsA was either quiescent or asymptomatic (based on rheumatological evaluation) in order to determine the sensitivity and specificity of the PASE tool amongst those with active symptoms only. In these remaining 180 individuals, a total PASE score of 47 gives a sensitivity of 93% (95% CI, 78–99%) and specificity of 80% (95% CI, 73–86%) (Fig. 1b).

Fig. 1.

a ROC curves for PASE total, symptom, and function scores of all participants. Total PASE AUC = 0.797, symptom score AUC = 0.814, function score AUC = 0.759. b ROC curves for PASE total, symptom, and function score of all participants with active psoriatic arthritis only. Total PASE AUC = 0.884, symptom score AUC = 0.876, function score AUC = 0.870

The PASE questionnaire missed nine participants with PsA because their total PASE score was below the 44 score cutoff for PsA. Of these nine participants, four had limited disease, two had quiescent disease, one had axial involvement, one participant received multiple intraarticular injections 10 days prior to PASE administration and another participant had been off systemic therapy for 5 months but began flaring at the time of PASE administration. Another 37 participants were screening test-positive for PsA but did not have the disease. Of these 37 participants, 18 had a history of other musculoskeletal conditions such as severe osteoarthritis/degenerative joint disease, spinal stenosis, carpal tunnel syndrome, chondromalacia, muscle strain, and muscle sprain. Another seven participants had undifferentiated arthritis, four had gout, two had fibromyalgia, one had peripheral neuropathy, one had spondyloarthropathy, and one had lupus. We did not have access to the medical records of the three remaining individuals.

Reliability

We administered PASE twice (>weeks apart) to a subset of 23 participants who had not received systemic therapy. Pearson correlation coefficients were used to assess the test–retest reliability of each PASE question. Correlation coefficients ranged from 0.35 to 0.80. All questions but two significantly correlated between the two time periods (P < 0.05) with some questions performing better than other questions (Table 5). For example, symptom questions four (joints swollen) and six (finger and toes like sausages) and function question four (trouble getting into and out of car) had the highest reliability, whereas function question two (joint problems affected ability to care for myself) had the lowest reliability. The ICC for the total PASE score was 0.90, which indicates acceptable test–retest reliability of the entire PASE questionnaire.

Table 5.

Test–retest reliability for each PASE question

Question	Description	Pearson's r	P
SQ1	Feel tired	0.41	0.0562
SQ2	Joints hurt	0.71	0.0002
SQ3	Back hurts	0.72	<0.0001
SQ4	Joints swollen	0.80	<0.0001
SQ5	Joints hot	0.60	0.0030
SQ6	Fingers/toes swollen	0.76	<0.0001
SQ7	Pain moves joint to joint	0.57	0.0061
FQ1	Work	0.61	0.0028
FQ2	Self-care	0.35	0.1090
FQ3	Wearing rings	0.70	0.0003
FQ4	Getting into car	0.76	<0.0001
FQ5	Not active	0.67	0.0006
FQ6	Morning stiffness	0.66	0.0009
FQ7	Morning worst time	0.57	0.0054
FQ8	Few minutes to get moving	0.69	0.0004

PASE psoriatic arthritis screening and evaluation; SQ symptom question

FQ function question

Sensitivity-to-change

In order to assess sensitivity-to-change, a total of 24 (13%) of the 190 participants completed the PASE questionnaire after receiving systemic therapy with either methotrexate or a biologic agent. Of these, 13 had psoriasis and 11 had PsA. A board-certified rheumatologist performed a repeat clinical assessment of the participants with PsA in order to measure therapeutic response in all cases except one. The mean change in PASE score was calculated in each group (Fig. 2). In the PsA group, post-treatment PASE scores were significantly lower than pre-treatment scores (P = 0.034). In the non-PsA group, post-treatment PASE scores did not differ significantly from pre-treatment scores (P = 0.181). The greatest improvement in scores was reported for function question four (trouble getting into and out of a car) and function question eight (few minutes to get moving) (data not shown).

Fig. 2.

Sensitivity-to-change of the PASE after median time of 19 weeks. Mean (SD) score reduction after systemic treatment for PASE total score, function domain score, and symptoms domain score. A total of 24 participants were administered the PASE before and after treatment with systemic therapy. The bars show a greater reduction in PASE scores within the psoriatic arthritis group compared to the non-psoriatic arthritis group

Discussion

We have shown that the PASE questionnaire can distinguish active PsA from non-PsA with high sensitivity and specificity. Furthermore, PASE has excellent test–retest reliability and monitors an individual's improved response to therapy. Other published screening tools for PsA have demonstrated a lower sensitivity and specificity [2], with the exception of the ToPAS instrument, and do not track an individual's response to therapy [2, 9]. Both the PAQ and PASE questionnaires were designed to screen for PsA among individuals with psoriasis. The ToPAS questionnaire, however, was designed to screen for PsA in any population, not just amongst individuals with psoriasis. Furthermore the ToPAS has been validated in multiple settings such as in family medicine, general rheumatology, general dermatology, psoriasis clinics, and PsA clinics. The PASE questionnaire measures function and can track an individual's response to therapy, whereas the other questionnaires only attempt to classify patients as having PsA [17]. This validation study in a larger population improves upon our previous work that demonstrated PASE helps identify inflammatory arthritis among psoriasis patients [13].

The PASE questionnaire fulfills many properties of an effective screening tool. PASE is brief and self-administered. It is free of discomfort or risk and performs a convenient musculoskeletal review of systems for dermatologists. It increases the level of PsA detection, which may help curb high healthcare costs associated with missed PsA cases [3, 10, 12]. The PASE also helps clinicians decide who among many psoriatic individuals with arthralgias will benefit from prompt rheumatologic referral and treatment. PASE may not detect treated PsA or PsA that is in remission. Also, PASE serves only as a screening tool for PsA and does not substitute for a thorough examination by a rheumatologist.

In terms of limitations, all participants were recruited from a tertiary care referral center. The PASE could benefit from further validation in a non-tertiary care referral setting, such as a community clinic. We also acknowledge the limited data for sensitivity to change. We only had a small number of participants on whom responsiveness data were available. Despite these limitations, we believe the PASE will help clinicians screen for PsA among psoriatic individuals who present with symptomatic joint pain. Specifically, those individuals who score high on the PASE (i.e. total score ≥44) will most likely benefit from a rheumatology referral.

Many challenges were inherent in designing a tool that screens for a chronic disease. PsA is a condition that may be difficult to diagnose, hence, sensitivity and specificity are considered optimal for a tool such as PASE. In an effort to reduce reporting bias, we required every patient with PsA to be newly diagnosed prior to initiation of systemic therapy. In order to assess sensitivity-to-change we had to meet the challenge of tracking a subset of participants, re-administering PASE after treatment with systemic therapy, and performing a repeat clinical examination by a rheumatologist. Furthermore, the gold standard for diagnosis of PsA was a rheumatologic evaluation using the Moll & Wright criteria; as such, this study would be difficult to conduct outside of a combined dermatology-rheumatology clinic.

The 15-item PASE was designed to be a self-administered and easy-to-use screening tool that dermatologists may use to screen psoriasis subjects for PsA in a busy community dermatology practice. This study is the second part of a multi-stepped process to evaluate the performance and properties of the PASE. PASE has already been translated into several other languages beside English; however, these questionnaires have not been tested. Future validation efforts will include a factor analysis to determine the usefulness of the symptom and function subscales.

Administration of a well-designed and validated screening tool can increase detection of PsA in psoriasis patients, determine the prevalence of PsA in a given population; capture clinical data for genotype–phenotype studies [18], and monitor response to therapy. We have shown that the PASE questionnaire is a valid and reliable tool to screen for active PsA among individuals with psoriasis and that it may be used as a marker of therapeutic response after systemic therapy.