The Psoriatic Arthritis Impact of Disease 12-item questionnaire: equivalence, reliability, validity, and feasibility of the touch-screen administration versus the paper-and-pencil version

Abstract

Background

Over the last few years, there has been a shift toward a more patient-centered perspective of the disease by adopting patient-reported outcomes. Touch-screen formats are increasingly being used for data collection in routine care and research.

Objectives

The aim of this study is to examine the equivalence, reliability, validity and respondent preference for a computerized touch-screen version of the Psoriatic Arthritis Impact of Disease 12-item (PsAID-12) questionnaire in comparison with the original paper-and-pencil version, in a cohort of patients with psoriatic arthritis (PsA).

Methods

One hundred and fifty-nine patients with PsA completed both the touch screen- and the conventional paper-and-pencil administered PsAID-12 questionnaire. Agreement between formats was assessed by intraclass correlation coefficients. Spearman’s rho correlation coefficient was used to test convergent validity of the touch screen format of PsAID-12, while receiver operating characteristic curve analysis was performed to test discriminant validity. In order to assess the patient’s preference, the participants filled in an additional questionnaire. The time taken to complete both formats was measured.

Results

A high concordance between the responses to the two modes of the PsAID-12 tested was found, with no significant mean differences. Intraclass correlation coefficients between data obtained for touch-screen and paper versions ranged from 0.801 to 0.962. There was a very high degree of correlation between the touch-screen format of PsAID-12 and composite disease activity indices (all at a P level <0.0001), Health Assessment Questionnaire, and Physician Assessment of disease activity. The discriminatory power of the touch-screen format of PsAID-12, assessed using the minimal disease activity – Outcome Measurements in Rheumatology Clinical Trials criteria, was very good, with an area under the receiver operating characteristic curve of 0.937 and a resulting cutoff value of 2.5. The touch-screen questionnaire was readily accepted and preferred. The mean time spent for completing the questionnaire on touch screen was 2 minutes and on paper was 2.7 minutes.

Conclusion

The touch-screen mode of administration of PsAID-12 can be a feasible and suitable alternative to the paper-and-pencil mode for the assessment of patients with PsA.

Introduction

Psoriatic arthritis (PsA) is a common chronic inflammatory joint disease. The prevalence of psoriasis is ~0.5% in the general population,1,2 and the prevalence of PsA among patients with psoriasis is reported aŝ6.2%–34.4%.3,4 PsA is a heterogeneous disorder, with an articular involvement quite distinct from rheumatoid arthritis (RA), including a possible combination of axial disease, peripheral arthritis, specific features such as enthesitis and dactylitis, as well as extra-articular manifestations in any individual patient. These protean characteristics of PsA, including many domains, are challenging for the clinical metrology of the disease, both from the patient’s perspective and from the physician’s perception.5–8 The core domains and instruments for use in clinical trials and in the care of PsA patients have been identified by the International Group for Research in Psoriasis and Psoriatic Arthritis (GRAPPA), and at the Outcome Measurements in Rheumatology Clinical Trials (OMERACT) meeting, and a preliminary work has been undertaken in developing a more comprehensive disease activity instrument for psoriatic disease.9,10 The European League Against Rheumatism supported the development of a new questionnaire and score to better assess the patient-reported outcomes (PROs) in this disease, in line with the existing similarly focused tools for patients with RA.11 Two Psoriatic Arthritis Impact of Disease (PsAID) questionnaires have been developed with both physical and psychological domains: one for clinical practice (12 domains of health) and one for clinical trials (nine domains).12 Developments in information and communication technologies have enabled the electronic delivery of self-administered survey questionnaires.13–15 However, there are disadvantages associated with the use of paper-and-pencil questionnaires;13,14 this modality tends to be time-consuming, presents a high risk of entry-related errors, has large data storage requirements, lacks security and flexibility, and is difficult to distribute across geographically dispersed users.15 Electronic modes of delivery could maximize both the speed and scalability of data collection, reducing its costs, without compromising the data quality.13–16 The use of touch-screen mode of data collection may affect the interaction among the questionnaire, the respondents, and the way of delivery, thus altering the properties of survey questionnaire responses.17 Therefore, the adaption of any questionnaire to a new mode of delivery should be accompanied by evidence demonstrating the equivalence between the two different modes,18 and this new mode of delivery needs to undergo scientific evaluation of the applicability in its intended setting, to assess if it meets the standard criteria of validity, reproducibility, and feasibility.19

In accordance with the OMERACT 7 recommendations,20 we seek to assess the measurement equivalence, validity, reliability, and feasibility of the paper-and-pencil and touch-screen Italian version of the PsAID 12-item (PsAID-12) questionnaire in patients with PsA in standard clinical care. To the best of our knowledge, the psychometric properties and feasibility of the touch-screen version of this questionnaire for patients with PsA have not been established yet.

Materials and methods

Patients

One hundred and fifty-nine patients aged ≥18 years with a diagnosis of PsA, defined by the Classification Criteria for Psoriatic Arthritis21 or by the Assessment of SpondyloArthritis international Society classification criteria22,23 in case of prevalent axial involvement, were enrolled in the study. The patients were grouped into subclasses according to disease pattern observed during their last visit of the study: monoarthritis, oligoarthritis (two to four joints affected), polyarthritis (five or more joints affected), and psoriatic spondylitis.24 Radiographs were scored for sacroiliitis according to the modified New York criteria.25 The anatomical region of the axial skeleton evaluated by magnetic resonance imaging (MRI) was chosen by both the rheumatologist and the radiologist after consensus, according to the patient’s symptoms.26 MRI of the sacroiliac joints was performed in 39 patients, and MRI of the sacroiliac joints plus the spine was performed in a further eight patients. Patients with active skin disease other than psoriasis that would interfere with the assessment of a target lesion, other active concomitant musculoskeletal diseases (such as gout or calcium pyrophosphate deposition), history of cancer or lymphoproliferative disease, uncontrolled diabetes, unstable ischemic heart disease, congestive heart failure, active inflammatory bowel disease, positive serology for hepatitis-B, and history of active tuberculosis were excluded. Additionally, those patients who fulfilled the classification criteria for fibromyalgia were excluded because the composite indices could be flawed. One hundred and forty-one patients (88.7%) with PsA were taking disease-modifying antirheumatic drugs, such as methotrexate, leflunomide, and sulfasalazine, and/or tumor necrosis factor inhibitors, namely, etanercept, adalimumab, golimumab, and infliximab. Fifty-two patients (32.7%) were additionally treated with low-dose corticosteroids (<10 mg/day of prednisolone or equivalent). Additional drug therapy included nonsteroidal anti-inflammatory drugs on an on-demand basis and analgesics, such as acetaminophen. Moreover, local skin treatment comprising corticosteroid preparations was administered in 61 patients (38.4%). Considering that it was not a randomized trial, drug therapy was chosen by the managing clinician as considered appropriate. All the patients were attending the outpatient and inpatient clinics of the Rheumatology Department of the Polytechnic University of Marche (Ancona, Italy), and they represented a “real-life” sample of PsA referred to our department. All subjects gave written informed consent to participate in the study, which was performed according to the criteria of the Declaration of Helsinki and approved by the local Institutional Research Ethics Committee (Comitato Unico Regionale–ASUR Marche).

PsAID-12 questionnaire

Two PsAID questionnaires have been developed with both physical and psychological domains: one for clinical practice (12 domains of health) and one for clinical trials (nine domains).12 The PsAID questionnaires have been developed, translated, and validated across several countries and are free of charge. Compilation is quick and straightforward, making it feasible and widely applicable.12 The longer questionnaire contains components for assessing 12 domains that are perceived by patients to be particularly important for their health, each based on a 0–10 numerical rating scale (NRS) and with a different weight. The PsAID-12 final value is obtained from the following: (PsAID1 [pain] NRS value [range 0–10] ×3) + (PsAID2 [fatigue] NRS value [range 0–10] ×2) + (PsAID3 [skin] NRS value [range 0–10] ×2) + (PsAID4 [work and/or leisure activities] NRS value [range 0–10] ×2) + (PsAID5 [function] NRS value [range 0–10] ×2) + (PsAID6 [discomfort] NRS value [range 0–10] ×2) + (PsAID7 [sleep] NRS value [range 0–10] ×2) + (PsAID8 [coping] NRS value [range 0–10] ×1) + (PsAID9 [anxiety] NRS value [range 0–10] ×1) + (PsAID10 [embarrassment] NRS value [range 0–10] ×1) + (PsAID11 [social life] NRS value [range 0–10] ×1) + (PsAID12 [depression] NRS value [range 0–10] ×1). The total score is divided by 20. The final score has a range from 0 to 10 (where 10 represents the worst health score). The PsAID scores had satisfactory psychometric properties in the international validation study.12 The patient-acceptable symptom state (PASS) cutoff was assessed in 274 patients and was found to be a PsAID-12 value ≤3.95. Thus, the proposed PASS cutoff is 4.12 A change of three or more points is considered a significant absolute change. This score can be used in clinical trials as a new composite index that captures relevant information in patients with PsA.

Key features of the touch-screen PsAID-12 questionnaire

We developed a multimedia touch-screen tablet application, denominated PsAID-12 touch-screen, according to the set of requirements for designing hand-held computer systems for electronic collection of patient’s diary and questionnaire data.15,16,18 PsAID-12 touch-screen app (software application for mobile devices) is a web single-page application, comprising multiple files relying on Javascript and HTML5 technologies. A single personal home page file is utilized for caching purposes only, enabling the web app to work offline. Designed to fit on medium (tablet)-to-large (PC) devices, PsAID calculator runs on every operating system (Windows, MacOS, Linux, Android, iOS) and modern browser (eg, Chrome, Safari, IE10+). To collect our data, we used a Lenovo Yoga Table 2 Pro’s 1.33 GHz quad-core Intel Atom Z3745 processor. The 12 questions were answered by touching one of the eleven radio buttons representing the NRS on the screen. The wording of the questions on the computerized touch-screen system was identical to the wording of the questions proposed for the paper-and-pencil format. Each patient was asked to complete both the versions of the questionnaire in a randomized order. There was a gap of 2 hours between filling the two formats in order to reduce the effect of memory on the subjects’ responses. Prior to proceeding with compilation of the electronic/computerized touch-screen questionnaire, all patients received a brief information/training session to familiarize them with both tablet or PC components and technical aspects for the responses. In addition, a real-time trained facilitator was available on the system to provide procedural assistance in case of need. To assess the patient’s acceptance and feasibility of computer-based questionnaires, the participants filled out an additional questionnaire. The patient’s acceptance was established by asking the following questions: (a) Is the touch-screen format easy to use? (b) Is the touch-screen format user-friendly? (c) In general, are you satisfied with using the touch-screen format? Finally, feasibility was evaluated by the time taken to complete the paper-administered format, which was recorded by a research assistant using a stopwatch, and the time taken to complete the touch-screen format, which was recorded by the software time registered on the computer.

Table 2.

Demographic and clinical characteristics of the study population and distribution of scores for each questionnaire (n=159)

Demographic and clinical variables	Mean	SD	Median	IQR
Age (years)	56.49	11.65	56.00	49.00–65.00
Educational level (years)	11.07	3.27	13.00	8.00–13.00
Disease duration (years)	8.40	5.21	8.00	3.00–12.00
Number of comorbidities	2.57	1.92	2.00	1.00–4.00
SCQ (range 0–39)	4.75	3.60	4.65	1.45–7.00
TJC (68 joints)	5.99	5.96	4.00	0.00–11.00
SJC (66 joints)	3.78	4.05	4.00	0.00–6.00
ESR (mm/h)	25.15	17.99	22.00	11.20–34.00
CRP (mg/dL)	3.56	3.37	2.22	1.02–5.51
Pain (range 0–10)	4.44	2.69	5.00	2.00–6.00
Dactylitis count (range 0–20)	2.01	2.28	1.00	0.00–4.00
LEI (range 0–6)	1.39	3.36	1.00	0.00–2.00
PASDAS (range 0–10)	4.44	1.77	5.00	2.98–5.74
DAPSA (range 0–164)	21.76	14.44	25.00	7.00–32.12
PASI (range 0–72)	5.36	5.08	4.40	1.12–8.47
PhGA (range 0–10)	3.98	2.70	5.00	1.00–6.00
HAQ (range 0–3)	1.07	0.70	1.12	0.50–1.62
SF-36 PCS (range 0–100)	38.87	10.32	36.29	31.63–46.61
SF-36 MCS (range 0–100)	44.29	12.26	42.22	33.4–54.58

Abbreviations: SD, standard deviation; IQR, interquartile range; SCQ, Self-administered Comorbidity Questionnaire; TJC, tender joint count; SJC, swollen joint count; ESR, erythrocyte sedimentation rate; CRP, C-reactive protein; LEI, Leeds Enthesitis Index; PASDAS, Psoriatic Arthritis Disease Activity Score; DAPSA, Disease Activity Index for Psoriatic Arthritis; PASI, Psoriasis Area and Severity Index; PhGA, Physician Assessment of disease activity; HAQ, Health Assessment Questionnaire; SF-36 PCS, Physical Component Summary score of the Medical Outcome Survey Short Form-36; SF-36 MCS, Mental Component Summary score of the Medical Outcome Survey Short Form-36.

Assessment of variables

A comprehensive questionnaire package, including sociodemographic data, quality-of-life items, and disease-related variables, was administered to the patients. The sociodemographic variables included age and sex. Disease-related characteristics included disease duration, comorbidity, and composite score used to evaluate the disease activity. We chose domains and instruments that had in general performed well in the previous studies and had been chosen by GRAPPA members and established at the various OMERACT conferences10,27 as being essential components of psoriatic disease documentation. These evaluations of PsA included the following domains: peripheral joint assessment – 68 joints for tenderness (68 tender joint count [TJC]) and 66 joints for swelling (66 swollen joint count [SJC]), patient-reported pain on an eleven-point NRS, physician and patient assessments of disease activity (Physician Assessment of disease activity [PhGA] and Patient Assessment of disease activity [PtGA], respectively) on an eleven-point NRS, patient’s general health status (on a 0–100 NRS), dactylitis – a simple dactylitic digit count was applied, enthesitis evaluated with the Leeds Enthesitis Index,28 physical function – as measured by the Health Assessment Questionnaire29 and by Physical (PCS) and Mental Component Summary (MCS) score of the Medical Outcome Survey Short Form-36 (SF-36),30,31 a measure of severity of psoriatic lesions, as evaluated by the Psoriasis Area and Severity Index (PASI),32 acute phase reactants (ie, C-reactive protein [CRP] and erythrocyte sedimentation rate), and rheumatoid factor measured by nephelometry. These variables were used to calculate fulfillment of the minimal disease activity (MDA) and remission criteria and all composite disease activity indices.9 We defined criteria for MDA by the following definition: a patient is classified as achieving MDA when he/she meets five of the following seven criteria: 68 TJC ≤1, 66 SJC ≤1, PASI ≤1, NRS for pain ≤1.5, PtGA score ≤2, HAQ score ≤0.5, and tender entheseal points ≤1.9 Finally, the presence of comorbidities was ascertained with the Self-administered Comorbidity Questionnaire (SCQ).33 The original SCQ lists 13 common medical conditions, including osteoarthritis and RA, as well as providing space to specify three optional health conditions in lay terms. An individual can receive a maximum of three points for each medical condition: one point for the presence of the problem, another point if he/she receives treatment for it, and an additional point if the problem causes a limitation in functioning. The SCQ produces a score between 0 and 39. We adapted this instrument by removing the item “rheumatoid arthritis” from the list of possible comorbidities and adding “inflammatory bowel disease”.

Composite disease activity indexes

In order to measure PsA disease activity, the Disease Activity Index for Psoriatic Arthritis (DAPSA)34 score and Psoriatic Arthritis Disease Activity Score (PASDAS)35 were calculated. We chose instruments that had in general performed well in the previous studies and had been chosen by GRAPPA members and established at the various OMERACT conferences as being essential components of psoriatic disease documentation.27,36 DAPSA was adapted from the Disease Activity Index for Reactive Arthritis, a score developed and validated for reactive arthritis. DAPSA was created from a clinical cohort34 and validated using clinical trial data.37 It comprises 68 TJC and 66 SJC, PtGA score, NRS for pain, and CRP level (in mg/dL). The composite score is a simple sum of the scores. The advantage of using the DAPSA is that values can be calculated at the time of the visit to the rheumatology clinic while the patient is being seen by the rheumatologist.38 PASDAS, a new composite index for measuring disease activity in PsA, was developed by multiple linear regression.35 Compared with existing indices, PASDAS is better able to discriminate between high and low disease activity of patients with PsA.35 The PASDAS comprises seven domains: evaluator and patient assessments of disease activity (PhGA and PtGA, respectively) on an eleven-point NRS, skin, peripheral joint counts, dactylitis, enthesitis, acute phase response, and SF-36 PCS.

Statistical analyses

Data recorded in the tablet or PC were transferred through a wireless local-area network to a computer and subsequently imported as an excel file into MedCalc^® version 16.0 (MedCalc Software bvba, Ostend, Belgium) for Windows XP for analysis. Paper data were entered manually into the database. We chose to calculate and display both parametric and nonparametric statistics for all questionnaires because not all data met the requirements of being normally distributed and/or continuous. To check for significant systematic differences between questionnaire versions, Wilcoxon’s signed rank test and intraclass correlation coefficients (ICCs) with 95% confidence intervals (CIs) for mean values were employed. ICCs >0.75 were considered. Agreement between scores was also illustrated by Bland and Altman plots, in which the difference between scores was plotted on the y-axis against the average of scores on the x-axis. For a reliability assessment of the touch-screen format scales of PsAID-12, internal consistency (Cronbach’s alpha) was calculated. According to Cicchetti, a value of 0.7 is usually regarded as acceptable.39 Items with item-total correlations <0.4 were rejected. The construct validity of the touch-screen version of PsAID-12 as a measure of health of patients with PsA was investigated in two ways. First, we explored the convergent validity of the questionnaire. Convergent validity examines the extent to which a particular measurement relates to other measurements that are believed to be assessing the same construct. In the absence of a true “gold standard” against which to assess criterion validity of the touch-screen PsAID-12, we compared this questionnaire with commonly used external measurements likely to reflect the impact of PsA. Thus, correlation between the PsAID-12 and the HAQ score or PCS and MCS scores of the SF-36, composite disease activity indices, and other clinical measurements (ie, patient’s pain, joint count, global assessment, and physician’s global assessment of disease severity) was measured. Spearman’s correlation coefficient was used to test convergent validity of the touch-screen format of PsAID-12. Correlation values ≥0.40 were considered satisfactory (rho =0.81–1.0 as excellent, 0.61–0.80 very good, 0.41–0.60 good, 0.21–0.40 fair, and 0–0.20 poor).40 Second, we evaluated discriminative performance in terms of the ability of the touch-screen format of PsAID-12 to distinguish between patients in different activity grades. To test this discriminant validity, we performed the receiver operating characteristic (ROC) curve analysis. Since ROC curve analysis requires external criteria to be dichotomous, the criteria for MDA were applied as external criteria.9 ROC curves were created by plotting the true-positive proportion (sensitivity) versus the false-positive proportion (100 – specificity) for multiple cut-off points for discriminating between the inactive and active patients. The area under the ROC curve (AUC) was calculated to quantify the discriminative accuracy. According to Swetz,41 AUCs from 0.50 to ~0.70 represent poor accuracy, those from 0.70 and 0.90 are “useful for some purposes”, and higher values represent high accuracy. From the ROC curves, the optimal cutoff point corresponding to the maximum sum of sensitivity and specificity was computed. The nonparametric Wilcoxon’s signed ranks test was used for calculation and comparison of the areas under the ROC curves. We also assessed overall patient acceptability and preference by calculating the proportion of patients stating either a preference for the paper version or no preference, along with a 95% CI. To calculate the proportions in each of the preference categories, we used the Fisher’s exact test for categorical variables. Finally, we examined the effect of the mode of administration on the time taken for completion of the two versions by the two-sample Student’s t-test.

Results

Patient characteristics

One hundred and fifty-nine patients (97 females, 62 males) fulfilling the Classification Criteria for Psoriatic Arthritis21 and Assessment of SpondyloArthritis international Society classification criteria22,23 were included in the analysis. The PsA cohort had oligoarticular disease (40.1%) or polyarticular disease (59.9%). A total of 18.2% of subjects with peripheral arthritis exhibited an isolated distal interphalangeal joints recruitment. In 43.4% of our patients, at least one enthesis involved had also been detected, while an exclusive spondylitis was documented in only 6.9% of our patients (Table 1).

Table 1.

Characteristics of patients in the study

Demographic and clinical variables	Patients with PsA
N of patients	159
Sex
Male, n (%)	62 (39)
Female, n (%)	97 (61)
Proportion with arthritis, n (%)	147 (92.5)
Oligoarthritis, n (%)	59 (40.1)
Polyarthritis, n (%)	88 (59.9)
DIP joint involvement, n (%)	29 (18.2)
Proportion with enthesitis, n (%)	69 (43.4)
Proportion with spondylitis, n (%)	11 (6.9)

Abbreviations: PsA, psoriatic arthritis; DIP, distal interphalangeal.

Table 2 summarizes the demographic and clinical characteristics of the study population as well as descriptive statistics of the distribution for the scales. These data are expressed as mean (standard deviation [SD] and median with interquartile range [IQR]). The mean age of the participants was 56.49 (SD 11.65) years ranging from 19 to 77. The mean duration of the disease was 8.40±5.21 years. Most subjects had more than one comorbid condition with a median number of 2 (range from 1 to 4). One hundred and forty-one patients (88.7%) with PsA were taking disease-modifying antirheumatic drugs and/or biological agents, and 52 patients (32.7%) with PsA were additionally treated with low-dose corticosteroids. Table 2 also outlines the descriptive statistics of all clinical variables, composite disease activity indices, and disability and health-related quality of life (HRQoL) scores.

The comparison of the demographic and clinical characteristics of the study population, carried out with the Wilcoxon’s signed rank test, did not show significant differences between female and male. The only statistically significant difference was observed in TJC and found to be higher in women (women 6.77 vs men 4.77, P=0.04) (Table 3).

Table 3.

Comparison of demographic and clinical characteristics of the study population and distribution of scores for each questionnaire in female and male

Demographic and clinical variables	Sex
	Female				Male
	Mean	SD	Median	IQR	Mean	SD	Median	IQR
Age (years)	55.67	12.01	55.00	48.7–65.25	57.77	11.04	60.00	51.00–65.00
Educational level (years)	10.78	3.26	13.00	8.00–13.00	11.53	3.27	13.00	8.00–13.00
Disease duration (years)	8.35	5.08	8.00	3.00–12.00	8.48	5.44	8.00	4.00–13.00
Number of comorbidities	2.51	1.82	2.00	1.00–4.00	2.67	2.09	3.00	1.00–4.00
SCQ (range 0–39)	4.65	3.74	4.40	1.30–7.16	4.92	3.38	5.45	1.85–6.95
TJC (68 joints)	6.77	6.22	8.00	0.00–11.25	4.77	5.36	2.00	0.00–8.00
SJC (66 joints)	3.92	4.349	4.00	0.00–6.00	3.56	3.56	3.00	0.00–6.00
ESR (mm/h)	25.71	18.05	22.00	12.00–34.00	24.29	17.99	18.50	10.00–36.00
CRP (mg/dL)	3.59	3.54	2.11	1.20–5.35	3.51	3.11	2.38	0.90–5.70
Pain (range 0–10)	4.43	2.53	5.00	3.00–6.00	4.46	2.94	5.00	1.00–7.00
Dactylitis count (range 0–20)	2.12	2.28	2.00	0.00–4.00	1.83	2.29	1.00	0.00–3.00
LEI (range 0–6)	1.58	4.16	1.00	0.00–2.00	1.09	1.37	1.00	0.00–2.00
PASDAS (range 0–10)	4.46	1.77	5.00	2.99–5.70	4.40	1.77	5.06	2.42–5.74
DAPSA (range 0–164)	22.66	14.57	26.37	9.25–32.91	20.34	14.24	21.10	5.47–31.68
PASI (range 0–72)	5.57	5.18	4.40	1.12–8.82	5.02	4.93	4.00	1.10–7.30
PhGA (range 0–10)	3.96	2.69	4.50	1.00–6.00	4.02	2.73	5.00	1.00–6.00
HAQ (range 0–3)	1.13	0.69	1.12	0.59–1.71	0.98	0.72	0.92	0.32–1.60
SF-36 PCS (range 0–100)	37.65	10.45	34.86	29.85–44.22	40.78	9.90	37.87	32.92–47.30
SF-36 MCS (range 0–100)	43.51	12.00	41.59	33.45–53.17	45.52	12.65	46.47	35.09–56.78

Abbreviations: SD, standard deviation; IQR, interquartile range; SCQ, Self-administered Comorbidity Questionnaire; TJC, tender joint count; SJC, swollen joint count; ESR, erythrocyte sedimentation rate; CRP, C-reactive protein; LEI, Leeds Enthesitis Index; PASDAS, Psoriatic Arthritis Disease Activity Score; DAPSA, Disease Activity Index for Psoriatic Arthritis; PASI, Psoriasis Area and Severity Index; PhGA, Physician Assessment of disease activity; HAQ, Health Assessment Questionnaire; SF-36 PCS, Physical Component Summary score of the Medical Outcome Survey Short Form-36; SF-36 MCS, Mental Component Summary score of the Medical Outcome Survey Short Form-36.

Central tendency and distributions PsAID-12 questionnaires

The composite scores of the items of PsAID-12 were not normally distributed (Kolmogorov–Smirnov test), and the distribution in all cases was a bimodal type, probably relating to the different types of cases enrolled (40.1% oligoarticular and 59.9% polyarticular). The medians (IQR) were as follows: paper-and-pencil format of PsAID-12, 3.60 (1.96–4.78); and touch-screen format of PsAID-12, 3.17 (1.93–4.54).

Agreement between paper-and-pencil and touch-screen format of PsAID-12

There was good comparability between paper-and-pencil and touch-screen scores. There were no significant differences between mean touch-screen and paper-and-pencil scores for all PsAID-12 items. The five highest scoring items (greater disease impact) were related to the following symptoms: pain, work and/or leisure activities, functional capacity, discomfort, and sleep disturbance (Figure 1).

Figure 1.

Spidergram comparison of the paper-and-pencil and touch-screen domains of PsAID-12 in the overall cohort of patients with PsA (n=159).

Note: The domain scores are plotted from 0 (best at the center) to 10 (worst at the outside).

Abbreviations: PsAID-12, Psoriatic Arthritis Impact of Disease 12 items; PsA, psoriatic arthritis.

Equivalence between the two modes of administration was also measured by calculating single-measurement ICCs between corresponding scales. These results are shown in Table 4. The ICCs ranged from 0.802 (“skin problems” domain) to 0.962 (“pain” domain) for all the domains in the PsAID-12, indicating excellent agreement for each domain between two modes. All scales met Cicchetti’s criterion of 0.75.39

Table 4.

Agreement between touch-screen and paper-and-pencil PsAID-12 scores assessed by ICC

PsAID-12 scales	ICC
1. Pain	0.962
2. Fatigue	0.849
3. Skin problems	0.802
4. Work and/or leisure activities	0.882
5. Functional capacity	0.907
6. Discomfort	0.808
7. Sleep disturbance	0.921
8. Coping	0.806
9. Anxiety	0.918
10. Embarrassment and/or shame	0.859
11. Social participation	0.890
12. Depression	0.897

Abbreviations: PsAID-12, Psoriatic Arthritis Impact of Disease 12 items; ICC, intraclass correlation coefficient.

Agreement between scores was also illustrated by Bland and Altman plots, in which the difference between scores was plotted on the y-axis against the average of scores on the x-axis. According to Bland and Altman analysis, there was no systematic error in paper-and-pencil and touch-screen scores of PsAID-12 (Figure 2).

Figure 2.

Agreement between scores obtained by the touch-screen and paper versions illustrated by Bland–Altman plots for PsAID-12 (1.96 SD limits of agreement, −0.60 to 1.04; mean difference, 0.22).

Notes: The graphs display differences between formats using the paper format as the gold standard for each individual. Horizontal line gives mean difference between formats. Ninety-six percent of the differences against the mean values were less than two SDs (dotted lines).

Abbreviations: PsAID-12, Psoriatic Arthritis Impact of Disease 12 items; SD, standard deviation.

Internal consistency of the PsAID-12 (touch-screen) health status subscales

Cronbach’s alpha as a measure of internal consistency of the scales for both the versions of the PsAID-12 questionnaire is presented in Table 5. Internal consistency ranged from 0.871 to 0.895. Cronbach’s alpha for the touch-screen PsAID-12 was nearly identical to the paper-and-pencil questionnaire (range 0.869–0.991) and exceeded 0.70 in all scales.

Table 5.

Reliability of the PsAID-12 touch-screen and paper-and-pencil subscales

PsAID-12 (touch screen) scales	Internal consistency of PsAID-12
	Touch-screen	Paper-and-pencil
1. Pain	0.885	0.876
2. Fatigue	0.880	0.866
3. Skin problems	0.895	0.870
4. Work and/or leisure activities	0.873	0.890
5. Functional capacity	0.871	0.872
6. Discomfort	0.874	0.875
7. Sleep disturbance	0.886	0.866
8. Coping	0.887	0.872
9. Anxiety	0.887	0.869
10. Embarrassmentand/or shame	0.895	0.869
11. Social participation	0.892	0.875
12. Depression	0.893	0.891

Abbreviation: PsAID-12, Psoriatic Arthritis Impact of Disease 12 items.

Convergent validity

There was a very high degree of correlation between the touch-screen format of PsAID-12 and composite disease activity indices (all at a P level <0.0001). The highest correlations were seen between touch-screen PsAID-12 and PASDAS (rho =0.667) and DAPSA (rho =0.644) (Figure 3A and B). In addition, touch-screen PsAID-12 had similar correlations with the HAQ and PhGA (rho =0.662 and 0.637, respectively). High correlations (P<0.0001) were also seen between the touch-screen PsAID-12 questionnaire and the SCQ (rho =0.485) (Figure 3C). Significant but less robust correlations (P<0.001) were found with SF-36 PCS and SF-36 MCS (rho =0.474 and 0.218, respectively) and other comparators, such as dactylitis count (rho =0.485), Leeds Enthesitis Index (rho =0.387), PASI (rho =0.251), and erythrocyte sedimentation rate/CRP (all at a P level <0.0001). The touch-screen PsAID-12 questionnaire showed no significant relationship with age, sex, or disease duration.

Figure 3.

Correlations between the touch-screen PsAID-12 and composite disease activity scores and comorbidity score.

Notes: (A) Scatter plot of PASDAS and touch-screen PsAID-12 values with a regression line. Each circle shows a single patient’s data. (B) Scatter plot of DAPSA and touchscreen PsAID-12 values with a regression line. Each circle shows a single patient’s data. (C) Scatter plot of SCQ and touch-screen PsAID-12 values with a regression line. Each circle shows a single patient’s data.

Abbreviations: PASDAS, Psoriatic Arthritis Disease Activity Score; PsAID-12, Psoriatic Arthritis Impact of Disease 12 items; DAPSA, Disease Activity Index for Psoriatic Arthritis; SCQ, Self-administered Comorbidity Questionnaire.

Discriminant validity

The ROC curves plotted to discriminate the ability of touch-screen PsAID-12, HAQ, SF-36 PCS, and composite disease activity indices to distinguish patients with active (Group A) and inactive disease (Group B) were similar. The discriminatory MDA power of touch-screen format of PsAID-12 was very good, without a significant difference, with an AUC of 0.937 (95% CI 0.898±0.975) (Table 6).

Table 6.

AUC-ROC values (standard error and 95% CIs), calculated to distinguish patients with active (Group A) and inactive disease (Group B), were similar for touch-screen PsAID-12, HAQ, SF-36 PCS, and composite disease activity indices

Disease activity, function and health status indices	AUC	SEa	95% CIb
CPDAI	0.941	0.0175	0.907–0.975
PASDAS	0.882	0.0425	0.799–0.965
DAPSA	0.993	0.0057	0.982–1.000
HAQ	0.928	0.0202	0.889–0.968
SF-36 PCS	0.847	0.0332	0.781–0.912
PsAID-12 (touch-screen)	0.937	0.0196	0.898–0.975

Notes:

^aHanley and McNeil.42

^bAUC ±1.96 SE.

Abbreviations: AUC, area under the curve; ROC, receiver operating characteristic; CI, confidence interval; PsAID-12, Psoriatic Arthritis Impact of Disease 12 items; HAQ, Health Assessment Questionnaire; SF-36 PCS, Physical Component Summary score of the Medical Outcome Survey Short Form-36; SE, standard error; CPDAI, Composite Psoriatic Disease Activity Index; PASDAS, Psoriatic Arthritis Disease Activity Score; DAPSA, Disease Activity Index for Psoriatic Arthritis.

From these data,42 we obtained the sensitivity and specificity for the possible threshold values, and we chose those with the highest diagnostic accuracy (minimal false-negative and false-positive results). The resulting cutoff value for touch-screen PsAID-12 was 2.5 (sensitivity 86.2%, specificity 91.7%) with an positive likelihood ratio of 10.3, when MDA-OMERACT were used.9

Acceptance and feasibility of touchscreen mode

Approximately 95% of patients reported that the touch-screen questionnaire was easy to use and 97% thought that the user interface was friendly, while 92% of patients stated that they liked using the touch screen to complete the questionnaire. Moreover, 84% of patients preferred the touch-screen mode to the paper-and-pencil format, and 13% of subjects had no preference. The mean time spent for completing the questionnaires on touch-screen was 2 minutes (95% CI for the mean, 1.71±2.21 minutes), and on paper, 2.7 minutes (95% CI for the mean, 2.25±2.88 minutes). The difference was significant (t-test =−3.18, P=0.002). Despite the presence of an instructor during the trial, no patients required any tutoring related to the touch-screen version of the questionnaire. Age and/or education exerted no impact on the differences between questionnaire versions; we found an unsystematic pattern of nonsignificant rho correlations ranging from 0.11 to 0.26 (P>0.05).

Discussion

Over the past decade, researchers have greatly developed the electronic data collection systems, platforms, or registries of electronic PROs.17,43,44 Self-administered questionnaires via tablet or PC are an easy and capable option in patients with rheumatic diseases for monitoring disease activity, efficacy, and safety of the treatment.45,46 Touch screens in clinics47 and hand-held computers known as personal digital assistants48 represent available technological tools that can be used for frequent disease assessments.49 Touch-screen technology has been used to develop a highly successful, efficient, and effective process for the routine collection of PROs in a busy, complex, and resource-depleted academic practice and in a typical private practice.11 The wireless connectivity capabilities of these personal mobile devices can enable the immediate transfer of data without temporal or geographical constraints.

We have shown for the first time that PsAID-12 performed equally well on paper and on touch screen in 159 patients in the waiting room as part of routine care in a busy clinic. According to guidelines of Cicchetti39 for interpreting Cronbach’s alpha, all the 12 scales demonstrated good (>0.80)-to-excellent (>0.90) internal consistency. However, as already mentioned, scales of the paper-and-pencil version showed a similar alpha coefficient in the current study and in earlier research.50 Thumboo et al suggested that the variability associated with computerized HRQoL assessment in clinical research in rheumatology is reduced. This evidence could result in smaller sample size requirements, with potential reductions in cost and in time recruitment when computerized tools are employed in clinical trials and cohort studies.51

Previously, we had already shown that HRQoL questionnaires using touch-screen technology are an acceptable approach for presenting information.15,16 Our present results are consistent with the findings of other studies and with the conclusions of systematic review and meta-analysis that showed equivalence and acceptance of electronic questionnaires in different areas of health assessment.43 Moreover, computerized assessments were preferred by patients14,45,46 and took no longer than paper versions.18,45,46,52,53 Further, agreement between computer and paper-and-pencil formats was independent of age or sex. Usually, computer preference is related to age but in a negative and gradual way. In a previous work, the youngest group was in favor of the PC-administered questionnaire, the middle-aged group had no preference, and the oldest group still preferred the traditional questionnaire. In an earlier research,53 it has already been shown that the older the individuals, the less comfortable they are with completing computerized questionnaires. However, it seems reasonable that in the future, computer usage by seniors will be as common as it is now by younger and middle-aged adults.

The construct validity of the questionnaire has been demonstrated by correlating touch-screen scores of PsAID-12 with all other composite indices. The correlation among scores obtained using the different disease activity measurements and other functional indices is very good, with the smallest rho correlation coefficient being 0.778. The touchscreen PsAID-12 questionnaire is also correlated with an SCQ at levels of P<0.0001.

Self-explanatory touch screens based on the Danish DANBIO registry generate valid results in ankylosing spondylitis and patients with RA on measurement of Bath Ankylosing Spondylitis Disease Activity Index, Bath Ankylosing Spondylitis Functional Index, HAQ, and visual analog scale scores for pain, fatigue, and global health when compared with the traditional paper form.50 We further explored the ability of the touch-screen PsAID-12 to discriminate between disease activity levels using ROC analysis. The AUC of the touch-screen PsAID-12 gives identical results to those provided by HAQ or other composite indices. The 2.5 cutoff points of touch-screen PsAID-12 correspond to fulfillment of the MDA-OMERACT. This observation provides clinically useful information, as it represents a stricter cutoff value compared to 4 for the PASS, calculated in the original validation work.12

Another goal consisted in determining individuals’ preference between both administration modes. The majority of subjects prefers completing the touch-screen questionnaire in comparison with the paper-and-pencil PsAID-12 questionnaire, which is in line with our previous study results.15,16 Compliance is a particularly important consideration for diary research. Stone et al, in a group of chronic pain patients, showed that protocol adherence with paper diaries was very low, whereas adherence with an electronic diary was excellent.54 Our results find that 95% of patients report that the touch-screen questionnaire is easy to use. This finding is similar to the results reported by Pouwer et al,55 and the time required to complete the questionnaire on the touch screen was slightly shorter than the time required for the paper versions, with an average of 2 minutes.

This study has some limitations. A primary limitation which must be emphasized is that repeatability of touchscreen PsAID-12 questionnaire has not been studied. A further potential limitation that has to be considered regarding the presented results is the nonrandomly selected primary care sample. It can be assumed that the motivation of the patient’s voluntary participation in a study differs in random population samples imputing to randomly ascertained patients a tendency to aggravate self-perceived severity. Finally, it has specifically explored the use of touch-screen technology in clinic patients with PsA where lack of familiarity with computers is an important factor to be considered in the design and evaluation of electronic questionnaires.

In conclusion, there are several advantages of the touchscreen computer format of the PsAID-12. First, the touchscreen format allows direct data entry and immediate display of results, which may improve patient monitoring in research and in clinical practice. Second, the touch-screen format is reliable across subgroups of patients, including the elderly and both sexes. Third, the majority of subjects in this study found the computer format easier to use, or as easy as the paper format, and among those who expressed a preference, approximately five times as many preferred the computer format over the paper format. Future applications exist for use of validated touch-screen versions of PsAID-12 questionnaire. These include web-based clinical and research applications, and the rapidly growing field of telemedicine.