Qualitative Interviews to Assess the Content Validity and Usability of the Electronic Raynaud Diary in Patients with Systemic Sclerosis

Abstract

Objective

To better understand the symptoms and impacts of Raynaud phenomenon (RP) in patients with systemic sclerosis (SSc) and to evaluate the content validity and usability of a new electronic patient‐reported outcome (PRO) measure for RP: the Raynaud Diary.

Methods

The Raynaud Diary was developed as a daily eDiary for assessing the number and duration of symptomatic Raynaud attacks; worst pain, numbness, tingling, and discomfort in the fingers; and overall disease severity, captured using the Raynaud's Condition Score. The Raynaud Diary was debriefed in two waves of qualitative interviews with adults with self‐reported RP secondary to SSc. All interviews included open‐ended questions about participants' experiences of RP.

Results

Participants (N = 39) had a mean age of 55.1 years, and 87% were female. Frequently reported RP symptoms were color change (reported by all participants), numbness (90%), tingling (82%), pain (77%), and discomfort (72%). Common attack triggers included temperature‐related factors and stress. Participants reported being unable to be outside or do outdoor activities and had problems gripping objects. All participants demonstrated understanding of the Raynaud Diary instructions. Most participants indicated that they would be able to use the Raynaud Diary to record the worst severity of individual RP symptoms in the previous 24 hours.

Conclusion

Patients with RP secondary to SSc bear a heavy symptom burden. The Raynaud Diary is a content valid PRO measure that captures the most frequent symptoms of RP in patients with SSc.

INTRODUCTION

Systemic sclerosis (SSc) is a rare, heterogeneous autoimmune rheumatic disease whose pathogenesis involves small vessel vasculopathy and autoantibody production, and it is associated with substantial morbidity, reduced survival, and poor quality of life (QoL) (1, 2, 3, 4). Raynaud phenomenon (RP)—ischemic digital episodes (or “attacks”) resulting from aberrant vasoconstriction of the cutaneous vessels involved in thermoregulation (5)—is the most common manifestation of vascular dysfunction in SSc. In SSc, RP often develops in people over the age of 30 (5, 6) and is a nearly universal feature of the disease, affecting more than 95% of patients (7, 8). RP secondary to SSc is generally more severe than primary RP (5) and is more common in women than in men (1).

Raynaud attacks typically occur with exposure to cold temperatures or stress. The distinctive features of a symptomatic Raynaud attack include pain, numbness, tingling, and discoloration of the digits. Attacks typically first affect one or more digits before spreading to all fingers of both hands. They end when the vasoconstriction is relieved and blood returns to the digits. Although the ischemia in RP is typically transient, RP in SSc can also reflect underlying vasculopathy, which can progress to ulceration or gangrene and amputation (5).

RP is associated with significant disability and a major psychological impact in patients with SSc. Patients experience significant anxiety, often expressing fears of ischemic digital ulcers and autoamputation (9).

Patient‐reported outcome (PRO) measures can be used to capture unique disease information not obtained through other clinical assessments. To our knowledge, the only disease‐specific PRO measure to be psychometrically validated for use in RP secondary to SSc is the Raynaud's Condition Score (RCS), which is used to assess the level of difficulty experienced due to RP each day (9, 10). However, as a nonspecific global assessment, the RCS is not intended to capture or distinguish between the different symptoms of RP. There is a clear need for a validated multi‐item PRO measure for evaluating RP secondary to SSc.

When developing a new PRO measure for a disease, it is important to ensure that the PRO measure adequately captures the experiences that are most important to patients. This requires a clear understanding of how patients experience the disease. Our understanding of how patients with SSc experience RP is currently limited by a paucity of qualitative research (11, 12).

We developed the Raynaud Diary as a new electronic instrument to be used for clinical trials in RP. The Raynaud Diary is a diary that incorporates elements of the RP attack diary developed by Merkel and colleagues (9)—including the RCS, frequency and duration of RP attacks, and frequency of going outdoors—and adds measures of severity for the following four key symptoms: pain, numbness, tingling, and discomfort. We then conducted qualitative interviews with patients with symptomatic RP secondary to SSc to better understand the symptoms and impacts of RP and to evaluate the comprehensibility and relevance of the Raynaud Diary items and response scales. It was also particularly important to assess the usability of the electronic device used to administer the Raynaud Diary in this population because of compromised hand function related to RP and SSc.

MATERIALS AND METHODS

Overall study design

The Raynaud Diary was refined through two waves of qualitative one‐on‐one interviews conducted with people with RP secondary to SSc (Figure 1). The first wave of interviews, which were conducted in person, assessed the content validity of the Raynaud Diary through concept elicitation and cognitive debriefing. Afterward, some diary questions were reworded and new questions were added, and the interview guide was revised accordingly. The second wave of interviews, some of which were conducted via telephone because of the COVID‐19 pandemic, combined concept confirmation and debriefing of new and modified diary items.

Figure 1.

Overall study design. IRB, institutional review board; PRO, patient‐reported outcome; RP, Raynaud phenomenon; SSc, systemic sclerosis.

Raynaud Diary

The Raynaud Diary is a daily symptom diary with a 24‐hour recall period. It includes the following questions assessing the number and duration of symptomatic Raynaud attacks: the number of times the participant went outdoors (to assess exposure to temperature changes); worst pain, numbness, tingling, and discomfort in the fingers, each assessed using a numeric rating scale (NRS); and overall disease severity (RCS). The RCS asks respondents to rate how much difficulty they had with RP “today” on an 11‐point NRS from 0 (no difficulty) to 10 (extreme difficulty) (9, 13), taking into account the number, duration, and severity of Raynaud attacks and how much RP affected their ability to use their hands. The user instructions for the Raynaud Diary defines a Raynaud attack as “at least one color change of your finger(s) (blue, white, and/or red) associated with pain, numbness, tingling, and/or discomfort of the finger(s).”

Participants

Participants were adults (≥18 years) with a self‐reported diagnosis of SSc and self‐reported RP attacks, as defined in the user instructions for the Raynaud Diary. They were recruited by convenience, snowball, and sampling. Advertisements were distributed in settings with high numbers of potentially eligible patients, including specialty clinics and patient advocacy groups. Interested patients contacted the study team directly using the toll‐free telephone number or email address provided in the study advertisement. They were then screened via telephone and scheduled for an interview.

Interviews

Interviews were conducted in fall/winter (September to February) by researchers trained in qualitative interviewing techniques using a semistructured qualitative interview guide (Supplementary Table 1). The first part of the interview for both waves included open‐ended questions about participants' experiences of RP symptoms. In the first wave of interviews, participants then completed the diary using a provided electronic PRO (ePRO) device (handheld tablet) and were subsequently cognitively debriefed on the diary to assess the understandability and relevance of the instructions, items, recall period, and response scales (Figure 1). They were also asked about usability of the ePRO device for completing the diary.

During the second wave of interviews, participants completed the Raynaud Diary using smartphones. Telephone interview participants used an application they had downloaded to their own smartphone before the interview. Participants in the telephone interviews were asked what their most bothersome and second most bothersome symptoms were.

Interviews lasted approximately 90 minutes and were audio recorded and subsequently transcribed. At the end of the diary debriefing, participants were asked whether it would be feasible to complete the diary at the same time each evening for 2 weeks (wave 1) or 6 to 7 weeks (wave 2). After completing the interview, study participants completed a questionnaire comprising questions on their sociodemographics and current health.

Analytic approach

Descriptive statistics were calculated for participants' sociodemographic and clinical characteristics.

In the qualitative analysis, one of the researchers first reviewed the interview transcripts for clarity and completeness and removed identifiable information. Concepts that emerged from the qualitative interviews were then analyzed using ATLAS.ti (version 8.3 or higher), which allows users to systematically conceptualize and compare qualitative data as concepts or “codes.”

Transcripts were coded by the constant comparative method (14). For each wave of interviews, the research team developed a coding dictionary based on the structure of the interview guide and entered it into ATLAS.ti for use in the analysis. The researchers involved in the coding of interview transcripts were trained to ensure agreement on the meaning and purpose of each code and to ensure consistency between coders. They then independently coded the first transcript. The coding was compared for consistency and was used to revise the coding dictionary. Once the coding dictionary had been finalized, the remaining transcripts were coded by the coding team. Finally, a senior researcher performed a quality review of all coded transcripts.

Ethics approval

An institutional review board (Advarra) granted exemptions for the in‐person interviews. Approval for the telephone interviews was received on September 8, 2020 (Advarra reference: Pro00029887). Each participant provided written informed consent before their interview.

RESULTS

Participants

Thirty‐nine people with self‐reported RP secondary to SSc participated in interviews: 20 in wave 1 (all in person) and 19 in wave 2 (seven in person, 12 via telephone). Mean (SD) participant age was 55.1 (13.0) years, and 87% of participants were female (Table 1). Median time since diagnosis of RP was 6 years (range 1‐43) and the most recent Raynaud attack had occurred a median of 1 day (range 0‐360) previously.

Table 1.

Self‐reported sociodemographic and clinical characteristics

Characteristics	N = 39
Age, y
Mean (SD)	55.1 (13.0)
Gender, n (%)
Female	34 (87)
Ethnicity, n (%)
Hispanic or Latino	3 (8)
Not Hispanic or Latino	36 (92)
Race, a n (%)
Asian	2 (5)
Black or African American	4 (10)
White	33 (85)
Other (not specified)	1 (3)
Highest education completed, n (%)
Secondary/high school	2 (5)
Some college	10 (26)
College degree	16 (41)
Postgraduate degree	11 (28)
Time since diagnosis of systemic sclerosis, y
Median (range)	5 (1‐32)
Scleroderma type, n (%)
Limited (or CREST syndrome)	10 (26)
Diffuse	20 (51)
Overlap	5 (13)
Don't know	4 (10)
Time since diagnosis of Raynaud phenomenon, y
Median (range)	6 (1‐43)
Time since last Raynaud attack, d
Median (range)	1 (0‐360)
Severity of Raynaud phenomenon in past 7 d, n (%)
Very mild	2 (5)
Mild	12 (31)
Moderate	16 (41)
Severe	8 (21)
Very severe	1 (3)
Digital ulcers within the past 6 mo, n (%)
Yes	16 (41)
No	23 (59)
Current treatments for Raynaud phenomenon, a , b n (%)
No current treatment	8 (21)
Prescription pharmacologic therapy
Phosphodiesterase 5 inhibitors	15 (38)
Calcium channel blockers	13 (33)
Antihypertensives	2 (5)
Prostacyclins	2 (5)
Analgesics	1 (3)
Botulinum toxin	1 (3)
Endothelin receptor antagonists	1 (3)
Immunosuppressants	1 (3)
Proton‐pump inhibitors	1 (3)
Selective serotonin reuptake inhibitors	1 (3)
Statins	1 (3)
Nonprescription pharmacologic therapy
Aspirin	2 (5)
Unnamed or generic ointments	2 (5)
Cannabidiol oil	1 (3)
Fish oil	1 (3)
Ibuprofen	1 (3)
Other (nonmedical) symptom management strategies
Hand warmers, warm water, gloves, staying warm	7 (18)
Essential oil massage	1 (3)
Finger supporters	1 (3)

Abbreviation: CREST, calcinosis, Raynaud phenomenon, esophageal dysmotility, sclerodactyly, and telangiectasia.

^a Not mutually exclusive.

^b Treatments were self‐reported by participants. There are no FDA‐approved pharmacologic therapies for Raynaud phenomenon.

Disease‐related terminology used by participants

Most participants used some variation of “Raynaud's” to refer to their condition. A smaller number of participants referred to their condition as just “scleroderma.” In describing their RP, many participants mentioned poor circulation, coldness, and color change as a result of specific triggers such as cold temperatures and stress. Some also mentioned other symptoms, including pain and numbness. Separate from their Raynaud symptoms, participants described their attacks using terms such as “attack,” “flare‐up,” and “episode.” Some participants referred to their attacks as “death hands” or “hands going dead.” Others did not have a specific term for their attacks.

Attack triggers

When participants were asked to identify what triggers their Raynaud attacks, changes in temperature (most commonly from warm to cold) were reported by all 39 participants (Table 2). This could be going outside in cold weather, entering an air‐conditioned building or the freezer aisle of a grocery store, touching something cold, touching or consuming a cold drink or cold food, or using the freezer or the fridge. Stress was another frequently reported trigger, reported by 19 participants (49%). Attack triggers varied both within and between participants.

Table 2.

Reported triggers for Raynaud attacks from the qualitative interviews

Trigger	Wave 1	Wave 2	Overall	Sample quotations
	(n = 20)	(n = 19)	(N = 39)
Temperature‐related triggers
Touching cold things	12 (60)	18 (95)	30 (77)	“…I can't hold a glass of water with ice in it” (001‐024)
				“…Sometimes it can be as simple as putting your hands into the freezer […] or even the refrigerator” (300‐016)
Cold weather or cold outdoor air	17 (85)	12 (63)	29 (74)	“The winter is worse” (300‐016)
				“…My hands will start to turn blue then white […] because I live in a cold climate” (300‐017)
Air conditioning or cold indoor air	14 (70)	8 (42)	22 (56)	“…If […] air conditioning blows on me or a fan [it] can trigger an attack” (001‐018)
				“…Sometimes the summer is worse too because you're going in air conditioning, out of air conditioning” (300‐016)
General changes in temperature (including from cold to warm)	3 (15)	3 (16)	6 (15)	“I mean I can walk into the grocery store, the freezer aisle and it's like instant” (002‐002)
				“…If I'm inside and it's 70 or something and I go outside and it's 60, that change although not huge is enough to trigger it” (001‐010)
All temperature‐related triggers	20 (100)	19 (100)	39 (100)
Stress and emotions (including anxiety and nervousness)	10 (50)	9 (47)	19 (49)	“…High stress is definitely a trigger, it's not always though…” (001‐002)
				“Stress brings it on…” (001‐024)
				“…Sometimes stress triggers it as well” (300‐001)
				“…If I'm a little stressed or anxious, [my fingers] will start to turn blue” (300‐017)
				“…Maybe extreme emotion. Like I can remember during my dad's death, at times I'd be like, wow, I didn't feel stressed out, so maybe it's emotion” (001‐002)
Other triggers
Eating or drinking certain foods or drinks (eg, caffeine)	3 (15)	2 (11)	5 (13)	“Drinking cold things or after eating.” (INTERVIEWER: After eating cold things?) “After eating anything, yeah.” (INTERVIEWER: You haven't noticed any certain foods trigger or…) “Cold drinks I do, but other than that or maybe caffeine because it's always my Diet Coke that it happens [laughs]” (INTERVIEWER: As far as food goes, it can be any temperature or any type of food? ) “Yeah, I guess so, but probably cold foods. I mean obviously if you're eating a hot bowl of soup it's not going to happen as much I guess.” (001‐021)
				“…But also caffeine can do it, yep, because of vasoconstrictor. So I usually like now in the winter, if I'm going to have coffee it's going to be decaf and avoid the chocolates.” (INTERVIEWER: Can you tell me more about that again, how the caffeine specifically can trigger an attack? ) “It's a vasoconstrictor, so since my vessels are already compromised, you avoid those types of things that are going to cause it to pipe down or get smaller and I avoid those.” (001‐020)
Lack of movement	1 (5)	1 (5)	2 (5)	“…If I'm stationary for too long. Like if I'm, if I'm just sitting or whatever and it's, it's been a long time, I, I feel that can sometimes bring it on…” (300‐026)
Physical pressure	1 (5)	0 (0)	1 (3)	“…Sometimes pressure will cause an attack if I'm doing something and there's a little bit of pressure…” (001‐007)
Exercise	1 (5)	0 (0)	1 (3)	“…Exercise strangely enough seems to make it worse” (001‐021)
Not eating or feeling hungry	1 (5)	0 (0)	1 (3)	“I do notice when I'm hungry it triggers it” (001‐014)

Note: Data are presented as n (%).

Attack frequency and duration

The reported frequency of Raynaud attacks varied considerably between participants, from one attack per day to several attacks per hour. Some participants reported having gone days or more than a week between attacks. Most participants (n = 28; 72%) described experiencing more attacks in the winter, but seven participants (18%) noted that any seasonal change could increase the frequency of attacks (“…any change of season, even spring into summer, any sort of change, especially fall into winter…” [participant 001‐024]). The reported duration of a typical attack ranged from less than a minute to several hours, with most participants (n = 22; 56%) reporting attacks lasting about 5 to 20 minutes. Almost all participants reported that the duration of their attacks varied from attack to attack. Participants reported different methods for shortening their attacks, including using heated pads and rubbing their hands together or placing them in warm water.

Endorsed symptoms occurring during Raynaud attacks

Table 3 shows sample quotations for the most frequently endorsed symptoms, and Figure 2 illustrates the words used during discussions of symptoms. All participants (100%) reported color change and at least one symptom. Frequently endorsed symptoms were numbness (90%), tingling (82%), pain (77%), and discomfort (72%).

Table 3.

Endorsed symptoms of Raynaud phenomenon secondary to systemic sclerosis

Symptom	Participants reporting the symptom, n (%) a	Most bothersome symptom, n (%) b	Second most bothersome symptom, n (%) c , d	Sample quotations
Color change	39 (100)	0	0	“So red and white is normally what it is, sometimes purple. There are certain fingers that tend to purple more than other ones…” (001‐024)
				“Definitely white first, sometimes blue and then red, but always white first” (001‐021)
				“I think white is the worst. It's purple first. They just start to turn blue and purplish. And then white after that” (300‐001)
				“It varies. Dark, you know, like purplish bluish, you know, that particular color, white, yeah, those are pretty much the colors that I experience.” (INTERVIEWER: Mm‐hmm, does it usually start off one color before changing for another? Or what have you noticed? ) “Um, I think it depends. Sometimes like last night, um, they would start off dark, you know, the bluish purplish color. So, um, I don't think I have ever really paid attention to the, you know, what color it starts first or whatever but mostly is dark” (300‐006)
Numbness	35 (90)	6 (46)	3 (30)	“…You feel like there's nothing there. Like when it gets extremely cold or something […], you can sit and poke it […] and there's nothing there, it just feels like a piece of dead skin that you're trying to wake up and it's not waking up” (001‐014)
				“…Numbness I feel in my opinion is losing feeling” (001‐023)
				“…I really feel like I have no control over what I'm doing with my hands…” (300‐015)
				“Um, I would say that the numbness is probably the most bothersome, um, just because it's hard to do everyday things when your hands are numb and you can't really grab things…” (300‐016)
Tingling	32 (82)	2 (15)	2 (20)	“The tingling is like little pins and needles…” (001‐013)
				“…The tingling is crazy, it's a nightmare, it's feeling like someone is stabbing you with 50,000 needles, it's aggravating” (001‐015)
				“…As if someone is kind of tickling your fingers…” (001‐023)
				(INTERVIEWER: … which of the symptoms that you experience during your Raynaud's is most bothersome for you? ) “The tingling.” (INTERVIEWER: And why do you say that? ) “It's distracting […] it takes my attention away from whatever else I'm doing” (300‐013)
				“…Sometimes there can be like tingling sensation, like, um, kind of like sparks […] like you get a zing…” (300‐016)
Pain	30 (77)	4 (31)	3 (30)	“Well, if I get pain it's pretty severe, it's a stabbing pain, sharp pains” (001‐002)
				“And it's that rush of warm blood […] that hurts, I mean seriously […] it felt like you were being stung just by bees” (001‐011)
				“…Not a severe pain, like a dull pain, not a sharp pain…” (001‐013)
				“…If you can imagine a labor pain…” (001‐018)
				“It's real achy I guess” (001‐023)
				“Well they're more sensitive when I touch something wrong. When I bump my fingers when they're in an attack, it seems hyper. The pain is a lot worse” (300‐015)
				“Um, it, it's kind of a constant throbbing pain. Um, and so sometimes it's a relief if the numbness—the numbness doesn't always set in” (300‐029)
				“I would say the tips of two of the ten fingers tend to be sensitive to the touch.” (INTERVIEWER: Sensitive to touch as in like painfulness, tingling? ) “Oh, excruciating pain…” (300‐031)
				“(INTERVIEWER: And what makes the pain [ … ] most bothersome … ? ) “…Your fingers just hurt and they feel like they're, um, being smashed, I guess. Like I'm going to describe that pain as, you know, someone probably slamming the door on your fingers” (300‐035)
Discomfort	28 (72)	1 (8)	1 (10)	“…I have discomfort in my hands almost all the time” (001‐002)
				“…That sort of burning sensation is quite uncomfortable” (001‐010)
				( INTERVIEWER: Would you say you experience pain along with the color changes? ) “I don't know if it's pain, but it's like your hands getting cold, like really cold. So I don't know if you call that pain, discomfort maybe is a better word” (001‐017)
				( INTERVIEWER: … Do you experience any pain before the sensation that the blood is coming back? ) “Not usually, it's more just very uncomfortable” (002‐002)
				“…It's an unpleasant feeling to have cold hands” (300‐027)

^a There was no limit on the number of symptoms participants could report.

^b Thirteen participants in the second wave of interviews (all 12 telephone interview participants and one of the in‐person interview participants) reported their most bothersome symptom.

^c Ten participants in the second wave of interviews reported their second most bothersome symptom.

^d Hands feeling cold was the second most bothersome symptom for one participant.

Figure 2.

Word cloud illustrating words used during discussions of Raynaud phenomenon symptoms during the qualitative interviews. The word cloud was created using ATLAS.ti and captures words used by both interviewers and patients. The more frequently a word was used, the larger it appears.

Color changes

When participants were probed about the finger color changes during an attack, the most frequently reported colors were combinations of white, purple, blue, and red. The sequence of color changes, and the parts of the fingers affected, varied between participants. Some participants also reported color changes in their toes and feet.

Numbness

Participants described numbness as a “dull coldness” or “loss of feeling.” Many participants characterized numbness as one of the first symptoms of an attack and as very debilitating, leaving them with “no control over what [they're] doing with [their] hands” (300‐015) and making it “hard to do everyday things” (300‐016).

Tingling

Participants described tingling as “tickling,” “pins and needles,” or “sparks.” Participants frequently associated it with blood circulation being restored to their hands: “…when those little blood vessels open back up and you get that rush of blood back in…” (001‐018). Tingling was often described as occurring with or directly prior to pain: “The tips of my fingers start to tingle and then they hurt” (300‐001).

Pain

Pain was described as “stabbing,” “burning,” “aching,” “throbbing,” or “stinging.” Like tingling, it was often associated with the end of an attack: “…the pain and tingling come when they're starting to warm up, when the blood is coming back” (001‐021). Some participants described the pain as very severe: “excruciating” (300‐031), like “someone […] slamming the door on your fingers” (300‐035); others reported it as being more mild: “…not a severe pain, like a dull pain…” (001‐013). Several participants reported that the pain could be relieved if they warmed their hands: “It will subside as soon as I restore warmth to the fingers” (300‐002).

Discomfort

Discomfort was characterized as “extreme sensitivity” or a “dull ache” and was often considered to be a milder form of other symptoms such as numbness or pain: “…I have discomfort in my hands almost all the time, but you learn to live with the pain level…” (001‐002). Some participants experienced discomfort as a cold feeling in their hands: “I don't know if it's pain, but it's like your hands getting cold, like really cold” (001‐017); “…It's an unpleasant feeling to have cold hands” (300‐027).

Distinguishing between symptoms

Participants in the wave 1 interviews were asked whether they could distinguish between certain pairs of symptoms. Although most participants could distinguish between pain and discomfort, describing discomfort as “annoying” but ultimately more bearable than pain, six (30%) reported that pain and discomfort were the same, in some cases because they occur at around the same time. Nearly all participants (n = 19; 95%) could distinguish between numbness and tingling.

Most bothersome symptoms

Thirteen participants in the second wave of interviews reported their most bothersome symptoms during Raynaud attacks. Numbness was reported as the most bothersome symptom by six participants (46%), pain by four participants (31%), tingling by two participants (15%), and discomfort by one participant (8%) (Table 3). One participant reported hands feeling cold as their second most bothersome symptom. No participants reported color change as their most bothersome or second most bothersome symptom.

Impacts of RP

Reported impacts of RP included being unable to be outside generally, especially during cold weather; inability to do outdoor activities, including winter sports; always needing to wear gloves or hand warmers (even in warmer weather); problems holding or gripping things; general loss of dexterity; and stigmatization. Individual participants described having to “give up their ice skating and […] winter sport activity” (300‐031) and being no longer able to “swim in the ocean” (300‐017). Another participant expressed frustration at always “having to remember to bring gloves, even when it's like 50 degrees” (001‐023). Numbness was a particular problem, leaving some participants unable to hold things: “I can't hold a pen, I can't hold a cup, I've dropped a lot of things because of [numbness]” (002‐002). One participant who reported problems holding things (“I drop everything”) also described a progressive loss of dexterity: “That has progressed, it used to be I couldn't open a [new] jar, now I can't open a jar that's even pre‐opened” (001‐024). For another participant, the impact of RP was comprehensive: “I mean think of everything you do with your hands, yeah, and it affects it” (001‐011). Finally, the color change during an attack could be stigmatizing: “Everyone notices it and they ask about it […] and I always have to explain it” (001‐023).

Cognitive debriefing of the Raynaud Diary

Instructions

All participants in the first wave of interviews demonstrated understanding of both the instructions to record their Raynaud attacks over the previous 24 hours and the provided definition of a Raynaud attack. However, three participants stated that the symptoms defining an attack should be more clearly emphasized. Therefore, the instructions were changed to emphasize that the Raynaud Diary should record “symptomatic” attacks. All participants in the wave 2 interviews demonstrated understanding of the instructions.

Attack frequency and duration

Most participants in the wave 1 interviews indicated that no changes were needed for the diary items relating to attack frequency (number of attacks in the previous 24 hours) and duration of each attack. Several participants indicated they might need to write notes throughout the day to help them remember the frequency or duration of their attacks, especially during the winter, when attacks are more frequent. However, overall, participants reported that answering these items as part of a daily diary would be feasible.

Number of times outdoors

Two participants in the wave 1 interviews indicated that it would be difficult to remember how many times they went outside in the previous 24 hours. Also, inconsistencies were noted in how participants interpreted and counted the number of times they “went outdoors.” One participant questioned whether it was relevant to record periods of time spent outdoors that are too short to trigger an attack. As a result, the item was modified to clarify that respondents should consider each new time they are exposed to the weather outside as a separate time of being outdoors. All participants in the wave 2 interviews demonstrated understanding of the modified item.

Worst symptom severity

In the Raynaud Diary, worst severity of pain, numbness, tingling, and discomfort in the previous 24 hours are each scored on an 11‐point NRS from 0 (no symptom) to 10 (extreme severity). Participants in the first wave of interviews only were asked to provide feedback on the NRSs. For pain, most participants indicated that they were capable of identifying the worst pain experienced across multiple attacks in the same day, although a few participants indicated that this would be slightly difficult. All participants indicated that they could identify the worst numbness experienced across multiple attacks in the same day. Seventeen participants (85%) stated that it would be easy to rate their worst tingling, and 19 (95%) said that it would be easy to rate their worst discomfort.

ePRO device usability

All participants in the wave 1 interviews stated they liked the tablet provided to complete the Raynaud Diary because it was easy or straightforward (n = 18; 90%) or because it was fast (n = 2; 10%). When asked to rate ease of use on a 5‐point scale, all 20 participants gave a rating of 1 (very easy). All participants in the in‐person wave 2 interviews indicated that the smartphones were easy to use.

Nine of the 12 participants who participated in telephone interviews (75%) rated ease of completing the diary as very easy and three (25%) as easy. One participant indicated that a larger screen (ie, tablet or laptop) would be preferable, because it can be difficult for people with RP to use small touchscreens.

Feasibility

Most participants believed that they would be able to complete the diary daily at approximately the same time in the evening for up to 2 weeks (wave 1) or 6 weeks (wave 2). However, six participants (32%) in the wave 2 interviews suggested changes to the administration schedule, such as recording attacks as they occur, having flexibility in when to complete the diary each day, and not having to answer all questions every day as some things (eg, medication) do not change from day to day.

DISCUSSION

Directly from the patient's perspective, this qualitative interview study gives insight into signs, symptoms, and impacts that are meaningful to patients with RP secondary to SSc. The concept elicitation data support the content of a novel Raynaud Diary; furthermore, the cognitive interview data support the content validity of the diary. The electronic usability of the diary was assessed and found to have been easy to use.

The symptoms most frequently reported by interview participants—color change, numbness, tingling, pain, and discomfort—confirm the face validity of the Raynaud Diary items. These symptoms affected their lives in various ways, interfering with their ability to spend time outside or do outdoor activities, making it difficult to hold or grip things, and causing stigmatization. Content validity of the Raynaud Diary was also confirmed. The diary items and response options were relevant and clear to patients, and no major issues were identified. Although some patients suggested that the diary should capture average severity rather than worst severity, it can be difficult for patients to determine an average over a period as long as 24 hours. Asking the patient to recall their worst severity is common when assessing clinical outcomes (15). Anchoring to worst severity was therefore ultimately considered the best approach for the diary.

Participants indicated that the diary instructions were clear and that the 24‐hour recall period was acceptable, although some participants explained that they might need to keep notes to ensure that their responses were accurate. Most participants reported that it was easy to complete the Raynaud Diary using the provided ePRO devices, or to download the app and complete the diary on their personal smartphone, and that it would be feasible to complete the diary daily for 2 weeks or even 6 weeks.

RP as a complication of SSc can have major impacts on QoL and functioning. In addition to the RCS (9, 10), various NRSs and visual analog scales (VASs) have been used to assess the severity and impact of Raynaud attacks (2, 9, 16, 17). These include the patient RPactivity VAS (9) and a VAS from the Scleroderma Health Assessment Questionnaire (17). The RCS and patient RPactivity VAS are approved by the Outcome Measures in Rheumatoid Arthritis Clinical Trials (OMERACT) initiative for use in clinical trials in SSc (18). However, single‐item measures such as these are inadequate for capturing the patient experience. Moreover, clinical experts in SSc have raised concerns as to whether the RCS captures aspects of RP as intended and the likely influence of extrinsic factors on RCS ratings (19). The Raynaud Diary is a new tool for assessing disease burden that incorporates the RCS, other elements of the RP attack diary developed by Merkel and colleagues (9), and measures of severity for four key symptoms (pain, numbness, tingling, and discomfort). It can potentially be used to help evaluate treatment efficacy in clinical trials in patients with SSc with symptomatic RP and may also be useful in clinical practice for monitoring patient experiences and identifying the need to start or change therapy.

Changes to the diary and interview guide and the mode of data collection (in person vs. via telephone) over the course of the study meant that not all variables were collected consistently across interviews. A further limitation is that participants self‐reported their diagnosis of SSc and their RP and other clinical characteristics, which were not clinically confirmed. However, the recruitment of participants in settings with high numbers of target patients, together with the way participants described their condition, gives confidence that they did have RP. The sample included some ethnic and racial diversity but was largely made up of White, not Hispanic or Latino, participants, which is a limitation of the study.

In conclusion, the Raynaud Diary (Supplementary Table 2) is a useable, content valid ePRO measure, and completing it daily using a tablet or smartphone app is feasible. The next steps in its development will be to measure its psychometric performance using data from phase 2 and phase 3 trials. Definitive conclusions about possible applications of the Raynaud Diary will depend on the results of this psychometric evaluation. The diary was assessed among participants with RP secondary to SSc. Should the diary be used in primary RP or RP secondary to a condition other than SSc, further research (eg, content validity and psychometric assessment) would be needed for that target population.

ACKNOWLEGMENTS

We thank the participants with systemic sclerosis who participated in this study

AUTHOR CONTRIBUTIONS

Domsic, Pokrzwinski, Stassek‐ study conception and design, aquisition of data, analysis and interpretation, and drafting and revising the article. Benton, Vampola‐ study conception and design, analysis and interpretation, and drafting and revising the article and full access to all the data. Oliver, Furst, Chung, Steen, Mayes, Shah, Molitor, Nagaraja‐aquisition of data, analysis and interpretation, and drafting and revising the article. Khanna study conception and design, aquisition of data, analysis and interpretation, and drafting and revising the article, and full access to all the data.

ROLE OF THE STUDY SPONSOR

The funding source(s) had role in the study design and in the collection, analysis, or interpretation of the data, the writing of the manuscript, or the decision to submit the manuscript for publication. Publication of this article was determined by the funding source and authors.

Supporting information

Disclosure Form

Appendix S1: Supplementary Information