Refining the Stroke Risk Screening Scales (SRSS) Through Two-Round Cognitive Interviews with Stroke Survivors and Healthcare Providers in Thailand

Abstract

Background

Accurate stroke risk screening requires instruments that are both clinically valid and easily understood by end users. Although the Stroke Risk Screening Scales (SRSS) demonstrated strong content validity, challenges related to item interpretation and response processes may compromise measurement accuracy. This study aimed to examine these barriers and identify strategies to enhance the clarity, accuracy, and usability of the SRSS.

Methods

A qualitative cognitive interviewing approach was conducted with 30 purposively sampled participants, including 15 individuals with stroke and 15 healthcare providers. Each participant completed two interview rounds, with the same 30 participants re-interviewed in Round 2 (conducted within one month of Round 1) to confirm whether revisions made after Round 1 addressed the barriers identified, yielding a total of 60 interviews. Data were collected using a semi-structured verbal probing guide to explore participants’ understanding, reasoning, and response processes across 30 SRSS items. Interviews were audio-recorded, transcribed verbatim, and analyzed using inductive thematic analysis based on Braun and Clarke’s framework. Rigor was ensured through independent coding, iterative codebook development, triangulation, and maintenance of an audit trail.

Results

Two overarching themes emerged: (1) interpretation and response barriers, including technical terminology, ambiguous measurement units, recall difficulties, misaligned response options, lack of skip logic, and cultural mismatches; and (2) evidence-based refinement strategies to improve usability. Participants recommended simplifying terminology, incorporating descriptive examples, aligning question stems with response options, implementing skip logic, and standardizing measurement units and thresholds. These findings highlighted critical gaps between clinical terminology and lay understanding that may affect response accuracy.

Conclusion

Cognitive interviewing identified key interpretive and structural limitations in the SRSS and provided actionable, user-informed strategies for refinement. These findings offer preliminary response-process evidence supporting the interpretability and usability of the SRSS among stroke survivors and healthcare providers, and highlight the importance of user-centered pre-testing in the development of screening tools for diverse populations. Further psychometric evaluation in larger and more diverse samples is needed to establish the instrument’s reliability, validity, and clinical applicability.

Background

Stroke remains a leading cause of death and disability worldwide, with disproportionately high incidence and mortality in low- and middle-income countries.^1–3 Global trends indicate increasing prevalence across age groups, including younger populations, particularly in resource-limited settings where prevention and early detection remain suboptimal.^1,2 Despite recognition of modifiable stroke risk factors, current stroke screening and primary prevention practices are insufficient to adequately identify individuals at elevated risk.^4,5 Strengthening early detection through effective screening tools is therefore essential to reducing stroke-related morbidity and mortality.

As stated in previous study,⁶ existing stroke risk assessment tools, including the Framingham 10-Year Risk Score, the Stroke Riskometer™, QStroke, the My Risk Stroke Calculator, and the Stroke Risk Screening Tool developed by Thailand’s Department of Disease Control, are often limited by their inability to integrate a comprehensive array of cardiovascular and neurological risk factors, which results in estimation gaps for diverse groups, underscoring the need for a screening instrument that is both inclusive and culturally appropriate.^6–9

In response, Stroke Risk Screening Scales (SRSS) were developed as a comprehensive tool integrating 18 domains of stroke risk, encompassing both non-modifiable factors (eg, age, sex, race) and modifiable factors (eg, hypertension, diabetes, smoking, physical inactivity); the complete domain list is provided in the original domain-identification study⁶ across 30 items (as of current version 1.3), intended for interviewer- or self-administration in both community and hospital settings to identify individuals at elevated stroke risk; scoring and risk-classification thresholds will be established in the subsequent psychometric validation phase of this instrument’s development. Phase 1.1 identified relevant domains and generated the initial items,⁶ while Phase 1.2 established strong content validity through expert evaluation, with all 30 items demonstrating acceptable item-level content validity (I-CVI ≥ 0.78) and an average scale-level CVI of 0.95.¹⁰ These findings support the conceptual and content adequacy of the SRSS; however, technical accuracy does not always equate to respondent clarity. Instrument development best practices mandate that draft items undergo rigorous pre-testing with the target population to ensure that clinical constructs are interpreted as intended.¹¹

Cognitive interviewing is a critical methodological step in instrument development, enabling researchers to examine how respondents interpret, process, and respond to draft survey items.¹¹ This process generates response-process validity evidence, a foundational form of validity evidence in contemporary measurement frameworks, by revealing whether items are understood and answered as intended, and represents an essential early step in the broader validation pathway that precedes subsequent psychometric testing of reliability, factor structure, and predictive validity for a new risk screening scale. This approach helps uncover barriers to comprehension, ambiguous or misleading wording, and response options that may not align with participants’ lived experiences. By eliciting participants’ thought processes, cognitive interviews provide insight into how well each item functions and whether it captures the intended construct. These insights are essential for identifying problematic items and informing precise strategies to refine and strengthen the instrument before field testing. Furthermore, cultural and contextual mismatches, such as utilizing unfamiliar measurement units or complex multi-clause sentences, can compromise data quality and lead to inaccurate risk profiles.

While the SRSS has demonstrated high content validity among experts, its usability and interpretability among stroke survivors and frontline healthcare providers have not yet been established. Stroke survivors were included in this pre-testing phase because many SRSS items ask directly about prior stroke, transient ischemic attack, and stroke-related symptoms; individuals with this lived clinical experience are uniquely positioned to judge whether such items are worded in a way that accurately reflects their actual diagnostic and symptom history, a judgment that respondents without this experience could not meaningfully make for these items.

Extending cognitive interviewing to community members without a stroke history, the eventual target population for primary prevention screening, represents an important next step, discussed further in the Study Limitations. A two-round design was adopted to allow initial barriers to be identified in Round 1 and to verify, in Round 2, whether subsequent revisions resolved those barriers without introducing new ones. This study utilizes cognitive interviewing to look beyond the “what” of respondent answers to the “how” and “why” of their thought processes. By identifying specific linguistic, conceptual, and structural barriers, this research provides the evidence-based refinements necessary to ensure the SRSS is a clear, accurate, and user-friendly tool for clinical practice.

Objective

To identify barriers to item interpretation and develop evidence-based strategies for refining the Stroke Risk Screening Scales (SRSS) through a qualitative cognitive interviewing study with 30 participants (60 interviews), focusing on enhancing the instrument’s clarity, cultural relevance, and usability.

Methods

Study Design

This qualitative study employed cognitive interviewing to examine barriers and strategies for improvement in the development of SRSS. Cognitive interviewing was selected because it allows researchers to explore how respondents interpret and respond to draft survey items, revealing issues related to comprehension, wording, recall, judgment, and response formulation. This approach is consistent with recommended pretesting procedures for newly developed health measurement tools.¹¹

Participants and Recruitment

A total of 30 participants were recruited from Faculty of Medicine Siriraj Hospital, Mahidol University, Bangkok, Thailand, via purposive sampling to ensure a diverse range of perspectives relevant to stroke risk. Stroke survivors were identified through the hospital health record system, with interested individuals contacted by the research team; healthcare providers were identified through collaboration with department leadership and internal hospital communication channels. All prospective participants underwent eligibility screening before enrollment.

The sample was equally divided between stroke survivors (n=15) and healthcare providers (n=15). In accordance with best practices in scale development,¹¹ cognitive interviewing during the pre-testing phase should include individuals who represent both the target population and those with relevant domain expertise to evaluate whether items adequately reflect the construct and generate valid responses.

In this study, healthcare providers were included to assess the clinical relevance, conceptual accuracy, and domain coverage of the items, while stroke patients contributed to evaluating clarity, comprehension, and response processes. This multi-perspective approach enabled a comprehensive assessment of both content validity and response validity, and facilitated the identification of discrepancies between clinical intent and patient interpretation.

Inclusion criteria for survivors required a clinical diagnosis of ischemic or hemorrhagic stroke, Thai language proficiency, and age ≥18 years, with sufficient communication ability to understand and respond to study materials. Individuals with conditions substantially affecting cognition (eg, dementia, schizophrenia, severe cognitive impairment) or post-stroke depression were excluded; as all participants were required to provide their own written informed consent, individuals lacking consent capacity were not enrolled.

Healthcare providers were required to have at least one year of clinical experience in stroke care. Written informed consent was obtained from all participants before data collection.

Data Collection Procedure

The study was designed as a two-stage iterative process to enhance the depth, credibility, and interpretive rigor of the findings. In Round 1, 30 participants (15 survivors, 15 providers) were interviewed to identify initial barriers to comprehension and item interpretation. Following the analysis of Round 1, the research team made preliminary adjustments to the SRSS based on participant feedback. Round 2 interviews were conducted with no more than a 1-month gap following Round 1, in accordance with the study’s schedule of procedures.

In Round 2, the same 30 participants were re-interviewed using the revised items to verify whether the modifications successfully resolved the previously identified barriers and to ensure no new ambiguities were introduced. This longitudinal, two-round approach yielded a total of 60 cognitive interviews, providing a deep, validated understanding of the instrument’s performance.

Interviews were conducted by the principal investigator, co-investigators, or trained research nurses with a minimum of two years of clinical experience, each of whom completed a structured training program covering study procedures, informed consent, and interview technique before data collection. Each interview lasted approximately 40–60 minutes and followed a semi-structured verbal probing approach, in which the interviewer posed follow-up questions after each item rather than requesting concurrent think-aloud verbalization.

At the beginning of the first session, the interviewer collected personal data using the demographic form before proceeding to the cognitive interview. Data were collected and managed using REDCap (Research Electronic Data Capture),¹² a secure, web-based platform hosted at the Faculty of Medicine Siriraj Hospital, Mahidol University.

During the interview, the interviewer read all 30 SRSS items aloud, and participants were asked to describe their understanding, reasoning, and thought processes as they formulated their responses. Probing questions explored issues related to clarity, comprehension, cultural appropriateness, recall demands, difficulty, and the adequacy of response options. The verbal probing guide (developed in Thai) is available from the corresponding author upon reasonable request. All interviews were audio-recorded with participants’ permission, and field notes were taken to document nonverbal cues and initial reflections relevant to the interpretation of responses.

Data Analysis

All audio-recorded interviews were transcribed verbatim and de-identified before analysis to ensure participant confidentiality. The transcripts were organized systematically for qualitative analysis. Data were analyzed using an inductive thematic analysis approach guided by Braun and Clarke’s six-phase framework.¹³

The analysis began with repeated, in-depth reading of all transcripts to achieve immersion in the data and to gain a comprehensive understanding of participants’ perspectives. Initial coding was conducted inductively without a predefined coding framework. Meaningful segments of text were identified and assigned descriptive codes that closely reflected participants’ expressions and interpretations, particularly regarding comprehension, interpretation, and response processes related to the SRSS items.

Codes were then examined and compared across transcripts, allowing for refinement, merging, or differentiation as needed. Through an iterative process, related codes were grouped into broader categories based on conceptual similarities. These categories were further developed into subthemes and higher-order analytical domains that captured patterns in barriers to item interpretation and strategies for improving clarity, accuracy, and usability of the instrument.

Throughout the analytic process, constant comparison was applied to enhance the depth of interpretation and analytic coherence across codes and domains. The analysis remained iterative and reflexive, with ongoing review of the data to verify that interpretations accurately represented participants’ perspectives. An audit trail was maintained to document coding decisions, theme development, and revisions over time, thereby enhancing transparency and methodological rigor.

Coding and theme development were conducted collaboratively by the research team through iterative discussions, allowing multiple perspectives to inform interpretation and deepen analytic insights. Two researchers independently coded each transcript; discrepancies were resolved through discussion, with a third team member adjudicating any disagreements that could not be resolved between the two coders. Coding was conducted manually using Microsoft Word and Excel. An evolving coding framework was developed and refined iteratively to support ongoing interpretation and organization of the data. Data collection and analysis proceeded until sufficient depth and richness of data were achieved to support robust and meaningful thematic interpretation. Final themes and subthemes were established through collaborative review and critical reflection to ensure alignment with the original data. Throughout the analytic process, researchers engaged in reflexive discussion to critically consider how their perspectives and disciplinary backgrounds informed data interpretation.

Rigor and Trustworthiness

To ensure the integrity of the findings, the study followed established criteria for qualitative rigor.¹⁴ Credibility was established through data triangulation, integrating perspectives from both stroke survivors and healthcare providers, and prolonged engagement consisting of 60 total interviews across two iterative rounds. This two-round design allowed for member-checking, as researchers could verify in Round 2 whether the refinements made after Round 1 accurately addressed participant concerns. The sample of 30 participants was determined based on information power rather than statistical power, given the qualitative, non-inferential aim of this study; this sample size was considered sufficient to capture diversity of interpretation across two distinct respondent groups and to reach saturation of item-level comprehension problems across the 30 SRSS items, consistent with recommended ranges for cognitive interviewing in survey pre-testing.

Dependability was maintained through a systematic audit trail, documenting every modification made to the SRSS items alongside the specific participant feedback (PID No) that prompted the change. Confirmability was supported through collaborative analytic discussions and reflexive engagement within the multidisciplinary research team (comprising nursing and medical experts), helping to mitigate individual bias during thematic development. Finally, transferability was supported by providing thick descriptions of the Thai clinical context and detailed participant demographics, allowing for the application of these findings to similar screening tool development in other resource-limited settings.

Results

Participant Characteristics

Table 1 summarizes demographic characteristics of the 30 participants, comprising 15 stroke survivors and 15 healthcare providers. Stroke survivors were older (59.2 ± 11.4 vs 38.8 ± 8.7 years), more often male (66.7% vs 20.0%), and had a higher mean BMI (25.90 ± 6.13 vs 23.62 ± 3.03 kg/m²) than healthcare providers. Marital status, insurance coverage, and smoking and alcohol use also differed by group, consistent with the two cohorts’ distinct age and employment profiles.

Table 1.

Demographic Characteristics of Study Participants

Characteristic	Stroke Survivors (n = 15)	Healthcare Providers (n = 15)	Total (N = 30)
Age, Mean (SD)	59.2 ± 11.4	38.8 ± 8.7	49.0 ± 14.36
Age, Range	44–84	28–55	28–84
Sex, n (%)
Male	10 (66.7%)	3 (20.0%)	13 (43.3%)
Female	5 (33.3%)	12 (80.0%)	17 (56.7%)
Body Mass Index, Mean (SD)	25.90 ± 6.13	23.62 ± 3.03	24.76 ± 4.89
Marital Status, n (%)
Single	3 (20.0%)	11 (73.3%)	14 (46.7%)
Married	9 (60.0%)	4 (26.7%)	13 (43.3%)
Divorced/Separated	2 (13.3%)	0 (0.0%)	2 (6.7%)
Widowed	1 (6.7%)	0 (0.0%)	1 (3.3%)
Health Insurance, n (%)
Social Security Scheme	1 (6.7%)	12 (80.0%)	13 (43.3%)
Universal Coverage Scheme	10 (66.7%)	1 (6.7%)	11 (36.7%)
Civil Servant Medical Benefit	3 (20.0%)	2 (13.3%)	5 (16.7%)
Self-paid	1 (6.7%)	0 (0.0%)	1 (3.3%)
Occupation, n (%)
General Employee	5 (33.3%)	—	—
Business owner	4 (26.7%)	—	—
Homemaker	3 (20.0%)	—	—
Teacher, Monk, Police (1 each)	3 (20.0%)	—	—
Nurse	—	7 (46.7%)	—
Physician	—	3 (20.0%)	—
Pharmacist	—	3 (20.0%)	—
Physical therapist	—	2 (13.3%)	—
Smoking Status, n (%)
Never smoked	7 (46.7%)	15 (100.0%)	22 (73.3%)
Former smoker	6 (40.0%)	0 (0.0%)	6 (20.0%)
Current smoker	2 (13.3%)	0 (0.0%)	2 (6.7%)
Alcohol Consumption, n (%)
Never	13 (86.7%)	10 (66.7%)	23 (76.7%)
Once a month or less	0 (0.0%)	5 (33.3%)	5 (16.7%)
4 times or more per week	2 (13.3%)	0 (0.0%)	2 (6.7%)

Note: Values are n (%) unless otherwise indicated.

Cognitive Interview Findings

A total of 60 cognitive interviews were conducted with 30 participants, each completing two rounds. Findings from Round 1 interviews primarily informed the identification of barriers to item interpretation and response processes, which formed the basis for initial coding and analytic development. Round 2 interviews were conducted to validate and refine these findings by assessing whether the revised items addressed previously identified issues and to confirm the clarity and applicability of the modifications.

While most participant concerns identified in Round 1 were resolved following revision, a subset of items continued to elicit concerns in Round 2, particularly those involving differentiation of stroke-related symptoms from non-stroke conditions, measurement units, and definitional clarity; a revised version of SRSS and full item-level breakdown, including the number of participants who raised each concern, is provided in Supplementary Data 1–2. This iterative process strengthened the credibility and confirmability of the analytic structure by distinguishing barriers that were successfully resolved from those requiring further refinement.

The analysis identified two major analytical domains derived from the data: (1) interpretation and response barriers, and (2) evidence-based refinement strategies. Within each domain, multiple inductively developed subthemes captured patterns in participants’ experiences and recommendations.

Domain 1: Interpretation and Response Barriers

Participants reported several challenges that interfered with their ability to accurately interpret and respond to the SRSS items. These challenges related to linguistic complexity, conceptual ambiguity, inconsistencies in response structures, and cognitive demands required to recall clinical information.

Unclear or Technical Wording

Participants frequently had difficulty understanding specialized medical terminology. Respondents noted that such terminology is rarely used in everyday conversation, leading to uncertainty about how to interpret the question. This issue was more prominent among non–healthcare participants, who felt unsure whether the items referred to medical confirmation or personal experience.

Representative quotes illustrating this theme included:

If we’re asking the general public, we could just say it plainly — has a doctor told you that you have high blood pressure? We don’t need to use the word ‘diagnosed’… some people, when they first hear ‘diagnosed,’ get confused about whether they have it or not. (Healthcare provider, PID 202406)

If I were explaining this to someone else, I’d have to tell them what’s what — which one is the ‘good’ fat and which is the ‘bad’ fat — because they wouldn’t understand otherwise. I only started to understand after I had my stroke, and then watched doctors’ videos explaining it. Since we’re not healthcare professionals, some medical terms — we can hear them, but we don’t know what they mean. (Stroke survivor, PID 202416)

Ambiguous Concepts and Insufficient Definitions

Several items lacked clear definitions, making it difficult for participants to determine whether their experiences fit the intended categories. For example, participants were uncertain about what severity of head injury should be counted. Lack of explicit criteria or time frames, particularly for TIA-related items, was a common source of confusion.

Representative quotes illustrating this theme included:

Sometimes it’s hard to say — transient ischemic attack, TIA… well, that’s the Thai term, okay. But maybe there should be a definition, like less than 24 hours or something? Because some people have temporary symptoms, but they don’t know what counts as this timeframe. (Healthcare provider, PID 202411)

For head injury, I’d want it to specify what kind — was it a fall, or something striking the head, or was there surgery? Because there are many types of injury, ranging from mild to severe. (Healthcare provider, PID 202412)

Inconsistent or Misaligned Response Options

Participants frequently experienced confusion when question stems were written in one format (such as “ever/never”) but the response options were provided in another (such as “yes/no”). This inconsistency required respondents to reinterpret the item, creating hesitation and reducing confidence in their answers. Many emphasized the need for consistent pairing of stems and response choices to maintain clarity.

Representative quotes illustrating this theme included: “I think the question uses the word ‘ever’ — ‘have you ever received…’ — so the answer choices should be ‘ever/never’ to be consistent.” (Healthcare provider, PID 202413). “Should be answered as ‘ever’ or ‘never.’” (Stroke survivor, PID 202426).

Lack of Skip Logic

Participants reported difficulty when items required answers that were irrelevant to their health status. For example, individuals without diabetes were still prompted to answer questions about diabetes medication, and participants without recent laboratory testing were unable to respond to items referencing lipid or glucose values. Respondents noted that such situations caused confusion and increased the cognitive burden of the questionnaire. The electronic REDCap platform used for data collection required a response to every item; when a question did not apply, interviewers recorded the response as “not applicable” on the paper case report form. Despite this workaround, participants still experienced being presented with inapplicable items as burdensome and confusing.

Representative quotes illustrating this theme included:

If you answer ‘no’ [to not having diabetes], shouldn’t this item be skipped? Since it’s asked right after, it’s confusing whether it should apply. (Healthcare provider, PID 202412)

It should ask from the start whether blood was drawn for testing — if not, it should skip directly. (Stroke survivor, PID 202421).

Difficulty Recalling Numeric or Clinical Information

Items requiring recall of numeric data, such as laboratory values, waist circumference, or blood pressure readings, were challenging for many participants. Several noted that although they were aware of having “high cholesterol” or “high blood sugar,” they could not recall the exact values. Others were uncertain about measurement units in everyday use versus clinical documentation. These challenges raised the likelihood of inaccurate or estimated responses.

Representative quotes illustrating this theme included:

Here, maybe we’d need to ask them, ‘what is your blood fat level?’ … I mean, remembering the exact number isn’t possible, but if that’s the case, the second question could offer relative options — higher or lower than a value — which would be easier than asking for an exact number they can’t recall. (Healthcare provider, PID 202406)

The question is too difficult. Sometimes I don’t remember [the value]. (Stroke survivor, PID 202427).

Cultural and Contextual Mismatch

Some items used units or formats that did not align well with participants’ daily experiences. Examples included quantifying exercise in minutes per week, using number of “drinks” to measure alcohol consumption, or requiring waist measurements in centimeters when individuals were more familiar with inches. These mismatches resulted in substantial variability in how participants interpreted the items.

Representative quotes illustrating this theme included:

It’s usually 150 minutes, right? Or is it 150 minutes — because ‘2 hours 30 minutes’ is confusing and hard to use. We don’t do it all at once; we accumulate it little by little, adding up to, say, 150 minutes per week. (Healthcare provider, PID 202409)

It should break down the volume/content a bit more — generally, people don’t drink beer at ‘20 glasses a week.’ This question is difficult, because when we assess a patient’s range, we think of it differently. (Healthcare provider, PID 202410)

Overly Long or Complex Questions

Participants described several items as too long or containing multiple pieces of information, making them difficult to follow. Long sentences and multiple embedded concepts required sustained concentration and contributed to misinterpretation, particularly among those with lower health literacy.

Representative quotes illustrating this theme included:

When it was first read aloud, it might be confusing. The length of the question should be revised to be more concise. (Stroke survivor, PID 202430).

The question is too long, might be difficult for some people, like villagers or farmers. (Stroke survivor, PID 202421).

Domain 2: Evidence-Based Refinement Strategies

Participants also offered numerous practical recommendations for improving the SRSS items. These suggestions focused on simplifying language, improving structure, and enhancing conceptual clarity.

Simplifying Language

Participants strongly recommended replacing technical terminology with everyday language. This adjustment was viewed as more accessible for community-based respondents.

Representative quotes illustrating this theme included:

For some phrasing, we could just ask directly — did the doctor say you have high fat in your blood, and are you taking medication for it? (Healthcare provider, PID 202406).

Providing Definitions, Examples, and Clarifying Phrases

Many participants suggested including brief definitions or examples to clarify symptom descriptions, disease categories, or clinical time frames. These additions could help respondents distinguish between mild and severe injuries, identify typical stroke symptoms, or understand what constitutes a TIA. Providing such context was seen as essential for improving accuracy among non–healthcare respondents.

Representative quotes illustrating this theme included:

For people who aren’t in the stroke field, there should be a slightly more detailed definition — specifying what ‘transient’ means. Transient ischemic attack — what does that mean, exactly?. (Healthcare provider, PID 202412)

Improving Consistency in Response Options

Participants recommended ensuring that question stems and response choices align structurally. For example, items written in “ever/never” format should use matching response options, and “yes/no” items should follow the same pattern. Consistency across items was viewed as critical for minimizing confusion.

Representative quotes illustrating this theme included:

Same as the item above — it should be ‘ever’ or ‘never,’ or if the choice is yes/no, then the question stem should ask directly: ‘Have you ever experienced weakness of the face, arm, or leg on one side?’ — that’s it, done. (Healthcare provider, PID 202413)

Should be answered as ‘ever’ or ‘never.’ (Stroke survivor, PID 202426).

Incorporating Skip Logic

Participants emphasized the importance of enabling respondents to skip questions that do not apply to them. For example, medication-related questions should only appear if the respondent confirms having the corresponding condition, and laboratory-related items should be presented only to participants who have undergone testing within the specified period. This structure would reduce respondent burden and enhance efficiency.

Representative quotes illustrating this theme included:

If you answer ‘no’ [to the prior item], shouldn’t this item not be asked — it should be skipped. As recommended, if you answer ‘no,’ then skip, because if we keep asking, it will confuse and puzzle the respondent. (Healthcare provider, PID 202412)

It should ask from the start whether blood was drawn for testing — if not, it should skip directly. (Stroke survivor, PID 202421).

Increasing Specificity Through Units, Thresholds, and Structured Options

Participants recommended clarifying measurement units, physical activity thresholds, alcohol volume definitions, and cigarette types to reduce variability in interpretation. Examples included converting time into minutes, offering step-count alternatives, or defining alcohol intake using standardized drink units. Clear parameters were seen as essential for improving response accuracy.

Representative quotes illustrating this theme included:

Could we change it to number of steps? Because nowadays there are smartwatches that count steps, which would give a clearer picture. With kilometers, I’m not sure — sometimes we’re not walking specifically to exercise, just walking around for work. (Healthcare provider, PID 202412)

Adding Symptom or Scenario Examples

Respondents encouraged the inclusion of short examples to illustrate key symptoms or conditions to help the general population better understand what the questions refer to. These examples would support more accurate self-assessment among respondents.

Representative quotes illustrating this theme included:

It might need to add examples, like valve leakage, valve stenosis, or coronary artery disease — maybe say ‘narrowing of the heart’s blood vessels’ or similar, because sometimes patients or the general public wouldn’t know [the medical terms]. (Healthcare provider, PID 202412)

Reducing Item Length and Improving Brevity

Participants consistently recommended shortening long or complex items to enhance readability and comprehension. They emphasized that focusing on a single concept per question would prevent cognitive overload and reduce misinterpretation.

Representative quotes illustrating this theme included:

When it was first read aloud, it might be confusing. The length of the question should be revised to be more concise. (Stroke survivor, PID 202430).

Summary of Item Revisions

The qualitative evidence gathered from 60 cognitive interviews served as the foundation for a systematic, iterative refinement of the SRSS (Supplementary Data 1). These revisions were not merely stylistic but were designed to reduce cognitive load and minimize measurement error.

Key modifications fell into four categories. (1) Linguistic and conceptual refinement: clinical terminology was translated into lay language, and abstract concepts were anchored with concrete examples. (2) Structural optimization: based on barriers identified across both interview rounds, skip logic was incorporated into the finalized SRSS (Version 1.3) for items conditional on prior responses (eg, diabetes medication history, following from Q12), reducing respondent burden and irrelevant data entry in future administrations. (3) Standardization of response metrics: inconsistent response scales were harmonized into a uniform or frequency-based format, and ambiguous measurement units were clarified by providing dual units and standardized volume references for alcohol consumption. (4) Syntactic simplification: multi-clause items were decomposed into shorter, single-focus questions to prevent participant confusion and ensure that each risk factor was assessed independently.

These modifications targeted the barriers identified in Round 1, most of which were resolved by Round 2. However, a smaller number of items (eg, differentiating stroke-related symptoms from common non-stroke conditions, standardizing alcohol volume units, and specifying diagnostic criteria for traumatic brain injury) continued to elicit participant concerns in Round 2 despite revision. In addition, because skip logic was incorporated into the finalized instrument (Version 1.3) after both interview rounds were completed, it was not itself tested within this study’s interview process. A complete item-by-item mapping of Round 1 barriers, suggested revisions, final wording, and Round 2 status is provided in Supplementary Data 2. This refinement process supports the clinical comprehensiveness, contextual appropriateness, and usability of the SRSS across diverse populations. The current version of the refined SRSS is provided as Supplementary Data 1.

Discussion

This study examined how respondents interpreted and responded to the items in the newly developed SRSS. Findings revealed several key barriers, including unclear or technical wording, ambiguous concepts, inconsistent response options, and difficulties recalling clinical information. These issues affected both stroke survivors and healthcare providers and align with prior research showing that respondents often struggle with medical terminology, complex phrasing, and items that demand specialized knowledge or precise numeric recall.^15–17 Similar challenges have been reported in early-stage testing of cardiovascular and neurological screening tools, emphasizing the importance of simplifying language and ensuring cultural and contextual relevance.^18,19 Notably, this qualitative pretesting approach distinguishes the SRSS from most existing stroke risk assessment tools: instruments such as the Framingham Stroke Risk Score, QStroke, and the Stroke Riskometer™ were derived primarily through statistical modeling of large epidemiological cohorts, with respondent comprehension and interpretability evaluated, if at all, only after deployment rather than during item development.^7–9 This suggests that response-process barriers similar to those identified in the present study may be present but undocumented in other widely used stroke risk instruments, underscoring the broader value of qualitative pretesting in this field.

Participants also provided practical strategies for improving item clarity and usability, such as replacing technical terms with everyday language, adding brief definitions or examples, aligning question stems with response options, incorporating skip logic, and clarifying measurement units. These suggestions are consistent with previous studies in instrument development, which recommend iterative refinement based on user feedback to enhance content validity and respondent comprehension.^20–22 Importantly, incorporating culturally meaningful formats, such as familiar units, examples, and structured options, addresses known barriers in applying screening tools across diverse populations.^23,24

The revisions informed by these interviews improved the readability and structural consistency of many SRSS items; however, as detailed in Supplementary Data 2, a subset of items, particularly those requiring differentiation between stroke-related symptoms and common non-stroke conditions, standardized alcohol volume units, and clearer diagnostic criteria for traumatic brain injury, continued to elicit participant concerns despite revision, indicating that further refinement is needed for these specific items. Future research should evaluate the revised SRSS through larger-scale psychometric testing, consistent with the planned scale-development pathway (Phase 2.2–2.4: survey administration, item reduction, and exploratory factor analysis; Phase 3.1–3.3: dimensional testing, reliability testing [eg, internal consistency, test–retest], and validity testing [eg, criterion and predictive validity against clinical stroke outcomes]), to assess factor structure, reliability, and predictive validity. Additional testing in diverse community settings may further enhance applicability and ensure consistent performance across populations with varying health literacy levels.

It is important to distinguish what this study’s findings can and cannot establish. Cognitive interviewing provides response-process validity evidence, evidence that respondents interpret and answer items as intended, which builds on and is conceptually distinct from the content validity established in earlier development phases.^6,10 However, response-process evidence cannot establish reliability, factor structure, predictive validity, clinical utility, or diagnostic performance; these require the subsequent psychometric evaluation phases described above.

This study also surfaced a recurring tension between clinical precision and lay comprehension. Several participant-recommended simplifications altered surface wording without changing the underlying clinical construct (eg, rephrasing “diagnosed with” as “has a doctor told you that you have” preserves reference to the same clinician-confirmed diagnosis). In other cases, simplification was not straightforward: participants repeatedly requested a clearer, lay-accessible definition of “transient ischemic attack”, but no simplified wording was ultimately adopted, reflecting the difficulty of clarifying this term for lay respondents without risking imprecision in a diagnosis that is itself clinically nuanced.

Beyond identifying barriers, this study also contributes methodologically by illustrating how a two-round cognitive interviewing approach can be used not only to detect interpretive issues but also to iteratively test whether revisions resolve them. This approach strengthens response-process validity by explicitly addressing discrepancies between clinical intent and patient interpretation. Furthermore, the inclusion of both patients and healthcare providers enabled a more comprehensive evaluation of instrument performance, integrating perspectives on both usability and conceptual accuracy.

Study Limitations

Despite the methodological rigor of this study, several limitations warrant consideration. First, the purposive sampling was conducted within a single urban healthcare network, which may result in geographic homogeneity. Consequently, the findings regarding linguistic preferences and interpretive barriers may not be fully transferable to rural populations or regions with distinct socio-cultural dialects. Second, while the inclusion of both stroke survivors and healthcare providers offered data triangulation, the survivor cohort consisted of individuals already engaged with clinical services. This population may possess higher health literacy or different cognitive schemas than the general community or undiagnosed individuals, potentially masking barriers that might exist in a primary screening context.

Third, the study was conducted exclusively in the Thai language. While this was essential for the cultural and linguistic adaptation of the SRSS for its primary target population, it necessitates further cross-cultural validation and psychometric equivalence testing before the instrument can be adapted into other linguistic contexts. Finally, although the two-round interview design was a structural strength, the repetition of items across sessions may have introduced a slight learning effect. Additionally, as with all qualitative inquiries, responses may be subject to social desirability bias, particularly among healthcare providers, and recall bias regarding past clinical events. Finally, a subset of items continued to draw participant concerns in Round 2 despite revision; consistent with the planned multi-phase SRSS development pathway, these items will be further refined and re-evaluated in the subsequent instrument development phase before psychometric testing.

Suggestions for Future New Tool Development

The findings from this study highlight several considerations that can inform the development of future health assessment tools. First, incorporating cognitive interviewing early in the process is essential for identifying interpretive problems, linguistic ambiguity, and usability challenges before large-scale testing. Second, item wording should prioritize plain language, cultural relevance, and clear definitions or examples to enhance comprehension across diverse user groups. Third, consistency between question stems and response options, along with the use of appropriate skip logic, helps reduce respondent burden and improve data quality. Fourth, measurement units and thresholds should be clearly defined and aligned with everyday practices to support accurate reporting. Finally, newly developed tools should undergo iterative pilot testing and subsequent psychometric evaluation to ensure reliability, validity, and practical utility.

Conclusion

This study underscores the critical importance of cognitive interviewing in the lifecycle of instrument development, revealing that expert-validated content does not always translate to layperson comprehension. By systematically identifying interpretive barriers, such as technical jargon and high cognitive recall demands, we generated preliminary response-process evidence to inform refinement of the Stroke Risk Screening Scales (SRSS). This study identified evidence-based, user-informed strategies, including linguistic simplification, incorporation of skip logic, and the use of culturally familiar measurement anchors, to enhance the instrument’s clarity and usability in future iterations. These refinements are intended to improve the SRSS’s comprehensibility and accessibility across diverse populations; however, their impact on measurement performance has not yet been established. This research provides preliminary response-process evidence and user-informed refinements that should now be evaluated in larger and more diverse samples, and serves as a methodological roadmap for developing patient-centered screening tools in resource-limited clinical settings. The SRSS should not be considered ready for broad clinical implementation until reliability, construct validity, predictive validity, scoring performance, and feasibility in target screening populations have been established. Importantly, this study also highlights the value of integrating multiple stakeholder perspectives and iterative refinement within cognitive interviewing, offering a practical and transferable framework for enhancing the development of patient-centered screening instruments.

Funding Statement

This research project is supported by Mahidol University (Strategic Research Fund): 2024.

Data Sharing Statement

The data generated, analyzed, and reported in this study are not publicly available because of the privacy of the participants. For further information on the data, simplified, de-identified data can be obtained by contacting the corresponding author.

Ethical Approval and Informed Consent Statements

This study was approved by the Institutional Review Board of the Faculty of Nursing, Mahidol University, Thailand (MU-MOU CoA No. IRB-NS2024/888.2809). All participants provided written informed consent before participation in the study, including consent for the publication of anonymized responses and direct quotations. Our study complies with the Declaration of Helsinki.

Disclosure

The authors declare no conflicts of interest in this work.