A standardised instrument for community resource mapping in social prescribing: development and content validation of the COMPASS™ tool

Abstract

Background

Social prescribing aims to address non-medical determinants of health by connecting individuals to community-based resources. However, its systematic implementation is constrained by the limited availability of structured and transferable approaches to map and characterise community resources in a consistent and transparent manner.

Methods

A multi-phase methodological study was conducted. Phase I involved instrument development and content validation using a two-round modified Delphi process with an international expert panel. Phase II comprised forward–backward translation and cross-cultural adaptation following established guidelines. Phase III involved preliminary field testing of the Portuguese version in a Lisbon municipality, including assessment of item completion, time to completion, perceived completeness and usefulness, and exploratory analyses of inter-rater reliability and temporal stability.

Results

The initial 36-item version of the COMPASS™ tool achieved a scale-level content validity index of 0.85, informing revision and reduction to 21 items. In Round 2, near-perfect consensus was achieved (S-CVI = 0.98 for both clarity and relevance). Cross-cultural adaptation demonstrated strong semantic and conceptual equivalence. Field testing across 16 community resources showed high item completion (95.5%) and a mean completion time of 14.8 min. The tool was rated as highly complete (mean = 4.4/5) and useful (mean = 4.6/5). Preliminary analyses indicated substantial inter-rater agreement and high temporal stability.

Conclusion

The COMPASS™ tool demonstrated strong content validity, successful cultural adaptation, and preliminary evidence of feasibility for community resource mapping. As social prescribing becomes increasingly integrated into health systems, structured tools such as COMPASS™ may support more consistent identification and organisation of community resources, contributing to population health monitoring and to addressing the social determinants of health.

Trial registration

Not applicable.

Supplementary Information

The online version contains supplementary material available at 10.1186/s13690-026-01982-4.

Text box 1. Contributions to literature
• There is currently no standardised and transferable method to systematically map community resources for social prescribing within population health systems.
• This study develops and validates COMPASS™, a structured tool grounded in public health frameworks and expert consensus.
• The tool provides a practical and scalable approach to improve transparency, consistency, and comparability in community resource mapping.
• It supports the integration of community assets into health systems and strengthens responses to the social determinants of health.

Background

Social prescribing has emerged as a promising approach to address non-medical determinants of health by linking individuals to community-based resources that support wellbeing, social participation, and quality of life. As social prescribing initiatives expand across health and municipal systems internationally, with growing interest in the Portuguese context [1], there is a growing need for structured, transparent, and context-sensitive tools capable of systematically mapping community resources and supporting informed referral decisions, particularly in order to avoid inconsistent or “scattergun” referral practices and to reduce reliance on individual practitioners’ tacit knowledge [2–4].

From a public health perspective, social prescribing initiatives can be understood as mechanisms that mobilise community assets to address the broader social determinants of health. In line with the social determinants framework proposed by Dahlgren and Whitehead [5], community resources such as social groups, cultural activities, volunteering opportunities, and physical activity programmes may contribute to improving health and wellbeing through pathways related to social participation, social support, and access to supportive environments.

Despite increasing policy interest and practical implementation of social prescribing, the availability of validated instruments to systematically map and characterise community resources remains limited. Existing approaches are frequently fragmented, locally developed, or insufficiently documented, which constrains comparability, scalability, and integration into routine practice. Moreover, tools developed within specific cultural or organisational contexts may not be readily transferable to other settings without careful adaptation. Reviews of community asset mapping approaches have highlighted substantial heterogeneity in how community resources are identified, described, and evaluated, noting the absence of standardised frameworks, inconsistent terminology, and wide variation in methodological rigor [6]. Together, these limitations pose significant challenges to the systematic integration of community resources into social prescribing pathways.

To address this gap, the Community Resource Mapping Tool for Social Prescribing (COMPASS™) was developed as a structured instrument to guide the systematic collection and organisation of information about community resources relevant to social prescribing. In its current form, COMPASS™ is not a standalone software platform, but a standardised mapping template that can be completed in paper-based format, electronically, or embedded within future digital directories or social prescribing platforms. The tool comprises 21 items covering resource identification, target population, implementation characteristics, accessibility, costs, referral and coordination mechanisms, and evaluation procedures. It is intended for use by professionals involved in community resource mapping, including community health practitioners, municipal technicians, link workers, researchers, and future social prescribing teams.

This study represents an early stage in the development and validation pathway of COMPASS™, intended to establish content validity, cultural adaptation, and preliminary feasibility before future testing in operational social prescribing pathways.

Methods

Aim

The present study aimed to (i) develop and establish the content validity of the COMPASS™ tool through expert consensus, (ii) translate and culturally adapt the instrument for use in Portugal, and (iii) conduct preliminary field testing of the Portuguese version within a municipal context relevant to future social prescribing pathways. In doing so, the study contributes to the methodological infrastructure required to support systematic community resource mapping within emerging social prescribing systems.

Study design

This methodological study followed a multi-phase, sequential design in accordance with established recommendations for instrument development, content validation, consensus-based expert review, and cross-cultural adaptation [7–12]. The study comprised three phases: (1) development and content validation of the COMPASS™ tool using a two-round modified Delphi process; (2) cross-cultural translation and adaptation of the instrument for use in Portugal; and (3) preliminary field testing of the Portuguese version in a Lisbon-area municipality to assess feasibility, comprehension, and applicability, including exploratory analyses of inter-rater reliability and temporal stability (Fig. 1).

Fig. 1.

Development and preliminary testing process of the COMPASS™ tool

Phase I: Instrument development and content validation

Initial item generation and conceptual framework

The initial version of the COMPASS™ tool was developed drawing on the social determinants of health framework proposed by Dahlgren and Whitehead [5], which highlights the role of social, environmental, and community-level factors in shaping health outcomes. In line with this perspective, the conceptual structure of the instrument was further informed by the thematic domains outlined in the World Health Organization (WHO) Social Prescribing Toolkit [13], with the aim of ensuring comprehensive coverage of key areas relevant to the systematic mapping of community resources prior to expert review. Item generation was informed by a structured analysis of the toolkit, focusing on domains related to target population, type and format of activities, accessibility and participation requirements, potential barriers to engagement, costs, delivery setting, and safety considerations. These conceptual domains were translated into operational, closed-ended items suitable for standardised data collection at the municipal level.

Design

A two-round modified Delphi process was used to assess the content validity of the COMPASS™ tool. This approach was selected to systematically elicit expert judgement and achieve consensus regarding the clarity and relevance of the proposed items [10].

Expert panel and recruitment

An international panel of experts in social prescribing was convened. Eligibility criteria included: (1) recognised academic or professional expertise in social prescribing; (2) participation in relevant research, policy, or implementation projects; and/or (3) recommendation by senior experts in the field. Seven experts were identified through supervisory recommendation and purposive snowball sampling and invited to participate. Of these, five agreed and completed both Delphi rounds. Participation was anonymous and independent, and data were collected electronically using Google Forms.

Although this represents a relatively small panel, Delphi panel size should be considered in relation to the purpose of the study, the specificity of the topic, and the expertise required [14]. In the present study, the Delphi process was used for focused content validation of a structured instrument, rather than for broad stakeholder consensus or policy development. The panel was therefore purposively composed of highly specialised international experts with recognised experience in social prescribing research, policy, or implementation. Complete retention across both rounds further supported the consistency of the consensus process.

Round 1 procedures

In Round 1, experts evaluated the initial item pool for clarity and relevance using a 5-point Likert scale (1 = not clear/relevant; 5 = very clear/relevant). Although a 4-point scale has been proposed in classical content validity studies [7], a 5-point format was adopted to allow greater discrimination among responses in an international expert panel. Following Polit, Beck and Owen [15], ratings of 4 or 5 were dichotomised as positive.

Item Content Validity Index (I-CVI) values were calculated as the proportion of experts rating an item as 4 or 5. The Scale Content Validity Index (S-CVI/Ave) was computed as the mean of I-CVI values across items. Experts were also invited to provide open-ended comments, which were analysed using thematic content analysis [16]. A deductive coding framework addressing linguistic clarity, redundancy, and structural issues was applied, with inductive codes added when necessary. All comments were anonymised and coded in Excel by the principal investigator, who conducted the thematic categorisation according to the predefined coding framework. Codes were assigned to each comment as independent meaning units, with additional inductive codes introduced when new themes emerged during the analysis.

Revisions between rounds

Items with an I-CVI below the recommended threshold (< 0.78) or those for which experts suggested linguistic, structural, or conceptual refinements were revised. New content was introduced when recommended by more than one expert or deemed essential by the research team, while ensuring alignment with the conceptual domains of the WHO Social Prescribing Toolkit [13]. The revised instrument was redistributed to experts for Round 2 evaluation.

Round 2 procedures

In Round 2, the same five experts completed the evaluation, corresponding to a 100% retention rate. Experts reassessed the revised items using a 5-point Likert scale, and optional qualitative comments were collected to identify any remaining concerns. I-CVI and S-CVI/Ave were recalculated. Consensus was also assessed using the median and interquartile range of expert Likert ratings for each item, separately for clarity and relevance, with consensus defined as a median rating of ≥ 4 and an IQR ≤ 1.

Phase II: Cross-cultural translation and cultural adaptation

The Portuguese adaptation of the COMPASS™ tool followed established international guidelines for translation and cultural adaptation [11, 12]. Two independent forward translations were produced: one by a bilingual health professional with a clinical background in nursing and one by a professional translator without formal training in the health field. This complementary approach aimed to balance conceptual accuracy with linguistic neutrality.

An expert committee was then convened to synthesise the two forward translations into a single consensus version. This committee comprised the principal investigator and the senior academic experts and co-authors (specialist in social prescribing and gerontology). Inputs from the translators were considered during this process when linguistic clarification was required. The committee conducted an item-by-item comparison of both translations, resolving discrepancies through consensus and prioritising fidelity to the original content, linguistic adequacy to European Portuguese, and preservation of semantic equivalence [11, 12].

Subsequently, a back-translation of the consensus Portuguese version was produced, by a third professional translator. A second expert committee review was conducted to compare the back-translated version with the original English instrument. This committee, composed of the principal investigator and senior academic experts, systematically assessed semantic, idiomatic, experiential, and conceptual equivalence, in line with established recommendations [11, 12].

Phase III: Preliminary field testing

Phase III comprised two complementary components: feasibility testing and preliminary reliability assessment. Feasibility testing examined the applicability, comprehensibility, completion rate, completion time, and perceived usefulness of COMPASS™ in a real municipal context. Reliability assessment examined whether different evaluators could apply the tool consistently when using the same standardised information sources.

Feasibility testing

Feasibility testing was conducted from January to February 2026 in one Lisbon-area municipality. Although formal social prescribing pathways are not yet established in Portugal, a Lisbon-area municipality was selected as the field testing context because it represents a real municipal environment likely to support future implementation of social prescribing, allowing preliminary assessment of the tool’s applicability prior to the formalisation of referral pathways.

The Portuguese version of COMPASS™ was applied to 16 community resources identified as potentially relevant for future social prescribing pathways. Data were collected by the principal investigator through semi-structured interviews with initiative leaders or professionals responsible for each resource. During each interview, responses were entered directly into the COMPASS™ tool. Participation was voluntary, and verbal consent was obtained prior to data collection. Field notes were used to document missing information, items that generated uncertainty, terminology-related issues, and suggestions for refinement.

For each resource, item completion, time to completion, perceived completeness, and perceived usefulness were recorded. Perceived completeness and usefulness were rated by respondents using 5-point Likert scales. Minor refinements to response categories were introduced following preliminary field testing to enhance operational sensitivity [17, 18]. For example, after field testing, the original response category “21–50 participants” was subdivided into “21–30” and “31–50 participants” to better capture differences in participation capacity. Descriptive statistics were used to summarise feasibility indicators.

Preliminary reliability assessment

Preliminary inter-rater reliability analyses were conducted to explore the consistency of the COMPASS™ tool. IRR was assessed using Fleiss’ Kappa coefficients [19]. Three independent evaluators applied the instrument to the same set of ten community resources. The evaluators included two community health specialist nurses and one municipal community technician, all with experience in community-based initiatives.

To introduce contextual variability, the resources were selected from two different settings: five located in Odivelas (mainland Portugal) and five in São Miguel, Azores. All evaluations were conducted exclusively through documentary analysis. To ensure consistency across raters, the principal investigator provided all evaluators with the same set of documentary sources for each resource, including publicly available information such as institutional websites, programme descriptions, and online documentation. Each evaluator independently completed the COMPASS™ tool for the same ten resources based on these materials.

Temporal stability was also explored. Two of the three evaluators reassessed the same set of resources after a three-week interval, using the same documentary sources and application procedures. Temporal stability was assessed using Cohen’s κ coefficients to estimate agreement between the two administrations of the instrument. These analyses were intended as a preliminary exploration of the consistency and temporal stability of the instrument, rather than a comprehensive validation of the instrument’s measurement performance [19].

Results

Content validity: delphi process

Seven international experts in social prescribing were invited to participate in the Delphi process, of whom five agreed and completed the first round (response rate: 71.4%). These experts were based in the United Kingdom, Ireland, and Australia and had recognised experience in social prescribing research, policy, or implementation. The same five experts also completed the second round, ensuring consistency of the expert panel across Delphi rounds and incorporating perspectives from health systems with established social prescribing models.

Round 1 results

The initial version of the COMPASS™ tool comprised 36 items and was subjected to Round 1 of the modified Delphi process. The S-CVI/Ave was 0.83 for clarity and 0.86 for relevance, resulting in an overall mean content validity score of 0.85. Eight items fell below the recommended I-CVI threshold of 0.78 in at least one domain, indicating the need for further refinement.

Qualitative thematic analysis of expert comments in Round 1 identified six main themes: terminological clarity, ambiguity or redundancy, suggested additional content, cultural or contextual adaptation, structure and response format, and conceptual adequacy and operational relevance (Table 1). The most frequent themes concerned conceptual adequacy and operational relevance, structure and response format, and ambiguity or redundancy. Overall, experts recommended clearer and more consistent terminology, greater use of closed-ended response options, reduction of overlapping items, and refinement of participation, accessibility, cost, and referral-related domains.

Table 1.

Thematic analysis of expert qualitative comments in Delphi Round 1

Thematic category	Summary of expert feedback	Illustrative anonymised comments	Frequency
Terminological clarity	Experts suggested clearer and more consistent terminology, particularly regarding terms such as “resource”, “service”, “specific group”, “institutionalised”, and “approximate year”.	“Maybe ‘approximate year’ so that people don’t get stumped.” / “Consistency of terminology would be good.” / “What do you mean by institutionalised? This isn’t clear.”	6
Ambiguity or redundancy	Several comments identified overlap between items or questions that appeared to ask for similar information.	“This might crossover with the previous question.” / “You have already asked about the characteristics of the target audience.” / “Goes with prior question about co-design development.”	8
Suggested additional content	Experts recommended adding content domains not sufficiently captured in the initial version, including safety arrangements, individual or group-based formats, and actions to reduce barriers to participation.	“Did you ask if activities are solo or group based?” / “I would also ask if the necessary health and safety arrangements are in place.” / “You could also ask if any actions are being taken to reduce the barriers to participation.”	2
Cultural or contextual adaptation	Comments highlighted the need to consider cultural inclusivity, intergenerational and cross-cultural participation, and the contextual meaning of professional or cultural background.	“This goes with intergenerational?” / “What about cross-cultural participation?” / “Expertise or cultural background.”	3
Structure and response format	Experts suggested improving the structure, layout, order, and response formats of the tool, particularly by using multiple-choice or tick-box options and allowing “other” responses.	“Multiple click options would be better for easier coding later, with an ‘other’ option included.” / “Using tick boxes for responses would help clarify things.” / “Ensure there are options for virtual/digital locations as well as physical locations.”	10
Conceptual adequacy and operational relevance	Experts questioned the purpose or operational relevance of some items and suggested refining constructs such as participation, target age group, cost, sustainability, and required qualifications.	“What are you actually interested in?” / “Types of participation to me means how physically and mentally engaged I am.” / “Are you asking if the recipient needs to pay to attend? Or if the activity itself is funded?”	11

Frequencies refer to the number of qualitative comments coded within each thematic category. Some comments addressed more than one issue; however, each comment was assigned to the dominant theme for reporting purposes

These qualitative findings, together with the I-CVI results, informed substantive revisions to the instrument prior to Round 2. Revisions included harmonising terminology, merging or removing redundant items, adding or refining response options, clarifying participation categories, and strengthening items related to accessibility, barriers, digital or online participation, safety considerations, and referral pathways (Supplementary Material I).

Round 2 results

Following revision, the COMPASS™ tool was reduced from 36 to 21 items and re-evaluated in Round 2 of the modified Delphi process. This round yielded near-perfect consensus. The S-CVI/Ave reached 0.98 for clarity and 0.98 for relevance (Table 2).

Table 2.

I-CVI for clarity and relevance in Delphi Round 2 (only the final 21 items retained after Round 1 revisions are presented)

Item	Item description	I-CVI Clarity	I-CVI Relevance
1	Name of the initiative	1.00	1.00
2	Organisation responsible for managing the initiative	1.00	1.00
3	Approximate year in which the initiative was established	1.00	0.80
4	Type of initiative	1.00	1.00
5	Location of the initiative	1.00	1.00
6	Contact information for the initiative	1.00	1.00
7	Social media presence of the initiative	1.00	1.00
8	Main target population of the initiative	1.00	1.00
9	Main needs or areas addressed by the initiative	1.00	1.00
10	Mode of delivery of the initiative	1.00	1.00
11	Frequency of the initiative	0.80	1.00
12	Average number of participants per session	1.00	1.00
13	Level of participant involvement	1.00	1.00
14	Cost or payment requirements for participation	0.80	1.00
15	Barriers to participation and whether these are addressed	1.00	1.00
16	Accessibility and inclusion features available	1.00	1.00
17	Defined referral process for the initiative	1.00	1.00
18	Training or experience of those responsible for the initiative	1.00	1.00
19	Promotion or dissemination channels used by the initiative	1.00	1.00
20	Feedback mechanism between referral services and the initiative	1.00	1.00
21	Availability of formal evaluation or impact reporting	1.00	0.80

Median and IQR analyses confirmed that all items met the predefined criteria (median ≥ 4; IQR ≤ 1) for both clarity and relevance, with most items demonstrating an IQR of 0, indicating minimal variability in expert ratings (Supplementary Material II). Minor divergence remained in three items, none of which required further modification.

Qualitative feedback in Round 2 was limited and indicated that most concerns raised in Round 1 had been resolved. Remaining comments focused on minor terminology and response-option refinements, and no new major conceptual issues emerged. One expert commented that “the format is much clearer now—well done”, supporting the perceived improvements in clarity, structure, and usability. Given these results, a third Delphi round was deemed unnecessary.

Cross-cultural adaptation results

The first translation prioritised fidelity to the original wording, while the second emphasised natural language use in European Portuguese. Differences between the versions were primarily stylistic rather than conceptual. Both versions were subsequently analysed item by item, and the consensus version prioritized the combination of: (i) fidelity to the original content and structure; (ii) linguistic adequacy to contemporary European Portuguese; and (iii) maintenance of semantic equivalence without the introduction of new concepts. In cases of divergence, preference was given to the formulation that best preserved the meaning of the original English text, while simultaneously ensuring a fluent and natural reading for Portuguese users. For example, in the item referring to the mode of delivery of the initiative (“In-person / Online / Hybrid”), the literal option “Híbrido” from T1 was replaced in the consensus version by “Formato híbrido”, proposed in T2, as it represents a more natural and commonly used expression in European Portuguese, without altering the original meaning. Similar principles guided other harmonisation decisions.

Comparison between the original English version of the COMPASS™ tool and its back-translation demonstrated a high level of semantic and conceptual equivalence. The minor differences identified are limited to stylistic or synonymous lexical choices rather than true alterations of meaning. For example, the category “teenagers (13–17 years)” appears in the back-translated version as “adolescents (13–17 years)”, a terminological preference that preserves the exact same age boundaries and conceptual intent. Similarly, the expression “social support or befriending service” is rendered as “social support or companionship service”, where companionship remains a close semantic equivalent to befriending.

One point that merits discussion concerns the term “social prescribing”, which appeared as “social prescription” in the back-translation. Although linguistically understandable, this expression does not correspond to the terminology most commonly used in the international literature and policy documents, where social prescribing is the established term. The observed difference is therefore interpreted as a translation-related variation rather than a conceptual discrepancy and was resolved during the review process to ensure consistency with the original construct.

Preliminary field testing

Feasibility testing

The Portuguese version of COMPASS™ was applied to 16 community resources in a Lisbon-area municipality. The mapped resources included physical activity programmes, cultural and artistic activities, volunteering initiatives, and community-based social support settings. Overall item completion was very high. Out of a total of 336 expected item responses (21 items per resource), 321 items were completed, corresponding to an overall completion rate of 95.5%.

Missing data were limited and concentrated in a small number of items. Information regarding the year in which the initiative was established (Item 1.3) and the physical location of the resource (Item 1.5) was less consistently reported. In several cases, the latter could not be specified because activities do not take place in a single fixed venue; instead, locations vary depending on availability, seasonality, or logistical arrangements, making it difficult to indicate one stable physical location. In two community centres, the item referring to approximate cost for participants (Item 4.1) could not be completed, as costs were dependent on individual circumstances and therefore not available at the time of data collection. All remaining items demonstrated near-complete or complete response rates across resources.

The mean time required to complete the COMPASS™ tool was 14.8 min per resource, with completion times ranging from 11 to 19 min, indicating good feasibility for use in routine community resource mapping activities.

Perceived completeness of the information captured by the COMPASS™ tool was assessed by the interviewed respondents using a 5-point Likert scale (1 = very incomplete; 5 = very complete). The mean perceived completeness rating of the tool across resources was 4.4, indicating that respondents generally considered the instrument to capture complete or very complete information. Perceived usefulness was also rated by respondents on a 5-point Likert scale (1 = completely useless; 5 = very useful), yielding a mean score of 4.6, suggesting high perceived utility of the tool for systematically mapping community resources.

The mapped resources comprised a heterogeneous set of community initiatives, including physical activity programmes (e.g. exercise groups, walking groups, and organised senior outings), cultural and artistic activities (such as senior theatre, music and band participation, and reading or library-based clubs), volunteering opportunities, and community-based social support settings (such as parish or neighbourhood community centres and residents’ associations). Several initiatives combined elements across these domains, reflecting the multifaceted nature of community-based provision. This diversity ensured that the COMPASS™ tool was tested across a wide range of community resources relevant to social prescribing, despite the absence of a formally implemented social prescribing system at the time of data collection.

Qualitative field notes highlighted minor terminology-related suggestions, including a recommendation to replace the term “fitness/sports instructor” with “exercise professional”, which was considered more appropriate and aligned with national professional designations. Additional observations indicated that several activities were managed by the same responsible community professional (Table 3).

Table 3.

Summary of preliminary field testing results by type of community initiative*

Type of Initiative (major	Overall item completion (%)	Missing Data	Mean completion time (min)	Perceived completeness of instrument (1–5)	Perceived utility of instrument (1–5)	Selected qualitative field observations
Physical activity and movement-based programmes (exercise group, walking group)	90.4	Item 1.5 (location); occasionally Item 1.3 (year established)	14.7	4.3	4.6	Activities often occur in variable or outdoor locations depending on season and logistics
Cultural and artistic activities (senior theatre, music/band participation, senior university, reading and library-based clubs)	99.1	Item 1.3 (year established)	15.0	4.5	4.7	Structured programmes with stable organisation facilitated completion across diverse formats; Multiple initiatives managed by the same responsible professional.
Volunteering initiatives (local volunteering bank)	95.3	Item 1.5 (location)	15.0	4.4	4.6	Centralised coordination; multiple initiatives managed by the same responsible professional.
Community-based social support settings (parish day centres, residents’ associations)	97.2	Item 4.1 (approximate cost)	14.5	4.4	4.5	Cost information often dependent on individual assessment; shared management structures observed
Total	95.5	------------------	14.8	4.4	4.6	----------------------

*Initiative types are used for descriptive purposes only and reflect the predominant activity focus of each initiative

Field testing indicated that the participation range “21–50 participants” encompassed initiatives with heterogeneous participation capacities. Specifically, six out of the sixteen mapped initiatives (37.5%) reported participation levels within this range, with observed values spanning approximately 25, 30, 43, and close to 50 participants. Based on these observations, this response category was further refined in the final version of the instrument to improve operational sensitivity.

Preliminary reliability assessment

Inter-rater reliability was assessed using a subset of 10 community resources from two territorial contexts. IRR analysis showed substantial agreement across the instrument sections. Item-level Fleiss’ κ coefficients varied across items, with several items demonstrating perfect agreement (κ = 1.00). Section-level mean κ values ranged approximately from 0.65 to 0.72, indicating substantial agreement among raters (Table 4; Supplementary Material 3).

Table 4.

Inter-rater reliability of the COMPASS™ tool by instrument section (Fleiss’ κ)

Sections	Mean Fleiss’ κ
1- Social Prescribing Resource Identification	0.73
2- Target Population and Objectives	0.68
3- Implementation and Participation Characteristics	0.66
4- Accessibility and Inclusion	0.65
5- Referral, Coordination and Evaluation	0.72
Overall	0.69

Items with no variability across raters produced non-estimable κ values and were excluded from summary statistics.

Temporal stability analysis showed high consistency between the two administrations of the instrument. Item-level Cohen’s κ values ranged from − 0.18 to 1.00, with a median κ of 1.00. Percentage agreement ranged from 60% to 100%, indicating overall high stability of responses between the two time points (Table 5; Supplementary Material 4).

Table 5.

Temporal stability of the COMPASS™ tool (test–retest reliability)

Indicator	Value
Median Cohen’s κ	1.00
Range Cohen’s κ	−0.18–1.00
Percentage agreement	60–100%
Number of resources	10
Time interval	3 weeks

Discussion

This study describes the development, content validation, and cultural adaptation of the COMPASS™ tool, addressing a methodological gap in the operationalisation of community resource mapping within population health and social prescribing systems. The use of a modified Delphi process resulted in strong expert consensus, supported by both high content validity indices and complementary median and IQR analyses, confirming the clarity and relevance of the instrument’s content. As social prescribing becomes increasingly integrated into health systems internationally, the availability of structured and transferable mapping tools may support more systematic identification and organisation of community resources, contributing to improved population health monitoring and more consistent responses to social determinants of health.

The findings should be interpreted in light of the broader methodological challenges identified in the literature regarding community resource and asset mapping. As highlighted by Luo et al. [6], existing approaches are characterised by a lack of standardisation, conceptual inconsistency, and variable methodological rigor, which limit comparability across studies and settings. The development and validation of the COMPASS™ tool directly respond to these challenges by offering a structured, theory-informed instrument grounded in international guidance and refined through expert consensus. By operationalising key thematic domains into standardised, closed-ended items and demonstrating feasibility in a municipal context, the COMPASS™ tool contributes to improving methodological coherence and transparency in the assessment of community resources for social prescribing.

The cross-cultural adaptation process demonstrated that COMPASS™ can be translated into European Portuguese while maintaining semantic and conceptual equivalence with the original version. This is particularly important in social prescribing, where community structures, service organisation, referral practices, and cultural meanings of participation vary substantially across contexts. Tools developed within one health system may not be directly transferable to another without careful adaptation. For example, referral mechanisms, professional roles such as link workers, eligibility criteria, safeguarding expectations, cost arrangements, and the organisation of municipal or voluntary-sector services may differ substantially between settings. The identification and correction of minor terminological issues therefore strengthened the interpretability and potential transferability of the Portuguese version, while also highlighting the need for similar adaptation procedures before use in other cultural or health system contexts.

Although formal social prescribing pathways are not yet widely implemented in Portugal [1], this emerging context provides an opportunity to develop and test the methodological infrastructure required before referral pathways are formalised. The decision to translate and field-test COMPASS™ in Portuguese was therefore intended to examine whether the tool could capture the characteristics of community resources in a real municipal context that may support future implementation. However, because the tool was not tested within an operational social prescribing pathway, its usefulness for routine referral decision-making remains to be evaluated in future studies.

Findings from the preliminary field testing offer insights into the organisational characteristics of municipal community resources. Several activities were managed by the same responsible professional, pointing to potential efficiencies in future data collection, coordination, and resource management processes. From an implementation perspective, this suggests that community resource mapping initiatives may benefit from recognising shared governance or management structures, which could streamline updating procedures and reduce duplication of effort.

Difficulties in reporting a fixed physical location for some community resources further highlight an important characteristic of municipal-level initiatives, namely their contextual and often itinerant nature. Findings from the field testing suggest that, for such resources, contact details of the responsible person or organisation may constitute a more reliable and actionable point of reference than a precise geographical address. In practice, individuals seeking to engage with these activities may be expected to initiate contact by telephone or email, rather than attending a predetermined location. This observation has implications for how community resources are documented and accessed within social prescribing-informed systems, indicating that flexibility in how “location” is conceptualised may be necessary to accurately reflect real-world community provision.

An additional insight emerging from the field testing relates to the hybrid and multi-component nature of many community initiatives. Several mapped resources simultaneously combined physical, cultural, educational, and social elements, challenging rigid categorical distinctions. This observation aligns with broader understandings of community-based interventions as complex and context-dependent, rather than discrete or single-purpose services. From a methodological perspective, this reinforces the importance of flexible and inclusive mapping frameworks capable of capturing such complexity. Rather than forcing mutually exclusive classifications, COMPASS™ is designed to document multiple relevant dimensions of each resource, thereby accommodating the blended nature of real-world community provision.

Variability observed within the original participation range “21–50 participants” highlights an important operational consideration in community resource mapping. Although initially conceived as a single category, field testing demonstrated that initiatives within this range may differ substantially in terms of organisational demands, facilitation requirements, accessibility, and group management dynamics. In response to these observations, the participation capacity category was refined into two subcategories, “21–30” and “31–50 participants”, in the final version of the tool. This refinement did not alter the conceptual structure of the item, but improved its operational granularity based on empirical observations during field testing. Such iterative refinements are consistent with guidance on complex intervention development and instrument refinement, where pilot testing can inform adjustments to item wording and response formats [17, 18, 20, 21].

The difficulty in recording approximate costs for some resources also has implications for equity and implementation. Although COMPASS™ already allows users to indicate whether participation is free, paid, or falls under another arrangement, field testing showed that exact costs may not always be fixed or known in advance. In some community-based services, payment may depend on income, membership status, individual assessment, subsidies, or other case-by-case arrangements. This suggests that future guidance for COMPASS™ users should clarify how to record variable or means-tested costs, including how to identify the contact person or organisation responsible for confirming payment conditions.

The reliability analyses provide preliminary evidence that COMPASS™ can be applied consistently under standardised testing conditions. Section-level Fleiss’ κ values fell within the range typically interpreted as substantial agreement, suggesting that different evaluators were able to apply the instrument consistently when analysing the same community resources. Temporal stability findings also indicated high consistency between administrations. However, the high level of agreement observed for many items may partly reflect the factual nature of several instrument fields, such as organisational type, contact information, service format, or referral mechanisms. These variables are inherently less subject to interpretative variation and therefore tend to produce higher agreement. Future reliability studies should distinguish between factual and more interpretative items, as agreement on interpretative fields may provide a more stringent assessment of consistency across evaluators.

A small number of items presented lower or negative κ values. This pattern is commonly reported in reliability studies involving binary variables and small samples, where limited variability in responses can affect κ estimates even when observed agreement remains relatively high [19]. Overall, the reliability findings should therefore be interpreted as preliminary evidence of consistency rather than definitive validation of the instrument’s measurement performance.

Strengths and limitations

Strengths of this study include adherence to international methodological standards, the integration of qualitative and quantitative expert feedback through a structured Delphi process, and the explicit focus on real-world applicability of the instrument. In addition, preliminary reliability testing was conducted, including inter-rater agreement and temporal stability analyses, providing initial evidence regarding the consistency of the instrument when applied to community resources.

Several limitations should be considered when interpreting these findings. First, the Delphi panel was small. Although the panel was composed of highly specialised international experts and achieved complete retention across both rounds, a larger and more diverse panel may have generated additional perspectives on the content and applicability of the tool [14]. Future studies would benefit from including experts from additional countries, health systems, professional backgrounds, and community-sector roles.

Limitations also relate to the feasibility testing phase, which was conducted by the research team. The researcher’s dual role as developer and tester may have introduced bias in the interpretation of feasibility-related observations and implementation challenges. The high item completion rate may have been influenced by the resolution of ambiguities during application rather than their registration as missing data. Perceived usefulness ratings, collected during researcher-led interviews, may also reflect social desirability bias. Although the mean completion time of 14.8 min suggests preliminary feasibility, this estimate should be interpreted cautiously because the tool was applied by the research team rather than by practitioners routinely involved in social prescribing or community resource mapping.

Feasibility testing was conducted within a single municipality, which may limit the generalisability of findings to other geographical or organisational contexts. Although the reliability assessment included resources from two territorial contexts, Odivelas and São Miguel Island, Azores, the small number of cases does not allow conclusions regarding the tool’s performance across different settings. Replication in different municipalities, regions, and health system contexts will therefore be necessary to further assess transferability.

The reliability analyses were also conducted using a relatively small pilot sample of community resources, which may have contributed to instability in κ estimates for some items. In addition, the binary nature of several variables resulted in limited variability in responses, a factor known to influence κ statistics [19]. Items with no variability across raters produced non-estimable κ values and were excluded from summary statistics. These findings should therefore be interpreted as preliminary results from a pilot reliability assessment.

The use of documentary analysis for the reliability assessment was intended to ensure that all evaluators had access to identical information, thereby reducing variability attributable to differences in data collection interactions. However, this approach may not fully reflect real-world social prescribing practice, where practitioners may contact organisations by telephone, email, or in person to clarify eligibility, costs, accessibility, and availability. It may also under-represent community resources with limited online or documentary visibility. Future testing should therefore examine the use of COMPASS™ under more naturalistic mapping conditions.

Another limitation is that the original English version of COMPASS™ has not yet been field-tested in English-speaking settings. Although the instrument was originally developed in English to facilitate expert review by an international panel, its practical application was only tested in the Portuguese version. Empirical evidence regarding feasibility and usability in English-speaking social prescribing contexts is therefore not yet available.

A further limitation is the absence of formal patient and public involvement and engagement (PPIE) in the development and preliminary testing of COMPASS™. Although the tool is primarily designed for use by professionals mapping community resources, its ultimate purpose is to improve access to community support for individuals and populations. The perspectives of potential users, community members, and people who may be referred through social prescribing pathways are therefore essential. Future work should include PPIE to assess the acceptability, relevance, accessibility, and comprehensibility of the information captured by the tool.

Finally, minor refinements to response categories were introduced following preliminary field testing to improve operational sensitivity. Although these adjustments were empirically informed, they were not subjected to further expert consensus or validation testing and should therefore be interpreted as iterative refinements rather than formal revalidation.

Implications and future research

Future research should focus on examining the reliability and practical performance of the COMPASS™ tool in applied settings, particularly once formal social prescribing pathways are established. Further studies should assess inter-rater agreement and consistency of resource classification across larger samples of community resources and across different territorial contexts. Future reliability studies should also distinguish between factual and more interpretative items, as agreement on interpretative fields may provide a more stringent assessment of consistency across evaluators.

In addition, research should examine the usability and added value of COMPASS™ among practitioners who regularly map or use community resources, including link workers, community health professionals, and municipal teams. Such studies should compare COMPASS™ with existing informal approaches, such as spreadsheets or locally maintained resource directories, and explore its potential impact on referral quality, referral matching, and collaboration between health and community sectors.

Future development should include patient and public contributors, including community members and individuals who may be referred through future social prescribing pathways, to assess whether the domains captured by COMPASS™ reflect the information people need when deciding whether a resource is accessible, acceptable, affordable, and meaningful. This is particularly relevant in the Portuguese context, where social prescribing pathways are still emerging and where co-design with future users could inform both the content and format of the tool before wider implementation. Cross-national testing of the original English version would also provide valuable evidence regarding its applicability in diverse cultural and policy environments.

Finally, future studies could explore the integration of COMPASS™ within digital platforms for social prescribing and community resource directories, supporting systematic updating, interoperability with health information systems, and improved accessibility for practitioners. This study therefore represents an early stage in the development and validation pathway of COMPASS™, with future work needed to test the tool in routine social prescribing practice and assess how its outputs can support referral decision-making.

Conclusions

The COMPASS™ tool demonstrated satisfactory content validity and successful cultural adaptation for use in Portugal. Preliminary reliability analyses indicated substantial inter-rater agreement and high temporal stability under standardised testing conditions.

These findings provide initial evidence supporting the feasibility of COMPASS™ for structured community resource mapping, with potential relevance for population health systems seeking to integrate social prescribing and address the social determinants of health. Further testing in larger, more diverse, and operational social prescribing contexts will be necessary to confirm the robustness, usability, and wider applicability of the instrument.

Abbreviations

COMPASS™

Community Resource Mapping Tool for Social Prescribing

CVI

Content Validity Index

I-CVI

Item-level Content Validity Index

S-CVI/Ave

Scale-level Content Validity Index (average)

IQR

Interquartile Range

IRR

Inter-rater Reliability

PPIE

Patient and Public Involvement and Engagement

WHO

World Health Organization

Authors’ contributions

SS, PSC and AC contributed to the study design, data collection, statistical analysis, and drafting of the manuscript. PSC and AC supervised the study and guided the analytical strategy. SS finalised the manuscript. All authors read and approved the final manuscript.

Funding

The first author is supported by a doctoral scholarship from Fundação para a Ciência e a Tecnologia, Portugal (2025.03890.BDANA). No additional external funding was received for this study.

Data availability

The COMPASS™ tool is the intellectual property of the authors. The dataset generated and/or analysed during the current study is not publicly available due to the nature of the data but is available from the corresponding author on reasonable request. The instrument is available for non-commercial research and educational use upon request to the corresponding author. Permission is required for adaptation or commercial use.

Declarations

Ethics approval and consent to participate

This study involved voluntary participation of professionals and community initiative leaders and did not include the collection of personal or sensitive data from individuals. Information collected during the field testing phase concerned organisational characteristics of community initiatives rather than personal information. In accordance with institutional and national guidelines for low-risk research involving professionals and organisational data, formal ethical approval was not required. All procedures were conducted in accordance with relevant guidelines and regulations, including the principles of the Declaration of Helsinki. All participants were informed about the purpose of the study, assured of confidentiality, and provided verbal informed consent prior to participation. Participation was voluntary, and participants could withdraw at any time without consequence.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.