Information needs of early-stage prostate cancer patients: within- and between-group agreement of patients and health professionals

Abstract

Purpose

The aims of this study were to analyze agreement on information needs within a group of early-state prostate cancer patients and to compare information preferences of patients with the view of health-care professionals about patients’ needs.

Methods

Sample consists of patients (n = 128) and six subgroups of health-care professionals (urologists, n = 32; nurses, n = 95; radiotherapy technologists (RTTs), n = 36; medical oncologists, n = 19; radiation oncologists, n = 12; general practitioners (GPs), n = 10). Information needs have been assessed with 92 questions concerning prostate cancer and its treatment. Respondents judged the importance of addressing each question. Within- and between-group agreements of patients and health-care professional groups were estimated with raw agreement indices as well as chance-corrected Kappa and Gwet’s AC1 measures. Finally, group-specific core items rated with high importance as well as high agreement were defined.

Results

Patients rated on average (median) half, i.e., 51 out 92 items as essential (interquartile range (IQR) = 36–66), 26 items as desired (IQR = 14–38), and 10 items as avoidable (IQR = 2–22). Within-group agreement on the presented information topics is modest for any participating group (AC1_patients = 0.319; AC1_{professionals} = 0.295–0.398). Agreement between patients and professionals is low too (AC1 = 0.282–0.329). Defining group-specific core sets of information topics results in 51 items being part of at least one core set. Concordance of the item core sets of patients and professionals is moderate with κ = 0.38–0.66, sensitivity of professionals’ core sets for patients’ preferences varies between 56 and 74 %.

Conclusions

Results emphasize the need for dialogue between doctor/professional and patient in identifying the information needed by individual patients and support the importance of shared decision making.

Introduction

Patients newly diagnosed with localized prostate cancer are faced with several treatment options such as watchful waiting or active surveillance, radical prostatectomy, radiation therapy, as well as others. Each treatment has its benefits as well as its risks and harms. However, scientific evidence on the efficacy of these treatments is still limited, in particular, long-term randomized clinical trial data are lacking [1, 2]. Thus, to decide on a specific therapy is a complex undertaking for patients by having to consider a variety of often sophisticated data.

Some prostate cancer patients want to know as much as possible about their disease and its treatment [3–5] but are overwhelmed by the complexity of information presented at the same time [6–8]. Therefore, patients are strongly in need of advice by the physician, and most patients wish to be involved actively in the treatment decision process. It is therefore not surprising that satisfaction with information received on illness and therapy is positively correlated with quality of life of prostate cancer patients after treatment [3, 9].

However, patients often report that their information needs were not satisfactorily met [10–12]. There seems to be also much heterogeneity with regard to amount and quality of information delivered by professionals [6, 11]. A majority of studies are limited to the counseling of patients by urologists, whereas the specific role of other health professionals is neglected. However, the literature on multidisciplinary cancer care underlines the benefit for patients cared of by a multidisciplinary team of specialists [13–15]. In particular, some research proves that the use of specialist nurses in prostate cancer care results in better covering of patients’ information needs [16]. Besides, research suggests that the patients’ decision making on a specific therapy option is highly dependent on the professional background of the consulting health specialist [17–19]. To summarize, information needs of prostate cancer patients are broad and manifold, and health professionals counsel patients quite inconsistently as well. Therefore, the present study addresses two main complexes of questions:

1. Agreement within and between groups of patients and health professionals, i.e.: To what extent do patients agree among themselves on specific information topics they want to be addressed in relation to prostate cancer and its treatment? And to what extent do health professionals, being involved in the treatment and care of prostate cancer patients, agree on information to be delivered to patients? Finally, do professionals’ ratings of information topics correspond to patients’ needs?

2. Patients’ and professionals’ core sets of vital information topics: Is it possible to derive core sets of information topics on prostate cancer being essential from the view of patients as well as professionals? To what degree do patients’ and professionals’ core sets overlap?

Methods

Design

The present study comprises two groups of participants, patients with established diagnosis of localized prostate cancer and health professionals engaged in the treatment of these patients. Both groups were provided with the same questionnaire developed by a Canadian research group led by Feldman-Stewart [5] and found to be appropriate for prostate cancer patients in eight other countries [20]. It consisted of 92 information topics about prostate cancer and its treatment. Patients had to rate the importance of each item retrospectively with regard to their decision making on choosing a specific treatment option. Professionals had to rate the information topics the same way, however, by trying to adopt the patient view.

Samples

Patient sample

The study population encompasses patients with established diagnosis of prostate cancer fulfilling the following additional selection criteria:

• Localized tumor stage (T1 or T2, PSA < 20 ng/mL, Gleason score < 8),

• Having obtained diagnosis 3 to 24 months before date of study,

• Being treated in one of four urology departments of clinics in the German-speaking part of Switzerland.

Of the 330 patients fulfilling the abovementioned criteria for inclusion, 179 (54 %) answered the questionnaires. Finally, 51 patients had to be excluded from the study sample because of invalid questionnaires or time of diagnosis dating back more than 24 months. The final study sample consists of n = 128 patients (mean age = 66 years). An analysis of response rates by age and type of treatment indicates no age-related differences. However, the response differed by type of treatment with a low rate (36 %) for patients who had received radiotherapy.

Health professionals sample

In order to achieve comparability with the study by Capirci et al. [21], the same health-care professionals were to be included. Therefore, the study population encompasses health professionals involved in treating prostate cancer patients in the 12 months prior to study date. Samples of physicians were drawn from the registry of the Swiss Medical Association (FMH) with sample sizes of n = 100 per group (exception: radiation oncologists, n = 47). Nurses and radiotherapy technologists (RTTs) were recruited from the participating centers with n = 197 nurses and n = 45 RTTs contacted. The final professional sample that could be used for analysis consisted of the following: 32 urologists (response rate = 32 %; mean age = 47 years; proportion men = 100 %), 19 medical oncologists (19 %; 47; 68 %), 11 radiation oncologists (23 %; 47; 75 %), 36 RTTs (73 %; 35; 14 %), 11 general practitioners (GPs) (11 %; 54; 60 %), and 99 nurses (50 %, 38; 16 %). Total sample size was n = 208 health professionals.

Instrument

The questionnaire used was developed by Feldman-Stewart et al. [5] based on interviews with prostate cancer patients, their family members, and medical experts. It was revised in further studies and then used internationally. The authors of the instrument paid attention to cover all possible aspects that may be relevant for those involved in the therapeutic decision making. This work resulted in a set of 92 questions referring to the epidemiology and natural history of prostate cancer, the effectiveness and side effects of treatments, and the experience of the professionals. Patients were asked to go back in their mind to the time of treatment decision and to indicate for each item and how important it was for their decision making. The importance of items had to be rated by using four possible categories: “no opinion,” “avoid,” “desired,” or “essential.” Health professionals had to judge the same 92 items in the same way, however, by taking the view of a hypothetical prostate cancer patient prior to the treatment decision. In order to ensure that each participant refers to the same situation, he/her was provided with a vignette of a hypothetical prostate cancer patient. After judging the importance of addressing the various questions, the participants filled out a form providing further demographic and professional background information.

Statistical analyses

Two types of concordance were analyzed: (1) concordance of judgments within a specific group of participants (i.e., within patients, and within professionals) and (2) concordance of judgments between groups, e.g., patients and urologists. To express concordance, we use inter-rater agreement coefficients. The appropriate measure of inter-rater agreement for our data structure is Fleiss’ Kappa (κ) coefficient [22], an extension of the κ measure of Cohen [23]. However, κ statistics can be inadequately biased by the variation of the trait prevalence rate and by the magnitude of raters’ classification probabilities [24]. Gwet [25] proposed an alternative agreement measure, the AC1 for categorical data. We will report both measures, the unweighted Fleiss’-κ as well as the unweighted AC1 for multiple raters and multiple subjects.

Usually, agreement coefficients measure concordance of two or more raters’ judgments on one or more subjects. However, we are also interested in measuring the agreement between groups of raters, in particular, agreement of patients and professionals. Technically speaking, we need to analyze the judgments on multiple subjects of all possible pairs of one rater group with another rater group. We derived the between-group agreement measure according to the following equation:

$$C_{1\times 2}=\frac{C_{1+2}-\left(\left(C_1*w_1\right)+\left(C_2*w_2\right)\right)}{w_{1\times 2}}$$

with:

• C _1×2 = between-group agreement coefficient of group 1 ratings paired with group 2 ratings

• C ₁₊₂ = total agreement coefficient of pooled group 1 and group2 ratings

• C ₁, C ₂ = within-group agreement coefficients of group 1 and group 2

• w ₁, w ₂, w_1×2 = weights for group 1 and 2 as well as for the combination of both groups.

The calculation of group weights is presented in the Appendix of this paper. Statistical analyses as well as the calculation of Fleiss’ Kappa were done with STATA SE 11. AC1 coefficients were calculated by using AgreeStat, an EXCEL VBA program developed and distributed by Gwet.1

Results

Distribution of item ratings

Figure 1 illustrates that patients rated on average (median) around half, i.e., 51 out 92 items, as essential (interquartile range (IQR) = 36–66), 26 items as desired (IQR = 14–38), and 10 items as avoidable or not assessable (IQR = 2–22). The variation of patient responses is large. This pattern of large within-group variance holds mostly true for professionals too with the exception of urologists’ ratings being somewhat more consistent compared to other groups.

Fig. 1.

Boxplots of number of items judged as “essential”, “desired,” or “avoid”/“no opinion” by patients and health professionals (maximum number of items = 92)

In general, the variation of responses to each judgment category is much larger within groups than between groups of participants: one-way ANOVA of category-specific item responses by group of subjects provided small intraclass correlation such as ρ = 0.059 (95 % confidence interval (CI) = 0.000–0.161; F = 3.57; p = 0.002) for “essential.” Quite similar results are provided for the remaining response categories such as “desired” with ρ = 0.070 (0.000–0.187; 4.11; 0.001) and “avoid/no opinion” with ρ = 0.016 (0.000–0.012; 1.68; 0.124).

Within- and between-group agreement on importance of judgments

Due to small sample size, we merged medical and radiation oncologists to one group and excluded GPs from the following analyses. First, we calculated for each group of participants, their raw agreement as well as chance-corrected agreement of ratings using Fleiss’ Kappa (κ) and Gwet’s AC1. With regard to overall raw agreement, results (Table 1) indicate that participants within a specific group agree on their ratings by around half of the total item set. Inspecting category-specific ratings reveals substantial raw agreement on the item categories “essential” and “desired,” whereas agreement on the categories “avoid” and “no opinion” is weak. However, by correcting for chance agreement, coefficients become small to moderate depending on type of measure. Finally, the 95 % CIs indicate that groups do not differ significantly from each other with regard to the amount of within-group agreement.

Table 1.

Within- and between-groups agreement of patients and health professionals about information topics on prostate cancer and its treatment

	Chance-corrected agreement		Raw agreement
	Kappa (95 % CI)	AC1 (95 % CI)	Overall	_1	_2	_3	_4
Within groups
Patients	0.085 (0.067–0.104)	0.319 (0.277–0.361)	0.454	0.040	0.200	0.323	0.607
Nurses	0.103 (0.080–0.126)	0.349 (0.303–0.394)	0.476	0.061	0.170	0.359	0.622
Urologists	0.146 (0.117–0.175)	0.295 (0.251–0.338)	0.447	0.103	0.266	0.408	0.576
RTT	0.090 (0.067–0.112)	0.398 (0.349–0.447)	0.507	0.071	0.105	0.320	0.666
Oncologists	0.096 (0.064–0.127)	0.308 (0.281–0.336)	0.449	0.049	0.196	0.460	0.520
Between pairs of groups
Patient–Nurses	0.077 (0.057–0.096)	0.285 (0.242–0.328)
Patient–Urologists	0.096 (0.076–0.116)	0.282 (0.242–0.322)
Patient–RTT	0.067 (0.050–0.083)	0.329 (0.287–0.372)
Patient–Oncologists	0.077 (0.058–0.096)	0.294 (0.264–0.324)
Nurses–Urologists	0.099 (0.081–0.117)	0.288 (0.249–0.326)
Nurses–RTT	0.087 (0.069–0.105)	0.363 (0.321–0.404)
Nurses–Oncologists	0.086 (0.066–0.105)	0.301 (0.273–0.329)
Urologists–RTT	0.091 (0.075–0.108)	0.292 (0.248–0.336)
Urologists–Oncologists	0.109 (0.089–0.128)	0.287 (0.258–0.315)
RTT–Oncologists	0.085 (0.069–0.102)	0.311 (0.281–0.340)

Answer categories: _1 = “avoid,” _2 = “no opinion,” _3 = “desired,” _4 = “essential”

Kappa unweighted Fleiss’ Kappa, AC1 unweighted Gwet’s AC1 for multiple raters and multiple subjects

Second, we calculated between-group agreement (cf. “Methods” and Appendix). Agreement between patients and professionals varies between κ = 0.067–0.096 and AC₁ = 0.282–0.329 and is comparably small like within-group agreement. The same holds true for agreement between different professional groups varying from κ = 0.085–0.109 and AC₁ = 0.287–0.363.

Agreement between patients’ and health professionals’ item core sets

We tried to build core sets of significant items for each study group (i.e., patients, health professionals). We defined “significant” as follows:

• an item is rated with high importance (≥67th percentile of the distribution of item means) and

• high agreement (≥67th percentile of the distribution of item specific raw agreement) within a sample.

These two rules produce group-specific core sets. The accordance between the patients’ core set of important items and a professionals’ core set can then be calculated. Furthermore, the sensitivity (as well as specificity) of professionals’ core set with regard to patients’ core set can be estimated. Sensitivity is interpreted as a measure of how well the judgments of a specific professional group fit the “important” information needs of patients.

Table 2 displays the 27 items of the patients’ core set as well as sensitivity and specificity of the professional sets with regard to patients’ set. Overall, somewhat more than half (51) of the items belong to at least one core set. In particular, eight items of the patients’ core set are part of every professional core set too (e.g., “What is prostate cancer?”) and another 14 items are part of a majority of professional sets (e.g., “If the prostate cancer is not treated, will it affect my bladder functioning?”). Finally, five items belong to the patients’ core set but only to a minority or even none of the professionals’ sets (e.g., “If the prostate cancer is not treated, how long will I live?”).

Table 2.

Patients’ core set of items (28 out of 92) with high importance as well as high agreement in relation to professionals’ core sets

Items	${\overline{x}}_{\mathrm{i}}$	p i	Core sets	Number of sets
What is prostate cancer?	1.90	0.85	I–V	5
If the prostate cancer is not treated, will I die from it?	1.75	0.67	I–V	5
How long can I safely take to make up my mind?	1.74	0.63	I–V	5
How long will it take to finish all my treatment?	1.74	0.63	I–V	5
If I choose radiation treatment, will I have large areas of my body radiated?	1.75	0.68	I–V	5
How does the treatment work?	1.81	0.73	I–V	5
When and how will you know if the treatment is working?	1.80	0.71	I–V	5
Will the treatment affect my bladder control? If so, for how long?	1.83	0.77	I–V	5
What does PSA mean?	1.79	0.73	I, III–V	4
If the prostate cancer is not treated, how fast will it spread?	1.77	0.70	I–IV	4
If the prostate cancer is not treated, will it be painful?	1.61	0.52	I–IV	4
If the prostate cancer is not treated, will it affect my sexual functioning?	1.59	0.51	I, II, IV, V	4
What type of monitoring will there be if I choose no treatment?	1.65	0.58	I–IV	4
How long will I have to wait to start treatment?	1.63	0.55	I–IV	4
What kind of follow-up will take place after my treatment?	1.57	0.51	I, III–V	4
Will the treatment affect my sexual functioning? If so, when and for how long?	1.66	0.56	I, II, III, V	4
If the prostate cancer is not treated, what parts of my body could be affected?	1.73	0.64	I–III	3
If the prostate cancer is not treated, will it affect my bladder functioning?	1.73	0.63	I, II, IV	3
If I do not have the treatment now, can I have it when the cancer gets worse?	1.59	0.52	I, III, V	3
What do we do if the cancer comes back?	1.75	0.68	I, II, III	3
If the treatment is not successful, what are my options?	1.79	0.73	I, II, III	3
If I get one type of treatment, can I have any of the other treatments later?	1.62	0.56	I, III, V	3
If more than one treatment is needed, how much time does each treatment take?	1.62	0.53	I, V	2
When and how will you know if I have been cured?	1.74	0.66	I, II	2
If the prostate cancer is not treated, how long will I live?	1.57	0.51	I	1
Is the equipment at the hospital up to date for treating the prostate cancer?	1.59	0.52	I	1
How experienced is my doctor in treating patients with prostate cancer?	1.55	0.52	I	1
Concordance of professionals’ core set with patients’ core set (27 items)		Kappa	Sensitivity, %	Specificity, %
Urologists (28 items)		0.66 (0.43–0.79)	74.1 (53.7–88.9)	87.7 (77.2–94.5)
Nurses (29 items)		0.54 (0.35–0.73)	70.4 (49.8–86.2)	84.6 (73.5–92.4)
RTT (29 items)		0.38 (0.18–0.59)	59.3 (38.8–77.6)	80.0 (68.2–88.9)
Oncologists (22 items)		0.47 (0.27–0.68)	55.6 (35.3–74.5)	89.2 (79.1–95.6)

${\overline{x}}_i$: item mean of patients (−1: “avoid”; 0: “no opinion”; 1: “desired”; 2: “essential”); p _i: item-specific raw agreement of patients; core sets: I patients; II nurses; III: urologists; IV RTT; V oncologists

Table 3 shows another 21 items being part of at least one professional but not of the patients’ core set. Only a few items are part of most professionals’ but not of the patient’s set. A larger number of 10 items belong only to one professional set.

Table 3.

Items belonging to at least one professional core set but not to the patients’ core set

Items	${\overline{x}}_i$	p i	Core sets	Number of sets
Will I receive medication for symptom control?	1.46	0.44	II–V	4
If I choose radiation treatment, should I take hormone pills before the radiation?	1.54	0.54	III–V	3
If I delay treatment, is there a chance I can still be cured?	1.55	0.50	III–V	3
Will the treatment cause pain? If so, for how long?	1.55	0.49	II, III, IV	3
What options/treatments do I have if my impotency is permanent?	1.56	0.50	II, III, V	3
If the prostate cancer is not treated, will it disappear on its own?	1.55	0.52	II, IV	2
Where will I go for the treatment?	1.42	0.42	II, IV	2
Can I continue my exercise program during treatment?	1.48	0.45	II, IV	2
Do I continue taking medications (prescribed and over-the-counter) during treatment?	1.54	0.49	II, IV	2
If I delay treatment, will any treatment help me?	1.49	0.48	III, V	2
Could the treatment lead to my death?	1.49	0.50	III, V	2
Are my sons at risk of developing prostate cancer?	1.08	0.32	III	1
If the prostate cancer is not treated, will I still be able to carry on my usual activities?	1.49	0.46	II	1
Will I get the same level of care regardless of my treatment decision?	1.32	0.39	V	1
Can I have sex during the course of treatment?	1.46	0.44	II	1
Will I see my doctor during the treatment?	1.20	0.35	IV	1
Will I lose my gonads/testicles?	1.43	0.44	II	1
Will the treatment make me tired? If so, for how long?	1.23	0.36	IV	1
Will the treatment cause diarrhea? If so, for how long?	1.29	0.37	IV	1
Will the treatment cause bleeding? If so, for how long?	1.45	0.43	IV	1
Will the treatment affect my bowel control? If so, for how long?	1.45	0.44	IV	1

${\overline{x}}_i$: item mean of patients (−1: “avoid”; 0: “no opinion”; 1: “desired”; 2: “essential”); p _i: item-specific raw agreement of patients; core sets: I patients; II nurses; III: urologists; IV RTT; V oncologists

The concordance of the item sets of patients and professionals is moderate with κ = 0.38–0.66. The sensitivity of professionals’ core sets varies between 56 and 74 %. Specificity of professionals’ judgments is higher (80–89 %).

Discussion

Patients newly diagnosed with localized prostate cancer are faced with several treatment options. To decide on a specific therapy is a complex undertaking for patients by having to consider a variety of sophisticated data. The present study explored the correspondence of professionals’ information priorities with patients’ information needs. We adapted the design and instrument of studies by Feldman-Stewart et al. [5] to a Swiss sample of patients and health professionals. Patients were presented with 92 information topics on prostate cancer and its treatment in order to rate the relevance of these topics with regard to their decision for treatment. Six groups of health professionals involved in the treatment and care of prostate cancer patients were presented with the same items, professionals had to rate the items from the viewpoint of a hypothetical vignette patient. Results are in line with the findings of previous studies [3, 4, 10–12], in particular, of studies by Feldman-Stewart et al. [5, 20], but our study also reveals some further insights.

First, information needs of patients with localized prostate cancer are very heterogeneous: whereas some patients are interested in as much information as possible, others report quite limited information needs [5, 26–28]. In addition to the findings of previous research, we have quantified the low agreement among patients in their appraisal of information topics. The low agreement suggests that meeting the highly individual needs of a patient might be a challenging task for the consulting health professional.

Second, the heterogeneity of patients’ information needs, however, is mirrored by the information priorities of health professionals. Results of the few studies investigating the role of different health specialists in prostate cancer care [17–19] indicate that how professionals counsel patients as well as what kind of treatment option they favor is highly dependent on their professional background. Our study presents findings for six groups of specialists indicating that, no matter what specific professional background, health professionals do only weakly agree among each other on information topics of importance for patients. We did not corroborate the findings of Capirci et al. [21] indicating that the central tendencies of appraisals by professionals are more consistent among each other than with the central tendencies of the patients’ judgments; however, what is consistent with both that study and with Feldman-Stewart’s study of professionals in Canada [26] is the wide differences in the judgments within each profession. Taking into account the modest within-group agreement of patients as well as of all groups of participating health professionals, it is not surprising that between-group agreement is modest too, i.e.: not only do the appraisals of health professionals correspond only moderately with the patient needs but also different groups of professionals agree only modestly with each other on information priorities.

Third, despite the heterogeneity of the information needs of patients as well as the priorities of professionals, we tried to build core sets of items vital to at least one group of study participants. Applying the rule that a specific item has to be rated highly and with strong consensus results in around half (51 of 92) of the presented information topics being part of at least one core set of a specific group. Correspondence between patients’ and professionals’ ratings of core set information topics is more pronounced compared to the agreement on the entire item pool. The majority (81 %) of the information topics within the patient core set are also part of at least three out of five core sets of professionals. Thus, in addition to the findings of other studies [21, 26, 29], we can conclude that there are some information topics on prostate cancer and its therapy with high relevance and substantial consensus among patients as well as professionals.

Some methodological limitations of the present study are to be mentioned: (1) It can be argued that the heterogeneous appraisals of information topics on prostate cancer is at least in some part induced by the questionnaire being used. Because it was designed to reflect a broad range of information needs of patients [5], high or already substantial agreement on all of the 92 items presented can hardly be expected. (2) The rating scale of the information topics may not allow for a sufficiently differentiated appraisal. In fact, the scale does not seem to display the complete range of possible ratings: subjects were provided with two categories for positive (“essential,” “desired”), one category for indifferent (“no opinion”), and another one for negative (“to avoid”) appraisal. Moreover, the negative category does not seem to represent the same dimension with regards to content as the positive categories do. However, the fact that some participants identified some items to be “avoided” suggests that adding a negative response option is an important addition to the more typical assessments of information needs that just elicit ratings from “not important” to “very important” [4]. (3) The sample of patients selected for participation might be too heterogeneous with regard to onset of disease. (4) Patients were asked to rate the importance of each item retrospectively regarding their decision making on choosing a specific treatment option. Therefore, we have to take into account a recall bias, meaning that memories of what kind of information on treatment that had been important may differ substantially as a function of the time since diagnosis. (5) Finally, the previous course of disease and therapy will influence the appraisal of information on its nature and treatment options.

Conclusions

Information needs with regard to treatment decision of men suffering from curable prostate cancer are broad, but very individual too. Although it is possible to reduce information about this disease on a core set of topics vital for patients, this will always result in a substantial number of men with information needs not sufficiently covered [30]. Our findings emphasize the function of the dialogue between doctor/professional and patient in providing information on a highly individual level and support the importance of shared decision making [7]. Counseling of prostate cancer patients by standardized guidelines only but also require sensitivity to the patients’ needs [31].

Furthermore, results suggest that health professionals even within the same speciality counsel men suffering from prostate cancer quite inconsistently [6, 32]. A good working exchange between different doctors as well as other professionals caring for patients seems to be of particular importance. Case management approaches to the care and coaching of cancer patients [33] as well as multidisciplinary counseling could be helpful to reach this goal [13–15]. In addition, the use of web-based tools in the field of prostate cancer counseling [32] could offer new ways of individually and flexibly tailoring knowledge transfer from professionals to patients [34].

Appendix

Calculation of between-group rater agreement based on within-group rater agreement

$$C_{1\times 2}=\frac{C_{1+2}-\left(\left(C_1*w_1\right)+\left(C_2*w_2\right)\right)}{w_{1\times 2}}$$

with:

• C _1×2 = between-group agreement coefficient of group 1 ratings paired with group 2 ratings

• C ₁₊₂ = total agreement coefficient of pooled group 1 and group2 ratings

• C ₁, C ₂ = within-group agreement coefficients of group 1 and group 2

• w ₁, w ₂, w_1×2 = weights for group 1 and 2 as wells as for the combination of both groups.

w ₁ = weighted number of rating pairs in group 1

$$w_1=\frac{\frac{\left(n_1*\left(n_1-1\right)\right)}{2}}{\frac{\left(n_{\mathrm{a}}*\left(n_{\mathrm{a}}-1\right)\right)}{2}}=\frac{\left(n_1*\left(n_1-1\right)\right)*2}{\left(n_{\mathrm{a}}*\left(n_{\mathrm{a}}-1\right)\right)*2}=\frac{\left(n_1*\left(n_1-1\right)\right)}{\left(n_{\mathrm{a}}*\left(n_{\mathrm{a}}-1\right)\right)}$$

w ₂ = weighted number of rating pairs in group 2

$$w_2=\frac{\left(n_2*\left(n_2-1\right)\right)}{\left(n_{\mathrm{a}}*\left(n_{\mathrm{a}}-1\right)\right)}$$

w _1×2 = weighted number of rating pairs of group 1 and group 2 each

$$w_{1\times 2}=\frac{\left(n_{\mathrm{a}}-\left(n_{\mathrm{a}}-1\right)\right)-\left(\left(n_1-\left(n_1-1\right)\right)+\left(n_2-\left(n_2-1\right)\right)\right)}{\left(n_{\mathrm{a}}-\left(n_{\mathrm{a}}-1\right)\right)}$$

• n ₁ = number of raters in group 1

• n ₂ = number of raters in group 2

• n _a = total number of raters in group 1 and group 2