Design and Implementation Content Validity Study: Development of an instrument for measuring Patient-Centered Communication

Abstract

Introduction: The importance of content validity in the instrument psychometric and its relevance with reliability, have made it an essential step in the instrument development. This article attempts to give an overview of the content validity process and to explain the complexity of this process by introducing an example.

Methods: We carried out a methodological study conducted to examine the content validity of the patient-centered communication instrument through a two-step process (development and judgment). At the first step, domain determination, sampling (item generation) and instrument formation and at the second step, content validity ratio, content validity index and modified kappa statistic was performed. Suggestions of expert panel and item impact scores are used to examine the instrument face validity.

Results: From a set of 188 items, content validity process identified seven dimensions includes trust building (eight items), informational support (seven items), emotional support (five items), problem solving (seven items), patient activation (10 items), intimacy/friendship (six items) and spirituality strengthening (14 items). Content validity study revealed that this instrument enjoys an appropriate level of content validity. The overall content validity index of the instrument using universal agreement approach was low; however, it can be advocated with respect to the high number of content experts that makes consensus difficult and high value of the S-CVI with the average approach, which was equal to 0.93.

Conclusion: This article illustrates acceptable quantities indices for content validity a new instrument and outlines them during design and psychometrics of patient-centered communication measuring instrument.

Introduction

In most studies, researchers study complex constructs for which valid and reliable instruments are needed.¹ Validity, which is defined as the ability of an instrument to measure the properties of the construct under study,² is a vital factor in selecting or applying an instrument. It is determined as its three common forms including content, construct, and criterion-related validity.³ Since content validity is a prerequisite for other validity, it should receive the highest priority during instrument development. Validity is not the property of an instrument, but the property of the scores achieved by an instrument used for a specific purpose on a special group of respondents. Therefore, validity evidence should be obtained on each study for which an instrument is used.⁴

Content validity, also known as definition validity and logical validity,⁵ can be defined as the ability of the selected items to reflect the variables of the construct in the measure. This type of validity addresses the degree to which items‏ of‏ an instrument sufficiently represent the content domain. It also answers the question that to what extent the selected sample in an instrument or instrument‏ items‏ is a comprehensive sample of the content.^1,6-8This type validity provides the preliminary evidence on construct validity of an instrument‏.⁹ In addition, it can provide information on the representativeness and clarity of items and help improve an instrument through achieving recommend- dations from‏ an expert panel‏.^6,10 If an instrument lacks content validity, It is impossible to establish reliability for it‏.¹¹ On the other hand although more resources should be spent for a content validity study initially, it decreases the need for resources in the future reviews of an instrument during psychometric process‏.¹

Despite the fact that in instrument development, content validity is a critical step¹²and a trigger mechanism to link abstract concepts to visible and measurable indices,⁷ it is studied superficially and transiently. This problem might be due the fact that the methods used to assess content validity in medical research literature are not referred to profoundly‏ ¹² and sufficient details have rarely been provided on content validity process in a single resource.¹³ It is possible that students do not realize complexities in this critical process.¹² Meanwhile, a number of experts have questioned historical legitimacy of content validity as a real type of validity.^14-16 These challenges about value and merit of content validity have arisen from lack of distinction between content validity and face validity, un-standardized mechanisms to determine content validity and the previously its un-quantified nature.³ This article aims to discuss on the content validity process, to train quantifying of it with a example instrument. This is designed to measure the patient-centered communication between the patients with cancer and nurses as a key member of the health care providers in oncology wards of Iran.

Nurse-patient communication

For improving patients’ outcomes, nurses cannot perform health services such as physical cares, emotional support and exchanging information with their patients without establishing a relationship with them.¹⁷During recent decades, patient-centered communication was defined as a communication in which patients’ viewpoints are actively sought by a treatment team,¹⁸ a relationship with patients, based on trust, respect, and reciprocity, and with mutually negotiated goals and expectations that can be an important support and buffer for cancer patients experiencing distress.¹⁹

Communication serves to build and maintain this relationship, to transmit information, to provide support, and to make treatment decisions. Although patient-centered communication between providers and cancer patients can significantly affect clinical outcomes²⁰and as an important element improves patient satisfaction, treatment compliance, and health outcomes,^21,22 however, recent evidence demonstrates that communication in cancer care may often be suboptimal, particularly with regard to the emotional experience of the patient.²³

Despite the public acceptance, there is little consensus on the meaning and operationalization of the concept of patient-centered communication,^19,24 so that a serious limitation is caused by lack of standard instruments to review and promote patient-centeredness in patient-healthcare communication. Part of this issue is related to the extended nature of patient-centeredness construct that has led to creating different and almost dissimilar instruments caused by researchers’ conceptualization and psychometrics.²⁵ Few‏ instruments can provide a comprehensive definition of this concept in cancer care and in a single tool.²⁶ Whereas, reviewing the literature in Iran shows that this concept has never been studied in the form of a research study. As one of the research priorities is to conduct research on cancer,²⁷ no quantitative and qualitative study has been carried out and no instrument has been made yet.

It is obvious that evaluating abilities of nurses in oncology wards to establish a patient-centered communication and its consequences require application of a reliable‏ instrument based on the context and culture of the target group.²⁶ When a new instrument is designed, measurement and report of its content validity have fundamental importance.⁸ Therefore, this study was conducted to design and to examine content validity of the instrument measuring patient-centered communication in oncology wards in northwest of Iran.

Materials and methods

This methodological study is part of a larger study carried out through the exploratory mixed method research (qualitative-quantitative) to design and psychometrics the instrument measuring patient-centered communication in oncology wards in northwest of Iran. Data in the qualitative phase of study with qualitative content analysis approach was collected by semi-structured in-depth interview with 10 patients with cancer, three family members and seven oncology nurses in the Ali-Nasab and Shahid Ayatollah Qazi‏ Tabatabai‏ Hos- pitals of Tabriz and in the quantities phase of study, during a two-step process (design – judgment), the qualitative and quantities viewpoints of 15 experts were collected.³

Ethical considerations such as approval of the ethic committee of Tabriz University of Medical Sciences, Permissions of administrators of Ali-Nasab and Shahid Ayatollah Qazi‏ Tabatabai‏ Hospitals, anonymity, informed consent, withdrawal from the study, and recording permission was respected.

Stage 1: Instrument Design

Instrument design is performed through three-‏ steps process, including determining content domain, sampling from content (item generation) and instrument construction.^11,14 the first step is determining the content domain of a construct that the instrument is made to measure it. Content domain is the content area related to the variables that being‏ measured.²⁸ It can be identified by literature review on the topic being measured, interviewing with the respondents and focus groups. Through a precise definition on the attributes and characteristics of the desired construct, a clear image of its boundaries, dimensions, and components is obtained. The qualitative research methods can also be applied to determine the variables and concepts of the pertinent construct.²⁹ The qualitative data collected in the interview with the respondents familiar with concept help enrich and develop what has been identified on concept, and are considered as an invaluable resource to generate instrument items.³⁰ To determine content domain in emotional instruments and cognitive instruments, we can use literature review and table of specifications, respectively.³ In practice, table of specifications reviews alignment of a set of items (placed in rows) with the concepts forming the construct under study (placed in columns) through collecting quantitative and qualitative evidence from experts and by analyzing‏ data.⁵ Ridenour and Newman also introduced the application of mixed method‏ (deductive- inductive) for conceptualization at the step of content domain determination and items generation.³¹ However, generating items requires a preliminary task to determine the content domain an constract.³² In addition, a useful approach would consists of returning to research questions and ensuring that the instrument items are reflect of and relevant to research questions.³³

Instrument construction is the third step in instrument design in which the items are refined and organized in a suitable format and sequence so that the finalized items are collected in a usable form.³

Stage2: Judgment

This step entails‏ confirmation‏ by a specific number of experts, indicating that instrument items and the entire instrument have content validity. For this purpose, an expert panel is appointed. Determining the number of experts has always been partly arbitrary. At least five people are recommended to have sufficient control over chance agreement. The maximum number of judges has not been determined yet; however, it is unlikely that more than 10 people are used. Anyway, as the number of experts increases, the probability of chance agreement decreases. After determining an expert panel, we can collect and analyze their quantitative and qualitative viewpoints on the relevancy or representativeness, clarity and comprehend- siveness of the items to measure the construct operationally‏ defined by these items to ensure the content validity of the instrument.^3,7,8

Quantification of Content Validity

The content validity of instrument can be determined using the viewpoints of the panel of experts. This panel consists of content experts and lay experts. Lay experts are the potential research subjects and content experts are professionals who have research experience or work in the field.³⁴Using subjects of the target group as expert ensures that the population for whom the instrument is being developed is represented¹

In qualitative content validity‏ method, content‏ experts and target group’s recommendations are adopted on observing grammar, using appropriate and correct words, applying correct and proper order of words in items and appropriate scoring.³⁵ However, in‏ the quantitative content validity method, confidence is maintained in selecting the most important and correct content in an‏ instrument, which is quantified by content validity‏ ratio (CVR). In this way, the experts are requested to specify whether an item is necessary for operating a construct in a set of items or not. To this end, they are requested to score each item from 1 to 3 with a three-degree range of “not necessary,useful but not essential,essential”‏ respectively. Content validity ratio varies between 1 and -1. The higher score indicates further agreement of members of panel on the necessity of an item in an instrument. The formula of content validity ratio is CVR=(N_e - N/2)/(N/2), in which the N_e is the number of panelists indicating "essential" and N is the total number of panelists. The numeric value of content validity ratio is determined by Lawshe Table. For example, in our study that is number of panelists 15 members,‏ if‏ CVR is bigger than 0.49, the item in the instrument with an acceptable level of significance‏ will‏ be‏ accepted.³⁶

In reports of instrument development, the most widely reported approach for content validity is the content validity index.^3,34,37Panel members is asked to rate instrument items in terms of clarity and its relevancy to the construct underlying study as per the theoretical definitions of the construct itself and its dimensions on a 4-point ordinal scale (1[not relevant], 2[somewhat relevant], 3[quite relevant], 4[highly relevant]).³⁴ A table like the one shown below(Table 1) was added to the cover letter to guide experts for scoring method.

Table 1. The table added to the cover letter to guide experts for scoring method.

Relevancy	Clarity
1[ not relevant] 2[item need some revision] 3[relevant but need minor revision] 4[ very relevant]	1[not clear] 2[item need some revision] 3[clear but need minor revision] 4[very clear]

To obtain content validity index for relevancy and clarity of each item (I-CVIs), the number of those judging the item as relevant or clear (rating 3 or 4) was divided by the number of content experts but for relevancy, content validity index can be calculated both for item level (I-CVIs) and the scale-level (S-CVI). In item level, I-CVI is computed as the number of experts giving a rating 3 or 4 to the relevancy of each item, divided by the total number of experts.

The I-CVI expresses the proportion of agreement on the relevancy of each item, which is between zero and one^3,38 and the SCVI is defined as “the proportion of total items judged content valid”³ or “the proportion of items on an instrument that achieved a rating of 3 or 4 by the content experts”.²⁸

Although instrument developers almost never give report what method have used to computing the scale-level index of an instrument (S-CVI).⁶ There are two methods for calculating it, One method requires universal agreement among experts (S-CVI/UA), but a less conservative method is averages the item-level CVIs (S-CVI/Ave). For calculating them, first, the scale is dichotomized by combining values 3 and 4 together and 2 and 1 together and two dichotomous categories of responses including “relevant and not relevant” are formed for each item.^3,34 Then, in the universal agreement approach, the number of items considered relevant by all the judges (or number of items with CVI equal to 1) is divided by the total number of items. In the average approach, the sum of I-CVIs is divided by the total number of items.¹⁰Table 2 provides data for better understanding on calculation CVI and S-CVI by both methods. Data of table has been extracted from judges of our panel about relevancy items of dimension of trust building as a variable (subscale) in measuring construct of patient-centered communication. As the values obtained from both methods might be different, instrument makers should mention the method used for calculating it.⁶ Davis proposes that researchers should consider 80 percent agreement or higher among judges for new instruments.³⁴ Judgment on each item is made as follows: If the I-CVI is higher than 79 percent, the item will be appropriate. If it is between 70 and 79 percent, it needs revision. If it is less than 70 percent, it is eliminated.³⁹

Table 2. Calculation of I-CVI and S-CVI by two approaches of S-CVI/UA and S-CVI/Ave for items of trust building dimension.

Items	Relevant (rating 3 or 4)	Not relevant (rating 1 or 2)	I-CVIs *	Interpretation
1 2 3 4 5 6 7 8 9	14 12 13 12 11 14 12 8 14	0 2 1 2 3 0 2 6 0	1 0.857 0.928 0.857 0.785 1 0.857 0.571 1	Appropriate Appropriate Appropriate Appropriate Need for Revision Appropriate Appropriate Eliminated Appropriate

Number of items considered relevant by all the panelists=3, Number of terms=9, S-CVI/Ave^*** or Average of I-CVIs=0.872, S-CVI/UA^**=3/9=.333NOTE: ^*Item-Content Validity Items, ^**Scale-Content Validity Item/Universal agreement,^***Scale-Content Validity Item/Average Number of experts=14, Interpretation of I-CVIs: If the I-CVI is higher than 79 percent, the item will be appropriate. If it is between 70 and 79 percent, it needs revision. If it is less than 70 percent, it is eliminated.

Although content validity index is extensively used to estimate content validity by researchers, this index does not consider the possibility of inflated values because of the chance agreement. Therefore, Wynd‏ et al.,‏ propose both content validity index and multi-rater kappa statistic in content validity study‏ because, unlike the CVI, it adjusts for chance agreement. Chance agreement is an issue of concern while studying agreement indices among assessors, especially when we place four-point scoring within two relevant and not relevant classes.⁷ In other words, kappa statistic is a consensus index of inter-rater agreement that adjusts for chance agreement¹⁰ and is an important supplement to CVI because Kappa provides information about degree of agreement beyond chance.⁷Nevertheless, content validity index is mostly used by researchers because it is simple for calculation, easy to understand and provide information about each item, which can be used for modification or deletion of instrument items.^6,10

To calculate modified kappa statistic, the probability of chance agreement was first calculated for each item by following formula:

P_C= [N! /A! (N -A)!]* . 5^N.

In this formula, N= number of experts in a panel and A= number of panelists who agree that the item is relevant.

After calculating I-CVI for all instrument items, finally, kappa was computed by entering the numerical values of probability of chance agreement (P_C) and content validity index of each item (I-CVI) in following formula:

K= (I-CVI - P_C) / (1- P_C).

Evaluation criteria for kappa is the values above 0.74, between 0.60 and 0.74, and the ones between 0.40 and 0.59 are considered as excellent, good, and fair, respectively.⁴⁰

Polit states that after controlling items by calculating adjusted kappa, each item with I-CVI equal or higher than 0.78 would be considered excellent. Researchers should note that, as the number of experts in panel increases, the probability of chance agreement diminishes and values of I-CVI and kappa converge.¹⁰

Requesting panel members to evaluate instrument in terms of comprehensiveness would be the last step of measuring the content validity. The panel members are requested to judge whether instrument items and any of its dimensions are a complete and comprehensive sample of content as far as the theoretical definitions of concepts and its dimensions are concerned. Is it needed to eliminate or add any item? According to members’ judgment, proportion of agreement is calculated for the comprehensiveness of each dimension and the entire instrument. In so doing, the number of experts who have identified instrument comprehensiveness as favorable is divided into the total number of experts.^3,37

Determining face validity of an instrument

Face validity answers this question whether an instrument apparently has validity for subjects, patients and/or other participants. Face validity means if the designed instrument is apparently related to the construct underlying study. Do participants agree with items and wording of them in an instrument to realize research objectives? Face validity is related to the appearance and apparent attractiveness of an instrument, which may affect the instrument acceptability by respondents.¹¹ In principle, face validity is not considered as validity as far as measurement principles are concerned. In fact, it does not consider what to measure, but it focuses on the appearance of instrument.⁹To determine face validity of an instrument, researchers use respondents and experts’ viewpoints. In the qualitative method, face-to-face interviews are carried out with some members of the target groups. Difficulty level of items, desired suitability and relationship between items and the main objective of an instrument, ambiguity and misinterpretations of items, and/or incomprehensibility of the meaning of words are the issues discussed in the interviews. ⁴¹

Although content experts play a vital role in content validity, instrument review by a sample of subjects drawn from the target population is another important component of content validation. These individuals are asked to review instrument items because of their familiarity with the construct through direct personal experience.³⁷ Also they will be asked to identify the items they thought are the most important for them, and grade their importance on a 5-point Likert scale including very important⁵, important^4,2relatively important³, slightly important², and unimportant. In quantities method, for calculation item impact score, the first is calculated percent of patients who scored 4 or 5 to item importance (frequency), and the mean importance score of item (importance) and then item impact score of instrument items was calculated by following formula: Item Impact Score= frequency×Importance

If the item impact of an item is equal to or greater than 1.5 (which corresponds to a mean frequency of 50% and an importance mean of 3 on the 5-point Likert scale), it is maintained in the instrument; otherwise it is eliminated.⁴²

Results

Results of stage1: Designing patient-centered communication measuring instrument

In the one step of our research, which was performed through qualitative content analysis by semi-structured in-depth interview with ten patients with cancer, three family members and seven oncology nurses, the results led to identifying content domain within seven dimensions including trust building, intimacy or friendship, patient activation, problem solving, emotional support, informational support, and spiritual strengthening. Each of these content domains was defined theoretically by combining qualitative study and literature review. In the item generation step, 260 items were generated from these dimensions and they were combined with 32 items obtained from literature and the related instruments. In research group, the items were studied in terms of overlapping and duplication. Finally, 188 items remained for the operational definition of the construct of patient-centered communication, and the preliminary instrument was made by 188 items (pool items) within seven dimensions.

Results of stage 2: Judgment of expert panel on validity of patient-centered communic- ation measuring instrument

In the second step and after selecting fifteen content experts including the instrument developer experts (four people), cancer research experts (four people),nurse-patient communication experts (three people) and four nurses experienced in cancer care, an expert panel was created for making quantitative and qualitative judgments on instrument items. The panel members were requested thrice to judge on content validity ratio, content validity index, and instrument comprehensiveness. In each round, they were requested to judge on face validity of instrument as well. In each round of correspondences via e-mail or in person, a letter of request was presented, which included study objectives and an account on instrument, scoring method, and required instructions on responding. Theoretical definitions of the construct underlying study, its dimensions, and items of each dimension were also mentioned in that letter. In case no reply was received for the reminder e-mail within a week, a telephone conversation would be made or a meeting would be arranged.

In the first round of judgment, 108 items out of 188 instrument items were eliminated. These eliminated items had content validity ratio lower than 0.49, (according to the expert numbers in our study that was 15, numerical values of the Lawshe table was 0.49) or those which combined to remained items based on the opinion of content experts through editing of item. Table 3 shows a sample of instrument items and CVR calculation method for them.

Table 3. Calculating of CVR for a sample of instrument items at the first round of judgment.

Items	N e *	CVR **	Interpretation
1	9	0.0667	Remained
2	5	-0.333	Eliminated
3	10	0.3333	Eliminated
4	15	0.8667	Remained
5	9	0.2	Eliminated
6	13	0.6	Remained
7	7	-0.2	Eliminated
8	7	-0.067	Eliminated
9	13	0.6	Remained

NOTE: ^* Number of experts evaluated the item essential, ^**CVR or Content Validity Ratio = (N_e-N/2)/(N/2) with 15 person at the expert panel (N=15), the items with the CVR bigger than 0.49 remained at the instrument and the rest eliminated.

The remaining items were modified according to the recommendations of panel members in the first round of judgment and for a second time to determine content validity index and instrument modification, the panel members were requested to judge by scoring 1 to 4 on the relevancy and clarity of instrument items according to Waltz and Bussel content validity index.³⁸

In the second round, the proportion of agreement among panel members on the relevancy and clarity of 80 remaining items of the first round of judgment was calculated.

To obtain content validity index for each item, the number of those judging the item as relevant was divided by the number of content experts (N=14). (As one of the 15 members of panel had not scored some items, the analyses were made by 14 judges). This work was also carried out to clarify the items of the instrument. The agreement among the judges for the entire instrument was only calculated for relevancy according to average and universal agreement approach.

In this round, among the 80 instrument items, 4 items with a CVI score lower than 0.70 were eliminated. Eight items with a CVI between 0.70 and 0.79 were modified (Modification of items was performed according to the recommendation of panel members and research group forums). Two items were eliminated despite having favorable CVI scores, one of which was eliminated due to ethical issues (As some content experts believed that the item “I often think to death but I don’t speak about it with my nurses.” might cause moral harm to a patient, it was eliminated). On another eliminated item, “Nurses know that how to communicate with me”, some experts believed that if that item is eliminated, it would not harm the definition of trust building dimension. According to experts suggestions, an item (Nurses try that I face no problem during care) was added in this round. After modification, the instrument containing 57 items was sent to the panel members for the third time to judge on the relevancy, clarity and comprehensiveness of the items in each dimension and need for deletion or addition of the items. In this round, four items had a CVI lower than 0.70, which were eliminated.

The proportion of agreement among the experts was also calculated in this round in terms of comprehensiveness for each dimension of the construct underlying study. Table 4 shows the calculation of I-CVI, S-CVI and modified kappa for items in the instrument for 53 remaining items at the end of the third round of judgment. We also used panel members’ judgment on the clarity of items as well as their recommendations on the modification of items.

Table 4. Content Validity Index, Modified Kappa and comprehensiveness of instrument dimensions and total instrument at the third round of judgment.

Dimensions of construct of study: Patient-centered communication	Number giving rating of 3 or 4 to relevancy of item	I-CVI *	p c **	K ***	Interpretation	Comprehensiveness of instrument dimensions and total instrument
						Agree	Proportion of consensus
D1: Trust Building
D1-1	14	1	6.103	1	Excellent	14	1
D1-2	12	0.857	0.022	0.85	Excellent
D1-3	12	0.857	0.022	0.85	Excellent
D1-4	12	0.857	6.103	0.85	Excellent
D1-5	12	0.857	0.022	0.85	Excellent
D1-6	14	1	6.103	1	Excellent
D1-7	12	0.857	6.103	0.85	Excellent
D1-8	14	1	6.103	1	Excellent
D2:Intimacy/Friendship
D2-1	14	1	6.103	1	Excellent	13	0.928
D2-2	14	1	6.103	1	Excellent
D2-3	14	1	6.103	1	Excellent
D2-4	13	0.928	.0008	0.928	Excellent
D2-5	14	1	6.103	1	Excellent
D2-6	14	1	6.103	1	Excellent
D2-7	12	0.857	0.022	0.85	Excellent
D3: Patient activation
D3-1	12	0.857	0.022	0.85	Excellent	14	1
D3-2	13	0.928	0	0.928	Excellent
D3-3	13	0.928	0	0.928	Excellent
D3-4	13	0.928	0	0.928	Excellent
D3-5	14	1	6.103	1	Excellent
D4: Problem Solving
D4-1	14	1	6.103	1	Excellent	14	1
D4-2	14	1	6.103	1	Excellent
D4-3	12	0.857	0.022	0.85	Excellent
D4-4	12	0.857	0.022	0.85	Excellent
D4-5	13	0.928	0	0.928	Excellent
D4-6	14	1	6.103	1	Excellent
D4-7	14	1	6.103	1	Excellent
D5: Emotional support
D5-1	13	0.928	0	0.928	Excellent	14	1
D5-2	14	1	6.103	1	Excellent
D5-3	12	0.857	0.022	0.85	Excellent
D5-4	14	1	6.103	1	Excellent
D5-5	13	0.928	0	0.928	Excellent
D5-6	12	0.857	0.02	0.85	Excellent
D6: Informational support
D6-1	14	1	6.103	1	Excellent	13	0.928
D6-2	13	0.928	0	0.928	Excellent
D6-3	14	1	6.103	1	Excellent
D6-4	13	0.928	0	0.928	Excellent
D6-5	14	1	6.103	1	Excellent
D6-6	14	1	6.103	1	Excellent
D7:Spirituality strengthening
D7-1	12	0.857	0.022	0.85	Excellent	14	1
D7-2	12	0.857	0.022	0.85	Excellent
D7-3	12	0.857	0.022	0.85	Excellent
D7-4	14	1	6.103	1	Excellent
D7-5	14	1	6.103	1	Excellent
D7-6	12	0.857	0.022	0.85	Excellent
D7-7	12	0.857	0.022	0.85	Excellent
D7-8	13	0.928	0	0.928	Excellent
D7-9	14	1	6.103	1	Excellent
D7-10	13	0.928	0	0.928	Excellent
D7-11	14	1	6.103	1	Excellent
D7-12	13	0.928	0	0.928	Excellent
D7-13	13	0.928	0	0.928	Excellent
D7-14	13	0.928	0	0.928	Excellent
53 Items	S-CVI/Ave= 0.939S-CVI/UN= 0.434					Agreement on total comprehensiveness=14Comprehensiveness of entire instrument=1

NOTE: ^*I-CVI: item-level content validity index,^**p_c (probability of a chance occurrence) was computed using the formula: p_c= [N! /A! (N -A)!] ^*.5^Nwhere N= number of experts and A= number of panelists who agree that the item is relevant. Number of experts=14,^***K(Modified Kappa) was computed using the formula: K= (I-CVI- P_C)/(1- P_C). Interpretation criteria for Kappa, using guidelines described in Cicchetti and Sparrow (1981): Fair=K of 0.40 to 0.59; Good=K of 0.60 to 0.74; and Excellent=K>0.74

Face validity results of patient-centered communication measuring instrument

A sample of 10 people of patients with cancer who had a long-term history of hospitalization in oncology wards (lay experts) was requested to judge on the importance, simplicity and understandability of items in an interview with one of the members of research team. According to their opinions, to make some items more understandable, objective examples were included in an item. “For instance, the item “Nurses try not to cause any problem for me” was changed into “During care (e.g. preparation an intravenous line), Nurses try not to cause any problem for me”. The item “Care decisions are made without paying attention to my needs” was changed to “Nurses didn’t ask my opinion about care(e.g. time of care or type of interventions)”. In addition the quantitative analysis was also performed as calculating impact score of each item. Nine items had item impact score less than 1.5 and they were eliminated from the final instrument for preliminary test. Finally, at the end of the content validity and face validity process, our instrument was prepared with seven dimensions and 44 items for the next steps and doing the rest of psychometric testing.

Discussion

Present paper demonstrates quantities indices for content validity a new instrument and outlines them during design and psychometrics of patient centered communication measuring instrument. It should be said that validation is a lengthy process, in the first-step of which, the content validity should be studied and the following analyses should be directed include reliability evaluation (through internal consistency and test-retest), construct validity (through factor analysis) and criterion-related validity.³⁷

Some limitations of content validity studies should be noted, Experts’ feedback is subjective; thus, the study is subjected to bias that may exist among the experts. If content domain is not well identified, this type of study does not necessarily identify content that might have been omitted from the instrument. However, experts are asked to suggest other items for the instrument, which may help minimize this limitation.¹¹

Conclusion

Content validity study is a systematic, subjective and two-stage process. In the first stage, instrument design is carried out and in the second stage, judgment/quantification on instrument items is performed and content experts study the accordance between theoretical and operational definitions. Such process should be the leading study in the process of making instrument to guarantee instrument reliability and prepare a valid instrument in terms of content for preliminary test phase. Validation is a lengthy process, in the first step of which, the content validity should be studied. The following analyses should be directed include reliability evaluation (through internal consistency and test-retest), construct validity by factor analysis and criterion-related validity. Meanwhile, we showed that although content validity is a subjective process, it is possible to objectify it.

Understanding content validity is important for clinician groups and researchers because they should realize if the instruments they use for their studies are suitable for the construct, population under study, and socio-cultural background in which the study is carried out, or there is a need for new or modified instruments.

Training on content validity study helps students, researchers, and clinical staffs better understand, use and criticize research instruments with a more accurate approach.

In general, content validity study revealed that this instrument enjoys an appropriate level of content validity. The overall content validity index of the instrument using a conservative approach (universal agree- ment approach) was low; however, it can be advocated with respect to the high number of content experts that makes consensus difficult and high value of the S-CVI with the average approach, which was equal to 0.93.

Acknowledgments

The researchers appreciate patients, nurses, managers, and administrators of Ali-Nasab and Shahid Ayatollah Qazi Tabatabaee hospitals. Approval to conduct this research with no. 5/74/474 was granted by the Hematology and Oncology Research Center affiliated to Tabriz University of Medical Sciences.

Ethical issues

None to be declared.

Conflict of interest

The authors declare no conflict of interest in this study.