The reliability of self-reporting chronic diseases: how reliable is the result of population-based cohort studies

Summary

Objectives

To evaluate the reliability of self-reporting chronic diseases in the baseline data of the Ravansar Non-Communicable Diseases (RaNCD) cohort study in Kermanshah province, western Iran.

Methods

The study was conducted in RaNCD cohort study. To assess the reliability of self-report of chronic disease, a random sample of 202 participants were asked about some of chronic conditions 30-35 days (mean = 32) after recruitment.

Results

A range of kappa agreement between 39.52-100%, which the lower statistics was for hypertension and hepatitis and the higher one for cancer, cardiac ischemic, and diabetes.

Conclusion

The self-report of chronic diseases was relatively reliable. Therefore self-reporting data for some conditions can be used in situations where the validity is acceptable.

Introduction

There are several epidemiological studies on chronic conditions in both developed and developing countries that are based on self-reported information [1-4]. Although such studies provide valuable information about common frequency measures with lower cost compared to those in which researchers use clinical and paraclinical criteria, there are still strong argument against reliability of self-report data.

Chronic diseases are increasing in terms of number and rates in both developed and developing countries and are responsible for about 70% of mortality in Iran [5]. About 45% of the mortality and 85% of total burden of chronic disease occur in the population aged under 70 years [6]. To tackle this important public health issue, the deputy of research and technology in ministry of health in Iran with cooperation of 17 universities decided to conduct the Prospective Epidemiological Research Studies in Iran (PERSIAN) [7]. Ravansar Non-Communicable Diseases (RaNCD) study is one the 17 cohort studies in Kurdish people of Kermanshah province, west of Iran. Most of the information related to chronic diseases in the PERSIAN cohort is often self-reported data.

While previous studies showed the importance of the self-reported data, there are no general agreement on reliability of such data in different cultures and for different chronic diseases [8-10]. In fact, reliability of self-reported chronic morbidity is related to different socio-demographic variables as well as type of chronic diseases. Similar to reliability, validity of self report is different for chronic conditions. While for diabetes, stroke, hypertension, asthma, and cancer there are some reports on 90% specificity [11-13], for other chronic diseases such as hypo-hyperthyroidism and arthritis is lower [2, 14]. There are some reports on association between single item self-rated health and overall mortality [15, 16].

RaNCD is the only cohort from PERSIAN in Kurdish people and there is no published data on reliability of self-reported of chronic morbidity in PERSIAN. Hence, our study aimed to evaluate the reliability of self-reporting chronic diseases in a sample from RaNCD study in Kermanshah province, western Iran.

Methods

STUDY POPULATION

In order to investigate the reliability of self-report of chronic morbidity, a cross-sectional study conducted in the RaNCD study in 2017. Ravansar, located in west of Kermanshah province is one of 17 centers of the PERSIAN Cohort study [17]. Ravansar is a district with both urban and rural areas, close to the Iraqi border, with a total population of around 50,000. The majority of residents have Iranian Kurdish ethnicity.

There are two substudies in Ravansar: adult PERSIAN and PERSIAN Youth Cohort. While Youth cohort is mainly focus on mental and psychological disorders among those aged 15-35 years old, in adult PERSIAN, researcher investigate about non-communicable diseases and their determinants in people aged > 35 years [17, 18]. For the purpose of this study, we used a random sample of 202 participants from adult cohort.

DATA COLLECTION AND QUALITY CONTROL

The original design and sample size for the RaNCD was recruitment of 10,000 people living in Ravansar in Kermanshah province. However, for the purpose of this study and in order to assess the reliability of self-report of chronic morbidities, a random sample of 202 participants were reinvited to attend our center in Ravansar after 30-35 days (mean: 32 days) of their recruitment. According to the related guidline, a maximum sample size with 95% confidence level, power 80%, and ICC of 0.2 was calculated as 152 subjects [19]. Using the same questionnaire, center and staff, we tried to provide the same situation similar to their first attendance during their recruitment. The inclusion criteria for the RaNCD were the age range of 35-65 years, inhabitant in Ravansar county for one year or more and providing oral and written informed consent for participant in the cohort study. There was no any inclusion and exclusion criteria for present study.

CHRONIC DISEASE DETECTION

Chronic diseases are recorded based on self-report of participants, history of past and present treatment and medication plus physical exam by trained staff in all PERSIAN cohort sites (Fig. 1). In order to asses the reliability of self-report of chronic morbidities, we reinvited a sample of 202 participants and the whole process repeated 30-35 days later.

Fig. 1.

The proportion of self-reported and medical records of chronic diseases in RaNCD, Kermanshah, western Iran.

All stages of the study design and implementation were approved by Ethical committee of Kermanshah University of Medical Sciences (kums.res.1394.315).

STATISTICAL ANALYSIS AND CALCULATIONS

Reliability of self-report was assessed using kappa (K) statistics for each chronic disease. In addition, reliability was assessed using two-way random effects intraclass correlations coefficient (ICCs) and total agreement. All analyses were carried out at 95% confidence interval using STATA software version 14.1 (Stata Corp, College Station, TX, USA).

Results

Table I presents the characteristics of the study sample by gender. A sample of 202 subjects was selected. Majority (64.18%) of them were aged more than 46 years (Tab. I). From total, 59.9% were illiterate (73.6% of women and 26.4% of men).

Tab. I.

The characteristics of the study sample by sex in RaNCD, Kermanshah, western Iran.

Variables		Total N (%)	Female n (%)	Male n (%)	P value
Age group (years)	35-45	72 (35.82)	44 (59.72)	29 (40.28)	0.75
	46-65	129 (64.18)	74 (57.36)	55 (42.64)
Marital status	Single	31 (15.35)	27 (87.10)	4 (12.90)	< 0.001
	Married	171 (84.65)	91 (53.22)	80 (46.78)
Level of education (years)	Illiterate	121 (59.90)	74 (73.55)	32 (26.45)	< 0.001
	1-5	54 (26.73)	23 (42.59)	31 (57.41)
	> 6	27 (13.37)	6 (22.22)	21 (77.78)

The test-retest reliability with range of ICC between 90.59% and 100% was found for different type of chronic diseases. There was an ICC of 100% for cancer and hepatitis. The agreement for self-reported hypertension was 78.50%, while it was ≥ 89.97% for all other self-reported chronic diseases. The reported kappa was between 39.52%-100% with the lowest value for hypertension and hepatitis and much higher values for cancer, ischemic heart disease, and diabetes. While self-report of chronic morbidity had a moderate sensitivity (range between 25%-100%), the reported specificity ranged between 93.1%-100% (Tab. II).

Tab. II.

Reliability of self-reported chronic diseases in RaNCD, Kermanshah, western Iran.

	Internal consistency (ICC)	Total agreement (%) (95%CI)	Kappa (%) (95%CI)
Diabetes	90.59	95.02 (91.08-97.60)	82.33 (69.0-90.25)
Hypertension	94.06	78.50 (72.32-83.75)	39.96 (25.45-57.15)
Ischemic heart disease	96.53	89.97 (85.31-95.25)	86.76 (74.1-97.25)
Fatty liver	93.07	96.04 (92.35-98.27)	71.20 (49.2-85.0)
Hepatitis B	100	98.51 (95.72-99.69)	39.52 (6.7-78.0)
Cancer	100	100 (98.18-100)	100 (24.11-100)
Thyroid	97.52	96.52 (92.69-98.59)	65.04 (38.5-82.3)
Kidney stones	91.58	96.04 (93.26-98.89)	74.44 (30.30-95.37)

Discussion

Self-reporting health-related morbidities such as chronic diseases are frequently used in the epidemiological studies [20-23]. Both quality and reliability of these data are important to estimate the health indicies (such as incidence and prevalence) of chronic diseases and their determinants. In this study, for the first time, the reliability of self-reporting chronic diseases was evaluated in a small sample of RaNCD study in western Iran.

We found a good test-retest reliability with range of ICC between 90.59 and 100. Reliability was better for hepatitis, cancer, thyroid disease, and ischemic heart disease than for diabetes. We also observed a wide range of reliability score with calculating Kappa statistic (range 39.52-100). These differences exemplify the fact that the score of Kappa takes into account the possibility of the agreement occurring by chance. While in our study the Kappa score for diabetes (82.33%), ischemic heart disease (86.76) and cancer (100%) were high, for other condition, such as hepatitis (39.52%) and hypertension (39.96%) were low. A study to examine test-retest reliability of self-reported diabetes among 33,919 participants showed an acceptable Kappa score of 0.65 for both type 1 and type 2 diabetes diagnoses [2]. It should be noted that with such a low value for kappa among those who reported suffering from hypertension, there are some reports on undertreatment of such people. There is a report from China that only 25%-50% of hypertensive subjects who are aware of their disease receive appropriate medical treatment [24]. In fact, because of low public awareness regarding the chronic condition, the burden of diseases such as diabetes mellitus and hypertension are assumed as iceberg and most of the patients don’t know about their disease status. Moreover, the most of these patients don’t receive treatment [6]. Although, there are numerous reliability studies on different aspects of diabetes and hypertension diseases [2, 4, 6, 25], we found no similar study conducted on other chronic diseases for more comparisons. A study to determine the validity and reliability of self-reported stillbirth data in Australia [26] has concluded that self-reported data are important to resolve inconsistencies in administrative datasets.

To our knowledge, this is the first study on assessment of reliability of self-report of chronic morbidities in PERSIAN cohort. The ICC and the kappa values in our study may not be comparable to other studies in other countries or even in other provinces in Iran. The reliability indices is closely related to factors such as culture, public awareness, socioeconomic factors, and lifestyle, as well as type of chronic condition, severity and even prevalence in population. Another factor may be the time interval between test and retest studies. In the present study, the average time of retest was 32 days after the recurement whereas a time interval of seven years was used in the study by Sheikh MA et al (2016) [2]. In a test–retest reliability of diabetes diagnosis with time interval of 30 days the kappa agreement was found to be 0.48 [27]. However, our results can be considered as a reference for other PERSIAN studies with some consideration.

In conclusion, with increase in public awareness regarding the chronic conditions, the self-report of chronic morbidities can provide a highly reliable feature for estimation of burden of such morbidities among participants aged 35-65 years in RaNCD. Although these measures vary among different chronic diseases, these scores provide an important information on self-reported questions and can be useful for increasing precision in the PERSIAN cohort studies.