What is the test-retest reliability of the Malay version of the Hypertension Self-Care Profile self efficacy assessment tool? A validation study in primary care

Abstract

Objectives

Self-efficacy and self-care measures are key attributes to optimal control of essential hypertension. Self-efficacy can be measured by the Hypertension Self-Care Profile (HTN-SCP) tool but its utility is dependent on the literacy and understanding of the subjects. A Malay version of the HTN-SCP Tool was developed to assess self-efficacy of Malay-literate patients with hypertension in the multi-ethnic Asian population in Singapore. The study aimed to determine the test-retest reliability of this tool which has been translated in Malay language.

Methods and Material

145 Malay-literate patients, aged 41-70 years, with essential hypertension were recruited in a polyclinic (primary care clinic) in Singapore. Forty-three percent of them completed both the first and second HTN SCP tool online, with a period of two weeks in between. The Cronbach’s alpha and Intra-class correlation coefficient (ICC) were computed to assess its test-retest reliability and internal consistency.

Results

The Cronbach’s alpha/ICC for “Behavior” (0.851/0.664)), “Motivation” (0.928/0.655) and “Self-efficacy” (0.945/0.682) domains showed high internal consistency, fair to good reliability and stability. No floor or ceiling effect was found for the "behavior" and "motivation" domains. However, the borderline ceiling effect (15.2) for "self-efficacy" suggested limited discriminating power of the tool for patients with high self-efficacy. Positive association was shown between the HTN-SCP score and reported self-care measures but it was not statistically significant.

Conclusion

Overall, the translated HTN-SCP tool showed satisfactory test-retest reliability and internal consistency amongst the Malay-literate study population. Further research is needed for its application in general practice to identify patients with low self-efficacy for possible intervention.

Strengths and limitations of the study.

• This study used a web-based method in administrating the test, showing its potential application in clinical practice.

• Without incentive, the web-based method resulted in significant dropouts in the re-test segment of the study.

• Reliability assessment of the ‘self-efficacy’, ‘behaviour’ and ‘motivation’ domains may be limited by ceiling and floor effects.

• The study excluded subjects who were not internet-savvy and those who were not proficient in written Malay language.

Introduction

Hypertension is often the most prevalent risk factor for vascular diseases in developed and developing countries.¹ Maintaining optimal blood pressure among patients with hypertension is key to avert vascular complications such as stroke and cardiac diseases.²However, achieving stable blood pressure (BP) control requires these patients to undertake multiple measures consistently, such as salt and fat restriction in diet, regular physical exercises, reduction in consumption of alcohol, self-monitoring of BP and weight and adherence to prescribed BP lowering medications.^{3 4}

The capacity to conduct these self-care activities is related to an individual’s self-efficacy.^{5 6} Thus assessing self-efficacy of patients with hypertension provides the healthcare provider an insight into their vascular risk status, and decision support in designing individualised care plan. Questionnaire-based tools have been developed to facilitate the multi-faceted assessment of self-efficacy. One example is the English-based Hypertension Self-Care Profile (HTN-SCP, also known as HBP SCP) questionnaire developed by Han HR et al, which covers three key domains of behaviour, motivation and self-efficacy.⁷ Her team had used two validated theoretical approaches, Orem’s self-care model⁸ and Motivational Interviewing (MI)⁹ to develop the questionnaire. The Orem’s model depicts the enablement of individuals to embark on deliberate actions such as taking medication and physical activity as indicators of self-care.⁸Using similar constructs such as those from the Social Cognitive Theory and Health Belief Model, MI indicates a commitment to change (ie, motivation) and reflects the development of confidence for behaviour change.⁹ Self-efficacy predicts medication adherence, physical activity, diet and weight control.⁷ Han HR et al reported high reliability estimates and strong evidence of validity of the HBP SCP when the study was carried out in an inner city American population. Their results suggest that the questionnaire can be used to assess and identify gaps in self-care behaviour, motivation, and self-efficacy in patients with hypertension.

The test-retest reliability assessment of this English-based questionnaire had also been performed in Singapore, where English is the main language of communication of its population on the developed island state.¹⁰ Among the 5.61 million multi-ethnic Asian population in Singapore, 13.4% are of Malay ethnicity.¹¹ According to the Singapore 2010 Population Census, 37% of the local Malay population had up to ‘below secondary’ education or equivalent of grade six educational level.¹² The questions of the original version of HTN-SCP were developed to pitch at grade six level.⁷ Uncertainty arises if over a third of the overall Malay population could understand the English-based questionnaire. Moreover, hypertension is more prevalent among the more senior residents. Based on the same census, only 5.5% of Malays of age 55 years and older, and 4.4% of Malays with ‘below secondary education’ are proficient and use English at home.¹² Hence, approximately one in twenty Malay patients with hypertension would have potential difficulties in using the locally validated HTN-SCP tool.

Singapore is also facing a rapidly ageing population and the prevalence of hypertension is expected to rise significantly by the next decade. A Malay version of the HTN-SCP tool is postulated to ease the assessment of self-efficacy among the older Malay patients with hypertension. To ensure that they understand the content, the translated tool needs to be contextualised to the local setting and tested on stability and reliability among the local Malay-literate patients with hypertension. The tool will allow its application not only in local Malay patients with hypertension, but will also cater to the larger Malay-literate population in Malaysia and other parts of South-East Asia.

Methods

Aim

The study aimed to determine the test-retest reliability of the Malay version of the HTN-SCP among Malay hypertensive patients who were managed in a public primary care clinic in Singapore.

Development of the malay version of the HTN SCP tool

The original English-based HTN-SCP tool has been enhanced in selected items to contextualise to the dietary habits of the local Asian patients with hypertension.⁷ The enhanced version was subsequently validated in an earlier study, which showed satisfactory test-retest reliability, as reflected in the internal consistency coefficient (ICC) (0.671, 0.762, and 0.720 for the three domains) and Cronbach-alpha indicators (0.857, 0.948, and 0.931 respectively). Next, the investigators engaged a commercial agency, whose two independent professional and certified bi-linguists in English and Malay carried out the forward and backward translation of the content respectively. The draft Malay version of the HTN-SCP was then pilot-tested among independent bilingual staff within the institution department of research and nurses at the study site. Minor amendments were carried out to the tool after this preliminary assessment. The Malay version (HTN-SCP-M) was finalised after approval by the investigator team members, who are also bilingual in English and Malay.

Subjects

The target subjects were patients of Malay ethnicity with essential hypertension based on their diagnosis code in the electronic medical records at the study site. They were either Singaporean citizens or Permanent Residents.

Inclusion criteria

Subjects must be between the age of 41 and 70 years on enrolment. They were treated with blood pressure lowering medication for their essential hypertension for at least 1 year.

Subjects were screened by the investigators and research assistants to be able to read and understand the written language as a minimal standard of Malay proficiency. They had to be internet-savvy in order to access the web-based administration of the tool.

Exclusion criteria

Subjects who lacked proficiency in the written Malay language and access to internet, were unwilling to execute the test, or were screened to have cognitive, visual or auditory impairment were excluded from the study.

Recruitment of subjects at study site

The study was conducted from May 2016 to Dec 2016 in a typical public primary care clinic located in an estate in north-eastern Singapore serving a population of about 140 000 multi-ethnic Asian population (2015). The investigators and research assistants identified potential subjects by their Malay names or attire and approached them before or after their medical consultation at the study site. The subjects were next screened to confirm their ethnicity and fulfilment of their eligibility criteria. They were provided with information on the study procedure in Malay and their doubts were clarified before obtaining their written informed consent according to stipulations by the institution review board.

Ethics approval

Ethics approval was granted from the Centralised Institutional Review Board (CIRB) for this study. Ref no: 2016/2332.

Administration of the tool

To ensure anonymity, each subject was given a unique study identification number after enrolment, which was used to log in to a free web-based online tool (Qualtrics) for both the test and retest. They were instructed on the step-by-step procedure to administer the web-based HTN-SCP-M during the test. The web design ensured mandatory filling of the response to each question before the subject was allowed to progress to the next question. This format eliminated any missing data.

After the completion of the test, the subjects were provided with information sheet, instructing them to access the online platform for the retest using their personal electronic devices (smart mobile phone, tablet, laptop or computers) on a specified date 2 weeks later in order to complete the study. The investigators would send reminders via calls, phone messages and electronic mails to defaulters after the stipulated retest dates. Those that failed to complete the retest beyond a 1 week grace period were excluded from the reliability analysis. Participation by the subjects were voluntary, with no incentive provided for their study participation.

Sample size calculation

The COSMIN checklist recommended a good sample of between 50 to 99 study subjects for assessing their test-retest reliability.¹⁴ In view of a significant drop-out rate for the retest using a web-based approach from an earlier study by the principal investigator, the target recruitment size was increased to 150 to account for attrition.

Data management and statistical analysis

The demographics of study participants were reported. Differences between the overall HTN-SCP-M scores and recommended self-management activities were assessed using the independent t-test.

Ceiling and floor effects were derived from the percentage of respondents with the highest and lowest scale scores. Ceiling and floor effects were considered present when it is higher than 15%.¹⁵ Internal consistency indicates the extent to which the items in a subscale are correlated, in order to evaluate homogeneity using Cronbach’s Alpha statistic.¹⁶ A value of 0.7 was considered acceptable while above 0.8 to 0.9 was considered good to excellent internal consistency.¹⁷

The agreement between repeated measurements (Test-retest) was evaluated through the use of the intra-class correlation coefficient (ICC). An ICC coefficient of>=0.75 was considered as evidence of measurement stability. ICC between 0.4 and 0.75 indicates fair to good reliability, and ICC <0.4 indicates poor reliability.¹⁸ This analysis excluded those who dropped out from the study for the retest 2 weeks after their recruitment.

A p-value of less than 0.05 was considered as statistically significant. All analyses were performed using the IBM SPSS Statistics for Windows, Version 23.0, IBM Corp, released 2013.

Results

A total of 768 subjects were approached by the investigators and research assistants, of which 204 patients satisfied the eligibility criteria. With 59 refusal of consent, 145 of them were recruited for the test segment of the study, resulting in a response rate of 71%. Subsequently, 63 (43%) of the recruited subjects completed the retest of the HTN-SCP-M on-line. Reasons for drop-out include patients being unable to be contacted for reminder, patients' refusal to complete even after reminder, and patients’ completion of the retest after an extended period beyond two to 3 weeks.

The demographics of the respondents were presented in table 1. There were more females (66.9%) than males (33.1), and the majority of them (66.2%) had up to 10 years of education. Among the subjects, 19.3% of them had hypertension only, while the majority had other co-morbidities, such as dyslipidemia (59.5%) and type two diabetes mellitus (34.6%).

Table 1.

Baseline characteristics of study population

Demographic characteristics	n (%)
Gender
Male	48 (33.1)
Female	97 (66.9)
Age, Mean (SD)	58 (6.8)
Highest Level of education
Primary	17 (11.7)
Secondary	96 (66.2)
JC/ Polytechnic/ Institute of Technical Education	23 (15.9)
University	9 (6.2)
Other Comorbidities
Hypertension only	28 (19.3)
Diabetes	78 (53.8)
Dyslipidemia	95 (65.5)
Heart disease	10 (6.9)
Stroke	4 (2.8)
Others	17 (11.7)
Number of medication, Median (IQR)	1 (1–2)

Table 2 reveals self-care measures reported by the subjects, which are relevant to hypertension management. Majority of them (71.7%) measured their BP at home but only 57.9% documented these measurements regularly. The HTN-SCP-M scores of the subjects who recorded their BP readings were significantly higher than those without BP documentation. The same finding was observed in those who measured their weight regularly.

Table 2.

Association between Self-Reported Self-Care Activities and the Overall Mean Scores of the Hypertension Self-Care Profile (HTN-SCP)

		Overall Mean HTN-SCP Score (SD)
	N (%)	Yes	No	p-value
Do you measure your blood pressure at home	104 (71.7)	189.3 (27.3)	180.0 (24.9)	0.06
Do you record your blood pressure readings regularly	84 (57.9)	190.9 (27.7)	180.8 (24.8)	0.03*
Do you keep a food diary	18 (12.4)	187.8 (37.1)	186.5 (25.3)	0.85
Do you measure your weight regularly	86 (59.3)	190.5 (26.9)	181.1 (26.2)	0.04*
Do you record your weight regularly	39 (26.9)	190.1 (29.6)	185.4 (25.9)	0.35
Do you use any mobile apps to monitor your health or medical condition?	35 (24.1)	192.1 (28.9)	184.9 (26.1)	0.17
Has your doctor ever reduced your high blood pressure medications	43 (29.7)	186.3 (27.5)	186.8 (26.8)	0.93

The HTN-SCP-M scores of those who performed self-care measures, such as home BP measurements, keeping a food diary, documentation of weight and using health-related mobile phone application, were higher than those who without such activities, although these differences did not attain statistical significance (table 2).

Floor and ceiling effect

Table 3 The mean score for the ‘Self-Efficacy’ domain within the HTN-SCP-M was highest, compared with those in the ‘Behaviour’ and ‘Motivation’ domains. The ‘self-efficacy’ domain presented a borderline ceiling effect of 15.2% (table 3). The other two domains show minimal ceiling effect (<15%).

Table 3.

Reliability indices of the HTN-SCP-M tool

Domains	Mean (SD)	Ceiling effect proportion (%)	First assessment score, Median (IQR)	Second assessment score, Median (IQR)	Cronbach's alpha	ICC (95% CI)
Behaviour	50.3 (10.1)	0	55 (48–58)	58 (53–64)	0.851	0.664 (0.500 to 0.783)
Motivation	65.8 (10.4)	14 (9.7)	67 (60–74)	66 (60–74)	0.928	0.655 (0.487 to 0.776)
Self-efficacy	66.4 (11.2)	22 (15.2)	67 (61–76)	67 (60–74)	0.945	0.682 (0.524 to 0.795)

Note, Each domain has 20 items

Internal consistency

All three domains showed excellent internal consistencies: the Cronbach’s alpha for ‘Behaviour’ was 0.851, 0.928 for ‘Motivation’ and 0.945 for ‘Self-efficacy’ domains respectively.

Test-Retest reliability

The Intra-class Correlation Coefficient (ICC) for all domains ranged from 0.655 to 0.682, which suggested fair to good reliability and stability.

Discussion

The study demonstrated efforts to validate a tool in a specific Asian language, which has been originally developed in English in North America.⁷ It followed a successful test-retest reliability assessment of the HTN-SCP tool in English, which is contextualised to the multi-ethnic Asian community in a primary care setting in Singapore.¹⁰ The over-arching aim is to expand the scope of the tool to appraise the self-efficacy of patients with hypertension in diverse populations. In spite of incorporating innovative approach in this study riding on the rapidly and extensively internet-connected community, the validation framework is aligned to the STROBE guidelines.¹⁹

Overall, the Malay version of the HTN-SCP tool (HTN-SCP-M) has attained satisfactory test-retest reliability and internal consistency among the study population based on the Cronbach’s alpha and ICC indices. While the flooring effect was minimal, the results showed borderline ceiling effect for the self-efficacy domain. It suggests potential limitation to the discriminating power of the tool among those with high scores in the self-efficacy domain. However, this subset of the study population with higher capacity for self-efficacy to control their blood pressure is not the target patients of concern to clinicians. Attention should focus on the group of patients with lower self-efficacy capacity, reflected by the lower HTN-SCP-M scores, who are at risk of poor blood pressure control.

The next step will be to test the application of the HTN-SCP-M on the local Malay-literate patients with hypertension to determine the correlation between its scores and self-care activities. The total aggregated score in this study was significantly associated with patients’ self-reporting of their documentation of their home blood pressure monitoring and weight measurement. For other self-care measures such as keeping a food diary, the positive correlation between the total scores and reported self-care measures was not statistically significant.

Nonetheless, the sample size of the study population was not computed to determine the differences in self-care measures based on the score. Further adequately powered study with larger number of subjects and incorporating reliable and objective assessment of self-care measures is needed to assess if the tool can be used to stratify Malay-literate patients into varying capacities for health behaviour, motivation and self-efficacy.

The study has its strength and limitations. It excluded patients who lacked access or were not competent internet users. The use of a web-based approach in implementing the test-retest reliability evaluation enabled the patients to self-administer the tool remotely using their smart-phone, tablets, computer and laptop at their preferred timing and venue. Despite the convenience of this method, the uptake of the retest segment was suboptimal, even with the use of reminders by the investigators. In retrospect, the uptake could potentially be improved with incentives and nominal reimbursement for the effort and time by the participants, even though the cost in carrying out the on-line retest was minimal. Nevertheless, the demographic profiles, in terms of gender, ethnic group, age and number of medications, of those who dropped out of the study were similar to those who completed the study.

The study also excluded patients with poor literacy of written Malay language, and those with cognitive, auditory and visual impairment. Future research will target expanded application of the tool to bridge these gaps, leveraging on voice-annotated administration based on info-communication technology and related tool to capture observations of self-efficacy by caregivers and proxies.

The study shows viable web-based administration of self-efficacy assessment, and indicates its potential application in routine clinical practice using this approach. However, the significant dropouts in the subsequent re-test highlight the challenge in serial measurements if the intention is to chart improvement in self-efficacy after interventions. Either spacing out the intervals between repeat administrations of the HTN-SCP or the use of a shorter, user-friendly version of the tool can be possible solutions.

The findings in this study can potentially be extrapolated to the larger ethnically similar Malay population in the neighbouring countries. This Malay version of the tool can be utilised to evaluate the self-efficacy of Malay patients with hypertension in the community, including those who are managed by the public polyclinics in Malaysia (Kesihatan Klinik).

Conclusion

The Malay version of the HTN-SCP has satisfactory test-retest reliability and internal consistency. The total scores of the Malay-based tool have shown potential association with patients’ self-reporting of their self-care behaviour, which require further research for its validation.