Using a Web-Based Approach to Assess Test–Retest Reliability of the “Hypertension Self-Care Profile” Tool in an Asian Population

Abstract

Essential hypertension often requires affected patients to self-manage their condition most of the time. Besides seeking regular medical review of their life-long condition to detect vascular complications, patients have to maintain healthy lifestyles in between physician consultations via diet and physical activity, and to take their medications according to their prescriptions. Their self-management ability is influenced by their self-efficacy capacity, which can be assessed using questionnaire-based tools. The “Hypertension Self-Care Profile” (HTN-SCP) is 1 such questionnaire assessing self-efficacy in the domains of “behavior,” “motivation,” and “self-efficacy.” This study aims to determine the test–retest reliability of HTN-SCP in an English-literate Asian population using a web-based approach.

Multiethnic Asian patients, aged 40 years and older, with essential hypertension were recruited from a typical public primary care clinic in Singapore. The investigators guided the patients to fill up the web-based 60-item HTN-SCP in English using a tablet or smartphone on the first visit and refilled the instrument 2 weeks later in the retest. Internal consistency and test–retest reliability were evaluated using Cronbach's Alpha and intraclass correlation coefficients (ICC), respectively. The t test was used to determine the relationship between the overall HTN-SCP scores of the patients and their self-reported self-management activities.

A total of 160 patients completed the HTN-SCP during the initial test, from which 71 test–retest responses were completed. No floor or ceiling effect was found for the scores for the 3 subscales. Cronbach's Alpha coefficients were 0.857, 0.948, and 0.931 for “behavior,” “motivation,” and “self-efficacy” domains respectively, indicating high internal consistency. The item-total correlation ranges for the 3 scales were from 0.105 to 0.656 for Behavior, 0.401 to 0.808 for Motivation, 0.349 to 0.789 for Self-efficacy. The corresponding ICC scores of 0.671, 0.762, and 0.720 for these respective domains showed good test–retest reliability. The correlation of the HTN-SCP scores and patients’ reported self-management measures were significant, except for keeping their food diary.

HTN-SCP showed satisfactory internal consistency and test–retest reliability in an English literate Asian population. A web-based approach is feasible if similar studies are needed to validate its translated versions of the tool for wider application in the local multilingual population.

INTRODUCTION

Hypertension is a prime risk factor for cerebrovascular, coronary artery, and chronic renal diseases, leading to premature morbidity and mortality.¹ Despite being a common noncommunicable disease globally, patients with known hypertension often seek medical attention periodically, during which their healthcare providers generally would advise them on lifestyle changes such as weight management, increase in physical exercise, adoption of a healthy diet, and adherence to their drug therapy. Such activities are necessary to address the multiple factors influencing their blood pressure control, which have to be performed on a regular basis in between consultations. Whether the patients embark on these recommendations consistently will often be influenced by their perception of the ease or difficulty of achieving such a particular behavior. This is rooted in the concept of self-efficacy in Bandura's social cognitive theory.²

An individual's belief in their capabilities to produce given attainments is known as “self-efficacy.” It is a desirable quality to enable the person to take charge of their health and any long-term illness, such as hypertension.^3,4 It is associated with better clinical outcomes in hypertension treatment, such as better blood pressure control, reduced frequency of medical consultation, and consumption of fewer medications.³ Assessing patient's self-efficacy capability will provide guidance to physicians in their design of holistic patient-centric care plan for their patients.

Assessment of self-efficacy requires a review of patient's perceptions, willingness, and confidence to perform the various activities to achieve their blood pressure control. It thus necessitates a composite evaluation of these elements. Several tools have been developed to assess self-efficacy in different chronic conditions but none has been validated for usage in a predominantly Asian population. Ideally, the content of the tool should relate specifically to hypertension, validated, and contextualized to the socio-cultural norms of the target population, and provides assessment of the multiple domains of self-efficacy.

Searches were performed using PubMed, Scopus, and The Cochrane Library and eventually yielded 34 relevant articles and 12 instruments. All instruments were validated in specific populations and 2 instruments were specific to hypertension. The HTN-SCP encompasses most domains and assesses behavior, motivation, and self-efficacy in patients with hypertension. It provides an extensive coverage of self-efficacy, including medication adherence and lifestyle modifications. HTN-SCP has been validated in an American Korean population, whose cultural beliefs and lifestyle habits are close to that of the multiethnic Asian population in Singapore.⁵

As English is also the common language in the local population, the investigators hypothesize that the local hypertensive patients are capable of understanding the content of HTN-SCP tool in relationship to their perceived self-efficacy capability. It will require a subject to complete the tool twice based on the test–retest reliability assessment. Typically such a validation study is carried out by the subject filling out the print copy of the questionnaire twice in a face-to-face encounter between the subject and the investigator, which is inconvenient and incurs additional resources. The investigators decided to adopt an alternative approach to carry out the test–retest evaluation of a web-based questionnaire on the internet platform. Brigham et al had used such a method in a test–retest reliability assessment on a web-based tobacco-related tool on an American study population.⁶ The advantages of a web-based repeat assessment include minimizing entry error, maximizing the subject's involvement, cost-saving, and reduction of inconvenience of repeat visits to study site at close interval. As Singapore ranks as the nation with the highest smartphone penetration rate (85%) in the world by Google in 2014, the environment for such a web-based approach is favorable.⁷ Earlier web-based validation studies were performed largely in Western populations.⁶ There is a need to assess the feasibility of executing such a study in an Asian urbanized population.

Hence, this study primarily aimed to determine the internal consistency and test–retest reliability of the HTN-SCP tool amongst the English literate patients with hypertension in primary care in Singapore. The secondary aim was to assess the feasibility of conducting the study using a web-based approach.

Evaluation of the internal consistency and reliability of this tool will allow it to be used routinely as part of patient-centric management of hypertension amongst the multiethnic Asian population in Singapore, South-East Asia, and East Asian regions. If proven feasible and acceptable to subjects undertaking test–retest reliability evaluation of web-based health assessment instrument, such an approach will be a boon to internet-driven validation studies.

METHODS

Study Site

Hypertension is 1 of the top 3 chronic conditions managed in the public primary care clinic (polyclinics) in Singapore.⁸ The study was carried out in a typical polyclinic located in Pasir Ris, an urban estate on the north-eastern region of the island state. The polyclinic manages ∼7500 patients with hypertension in 2014 and is a branch of SingHealth Polyclinics, a public primary care institution, and a Family Medicine training centre certified by the Accreditation Council of General Medical Education.⁹ The study was conducted at the site from August 2014 to March 2015.

Study Population

The study population comprised of multiethnic Asian citizens or permanent residents, with clinical diagnosis of hypertension based on disease coding by the International Classification of Diseases (version 10 or ICD-10) and were managed at the designated polyclinic. Their diagnosis was confirmed with reference to their electronic health records at the study site. They included patients aged 40 years and above, of Chinese, Malay, Indian, and other ethnic origin and proficient in English. They reported willingness to access the internet via any form of devices to administer the web-based questionnaire twice on 2 separate occasions 2 weeks apart. Patients, who could only understand non-English local languages, did not have access to internet, not internet savvy or with known cognitive, visual and auditory impairment, were excluded.

Ethics Review

Ethics approval was granted from the Centralised Institutional Review Board (CIRB) for this study. Ref no: 2014/689/E.

Sample Size Calculation

A sample size of >100 subjects is considered adequate for assessing internal consistency and test–retest reliability.¹⁰ Hence, to account for the estimated 60% attribution rate for the retest,¹¹ due to anticipated lower re-test uptake rate from the use of the web-based tool, we inflated the sample to 160.

Tool

The investigators conducted a literature search, retrieved 34 studies, and reviewed 11 self-efficacy assessment tools. They evaluated their origins and development, validity, domains, and their relevance to hypertension, strengths, and weaknesses. The HTN-SCP was finally selected for this test–retest reliability study, based on its multidomain composition; content validity and its specificity to hypertension; and prior validation study in another ethnic Asian population, in which English is the common language. It is a self-administered English questionnaire, consisting of 60 items grouped into 3 sections. For each section, the questions commence with the following stems: (1) how often do you do; (2) how important you think it is to do; and (3) how confident are you in doing a particular task?

The questions cover multiple domains of self-efficacy, such as medication adherence, physical activity, diet restriction, alcohol consumption, smoking, home blood pressure monitoring, weight control, regular follow-up, and stress management. Each question uses a 4-point ordinal scale. The investigators have sought approval from the developer for its usage.

In anticipation of queries on food items and to contextualize to the diet habit of local patients, the investigators have inserted local food options of equivalent calories in the following questions within the HTN-SCP tool:

1. “Eat <1 teaspoon of table salt per day (6 g)”

2. “Limit total calorie intake from fat (<65 g) daily”

The investigators transformed the entire HTN-SCP questionnaire from print to digital format and expanded it to collect data pertaining to patient's demographic characteristics. They used the free Qualtrics platform to host the web-based questionnaire.¹²

Administration of Tool

Potential subjects were identified at a clinical monitoring and screening service point on a case-encounter basis before their medical consultation at the study site. They were provided with the study information by the investigators according to the approved protocol by the institution review board. Upon obtaining their informed consent, the enrolled patients were briefed on the use of the web-administration of HTN-SCP and proceeded to complete the “test” by administering the tool using the tablet provided by the investigators. To maintain anonymity, the subjects were provided with a study identification number, which would be used again during the “retest” 2 weeks later.

The patients were provided with instruction sheet detailing the administration procedure for the retest, which would be performed outside the study site. They had the option of using their smartphone or tablet or laptop to connect to the web-based HTN-SCP. Their contact information was obtained with their consent in order for reminders via calls, short message service (SMS), and emails to be sent to them to fill the same tool again during the retest.

Beyond the 2 weeks interval between test–retest, the investigators allowed an extra week for the subjects to carry out the retest. Those who failed to perform the retest despite reminders were excluded from the test–retest analysis. Data from the first test segment was computed to provide a preliminary cross-sectional perspective of the self-efficacy capability of this study population.

Statistical Analysis

Data were analyzed using the Statistical Package for Social Sciences (SPSS) version 22.0. Descriptive statistics were reported which included scores from the 3 domains of the HTN-SCP: (1) “behavior,” (2) “motivation,” and (3) “self-efficacy.” The chi-square test and Mann–Whitney U test were used to test for difference between the demographics of test and retest participants. To test the difference between the overall HTN-SCP scores and recommended self-management activities, t test was used on the continuous dependent variable against dichotomous independent variable. A P value of <0.05 was considered as statistically significant.

The internal consistency of the HTN-SCP was tested using Cronbach's Alpha statistic. Cronbach's Alpha value of >0.7 is considered acceptable. For good internal consistency, the value should be >0.8 and for excellent internal consistency, the value should be >0.9.¹³

To determine if any item in the scale is inconsistent with the averaged behavior of the other items, the item total correlation is tested. A mean item total correlation of 0.15 to 0.20 is desirable for scales that measure broad characteristics, whereas values of 0.40 to 0.50 are required for scales that have narrower characteristics, which is the case in the present study.¹⁴

Floor and ceiling effects were also examined to ensure that the tools can detect improvements in scores outside the range measured. A floor or ceiling effect of 15% is considered the maximum acceptable.¹⁵

To illustrate the extent to which similar scores are obtained when the scale is administered on different occasions separated by a relatively brief interval, the test–retest reliability is used. The interval between the first and second tool administration was 2 to 3 weeks. Test–retest reliability of the HTN-SCP and of its 3 domains, “behaviour,” “motivation,”’ and “self-efficacy,” was determined by calculating intraclass correlation coefficients (ICC). An ICC ≥0.75 indicates excellent reliability, 0.4 ≤ ICC < 0.75 indicates fair to good reliability, and ICC <0.4 indicates poor reliability.¹⁶

RESULTS

A total of 680 subjects were screened during the recruitment period (Figure 1): 254 were not previously diagnosed with hypertension, 92 were not proficient in English, and 20 of them (4.7%) were unwilling to perform the web-based questionnaire on a second occasion. The remaining 314 were invited to participate in the study: 97 expressed their disinterest, 37 (11.9%) did not have access to the Internet, 17 had previously been approached but declined. Three subjects were not recruited due to their exclusion criteria: not registered as a patient at the study site, being a foreigner, and had hearing impairment respectively.

FIGURE 1.

This figure summarizes the recruitment at different stages.

The remaining 160 subjects agreed to participate in the study with informed consent and completed the web-based questionnaire on the first occasion. Subsequently, 82 (51.3%) subjects did not proceed to the retest. Of the 78 subjects who completed the retest, 7 of them had invalid study identification number and were excluded in the analysis. Eventually, a total of 71 test–retest responses were used in the final reliability evaluation.

The demographic characteristics of subjects are shown in Table 1. Aside from hypertension, ∼2 out of 3 subjects had dyslipidemia, whereas 30% of them had concomitant type 2 diabetes mellitus. There is no statistical difference in the demographic profiles between the participants in the “test” and “retest” groups.

TABLE 1.

Demographics and Self-Management Practices of Participants

Table 2 shows the responses to the 60-item HTN-SCP questionnaire. Ninety-five to 96% of the subjects thought that taking the blood pressure medicine was important and were confident to do so, and this is reflected in their behavior as 95% of them sometimes/rarely/never forget to take their blood pressure medicine. “Read nutrition facts label to check information on sodium content?” is the lowest behavior, only 30.0% always/frequently do so. This is also the least number of subjects who felt that it is important to do so (116 or 72.5%). Only 60.0% of them are confident to “Read food nutrition facts label to check information on saturated (e.g. butter, red meats) and trans fat (e.g. lard, shortening)?”

TABLE 2.

Descriptive Statistics of 60-item Hypertension Self-Care Profile Instrument

Results on the overall score of the 60-items HTN-SCP and the recommended self-management practices show that subjects who carried out the recommended self-management activities had high HTN-SCP scores, compared to those who did not perform such activities (refer to Table 3). The relationship is statistically significant for all except “keeping a food diary.”

TABLE 3.

Relationship Between Self-Reported Self-Care Activities and the Hypertension Self-Care Profile Scores

Floor and Ceiling Effect

No floor or ceiling effect was found for the scores for the 3 subscales; only motivation has a ceiling effect of 11%, which is considered acceptable (Table 4).

TABLE 4.

Mean Scores and Ceiling Effects of Each Domain

Internal Consistency

Table 5 shows the results of the evaluation of the internal consistency of the scales within the HTN-SCP tool. The “test” data was used to determine the internal consistency of the items. For the items “How often do you forget to take your blood pressure medicine? (Forget to collect your medications?)” and “How often do you forget to fill your prescriptions?,” the options are reversed before conducting Cronbach's Alpha test as the questions are asked in a negative way. Cronbach's Alpha coefficient suggested overall high consistency across all 60 items (Cronbach's Alpha = 0.960), indicating that the items were closely related to the topic. For the individual domain, Cronbach's Alpha measurements were 0.857, 0.948, and 0.931 for “behaviour,” “motivation,” and “self-efficacy” respectively, showing a high level of internal consistency for the 3 scales.

TABLE 5.

Internal Consistency Reliability of Scales and Test–Retest Reliability of Stability

Item-Total Correlation (ITC)

The item-total correlation ranges for the 3 scales were from 0.105 (Item 15) to 0.656 (Item 3) for behavior, 0.401 (Item 34) to 0.808 (Item 25) for motivation, 0.349 (Item 54) to 0.789 (Item 47) for self-efficacy. The range shows a moderate-to-high degree for the perceptions in the questionnaire. Only Item 15 of the tool, “Forget to take your blood pressure medicine?” and “Forget to fill your prescriptions?” was considered a poor item (ITC<0.20).

Test–Retest Reliability

Table 5 also indicated the results of the test–retest reliability evaluation. The resulting coefficients for the 3 domains were high: ICC = 0.671, 0.762, and 0.720 respectively, showing excellent stability of measures over time. The 95% confidence intervals lie between (0.520, 0.781), (0.644, 0.845), and (0.586, 0.816).

DISCUSSION

The results of the test–retest reliability evaluation of this HTN-SCP tool suggest that it has potential application in English literate Asian patients with hypertension. This is feasible as the majority of the Singapore population uses English as the common language for communication. Nonetheless hypertension is a chronic disease that affects largely the middle-aged and elderly patients, some of whom may not be conversant with English. In fact, 1 in 5 (21.9%) patients declined participation in the study due to lack of English proficiency. There is thus a need to translate the HTN-SCP tool into Mandarin, Malay, and Tamil versions and validate them, if this tool is to be adopted for wider use amongst the local multiethnic Asian patients with hypertension.

Nonetheless, the lead investigator had anticipated local patients would face difficulties understanding several items in the tool. They specify quantitation of food ingredients such as salt, fat, and total calories. Local patients are less familiar with details such as 65 g of fat or 1 teaspoon of salt (6 g) to be reduced in their diet as such measurements are rarely done in their daily living. In anticipation of subjects’ high likelihood to raise questions during the administration of the tool, local food type, and quantity are enlisted to serve as surrogate food option with equivalent nutrient content. The local subjects are familiar with such examples, which minimize their misinterpretation of the items as this study relied on self-administration of the HTN-SCP tool.

The scores from HTN-SCP correlating with the subjects’ self-reported selected self-care activities such as home blood pressure and weight measurements were statistically significant, except for the maintenance of a food diary. The tool thus harbors potential to prognosticate their self-care actions. Further research can be considered to study the relationship between the HTN-SCP scores and the level of self-management activities and clinical outcomes, such as degree of adherence to pharmacotherapy for hypertension, blood pressure control status, and related cardiovascular events such as myocardial infarction and stroke.

The HTN-SCP tool also provides insight into the attributes of each subject in relation to the 3 scales for the domains of “behaviour,” “motivation,” and “self-efficacy.” This allows the development of domain-centred strategy to boost the self-care capability of the individual patient with specific deficiency. For example, managing a patient with lower score in the “motivation” domain, the attending physician or healthcare team may consider the use of specific counseling skill such as motivational interview technique to raise the level of motivation of the patient to embark on appropriate lifestyle changes.

Currently, the deployment of a web-based survey method to carry out the test–retest reliability assessment is relatively new. With the widespread use of smartphone in Singapore, the intent is to facilitate the performance of the retest at patient's convenience, reducing the need for them to return to study site, nor the inconvenience of mailing back the questionnaire to the investigator team. The eventual response to the retest was less than half (44.4%), despite sending reminders via email and SMS. Such electronic reminders were asynchronous, which were less taxing on manpower, less restriction on the selection of timing on telephone call and less costly than conventional mail requiring postage. The overall response rate of the web-based mode of validation was higher than those of most mailed questionnaire surveys in Singapore, which ranged from 20 to 30%.¹⁷

However, there are certain challenges in using a web-based approach to conduct the study. Patients with no internet access and/or who were proficient to use the internet were excluded from our study. Study identity codes were assigned to each subject to preserve their anonymity and confidentiality of their responses. Although the investigators had exercised their due diligence to indicate these identity codes on study information sheet for the subjects to use in the retest segment, some invalid identity codes were used, thus inflating the nonresponse rate, as they would be excluded from the data analysis.

Investigators contemplating a web-based method to field the instrument or questionnaire for their validation studies should plan a larger subject population due to the relatively high dropout rate for the retest portion of the reliability assessment. Nonetheless, Table 1 shows that the profile of the subjects who completed both test–retests were demographically similar to those who completed only the test segment.

Overall, this study shows that a web-based approach is feasible in conducting such studies. It has several advantages to the data collection. Aside from convenience, missing data is minimized as the subject would not be able to proceed with the questionnaire unless the required data field is filled. Transcribing data error is also abolished as the data is entered into an MS Excel spreadsheet directly. The questionnaire operates on a free web-based platform, and thus reduces the expenditure in executing this nonfunded study.