Development and Psychometric Assessment of the Methadone Therapy Experiences Questionnaire among Patients under Treatment

Abstract

Background

Patient experiences play an important role in the quality of health care and gathering patients’ experiences is common as part of quality measurement in health care. The present study was carried out with the aim of developing and psychometric analysis of the methadone therapy experiences questionnaire among patients under treatment with methadone.

Methods

This cross-sectional study was performed in 2018 and 200 patients referred to the addiction treatment clinics in Kerman, Iran, participated in this study. The convenient sampling method was employed. The validity was assessed using the opinions of 50 individuals similar to the target population and 8 experts. In addition, the Cronbach’s alpha coefficient was utilized to examine the test-retest reliability. Data were analyzed using descriptive and inferential statistics in the SPSS software.

Findings

The face validity of the questionnaire was acceptable in the present study and the values for content validity were higher than 0.79, indicating the appropriate content validity of the questionnaire [content validity index (CVI) = 0.82 and content validity ratio (CVR) = 0.83]. Moreover, the results confirmed the reliability or reproducibility of the questionnaire (Cronbach’s alpha = 0.83).

Conclusion

The methadone therapy experiences questionnaire was of a good validity and reliability among the patients. In the clinical area, the psychiatrists, psychologists, authorities, and staff in addiction treatment clinics can identify the quality of treatment and its strengths and weaknesses using this questionnaire.

Introduction

Drug abuse is a disaster for consumers, their families, and the community. Therefore, it is recognized as a major contributor to social, economic, health, and criminal problems.1 The trend of drug addiction is increasing in countries such as China, India, Indonesia, Russia, Malaysia, Pakistan, Iran, and many other countries worldwide, so that its global burden on health is estimated to be about 0.7% in recent years.2,3

The addiction directly affects the addict’s body and mind. One of the issues that is important in these individuals in the psychological field is how they interact with the environment and their adaptability mechanisms and behavior type that are in some cases the cause and in most cases, the effect of the phenomenon of addiction, behaviors that can cause very unpleasant consequences for them and other members of the society and are considered as high risk behaviors.4

Thus, addiction has raised concerns among policy makers and has become an important public health problem requiring appropriate interventions and plans. Despite the overwhelming effort of Iranian government over the past three decades on drug trafficking and use, statistics show that the prevalence of drug abuse is approximately 27 per 1000 population, well above other countries such as China, Germany, Finland, Luxemburg, and Lithuania; this confirms vulnerability of Iran to substance abuse in geographic and demographic sense.5 Nowadays, maintenance treatment with opioid drug compounds (especially methadone and buprenorphine) is considered as one of the most common and valuable therapies to reduce the risk of drug abuse.6

Methadone maintenance treatment began in 1965, with the first clinic demonstrated as part of the Addictions Research program in Rockefeller University. Since then, a lot of studies have shown methadone pharmacotherapy to be the safe, efficient, and effective therapy for heroin addiction.7 For the first time in Iran, this drug was introduced into the medical system of the country in 2005, and now more than 300 centers in this country have been providing it to patients based on the national treatment protocol. Since 2009, methadone has only been used in maintenance treatment.5

Methadone maintenance therapy (MMT) is an effective way to reduce heroin use and use of injection methods, in addition to reducing crimes, enhancing social functioning, and improving physical symptoms and thus improving the quality of life (QOL) of these people.8 Despite the benefits of using this type of treatment that reduce the adverse effects of substance abuse and protect the high risk groups from problems such as viral hepatitis or acquired immunodeficiency syndrome (AIDS), this drug may result in side effects among the consumers. The complications of methadone include the cerebrospinal conditions,9 gastrointestinal complications, genitourinary problems, skin, ear, nose, and throat complications, and vascular complications.10

On the other hand, the advantages of this treatment are also of interest. A study by Roohani et al. revealed that drug therapy and methadone treatment could be a good way to treat addicts and improve their QOL.11 Therefore, understanding the experiences of the patients during the MMT is important to continue their treatment and this can provide a better and deeper understanding of the concept of MMT and can also be used to revise standards of the methadone clinics and improve service quality.12

Studies have indicated that several different factors influence the rate of preservation of the patients in maintenance treatment, generally classified in three main categories including patient-related factors, treatment plan-related factors, and community characteristics.13 However, the quality of the maintenance treatment in Iran has received less attention. Few studies have addressed this issue, which may be due to the lack of appropriate tools for this purpose. Therefore, given the lack of instruments to measure the experiences of the drug-dependent individuals in MMT referring to treatment centers to quit addiction, this study was performed aiming to develop and standardize the methadone therapy experience questionnaire to develop a tool according to the cultural conditions of Iran and standardize it in this group.

Methods

This cross-sectional study was conducted on the patients referring to the addiction treatment clinics in 2018. The statistical population consisted of the individuals referring to addiction treatment clinics in Kerman, Iran. Since this questionnaire was prepared for the first time, a suitable sample of 200 people entered the study with the information obtained from the drug addiction clinics from among the population of 1000 people referring to these clinics using the Morgan sampling table. The inclusion criteria were: patient’s consent to participate in the study and being under maintenance treatment. Similarly, the exclusion criteria included lack of tendency to participate in the study and incomplete questionnaires.

The study subjects were selected from among the methadone-treated patients referred to methadone clinics in Kerman using the convenience sampling method. In the pilot phase, to examine the structure of the initial version, 50 individuals with methadone treatment referring to the methadone clinics in Kerman were selected. In the second stage, 200 patients were selected from the methadone treatment clinics in Kerman to investigate the structure of the final version.

The first phase of developing the questionnaire began with a review of theoretical resources on the experiences of opioid dependents. In addition, the records of patients referred to addiction treatment clinics were also reviewed to determine these dimensions. In the second phase, the important aspects were identified. Then, based on the recent recommendations on how to construct the questionnaire, of this study designed some questions for each of the above factors. The questions were designed in a way to be short and clear, without vulgar or technical terms, and understandable for people with minimal literacy. The designed questions were asked by three psychologists who were expert in the field of addiction, and the above factors were well included in the items. In this way, the initial version of the questionnaire was prepared.

To determine the face validity of the questions of the questionnaire, the participants were asked about the clarity and understandability of the questions. To assess the face validity of the questionnaire items, 50 people similar to the target population were polled about the clarity and understandability of the questions, and 8 experts were asked about the general form and understandability of the questions. The experts’ opinions were exploited to determine the content validity and they assessed each of the questionnaire items in three areas. These three areas were: suitability, transparency, and necessity. The content validity index (CVI) and content validity ratio (CVR) were calculated. Moreover, the expert panel was employed to measure the content validity. In this way, the questionnaire was provided to 8 experts and they were asked to examine the questionnaire items in terms of the issues as which questions needed to be included in the questionnaire, which questions were useful but unnecessary, and which questions were better to be eliminated. Since the number of experts was 8, the minimum value for CVR was considered as 0.75 according to the Lawshe table. Based on the corresponding formula of CVR = (ne - n/2)/(n/2), the CVR was calculated for each item. In this formula, n and ne were the total number of experts who participated in the questionnaire validity assessment and the number of experts who had chosen the necessary option for the item, respectively. Furthermore, the value set for CVI as higher than 0.79, 0.70 to 0.79, and less than 0.70 indicated suitability, requiring modification, and unacceptability, respectively.14,15

Internal consistency reliability was calculated using the Cronbach’s alpha method. This method is the most commonly used internal consistency reliability coefficient employed in most studies and represents the proportion of a group of items that measure a structure. The alpha value must be at least 0.70 or greater for a question to remain in a tool, and most researchers consider the point of 0.80 to be necessary for a question to remain in the tool. The Cronbach’s alpha was used in this project for the internal consistency.16 Data were analyzed using the descriptive and inferential statistics in the SPSS software (version 21, IBM Corporation, Armonk, NY, USA).

The proposal obtained the ethics code with IR.KMU.AH.REC.1397.085 number from the Ethics Committee of Kerman University of Medical Sciences, Kerman, Iran.

Results

In order to assess the face validity, the questionnaire was provided to 50 people similar to the target population to examine the clarity and understandability of the questions, as well as to 8 experts in the form of an expert panel to check the questionnaire expressions in terms of the clarity (using simple and understandable words) and use of a common language (avoiding specialized technical terms). After collecting the questionnaires, the comments were applied to the questionnaire.

The expert panel was employed to measure the content validity. In this way, the questionnaire was provided to 8 experts and they were asked to examine the questionnaire items in terms of the issues as which questions needed to be included in the questionnaire, which questions were useful but unnecessary, and which questions were better to be eliminated. Given the number of experts (8 individuals), the minimum value for CVR was considered as 0.75 according to the Lawshe table.15

To determine the CVI of each item, the opinions of the expert panel in the form of relevance were taken as a 4-point Likert scale for each criterion. To examine the relevance criterion, the four options of totally relevant, relevant, relatively relevant, and irrelevant were used. Finally, to calculate the CVI, the number of agreement of the expert panel members with the first two options of each criterion was calculated for each item, and the resulting number was divided by the number of experts, i.e., 8, and the CVI of each item was determined.

The initial questionnaire used in the pilot phase consisted of 61 questions. Based on the results of the data analysis, the status of the questions changed into the following form:

Unchanged and accepted questions: 2, 6, 14, 15, 16, 18, 19, 21, 23, 24, 27, 28, 37, 38, 44, 45, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61

Modified questions: 1, 3, 5, 11, 12, 41, 46

Deleted questions: 4, 7, 8, 9, 10, 13, 17, 20, 22, 25, 26, 29, 30, 31, 32, 33, 34, 35, 36, 39, 40, 42, 43, 47, 48, 49

Finally, the questionnaire with 35 questions was utilized in the second phase of the study (Table 1). In this questionnaire, questions 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 15, 16, 18, 21, 22, and 23 were negative and numbered inversely.

Table 1.

Content validity of the questionnaire using content validity ratio (CVR) and content validity index (CVI) before and after modification

Row	Question	CVR	CVI
1	Have you been tempted to use drugs during methadone treatment?	0.81	0.72
2	How much have you experienced severe hangover symptoms in the first few days of methadone treatment?	0.87	0.82
3	How much has the addiction of the individuals around tempted you to consume drugs?	0.83	0.71
4	Do you feel embarrassed to attend the clinic for methadone use?	0.79	0.75
5	Do you think quitting methadone will improve your health?	0.90	0.89
6	Does methadone cause skin problems for you?	0.76	0.77
7	Have you had any energy loss with methadone use?	0.79	0.75
8	How much has the concern about the side effects of methadone use reduced your drug use rate?	0.85	0.88
9	How much has the concern about the side effects of methadone use stopped you from its use?	0.79	0.81
10	How much has the concern about the methadone side effects and inability to quit it led you to stop using it?	0.90	0.83
11	How much does fear of liver problems reduce its use or stop you from taking it?	0.78	0.79
12	How much does fear of renal problems reduce its use or stop you from taking it?	0.80	0.82
13	How much does the fear of infertility reduce its use or stop you from taking it?	0.74	0.80
14	Have you been given sufficient information about the treatment of methadone on arrival at the center?	0.78	0.79
15	Have you ever exchanged the pills with drug outside the center?	0.75	0.83
16	How difficult was your admission at the center in the methadone plan?	0.78	0.79
17	How satisfied are you with the behavior of the center staff?	0.81	0.80
18	How much do you feel like using alcohol with methadone use?	0.87	0.84
19	How satisfied are you with the number of times of methadone administration?	0.79	0.80
20	Do you think methadone is better than drugs because of its purity?	0.78	0.75
21	Do you think you will be labeled as an addict with methadone use like drug abuse?	0.79	0.90
22	Do you think the side effects of methadone on health are more than those of drugs?	0.85	0.87
23	How much can you afford methadone costs?	0.76	0.72
24	How much has the ease of providing methadone encouraged you to quit?	0.82	0.79
25	How much has methadone adjusted your sleep?	0.89	0.88
26	How much has methadone relieved your sexual problems?	0.82	0.85
27	How much has methadone relieved your pains?	0.90	0.86
28	How much has methadone improved your mood?	0.89	0.86
29	How much has your confidence been improved with methadone?	0.85	0.84
30	How much do you feel healthier with methadone?	0.81	0.80
31	How much has your legal conflict decreased with methadone use?	0.87	0.96
32	How much has your job status improved with methadone?	0.88	0.95
33	How much has your family relationship improved with methadone use?	0.90	0.93
34	How much has your social relationship improved with methadone use?	0.88	0.90
35	How much has your appearance improved compared to before using methadone?	0.88	0.89

CVR: Content validity ratio; CVI: Content validity index

The internal consistency was employed to assess the reliability of the questionnaire. Internal consistency or correlation is the degree to which the questions in a questionnaire are correlated and summarized in a single index, with the most common method of calculation being the Cronbach’s alpha coefficient. Given this method, the instrument will have a good reliability when the alpha coefficient is greater than or equal to 0.70. In the current study, the Cronbach’s alpha coefficient was 0.83 and confirmed.

Discussion

Patient experiences play an important role in the quality of health care, and collecting patient experiences as part of quality measurement is common in health care.17 Questionnaires are the main tool for gathering basic data, which in addition to collecting quantitative data in a standard way, allow for the analysis of these data due to their internal consistency and coherence.18 The objective in the present study was to construct and standardize the methadone therapy experiences questionnaire among the patients under treatment. Some studies by designing and psychometric analysis of questionnaires have examined the experience and satisfaction of patients with the treatment received, which was related to general or population-specific therapies and was tailored to the type of treatment and purpose of the studies. It is evident that it was not possible to use these tools for the participants in the present study and it would not lead to a proper evaluation. Besides, the presence of methadone-treated individuals in research studies is rare and evaluation processes are usually not properly performed and the needs of this group are not met well, so access to a reliable and valid tool for investigating this issue is essential and helpful.

Using a tool is a very complex process that requires careful planning for maintenance of the content and its psychometric properties as well as its general credibility for the target population. During this process, there should be evidence of the semantic equivalence of each question and the psychometric properties of the version prepared. The tool also has to be culturally appropriate, meaning that it can be used in different cultural contexts. Ignoring any of the rules that are necessary in constructing a questionnaire leads to a difficulty in interpreting the results and this may affect clinical or educational performance.19

In the present study, the aspects of therapeutic effect and the overall satisfaction of the patients under methadone therapy, the patients’ mental status and perceptions of the effect and complications of treatment at the personal health levels, and their family and social relationships were examined, and for each of them, some questions were considered and designed in the form of the five-point Likert scale ranging from 1 (not at all) to 5 (very much). In Norway, Haugum et al. designed a questionnaire to examine patients’ experiences in interdisciplinary treatment of drug dependence (narcotics, alcohol, and drug); in this study, 51 closed-ended questions were designed in the Likert scale with lowest to highest scores of 1 to 5 to the answers of “not at all” to “very much”, respectively, in addition to two open-ended questions. The questionnaire included questions on waiting and admission time, environment and conditions of service, therapist, and medical staff, preparation for post-discharge period, and admission record. Compared to the present study, the questions were designed to cover a wide variety of the drug-dependent patients and their various aspects, treatment and medical personnel, and treatment environment and outcome, and were not focused on a particular type of treatment and patients.20

However, in the present study, in addition to the cases mentioned, the target group and the type of treatment were specifically determined. In the study by Kimman et al. carried out in the Netherlands, the authors designed a questionnaire to evaluate experiences and drug satisfaction among patients with pulmonary problems, with questions developed in several general and specific sections to evaluate drug use experience and patient satisfaction, taking into account common points in these questionnaires including effectiveness, side effects, ease of use, and overall satisfaction.21

The content and face validity are commonly used in designing the questionnaire for its apparent consistency and content range.22 The content validity, in addition to achieving suitability, transparency, and necessity, also contributes to improving the reliability indices of the instrument and reducing the financial resources and time required to prepare the questionnaire.23 Face validity is a valid method that is both simple and applicable and provides important and different views on the questionnaire. If the questionnaire is not apparently acceptable, the participants will be reluctant to answer the questions. Therefore, face validity is the most important principle for accepting and completing the questionnaire. In this study, to examine the face validity, a group of 50 people representing the target population as well as 8 expert panel members were employed to determine the face validity and the results were appropriate. Although the qualitative nature of the experts’ responses is a problem, the quantitative stages of the implementation process greatly reduce concerns about this issue.23 The content validity of the questionnaire was determined using standard guidelines and qualitatively by the experts’ opinions and quantitatively using CVR and CVI; and the obtained values were greater than 0.79, indicating the validity of the questionnaire (CVI = 0.82 and CVR = 0.83). In other words, the questionnaire was desirable in terms of accuracy, clarity, simplicity, comprehensibility, and proportionality with the target population.24

Internal consistency means the fixed responses of the subjects to all questions in the questionnaire. In other words, all questions represent the same basic structure, so the individual’s responses to all questionnaire questions should correlate with each other. In the present study, the Cronbach’s alpha was exploited to determine the internal consistency or reliability, which is appropriate and common to assess the reliability of the questionnaire in the Likert scale,25 and the results confirmed the reliability or reproducibility of the questionnaire (Cronbach’s alpha = 0.83). In other words, if the study is repeated under similar conditions, the scores obtained will be relatively similar.26

Given that the validity and reliability of this questionnaire were obtained in a particular community, there were some limitations to the collaboration, especially regarding the calculation of the reliability in the test-retest method, in which the information was confusing. Moreover, it was better to explore the constituent areas of the questionnaire using the statistical methods of the exploratory factor analysis (EFA) and confirmatory factor analysis (CFA). There have been no similar studies on the development of quantitative tools for measuring the quality of MMT, and most studies have addressed this issue in a qualitative manner. Accordingly, it was not possible to compare the findings of the present study with similar cases, and it was tried to compare the results with other studies done to investigate the experiences of patients in receiving treatment in other populations, and this was another limitation of the present study.

Conclusion

The validity and reliability of the questionnaire were acceptable. In the clinical domain, the psychiatrists, psychologists, and staff in addiction treatment clinics can identify the quality of treatment and its strengths and weaknesses using this questionnaire. Authorities can also use this tool in their executive processes. Since applying a new tool in any field of science requires training and awareness of how to use this new tool, the individuals who intend to use the tool should have the necessary training and a thorough knowledge on how to fill in the questionnaire correctly in the group under study.