A Self-Reported Version of the Measurements in the Addictions for Triage and Evaluation-Q: Concurrent Validity with the MATE 2.1

Abstract

Introduction

Substance abuse treatment centers require reliable and valid instruments to monitor treatment progress, to evaluate treatment effectiveness, and to initiate clinical trials. Currently the Measurements in the Addictions for Triage and Evaluation (MATE) 2.1, an instrument that serves these purposes, is considered quite lengthy and intensive, especially in the case of allocation to milder treatment intensity. Therefore, a self-reported version of the MATE-Q was designed for patients with mild to moderate substance-abuse and co-occurring problems. The aim of the present study was to assess concurrent validity with the interviewer version of the MATE (version 2.1).

Materials and Methods

Data were collected at 2 locations of a Dutch substance abuse treatment center, one location in a large city and one in a suburban area. A correlational design was employed, where each included participant completed a MATE-Q and a MATE 2.1 within 3 days or less (administered at intake, before treatment initiation). A total of 98 treatment-seeking patients were included (51.0% alcohol as a primary problem, 19.4% cannabis, 14.3% gambling and 6.1% cocaine). Measurements included the MATE-Q and the MATE 2.1. Intraclass correlation coefficients (ICCs) for single measures were calculated, deploying the 2-way mixed procedure with absolute agreement. Descriptives of scores comprise means and Cronbach's alpha for internal consistency.

Results

For the majority (15 out of 24) of the scores ICCs were equal or above 0.7. For 93 patients (95%), the primary problem substance or problem behavior was reported correspondingly. Nine MATE-Q mean scores differed significantly from their MATE 2.1 counterparts.

Discussion/Conclusion

For the majority of scores, the MATE-Q has acceptable concurrent validity for the assessment of patients with mild to moderate substance abuse and co-occurring problems.

Introduction

Substance use disorders are associated with serious public health problems, public safety problems, and economic loss [1]. Treatment systems require efficient, rational and transparent procedures and assessment instruments to inform patients about the available treatment options and to support allocation to appropriate levels of care [2, 3]. Subsequently, reliable and valid tools are needed to monitor treatment progress, to evaluate treatment effectiveness, and to initiate clinical trials. One of the instruments with these aims is the Measurements in the Addictions for Triage and Evaluation (MATE), a semi-structured interview, available in 6 languages and used in several European countries [4, 5, 6, 7, 8]. In the Netherlands, the MATE is implemented in a number of substance abuse treatment centers, in an intake- and outcomes version, both for adults and youth. The intake version is referred to as “the MATE,” of which version 2.1 is the latest. The MATE 2.1 is an intake-interview that is administered by trained assessors (usually addiction psychologists), and takes between 45 and 60 min to complete [9, 10]. For patients with mild to moderate substance abuse and co-occurring problems, this assessment duration is disproportionate to the subsequent allocated treatment duration. Therefore, the MATE-Q was developed: a shorter self-administered version, comprising 8 modules originating from the MATE 2.1 and an additional module assessing motivation for treatment [11].

The MATE-Q is designed for situations when personal contact with a professional is neither available nor appropriate, for instance, in the case of those seeking online treatment. The instrument collects information on substance use, addictive behaviors, as well as life domains like psychological and physical health, activities and participation, and comorbid problems. Rationale of the instrument is that it (1) provides information that can be used by remote assessors to allocate patients with mild to moderate problems directly toward low levels of care (on-line or face to face), and (2) provides information to be reviewed during a personal assessment interview for patients whose MATE-Q scores indicate more severe problems [11]. Both applications should result in a highly accessible, efficient allocation procedure for patients with apparently low to moderate addiction problems.

In 2014, feasibility of the MATE-Q was studied in a sample of 289 patients of an out- and inpatient treatment center in the south of the Netherlands. The far majority of the respondents reported alcohol (56.1%) or cannabis (22.8%) as their primary problem substance, and no patients with opiates as their primary problem substance were present. Completion time of the MATE-Q was 23 min on average, with a median of 20 min. Over 90% of the respondents reported a good uptake of the questions and over 85% affirmed the importance of the questions for problem assessment. Less than 8% indicated that the MATE-Q contained too many questions [12].

Another important prerequisite for an accessible, ­efficient allocation procedure is the validity of the ­MATE-Q: the self-administered questionnaire should yield similar results as would have been obtained for the same patient interviewed with the MATE 2.1. Although review of an assessor is always more or less present, deviations from the standard (the MATE 2.1) will result in a less accurate and less efficient allocation procedure. Therefore, this study examines the concurrent validity of the MATE-Q with the MATE 2.1 by assessing the agreement of the MATE-Q scores with corresponding scores derived from the MATE 2.1 and for the added module assessing treatment motivation.

Materials and Methods

Setting

Data were collected at two locations of a Dutch substance abuse treatment center (Jellinek), one location in a large city and one in a suburban area. Standard intake procedure consists of administering the MATE 2.1, supplemented with a module assessing treatment motivation. For reasons of clarity, from now on when referring to MATE 2.1, all modules including the added Motivation for treatment module are meant.

Yearly, about 1,800 patients apply for intake at the two treatment locations. Available treatment options vary in intensity levels, of which brief out-patient treatment is the least and in-patient treatment is at the most intensive level. Allocation takes place according to a structured matching and allocation protocol that takes scores on the MATE 2.1 as well as patient preferences into account [9, 13, 14, 15].

In this study, patients completed the MATE-Q, next to the standard intake procedure using the MATE 2.1. The MATE 2.1 was administered by trained intake assessors, using a software package supportive in data-management and outcomes monitoring and feedback. For the administration of the MATE-Q, the same software package was used and accommodated for self-report. The majority of the intake assessors (psychologists, social workers, or psychiatric nurses) received a booster training for the administration of the MATE 2.1 and an instruction on how to do this using the software package.

Participants

Data collection ran between November 15, 2016 and April 26, 2017. A total of 158 patients completed both the MATE 2.1 and the MATE-Q questionnaire. The average age of patients was 40.7 years (SD 13.8, min-max: 15–74 years), 75% were male and 80% were of Dutch descent. This is in concordance with the profile of the national treatment-seeking population [16] and similar to characteristics of the generally observed outpatient population [17].

Data were eligible if time between administration of the MATE 2.1 and the MATE-Q was 3 days or less. This was considered a safe option for the assessment of concurrent validity since ratings considered timeframes of 7 or 30 days. There were no other exclusion criteria. Data of 98 patients were included for analysis. No differences between included and excluded patients were found on primary problem substance or severity of dependence/abuse. Of the included patients, 73 (75%) completed the MATE-Q before the administration of the MATE 2.1 took place, and 25 patients (25%) completed the MATE-Q afterwards. According to the MATE 2.1, 51.0% reported alcohol as a primary problem substance, 19.4% cannabis, 14.3% gambling, and 6.1% cocaine. One patient reported opioids as a primary problem. Compared to the regularly known outpatient population [17], this is an overrepresentation of patients with gambling problems and an underrepresentation of patients with cocaine problems. For 83.7% of the patients it was the first or second treatment episode in 5 years (according to the MATE 2.1). This is indicative of a population eligible for lower level treatment intensities [15].

Procedure and Assessments

During registration for intake assessment, patients were informed about the study, both orally and through written information. Two weeks prior to the upcoming assessment, a research assistant contacted patients by telephone, re-informing them about the study, and asking for participation. Consenting patients received a link to a digital version of the MATE-Q. In order to assess potential order effects, 60% of the patients received the link within 2 days prior to the planned intake and the remaining 40% received the link just after their planned intake session. A telephone reminder was given if patients did not complete the MATE-Q. Patients who could not be reached by telephone, were sent the link (and information about the study) as well, and they completed both the intake interview and the MATE-Q implied consent for participation.

The MATE 2.1

Intake assessment consisted of the MATE 2.1, an intake interview aimed at establishing “[…] a professional assessment of the person at the start of a treatment program (p3)” [11]. The instrument comprises 10 modules, of which some are newly developed (like the module “Activities and participation”, based on the international classification of functioning [18]) and others are existing instruments like the 21-item Depression, Anxiety and Stress scales [19, 20, 21]. Assessment takes place face-to-face, based on information or experiences reported by the patient (interview), by observations of the assessor, or based on self-report. Three self-reported modules can be administered independently of the assessor, that is, filled out by the respondent, unless participants have cognitive difficulties or appear to be under the influence of alcohol or drugs. These are MATE 2.1 modules with the prefix Q (of questionnaire, see Table 1). In this setting, the MATE 2.1 Q modules were administered to participants via the computer display with the assessor capturing the participants' answers and entering those in the system. In cases where cognitive or other difficulties were expected, the “Q” modules were administered in the interview form.

Table 1.

Modules of the MATE-Q and their MATE 2.1 counterparts

MATE-Q	MATE 2.1
0a Primary problem substance or - behavior	1 Substance use
0b Substance use disorder	4 Substances: dependence and abuse
1a Lifetime substance use	1 Substance use
1b Substance use in the past 30 days
2 Craving for substances	Q1# Substances: craving
3 Physical and psychiatric complaints	5 Physical complaints
	2 Indicators for psychiatric or medical consultation
4 DASS	Q2# Depression, anxiety and stress
5 Previous treatment and prescription drugs	3 History of treatment for substance use disorders
	2 Indicators for psychiatric or medical consultation
6 Motivation for treatment	Q3# Motivation for treatment*
7 Difficulties and problems	7 Activities and participation
8 Circumstances	8 Environmental factors influencing recovery
9 Closing questions

^*Module Q3 is the additional module (part of the MATE-Crimi), all others are original MATE 2.1 modules.

^#For modules that are self-report in origin, in the MATE 2.1 manual the prefix “Q” (referring to “questionnaire”) is used.

The MATE 2.1 is available in the public domain. Completing the MATE 2.1 and applying the accompanying scoring algorithms from the manual results in 20 scores. In addition, the manual proposes an algorithm that results in a suggestion for allocation to levels of care [9, 11].

As mentioned before, an additional module was administered, called Motivation for treatment. This is the Dutch translation and adaptation of the Motivation for Treatment Questionnaire [22]: the original items concerning motivation for the actual, current treatment are adapted to items that capture the consideration for undergoing treatment. Originally this module is included in the MATE-Crimi, an instrument aimed at the assessment of characteristics of patients with alcohol, drug, or gambling problems accompanied with behavior that leads to criminal conviction [10].

The MATE-Q

The MATE-Q is a self-report questionnaire containing 9 modules and can be completed within approximately 20 min [11, 12]. Table 1 shows the modules and their MATE 2.1 counterparts. The questionnaire is concluded with 7 “closing questions” aimed at collecting additional information that can be supportive for the assessor, that is, comprehension of questions, the importance of questions for the patient, and if there is additional information in the patient's view that needs to be shared.

MATE 2.1 and MATE-Q: Similarities, Differences, and Scores

The MATE-Q is developed for occasions where face-to-face contact is not essential or desirable, for example, for online screening and assessment. Second, for occasions where problems turn out to be more severe than initially manifested, the results of the MATE-Q can serve a more efficient assessment procedure because they can be reviewed and thus serve as partial input for the full MATE 2.1 interview.

The instruments differ in composition. The MATE 2.1 contains a module assessing indications for personality disorders. This module is absent in the MATE-Q. Also, the modules Activities and Participation and External Factors (fully included in the MATE 2.1) are partly included in the MATE-Q because not all parts of the original are suitable for self-report [11, 12]. Additionally, since the MATE-Q is designed for persons (considering) seeking substance abuse treatment and because persons who fill out the questionnaire might have doubts on perceived helpfulness of treatment, information on motivation is collected with a motivation for treatment module (absent in MATE 2.1) [11].

Both instruments come with a manual providing algorithms for calculating several (sum)scores that can be derived from the modules, expanded with a procedure for the conversion of MATE-Q scores on substance use to corresponding MATE 2.1 scores [9, 10, 11, 23]. Answer options for both MATE-Q and MATE 2.1 instruments vary between and within modules, for example, “yes” and “no” answers, 5 or 7 point Likert scales or answer options for quantity-frequency questions. In total, there are 18 corresponding MATE-Q and MATE 2.1 scores.

Data Analysis

Patient records were extracted from the administrative systems, containing the MATE-Q modules and the corresponding MATE 2.1 modules. In order to guarantee anonymity, no extra patient characteristics were extracted at the individual level. Information on age, gender, and nationality was provided at the aggregated level, in order to draft a picture of the patient group under study.

Analyses were performed with SPSS 24.0. The 18 scores resulting from the corresponding MATE-Q and MATE 2.1 modules were calculated according to the algorithms in the manuals [9, 10, 11]. In addition, we analyzed 5 items from the Substance Use module and the reported Number of previous treatment episodes, since these items are (co-)determinative in the treatment allocation protocol [9, 10, 11, 13, 14, 15], resulting in 24 corresponding scores in total. For all scores (except problem substance or behavior), intraclass correlation coefficients (ICCs) for single measures were calculated, deploying the two-way mixed procedure with absolute agreement.

Differences between means of the MATE-Q and MATE 2.1 scores are tested with paired t tests to see whether they were similar for self-report and interview. Controlling for the family-wise error rate (that would be 69% percent in this case with 23 tests), we used the Benjamini and Hochberg method, allowing a false discovery rate of 10% [24, 25]. In addition, we checked whether ICCs of patients who completed the MATE-Q before the MATE 2.1 differed from the ICCs of patients who completed the MATE-Q after the MATE 2.1. A potential effect of completing the MATE 2.1 first can be that the MATE-Q is easier to self-administer, leading to higher ICCs for the patients who were interviewed first in comparison to patients who underwent the procedure in the reverse order.

Descriptives of scores comprise means and Cronbach's alpha for internal consistency, of which a value above 0.70 is regarded as acceptable [26]. ICCs lower than 0.70 were classified as “low agreement,” indices of 0.70 till 0.90 as “acceptable to good agreement,” and those of 0.90 and higher as “high or excellent agreement” [27].

Results

Table 2 shows means, Cronbach's alphas, and ICCs for corresponding MATE-Q and MATE 2.1 scores. For 93 patients (95%), the primary problem substance or problem behavior was reported correspondingly. The majority of Cronbach's alphas are acceptable, aside from those for Characteristics of physical and psychiatric comorbidity, Undergoing psychiatric or psychological treatment and Negative external influences for both MATE-Q and MATE 2.1, and for Severity of dependence/abuse for the MATE 2.1 score.

Table 2.

Means, Cronbach's alpha and ICC's for MATE-Q and MATE 2.1 scores

Module name/score		Participants (n = 98)
		MATE-Q mean (SD)	MATE 2.1 mean (SD)	ICC	95% CI	Cronbach's alpha$
0: Problem substance/behavior
	Primary problem substance or behavior			95%
S4.3	Severity of dependence/abuse*	5.5 (2.1)	6.0 (1.9)4	0.70	0.57–0.80	0.72/0.63
1: Substance use
	Substance use past 30 days Alcohol, standard units	108 (134.1)	137 (193.2)5	0.70	0.58–0.79
	Nicotine, standard units	276 (291.1)	271 (306.3)	0.95	0.93–0.97
	Cannabis, days of use	5.3 (10.4)	5.3 (10.8)	0.97	0.96–0.98
	Gambling, days	0.8 (3.5)	1.3 (5.4)	0.28	0.09–0.45
2: Craving for substances
SQ 1.1	Craving*	7.5 (4.1)	7.8 (4.8)	0.75	0.64–0.82	0.84/0.84
3: Physical and psychiatric complaints
S5.1	Physical complaints	10.9 (7.8)	9.3 (6.6)6	0.81	0.70–0.87	0.86/0.78
S2.1	Characteristics of physical comorbidity	0.60 (0.8)	0.37 (0.7)7	0.49	0.32–0.63	0.26/0.50
S2.3	Characteristics of psychiatric comorbidity	0.87 (1.0)	0.30 (0.8)8	0.51	0.17–0.70	0.36/0.60
4: DASS
SQ 2.1	Depression	15.8 (12.7)	13.0 (10.9)9	0.79	0.67–0.86	0.93/0.91
SQ2.2	Anxiety	9.2 (9.2)	7.9 (8.3)10	0.77	0.67–0.84	0.86/0.83
SQ2.3	Stress	14.8 (10.6)	13.6 (10.1)	0.78	0.69–0.85	0.91/0.89
SQ2.4	DASS total	39.8 (29.4)	34.4 (25.6)11	0.80	0.70–0.86	0.95/0.94
5: Previous treatment and prescription drugs
S2.2	Undergoing psychiatric or psychological treatment	0.63 (0.8)	0.75 (0.8)12	0.80	0.71–0.86	0.65/0.67
	Number of previous treatment episodes1, 2	1.08 (2.3)	0.71 (1.5)13	0.63	0.49–0.73
6: Motivation for treatment
SQ3.1	Problem recognition − general	3.5 (0.9)	3.5 (0.9)	0.80	0.71–0.86	0.71/0.79
SQ3.2	Problem recognition − specific3	2.3 (0.8)	2.3 (0.9)	0.70	0.58–0.79	0.70/0.75
SQ3.3	Desire for help	3.4 (0.8)	3.6 (0.7)14	0.75	0.63–0.83	0.79/0.78
SQ3.4	Treatment readiness	3.8 (0.6)	3.9 (0.6)15	0.65	0.51–0.75	0.73/0.79
7: Difficulties and problems
S7.1	Limitations − total	18.2 (13.1)	12.6 (9.0)16	0.53	0.28–0.69	0.90/0.81
S7.2	Limitations − basic	5.5 (5.7)	3.5 (3.7)17	0.47	0.27–0.62	0.84/0.69
S7.3	Limitations − relationships	4.9 (4.2)	3.6 (3.2)18	0.52	0.34–0.66	0.75/0.63
8: Circumstances
S8.2	Negative external influences	3.6 (3.4)	3.4 (3.0)	0.41	0.23–0.56	0.51/0.43

^*Since this module contains questions referring to the primary problem, patients with no agreement regarding primary problem on MATE-Q and MATE 2.1 were excluded (n = 5).

^$Cronbach's alpha is reported for MATE-Q/MATE 2.1.

¹Range 0–12 (92% of the patients reported 3 or less treatment episodes).

²MATE-Q before: ICC = 0.49 (95% BI 0.29–0.64); MATE-Q after: ICC = 0.97 (95% BI 0.92–0.98).

³MATE-Q before: ICC = 0.62 (95% BI 0.45–0.74); MATE-Q after: ICC = 0.88 (95% BI 0.75–0.95).

⁴t(92) = −3.20, p = 0.002 (0.0348);

⁵t(96) = −2.27, p = 0.026 (0.057);

⁶t(97) = 3.71, p = 0.000 (0.026);

⁷t(97) = 3.25,p = 0.002 (0.030);

⁸t(97) = 6.96,p = 0.000 (0.004);

⁹t(97) = 3.83,p = 0.000 (0.039);

¹⁰t(97) = 2.29,p = 0.024 (0.052);

¹¹t(97) = 3.18,p = 0.002 (0.039);

¹²t(97) = −2.16;p = 0.034 (0.065);

¹³t(97) = 2.22;p = 0.29 (0.061);

¹⁴t(95) = −2.88;p = 0.005 (0.043);

¹⁵t(95) = −2.56,p = 0.012 (0.048);

¹⁶t(97) = 5.46,p = 0.000 (0.009);

¹⁷t(97) = 4.16,p = 0.000 (0.002);

¹⁸t(97) = 3.82,p = 0.000 (0.005). Values in straight brackets are critical values according to Benjamini and Hochberg's method, taking a false discovery rate of 10% and 23 tests into account.

ICCs, Intraclass correlation coefficients; MATE, Measurements in the Addictions for Triage and Evaluation; DASS, Depression, Anxiety and Stress scales.

Most ICCs for scores are above 0.7, indicating acceptable to good agreement (printed in bold), of which the majority are pertaining to Substance use past 30 days (module 1b), Craving (module 2), Depression, anxiety and stress (module 4) and Motivation for treatment (module 6). For the single items Severity of dependence/abuse (S4.3), Undergoing psychiatric or psychological treatment (S2.2) and Physical complaints (S5.1), ICCs indicating good to excellent agreement were found.

Low agreement (ICCs below 0.7) was found for scores derived from Difficulties and problems (module 7), Circumstances (module 8) and part b. of module 3: Psychological and psychiatric complaints, as well as for the single scores Number of previous treatment episodes and Gambling days.

In addition, 15 MATE-Q mean scores differ significantly from their MATE 2.1 counterparts, with 10 of them being higher (t test statistics are shown at the bottom of Table 2). For Number of treatment episodes and Problem recognition − specific the ICCs of the group of patients that completed the MATE-Q prior to the MATE 2.1 were significantly lower than the ICCs of the group of patients that completed the MATE-Q afterwards (CIs shown at the bottom of Table 2).

Conclusion and Discussion

Main findings of the present study are that the majority (15 out of 24) of the scores of the MATE-Q showed acceptable to good agreement with their corresponding MATE 2.1 scores. The remaining 9 pairs of scores showed low agreement. For almost two-third of the scores, a higher mean score for the MATE-Q was found, indicating patients reporting more severe problems in self-report than in the interview setting. Modules containing these scores should be subject for revision. Almost all scores belonging to the three MATE 2.1 modules that are self-reported in origin (Craving, Depression, Anxiety and Stress Scales and Motivation for treatment), show high agreement with their MATE-Q counterparts. Scores with lower agreement concern mainly modules that are interview or observer scales in origin (Difficulties and problems, Circumstances, and part of the module Physical and psychiatric complaints). Origin might lie in the low reliability of the scores of these original MATE 2.1 modules: former research showed low (test-retest) reliability of the MATE 2.1 version of the Difficulties and problems (between 0.51 and 0.73) and Circumstances(0.52 for score S8.2) modules [18]. For these modules it is urgent to investigate both scale construction and (in the case of the MATE 2.1) need for and quality of interviewer training in order to enhance reliability − a prerequisite for validity.

The assessment of the quantities of cannabis and nicotine used over the past 30 days, show high agreement between self-report and interview. The agreement for units of alcohol used in the past 30 days is acceptable, but the mean for self-report turns out to be lower than the mean from the interview. An explanation for the lower mean of reported alcohol use in the last 30 days might lie in the fact that the question on frequency and quantity of substances is reported categorically in the MATE-Q, whereas this is reported in the number of days in the MATE2.1 [23]. In the case of alcohol, this conversion algorithm is relatively complex, and might account for deviations. Therefore, the categories and algorithm should be reviewed.

The acceptable and high agreement for substances used also shows that not all questions that are interview or observer in origin are prone to low validity when used in a self-reported nature. Agreement for days of gambling was very low. Explanation for this is a skewed distribution of scores, since relatively few people gamble and the number of gambling days reported is very low.

The number of previous treatment episodes is a topic that might require more elaboration for patients before administration, since agreement between the MATE-Q and the MATE 2.1 score appeared to be higher for the group of patients that was interviewed before the self-reported version was completed. Many patients in substance abuse treatment experience a multitude of problems [28, 29, 30] and people might have had counseling or therapy for this as well. It might be difficult for them to disentangle which help was applicable for what problem, and therefore be a difficult topic for self-report. This should be the subject of further study, just as for Problem recognition − specific.

What do the results tell about the clinical and research value of the MATE-Q? With the majority of the scores showing acceptable to good agreement, we can conclude that the modules containing these scores, that is, Primary problem substance (module 0), Substance use past 30 days (module 1b), Craving (module 2), Depression, anxiety and stress (module 4), and Motivation for treatment (module 6) do possess sufficient concurrent validity and can be used for clinical and research purposes. Other modules should be the subject of revision and research, since scores of self-report and interview either deviate in severity, agreement, or both.

Additional attention is important for the clinical use of the MATE-Q. The reported deviations can have consequences for results of allocation procedures, using present scores. The current, regional, allocation procedure [9, 10, 11, 13, 14, 15] builds on four indicators, and all are “affected” by the low concurrent validity of one or more MATE-Q scores. Although review by an assessor is required before allocation to any treatment level, it is questionable if and how assessors can correct deviations that are scattered over a multitude of indicators. Therefore, the next step should be a more detailed review of the magnitude of deviations from the allocation procedure resulting from the current state of (lack of) agreement of scores. Of course, parallel development should be review and further development of items (i.e., wording, sentence length, presentation).

In general, this study shows the difficulty of translating interviewer or observation instruments to self-report versions. In order for the MATE-Q to act as an instrument for remote allocation, all scores that are involved with the current allocation procedure should be reviewed. For the MATE-Q to provide information for face-to-face review, one can consider to limit the instrument to all scores with sufficient agreement, that is, Primary Problem, Substance Use, Physical complaints and Undergoing psychiatric or psychological treatment,and − with prudence − Severity of dependence/use.

The strength of this study is the day-to-day setting, which secured generalizable results. This also comes with some downsides: although the number of patients we included was sufficient in terms of power, the inclusion of only 98 out of over 800¹ potentially eligible patients raises questions on representativeness. Regarding gender, age, cultural background, and treatment history, the sample appears to represent an outpatient population, the target population for this study. Although the MATE-Q should be valid for help-seeking individuals with mild to moderate problems, it should also be valid for persons outside specialized care, who just consider or explore the possibilities of treatment. In addition, validity for specific patient sub-groups could be studied, that is, for patients who differ in primary problem substance, or who vary in severity of dependence or abuse. This would contribute to more insight into the strengths and difficulties of self-report on addiction problems and should be the focus of the next step.

Some scores showed deviations from normality (about 60%), most in skewness (to the right) and in a few cases in kurtosis (mostly peaked). This might influence the precision of the ICCs, since normality is assumed for this statistic. Transformation of scores did not result in satisfactory normality of distributions. Results of this study should be evaluated in the light of this limit, although the ICC is said to be robust for violations of normality [31]. In addition, for scores with relatively few categories, we calculated Weighted Kappa, which returned in similar results: Weighted Kappa for Characteristics of physical comorbidity and Characteristics of psychiatric comorbidity were 0.49 and 0.50 respectively. Another limitation concerns the standard in this study, the MATE 2.1. Although some modules are validated, other parts are in development. This is reflected in the lower test-retest and inter-rater reliabilities for the modules Limitations and Circumstances found in a previous study [18]. The higher internal consistencies found in this study point out that improvements since then have resulted in better psychometric quality, but the standard in this study needs further development and training as well.

Using quantitative and structured instruments in substance abuse treatment improves transparency and treatment allocation and evaluation during treatment [14, 15, 32, 33, 34, 35, 36, 37]. Instruments like the MATE 2.1 and the MATE-Q contribute to this in a multidimensional fashion, and should be further developed in order to provide reliable, valid, and comprehensible information both for informing and supporting patients in gaining control of their substance use as well as patients, professionals and organizations for the purpose of clinical decision making.

Supplementary Material

Supplementary data