Multicentre study for validation of the French addictovigilance network reports assessment tool

Abstract

Aims

The French health authority (ANSM) is responsible for monitoring medicinal and other drug dependencies. To support these activities, the ANSM manages a network of 13 drug dependence evaluation and information centres (Centres d'Evaluation et d'Information sur la Pharmacodépendance ‐ Addictovigilance ‐ CEIP‐A) throughout France. In 2006, the Nantes CEIP‐A created a new tool called the EGAP (Echelle de GrAvité de la Pharmacodépendance‐ drug dependence severity scale) based on DSM IV criteria. This tool allows the creation of a substance use profile that enables the drug dependence severity to be homogeneously quantified by assigning a score to each substance indicated in the reports from health professionals. This article describes the validation and psychometric properties of the drug dependence severity score obtained from the scale ( Clinicaltrials.gov NCT01052675).

Method

The validity of the EGAP construct, the concurrent validity and the discriminative ability of the EGAP score, the consistency of answers to EGAP items, the internal consistency and inter rater reliability of the EGAP score were assessed using statistical methods that are generally used for psychometric tests.

Results

The total EGAP score was a reliable and precise measure for evaluating drug dependence (Cronbach alpha = 0.84; ASI correlation = 0.70; global ICC = 0.92). In addition to its good psychometric properties, the EGAP is a simple and efficient tool that can be easily specified on the official ANSM notification form.

Conclusion

The good psychometric properties of the total EGAP score justify its use for evaluating the severity of drug dependence.

What is Already Known About this Subject

• In France, there is a dedicated system for the evaluation of drug abuse and dependence that detects and identifies problematic consumption cases.

• The assessment was performed using a recognized tool, the EGAP, which is used in routine practice to characterize drug dependence profiles.

What this Study Adds

• The good psychometric properties of the total EGAP score justify its use for evaluation of drug dependence severity. The EGAP is a simple and efficient tool consisting of merely eight items that can be easily specified on the official ANSM notification form.

• This tool provides an essential clinical approach.

Introduction

In France, the current consumption of psychoactive substances represents a serious public health problem. For the past several years, surveys have placed France amongst the leading countries in terms of the consumption of psychotropic drugs (anxiolytics, hypnotics, antidepressants, antipsychotics, among others) 1. In some patients, this consumption can lead to misuse, abuse and dependence 2. These drug dependencies can partly explain the chronic drug use. The identification of wayward drug use is very difficult because the boundary between therapeutic use and misuse/dependence is difficult to define. Drug dependence is rarely detected during clinical trials because such trials are short lived and include selected patients and formalized drug administration, preventing potential dosing increases in patients, which is one of the most obvious signs of drug dependence 3. The potential for abuse of and pharmacodependence on experimental drugs have not been extensively documented at the end of phase III clinical trials. The occurrence of these phenomena can be assumed by comparison with other drugs in the same pharmacological class but only ascertained during large scale assessments of ‘real life’ use after the drug has been introduced to the market.

Drug addictions are the subject of a surprising paradox. They are widely criticized yet seldom studied. However, the rapid detection and identification of problem consumption is necessary. Early detection and evaluation allow, on the one hand, patients to be guided towards optimized management and, on the other hand, the quantification of drug abuse and dependence potential to adopt the most secure prescription and dispensing rules 4.

The data available for illicit substances are extremely troubling. Cannabis is currently the most widespread illicit psychoactive substance used in France 5, 6. The figures for other substances are also worrisome 7. The regular appearance of new designer drugs, whose harmfulness and dependence‐inducing potential are completely unknown, is an extremely alarming phenomenon 8. In this context, the identification of problem consumptions would allow for better patient management, thus limiting the adverse consequences and co‐morbidities associated with consumption; it would also enable implementation of regulatory measures that limit the circulation of these substances.

In France, the French Health Products Safety Agency (Agence National de Sécurité du Médicament et des produits de santé ‐ ANSM) is responsible for monitoring medicinal and other drug dependencies. To support these activities, the ANSM manages a network of 13 drug dependence evaluation and information centres (Centres d'Evaluation et d'Information sur la Pharmacodépendance ‐ Addictovigilance ‐ CEIP‐A) throughout France 9.

These centres have three main tasks: to gather and evaluate cases of drug dependence, to inform healthcare professionals and to conduct research activities 9, 10. Their collection and evaluation activities are essential because they allow the early detection and identification of problem cases of consumption of medicinal or other substances. To undertake this drug dependence evaluation activity at the national level, the CEIP‐A have developed several original data collection protocols, together with epidemiological tools and methods for the analysis of various databases 11, 12, 13, 14, 15, 16, 17.

Independently of these reporting tools, all healthcare professionals (regardless of their field of expertise) are required to report anonymously cases of serious drug abuse and dependence associated with the use of substances or plants with psychoactive effects (articles R5132‐97 to R5132‐116 of the French Public Health Code). These spontaneous notification reports (NotS) by healthcare professionals are key for determining ‘real life’ drug misuse and abuse and for identifying new non‐medicinal drugs that present a risk to public health.

Following the analysis of these tools and notification methods, hypotheses have been proposed to evaluate the potential for abuse and pharmacodependence. The data produced by all CEIP‐A are aggregated and summarized. This networking requires that the data that are collected and evaluated by the CEIP‐A are homogeneous over time. The collection and analysis homogeneity for epidemiological tools are achieved by applying national procedural standards that are defined to ensure that all CEIP‐A collect data in the same manner. There was no tool available that permitted the common evaluation of NotS by the different centres. Each CEIP‐A collected and evaluated notifications from the healthcare professionals located in its assigned territory. However, it was essential to implement a common procedure for evaluating the severity of drug dependencies in NotS and to standardize the case reading. Thus, in 2006, the Nantes CEIP‐A created a new tool called the EGAP (Echelle de GrAvité de la Pharmacodépendance ‐ drug dependence severity scale), which was based on DSM IV criteria 10, 18. This tool is adapted to the CEIP‐A because it allows healthcare professionals to provide a substance use profile based on the NotS. It enables the homogeneous quantification of drug dependence severity by assigning a score to each substance indicated in the NotS and the CEIP‐A network has published work detailing its use for evaluating the substance dependence severity 19, 20, 21, 22, 23, 24.

In 2007, the Nantes CEIP‐A received funding for a national research programme (PHRC) to validate this tool. All CEIP‐A participated in the validation of this tool (clinicaltrials.gov Identifier NCT01052675).

This article describes the validation steps and psychometric properties of the drug dependence severity score obtained using the scale. The aims were to determine whether the EGAP is suitable for evaluating drug dependence based on NotS, is a reliable scale for the evaluation of drug dependence and the optimum conditions for its use.

Methods

Population

In this article, we shall not use the customary terms ‘population’ or ‘participants’. Instead, we shall use the term ‘spontaneous notifications’ (NotS), which is the term used by the ANSM to refer to reports of drug abuse or dependence spontaneously declared by healthcare professionals to their CEIP‐A. A notification is composed of three elements: a patient (age, gender, professional and family situation, medical history, history of abuse or dependence), at least one substance (name of the substance or substances, dosage, duration of consumption) and a problem situation (description of the patient's clinical situation).

The CEIP‐A evaluate the NotS that they receive, fill in the EGAP and assign a score for the substances that are considered problematic by the declaring healthcare professional. Each notification pertains to a single patient, but it may related to several substances (a subject may consume multiple substances). A notification may thus give rise to several EGAP scores. Between January 2008 and May 2011, all NotS (and the corresponding EGAP scores) from the CEIP‐A were entered into a computer database. Each substance indicated in the NotS was evaluated via the severity score. This was not a simulated collection but rather the usual collection of notifications from the CEIP‐A. The validation was performed using this collection of NotS (and the corresponding EGAP scores). EGAP scores without incomplete data were used for the validation steps.

Tools

Drug dependence severity scale (EGAP)

The EGAP scale is completed by the CEIP‐A for each NotS they receive. All health professionals working in the CEIP‐A network undergo training to complete the EGAP during a 2 day annual seminar. Formation to this method is mandatory for health professionals working in a CEIP‐A, although internal procedures also exist. Formation is overseen by the EGAP conceptualizers.

The received NotS are filled in by a healthcare professional in front of the patient, after the clinical interview and/or based on the patient's clinical file, in accordance with the applicable legislation. The CEIP‐A may need to contact the reporting healthcare professional to complete the NotS. The EGAP (Figure 1) was developed from the DSM IV diagnostic items. It is neither a diagnostic nor a screening tool, but its purpose is to provide a qualitative and quantitative evaluation of the severity of substance use based on items that are considered relevant by addictologists and pharmacologists.

Figure 1.

EGAP (Echelle de GrAvité de la Pharmacodépendance‐ drug dependence severity scale)

The development of such a tool was necessary for the CEIP‐A because the available tools were not designed for notification‐based evaluation and thus were ill‐suited to CEIP‐A practices (i.e. self‐questionnaires, screening/diagnostic tools, single‐substance evaluation tools, lengthy transmissions or those interpretation difficulties were invalid or not drafted in French). The CEIP‐A needed a tool that could quantify and qualify consumption behaviour severity and homogenize evaluation modes based on the NotS. It had to be possible to complete this tool based on the received notifications (thus excluding self‐questionnaires), and it needed to be applicable to all substances (lawful and illicit) and focus on the evaluation of drug dependence and not diagnosis, which is the role of the CEIP‐A.

To our knowledge, the EGAP is the only tool that meets all of these criteria. The items contained in the EGAP were derived from the DSM IV. A panel of experts designed the EGAP. This panel included pharmacologists, addictologists, psychiatrists and biostatisticians. The seven first items were derived from the official definition of pharmacodependence, whereas the last one (item 8) was added by the panel of experts.

The scale evaluates the physical and compulsive signs of dependence (items 1 to 4):

• Tolerance (item 1)

• Withdrawal symptoms when consumption is stopped, or consumption of another product to avoid these symptoms (item 2)

• Higher dose or duration than initially planned (item 3)

• Desire or failed attempts to stop (item 4)

It also evaluates the adverse consequences identified in the NotS by the healthcare professional (items 5 to 8):

• Substantial amount of time devoted to consumption (item 5)

• Family, professional, social, legal or financial difficulties associated with consumption (item 6)

• Persistence of consumption, even though the patient is aware of the consequences of consumption on his/her health (item 7)

• Transgressive behaviour with respect to the manner in which the drug is obtained or used (item 8)

This last item was not part of the DSM IV criteria but it needed to be included in the CEIP‐A evaluation because it is frequently a cause for notification by healthcare professionals. It corresponds to fraudulent manners of obtaining the substance (illicit substance purchases, doctor shopping, pharmacy shopping, prescription forgery, exaggeration of symptoms) or of substance misuse (unapproved use regarding the indication or route of administration, combination with another substance to enhance effects).

Calculation of the EGAP score is simple. The numerator corresponds to the number of positive items and the denominator is the number of items specified. There are three possible answers for each item, ‘yes’, ‘no’ and ‘not specified’ (NS). A ‘yes’ answer assigns 1 point to the numerator and 1 to the denominator, a ‘no’ answer does not assign a score to the numerator and assigns 1 point to the denominator and a ‘NS’ answer assigns no points to the numerator or the denominator. A complete EGAP score is a score with an eight point denominator.

Addiction severity index (ASI)

To test the concurrent validity, the EGAP scores were compared with those obtained using the addiction severity index (ASI). Because the ASI is the tool of reference for evaluating concurrent validity, only those NotS corresponding to patients who were also evaluated by the ASI were selected. These NotS arose from patients from the Nantes University Hospital Addictology Department (seeking outpatient or residential management of their addiction). This department uses the ASI to perform a clinically standardized evaluation of patients upon their arrival at the department and to monitor their progress during management.

This assessment is performed by health professionals who are specifically trained to evaluate the ASI. Training includes theoretical competence and practical exercises. For this study, trained professionals from the Nantes Addictology service performed the ASI assessment, which lasted approximately 1 h.

It is a semi‐structured interview that is routinely used in addictology to evaluate the severity of problem substance use by patients. It is used to gather information from seven areas that are likely to be affected by substance use or by addictive behaviour (medical condition, employment/financial resources, alcohol use, substance use (including smoking), family and social relations, legal situation, and psychological state). One special feature of the ASI is that it takes into consideration the patient's opinion concerning his/her difficulties and need for help in the seven aforementioned areas (self‐evaluation from 0 to 4). After the interview, the evaluator assigns a severity score of between 0 and 9 to each of the seven areas based on clearly defined answers to ASI items (items indicated in the ASI user manual), using the patient's self‐evaluation and the evaluator's judgement. An ASI severity score greater than or equal to 4 reflects a need for management in the concerned area. This tool is available in French 25, 26, 27.

Statistical analysis

The statistical methods that are generally used for psychometric tests were used to validate the EGAP.

The validity of the EGAP content had been taken into consideration during the design. It was appraised by a panel of experts, who determined the extent to which the EGAP items appeared to measure correctly the features they were intended to measure, based on a selection of items capable of measuring the fundamental features of drug dependence that were deemed consistent with theoretical knowledge of the studied phenomenon.

The construct validity of EGAP was determined by evaluating the dimensionality of the eight items using an exploratory factor analysis (EFA) and by validating the obtained structure using a confirmatory factor analysis (CFA). Dimensionality was determined by the number of eigenvalues greater than 1 in the EFA. Structural validation was defined by factor loadings greater than 0.4 for each item in its respective dimension and by using the general suitability indices of RMSEA and SRMR less than 0.05 with CFI and TLI greater than 0.95. A graphical representation of the correlations between items was created using a biplot.

The concurrent validity of the EGAP score for a substance (number of positive items / number of specified items) is intended to evaluate the compliance of the obtained result using the new scale with that obtained from more conventional measurements used to ascertain the same concepts. The ASI scale was used as the reference instrument. The correlations between the scores obtained using the two scales were measured by calculating the Pearson correlation coefficient. A correlation coefficient comparing the ASI severity score for the "substance use" area and the number of positive EGAP scale items greater than 0.4 was deemed significant. The two scores were assigned by two different evaluators, neither of whom had access to the other's score.

The discriminant validity of the EGAP score was tested by examining the correlation between the mean number of positive EGAP score items per substance and the number of notifications received for this substance by hypothesizing that the most numerous substance notifications reflected the most serious dependencies. A correlation greater than 0.4 was expected.

The coherence of the answers to items forming a dimension was analyzed using a Mokken scale. This scale, based on Loevinger's H coefficient, is validated when the H coefficient of a dimension and the H coefficients of each item are greater than 0.3.

The internal EGAP score consistency was measured using the Cronbach's alpha coefficient. A Cronbach alpha coefficient greater than 0.70 is generally deemed satisfactory.

Inter‐evaluator reliability was determined by calculating the intra‐class correlation coefficient (ICC). The ICC was calculated from evaluations that were determined for selected NotS. ICC values greater than 0.6 represent good score inter rater reliability and those greater than 0.8 represent very good inter rater reliability.

Fifteen NotS arising from the CEIP‐A network were selected in a manner that diversified the indicated substances (opiate maintenance treatments, benzodiazepines, hypnotics, analgesics, new designer drugs, cannabis, cocaine, heroin), the declaring healthcare professionals (community pharmacists, physicians, addictology specialist and non‐specialist healthcare professionals, medical advisors and poison control centres), the substance procurement methods (prescription, over‐the‐counter sale, drug deal) and the EGAP scores (complete and incomplete scores). This selection was used to mimic closely the conditions of use of the scale in the field in routine CEIP‐A practice. The 15 NotS were then combined into a single document. For each NotS, the CEIP‐A were required to evaluate one of the incriminated substances using the EGAP and the name of the substance they were to evaluate was specified in the document. Thus, for each CEIP‐A, one NotS corresponded to one EGAP score. All CEIP‐A received the same document listing the 15 NotS, along with the EGAP instructions for completion. A member of each CEIP‐A assigned the 15 EGAP scores without being aware of the scores assigned by the other CEIP‐A. For Nantes, the score designer filled in the EGAP scores (judge 10). Because this person possessed extensive scoring training, the influences of their answers on the ICCs was evaluated.

Results

General description

During the study period, 2669 scores from NotS were transmitted to and recorded by the Nantes CEIP‐A, including 324 (12%) for which all eight questionnaire items were completed, for scoring. One NotS usually corresponds to one patient, although one patient may use several drugs, as indicated in the description of the tool. The health professional who reports the NotS often asks the patient exhaustively about the drug that justified the report, i.e. the drug for which we generally have a complete EGAP. If the information collected on the related drugs is not exhaustive, the corresponding EGAP can be incomplete. The addictovigilance network is accustomed to proceed in this manner. However, the EGAP is generally very informative even if they are incomplete. In our study, 78% of the EGAP had at least four items completed. The most problematic drug was the most documented one.

In the context of the tool validation, only completed EGAPs were used. These were notifications involving the problematic use of opiates (heroin (13.8%), opiate maintenance treatments (OMTs) (21.9%), opiates excluding OMTs (10.8%), cannabis (18.5%), benzodiazepines and related drugs (18.2%), psychostimulants (7.7%), antidepressants (1.8%) and other substances (7.1%)). Depending on the drug and the positivity of the items, the average EGAP scores are presented in Table 1. From a quantitative perspective, the average number of positive items varied according to the drug. From a qualitative perspective, the most often positive items were not the same depending on the drug, allowing for the characterization of different drug profiles. The 324 complete EGAP scores, corresponding to 229 patients, were used to evaluate all parameters, with the exception of the concurrent validity and the inter‐evaluator reliability.

Table 1.

Average EGAP scores by drug and positivity of each item

	OMT	Heroin	Opiates excluding OMT and heroin	Cannabis	Benzodiazepines and related drugs	Psychostimulants	Antidepressants and other substances	All psychoactive substances
Number of completed EGAP	71	45	35	60	59	25	29	324
Item 1 n (%)	23 (32%)	42 (93%)	33 (94%)	48 (80%)	30 (51%)	13 (52%)	6 (21%)	195 (60%)
Item 2 n (%)	65 (92%)	44 (98%)	31 (89%)	40 (67%)	41 (70%)	8 (32%)	5 (17%)	234 (72%)
Item 3 n (%)	42 (59%)	45 (100%)	32 (91%)	50 (83%)	47 (80%)	16 (64%)	13 (45%)	245 (76%)
Item 4 n (%)	48 (68%)	44 (98%)	31 (89%)	45 (75%)	35 (51%)	16 (64%)	7 (24%)	226 (70%)
Item 5 n (%)	41 (58%)	42 (93%)	26 (84%)	54 (90%)	29 (49%)	17 (68%)	12 (41%)	221 (68%)
Item 6 n (%)	33 (47%)	44 (98%)	24 (69%)	44 (73%)	28 (48%)	14 (56%)	7 (24%)	194 (60%)
Item 7 n (%)	29 (41%)	17 (38%)	20 (57%)	38 (63%)	31 (53%)	11 (44%)	10 (35%)	156 (48%)
Item 8 n (%)	49 (69%)	45 (100%)	28 (80%)	60 (100%)	31 (53%)	25 (100%)	12 (41%)	250 (77%)
Number of EGAP‐positive items (mean, SD)	4.6 (2.1)	7.2 (1.0)	6.4 (2.0)	6.3 (1.8)	4.6 (2.9)	4.8 (2.6)	2.5 (2.5)	5.3 (2.5)

SD standard deviation; OMT opiate maintenance treatment

Over the study period, 101 patients who were evaluated by the ASI were the subject of NotS with complete EGAP scores. These 101 NotS were used to perform the concurrent EGAP score validation. These patients were from the Nantes addictology department who were requesting hospital or outpatient management for problems associated with the consumption of cannabis (52.5%), opiate maintenance treatment (OMT) (45.5%), heroin (29.7%), benzodiazepines (21.0%), cocaine (17.8%), codeine/morphine (5%) or hallucinogenic substances (1%). Approximately one half of the subjects were managed for a single substance (52.5%). The three EGAP score items that were most frequently positive were transgressive behaviour with respect to the manner in which the substance was obtained or used (item 8), higher consumption (dose or duration) than that defined for the patient (item 3) and a desire or unsuccessful attempts to stop (item 4).

EGAP construct validity

The EFA performed using all eight items revealed a strong main dimension. Indeed, the first eigenvalue was estimated at 3.79 vs. 0.98 for the second. The assumed questionnaire structure was thus unidimensional and corresponded to the unidimensionality hypothesis of the concept of drug dependence severity that was proposed when the questionnaire was designed. All CFA factor loadings were greater than 0.4 (minimum for item 2: 0.67), indicating good relationships between each item and the considered concept of drug dependence. The CFA suitability indices were satisfactory: RSMEA=0.043 CI90% [0.000; 0.074]; SRMS=0.025; CFI=0.989; TLI=0.98.

The biplot (Figure 2) shows that all EGAP items measure a common direction, confirming that the EGAP is unidimensional. The analysis was performed using Stata 13 (StataCorp, College Station, Texas, USA).

Figure 2.

Biplot representing the correlations between EGAP items. Item 1 = tolerance, item 2 = withdrawal symptoms when consumption is stopped, or consumption of another product to avoid these symptoms, item 3 = higher dose or duration than initially planned, item 4 = desire or failed attempts to stop, item 5 = substantial amount of time devoted to consumption, item 6 = family, professional, social, legal or financial difficulties associated with consumption, item 7 = persistence of consumption, even though the patient is aware of the consequences of consumption on his/her health, and item 8 = transgressive behaviour with respect to the manner in which the drug is obtained or used

EGAP score concurrent validity

In the population of 101 patients, the Pearson correlation coefficient between the EGAP score and the ASI ‘substance use’ severity score was estimated to be 0.70 (P<0.001), thus defining a satisfactory concurrent validity with the ASI score.

EGAP score discrimination validity

The correlation coefficient between the mean drug dependence severity score for each substance and the number of notifications concerning this substance was estimated to be 0.26 (P<0.0001). This value does not allow for validation of the hypothesis of a relationship between the number of notifications concerning a substance and the EGAP drug dependence severity score for that substance.

Coherence of answers to EGAP items

The Mokken scale matched well with the data, with a Loevinger H coefficient of 0.50 for the entire scale and Loevinger H coefficients for all individual items greater than 0.40. The answers thus showed good coherence between the EGAP items.

EGAP score internal consistency

The Cronbach alpha coefficient was estimated to be 0.84 for all complete EGAP scores, thus defining good internal consistency and hence good score reliability.

EGAP score inter rater reliability

The inter rater reliability of the EGAP score was very good for all NotS studied (ICC=0.92), for prescription drugs (ICC = 0.96), legal substances (ICC = 0.83), illicit substances (ICC = 0.98), regular uses (ICC = 0.87), complete EGAP (ICC = 0.97) and incomplete EGAP (ICC = 0.83). Inter‐rater reliability was good for occasional uses (ICC = 0.65). Inter rater reliability was poor, however, for over‐the‐counter drugs (ICC = 0.07). ICC values were only slightly affected by the presence or absence of a judge who was more highly trained in EGAP scoring (judge 10), thus demonstrating the ease of appropriation of this score by specialists. ICC values are presented in Table 2.

Table 2.

Inter‐rater reliability of the EGAP

	With judge 10		Without judge 10
	ICC	95% CI	ICC	95% CI
All NotS	0.92	[0.86, 0.98]	0.91	[0.85, 0.98]
Over‐the‐counter drugs	0.07	[0, 0.50]	0.08	[0, 0.56]
Prescription drugs	0.96	[0.93, 0.99]	0.96	[0.93, 0.99]
Illicit substances	0.98	[0.9519, 1]	0.98	[0.9473, 1]
Legal substances	0.83	[0.69, 0.97]	0.82	[0.67, 0.97]
Occasional uses	0.65	[0, 1]	0.63	[0, 1]
Regular uses	0.87	[0.77, 0.96]	0.85	[0.75, 0.96]
Complete EGAP	0.97	[0.94, 1]	0.97	[0.93, 1]
Incomplete EGAP	0.83	[0.65, 1]	0.81	[0.62, 1]

NotS spontaneous notifications; ICC intraclass correlation coefficient; CI confidence interval.

Discussion

Psychometric qualities of the EGAP

The results of the factorial analyses served to underscore the relevance of the tool for performing unidimensional evaluations of drug dependence severity for a given substance. The statistical analyses showed that our scale, consisting of eight items that were defined and clinically validated by an expert committee, provided a good evaluation of a single dimension concept and that the eight items were homogeneous for evaluating this concept. Thus, we have created a strong list of items for evaluating the concept of drug dependence; the good psychometric properties of the total EGAP score justify its use for evaluating drug dependence severity. The total EGAP score (number of positive items / number of specified items) proved to be a reliable and precise score for evaluating drug dependence (Cronbach alpha = 0.84; ASI correlation = 0.70; global ICC = 0.92). In addition to having beneficial psychometric properties, the EGAP is a simple and efficient tool consisting of only eight items that can be easily specified on the official ANSM notification form, that has been modified to include all necessary data 28 either from patient clinical files or directly via phone discourse with the healthcare professional. Instructions for using the EGAP are already available, and familiarization with the scale is a rapid process. When the most highly trained judge was removed from the inter rater reliability analysis, the mean ICC value dropped slightly but remained greater than 0.8.

The discrimination validity of the score was insufficient to validate the hypothesis that the higher the EGAP score for a substance, the greater was the number of NotS for the substance. Validation of this characteristic is not desired by CEIP‐A because their tool must be able to rapidly identify alert signals even with only been a few notifications for the substance. For example, the arrival of new and highly addictive substances (e.g. designer drugs) or of a new consumption practice for a known substance that enhances the risks of addiction or of the adverse consequences of its use (desocialization, deschooling, physical health risks), must be identified as soon as possible to implement an alert and specific prevention aimed at users and to reduce the public health impact. These frequently start as marginal behaviours that must be detected rapidly before they spread to other consumer spheres. The CEIP‐A network needed a tool that enabled it to homogeneously pool NotS evaluation results to be reactive as soon as a new substance or new practice emerges. The network must also be able to evaluate this substance in terms of public health risks.

The analysis of tool inter‐rater reliability demonstrated that the EGAP was well‐suited for the evaluation of substances (legal or illicit) in regularly consuming subjects, but the ICC was adequate for occasional use and less effective for over‐the‐counter medicines. While the evaluator inter rater reliability for these two parameters was unsatisfactory, it is possible that the tool is not appropriate for evaluating these particular types of consumption. Moreover, these types of consumption are less frequently reported by healthcare professionals. Inter‐rater reliability was similar for prescription drugs and illicit substances, but was lower for over the counter drugs. This is probably linked to the fact that the regulatory framework and the context of utilization (dose, duration, administration modalities) are less known for over the counter drugs than for prescribed drugs from a physician point of view. Notifications can therefore be misinterpreted in the absence of a prescription framework. Nevertheless, these difficulties could be easily corrected by perfecting the procedure provided with the EGAP so that it is better suited to occasional consumers and over‐the‐counter medicines.

EGAP and changes to the definition of substance use disorders (DSM 5)

Since its creation, the EGAP score has been used for quantitative purposes, in addition to its use for the qualitative characterization of consumption profiles. We never adopted a diagnostic approach, as used by the DSM IV. The previous DSM version proposed a diagnostic classification: the term dependence was used if at least three criteria were positive. The duty of the CEIP‐A is not to diagnose patients but rather to evaluate the problem associated with substance use by a patient. As such because the outset, we have not established any threshold for the number of positive criteria, but we have always considered the presence of numerous positive items to correspond to a more problematic situation. This continuous non‐category‐based (without a threshold value) measurement use of the EGAP is consistent with the new DSM 5 classification 29 for problem consumptions of psychoactive substances. The DSM 5 combines abuse and dependence criteria, and has added the notion of craving, recommending a quantitative evaluation. With two or more positive criteria, substance use is considered to be problematic. Two and three positive criteria, consumption is considered slightly problematic. Between four and five positive criteria, consumption is considered moderately problematic and at least six criteria positive criteria, consumption is considered to be very problematic.

We can legitimately raise the question of updating the EGAP since the publication of the DSM 5. The DSM 5 contains four additional items than the DSM IV: (i) craving, (ii) (iii) two items specifying social consequences. The first is recurrent use resulting in failure to fulfil major role obligations at work, school or home, where use is continued despite having persistent or recurrent social or interpersonal problems caused and exacerbated by consumption, the second is a social impairment item (i.e. important social, occupational, or recreational activities are given up or reduced because of use) that was already present in version IV thirdly recurrent use in situations where it is physically hazardous, an item taken from the definition of abuse. The CEIP‐A considered that it was not legitimate to change the EGAP because the items covering social impairment were already ranked in EGAP item 6 and the craving and hazardous use items cannot be evaluated from the notification files. Moreover, they involve the patient's own behaviour, not a direct effect of the substance.

EGAP and public health benefits

The EGAP score meets the needs of the CEIP‐A network. This tool, which is based on notifications received, is able to characterize problem substance use both quantitatively (total score between 1 and 8) and qualitatively (what are the positive items?). This evaluation provides an individual analysis of a patient's situation relative to a given substance, thus allowing his/her management to be optimized and providing an immediate benefit. Moreover, the language used for the evaluation is understood by the healthcare professionals involved in the management of addicts. Case reports analyzed using the EGAP have been published 21, 24, 30.

Beyond this individual benefit, the EGAP scores allow for the qualitative and quantitative analysis of the dependence profile for a given substance. During a survey commissioned by health authorities, it is possible to analyze all of the scores pertaining to a substance and to identify the qualitative and quantitative characteristics of dependence. It is also possible to compare different substances. This evaluation has been used for national surveys requested by health authorities 31, 32. For example, zolpidem and zopiclone are two hypnotic drugs related to benzodiazepines that are monitored by the French health agency. Their stories are similar. They were marketed as ‘ideal hypnotics’ and were thought to be safer than benzodiazepines because the results of clinical trials did not report abuse and dependence, but in 2004, after an assessment was launched by health authorities, the legal text of their summary product characteristics was changed. The results from the French addictovigilance network monitoring, data reported in the literature and epidemiological analyzes showed that ZZ drugs are in fact true false twins. The characteristics of the reported dependencies using EGAP are not the same for the two drugs, and the data acquired for zolpidem appear to be more concerning 33.

The advantage of analyzing this score was broadly recognized by the ANSM. In 2012, they funded project PROMESS to develop a computerized statistical tool to automate the EGAP score analysis based on the CEIP‐A database 34.

Beyond the individual benefits and the characterization of the dependency profile, the EGAP can be useful for generating alerts about problematic drugs. It has been demonstrated that only 12% of the EGAPs are complete, which is linked to the mode of NotS collection. The health professional who completes the NotS is aware of the drug justifying the report, i.e., the drug for which we generally have a complete EGAP. Logically, the more the patient uses drugs, the more incomplete are the associated EGAPs for the drugs. Complete and incomplete EGAPs are provided by the same health professionals. EGAPs corresponding to substances that give rise to specific consultations are usually complete. This is not related to the severity of the EGAP score, as shown by the lack of correlation between the number of notifications received for a drug and the severity of the response to the EGAP. Even incomplete, the EGAP is analyzed separately when the French health authorities (ANSM) require specific drug information. Indeed, an incomplete EGAP can highlight drug particularities or specific dangers associated with a drug or the method of utilization (for example, we may have incomplete NotS for an emerging drug because its consumers are desocialized or marginalized, but if the EGAP analysis for this drug reveals systematic serious health consequences, the EGAP scores could provide valuable data). In this case, the ANSM will be alerted and could implement preventive measures. Thus, incomplete EGAPs are as important as complete EGAPs.

In conclusion, the CEIP‐A currently uses the EGAP, a recognized tool in their routine practice that permits characterization of the drug dependence profile during substance abuse analysis. This tool is complementary to other epidemiological tools used to evaluate the dependence potential of drugs and substances, and it offers an essential clinical approach.

Competing Interests

All authors have completed the Unified Competing Interest form at http://www.icmje.org/coi_disclosure.pdf (available on request from the corresponding author) and declare support from the Programme Hospitalier de la Recherche Clinique for the work, no financial relationships with any organizations that might have an interest in the work in the previous 3 years and no other relationships or activities that could appear to have influenced the work.

We wish to thank the Programme Hospitalier de la Recherche Clinique for providing financing.

Victorri‐Vigneau, C. , Hardouin, J. B. , Rousselet, M. , Gerardin, M. , Guerlais, M. , Guillou, M. , Bronnec, M. , Sébille, V. , Jolliet, P. , and the French Addictovigilance Network (2016) Multicentre study for validation of the French addictovigilance network reports assessment tool. Br J Clin Pharmacol, 82: 1030–1039. doi: 10.1111/bcp.13044.