Assessing the quality of drug information provided by hospital pharmacies using a fictitious enquiry and simulated real-life conditions

Abstract

Objectives

Guidelines for drug information (DI) provided by hospital pharmacists call for quality assurance procedures; however, no method of evaluation is internationally agreed on. The procedure should be feasible, reproducible and representative for real-life quality. We tested a new approach using a fictitious enquiry under simulated real-life conditions for quality assessment of DI by German hospital pharmacists.

Methods

A fictitious enquiry was submitted under simulated real-life conditions (study part I; test week announced, but not exact day; response time given). An expert panel determined content-related (three essential, and up to seven additional items of useful information) and structural requirements for answers and performed blinded evaluations. To compare quality of routine DI answers (study part II), five recently answered routine enquiries could retrospectively be evaluated for plausibility (binary scale 0/1) and structural requirements.

Results

Of 62 hospital pharmacies opting to participate, 45 (71%) entered study part I and 18 (40%) entered study part II. In study part I, 28 participants (62%) presented three essential contents, 11 (24%) two, five (11%) one, and one none. Additional useful information was given in 44–80%. Structural requirements achieved mixed results with low scores for logical conclusion deduction and reference presentation. In study part II, plausibility for the 90 recently answered routine enquiries was rated good (median 0.91, range 0.53–1). Concerning structural requirements, overall comparable results were achieved with minor variations compared with study part I. Thus, the quality of DI was judged to be comparable between study parts I and II.

Conclusions

An open quality assessment procedure with a fictitious enquiry under simulated real-life conditions can successfully be used for quality measurement of DI of hospital pharmacists and identifies areas for improvement.

Introduction

The aim of drug information (DI) is to provide unbiased, evidence-based, useable and timely information on the rational use of drugs to improve quality and medication safety. Beyond searching and abstracting the scientific literature, this includes the interpretation of data for a specific clinical situation and discussion of possible solutions with the enquirer.¹ In Germany, according to regulatory requirements, every hospital pharmacy has to provide DI for medical staff at their institution.² The organisation of the DI service differs substantially between German hospital pharmacies. While some have a specialised DI department, in other institutions pharmacists answer enquiries in addition to their daily tasks in compounding or dispensing.

International practice guidelines on provision of DI demand quality assurance procedures on a regular basis.^3–7 Several approaches have been published for this task, such as periodic review of responses by internal or external experts, use of fictitious enquiries, or assessment of user satisfaction.^8–14 However, currently there is no established, standardised method internationally agreed on and there are no commonly accepted quality criteria for formulating answers.^{4 15} Reasons for this may be the different models of DI organisation between countries and hospitals, the general problem in assessing appropriateness of an answer to an enquiry in a special clinical situation, and time-consuming methods.

The use of fictitious enquiries is a promising method to assess quality of DI responses and has been tested in various settings.^{8–10 16} Ideally, the pharmacist is not aware of the test character of the enquiry and thus handles it as usual. However, if this cannot be realised, using an open test method under simulated real-life conditions may be a promising approach. Therefore, we tested an open quality assessment procedure with a fictitious enquiry under simulated real-life conditions. To verify whether results represent the quality of DI in every day life, content-related and structural requirements of the answer to the fictitious enquiry were compared with recently answered routine DI enquiries at the same hospital pharmacy.

Methods

Study design

The study was initiated by the DI Working Committee of the German Association of Hospital Pharmacists (ADKA e.V.) and consisted of two parts. Part I contained a fictitious enquiry to be answered under simulated real-life conditions. Part II contained an additional voluntary option: five anonymised, patient-related routine enquiries answered from within the last 2 weeks could be sent for retrospective external evaluation. A call for participation was placed twice on the internal mailing list of ADKA, which is open to all members.

All communication and anonymisation were performed by a communicating pharmacist, who did not take part in the evaluation of the received answers. An expert panel of six hospital pharmacists with extensive experience in DI and direct patient care was named for evaluation of enquiry answers. Experts were blinded regarding participants. Participating hospital pharmacies were numbered consecutively. All conversations were undertaken by email. A questionnaire was sent to all participants asking for the following hospital pharmacy characteristics: number of pharmacists, details on DI organisation, DI documentation, DI quality measures, and clinical experience of the pharmacist answering the test enquiry.

Study part I

A fictitious enquiry (enquiry level 2¹⁷) and background information on the fictitious patient (table 1) were formulated by the communicating pharmacist and agreed on with the expert panel. In addition, content-related and structural requirements of answers were defined by the communicating pharmacist and the expert panel. Structural requirements were developed referring to the literature.^{10 12} Content-related requirements were divided into essential (crucial for answering the enquiry) and optional (additional benefit for enquirer).

Table 1.

Fictitous enquiry and background information on the patient (study part I)

Fictitious enquiry	A physician at your hospital is calling. He wants to prescribe dexketoprofen for trigeminal neuralgia. The patient has renal impairment. Which dosage should be used?
Background information received on request	Female patient, aged 77 years, admitted to surgical unit for ablation of benign intestinal polyps. Renal function (CKD-EPI): eGFR=35 mL/min/1.73 m². Episode of trigeminal neuralgia 25 years ago; patient does not know how it was treated then. Symptoms re-appeared lately after being absent for many years. She had tried ibuprofen which did not help. Additional medication: ramipril/hydrochlorothiazide 5 mg/25 mg, levothyroxine 75 µg, atorvastatin 20 mg
Enquiry level(17)	Level 2: complex enquiries—multiple sources. Enquiries that require the use of more specialist resources and/or the interrogation of multiple sources. Enquiries where application of medicines information skills and knowledge is needed, but sources provide a reasonably clear answer or course of action to offer the enquirer.

CKD-EPI, Chronic Kidney Disease Epidemiology Collaboration; eGFR, estimated glomerular filtration rate.

To simulate real-life conditions, pharmacies were given the week, but not the day, of the enquiry and written answers had to be returned within a 5 hour interval. On Wednesday of the test week, the fictitious enquiry and background information on the patient was sent at 9 am to all registered participants by email.

The communicating pharmacist collected and anonymised all responses to the fictitious enquiry received in time. The anonymised responses were forwarded to the blinded expert panel. The experts rated all predefined requirements as fulfilled (1) or not (0) and, in addition, ascribed a subjective mark based on the overall impression of a written response (from 1=excellent to 6=failed).

Study part II

Hospital pharmacies opting for part II of the study sent five anonymised recent routine enquiries in addition to their answer to part I. The communicating pharmacist checked for and, if necessary, ensured anonymisation and forwarded each enquiry to three experts out of the panel of six. A rotation system ensured that not the same three experts rated the same sample of enquiries. Rating was performed for structural requirements as in part I (10 requirements) plus the additional criteria, if a formal answer existed (11 requirements in sum). Content was rated as plausible or implausible based on the experts’ experience and knowledge, and the presented answer and references. A thorough content-related evaluation including researching the literature and definition of essential contents was not performed, due to the extensive time necessary for an evaluation and sometimes because of missing information regarding the patient and/or the hospital’s individual recommendations/guidelines.

Data analysis and statistics

The experts returned all ratings to the communicating pharmacist, who analysed the results. For every participating hospital pharmacy, the median score per criteria was calculated from the ratings of the six (part I) or three (part II) experts. This basic score per participant per criteria was used to calculate median and mean scores over all participants. For part II, the basic score per criteria was calculated over the five enquiries from one hospital pharmacy sent for retrospective evaluation.

The participating hospital pharmacies received their own results in comparison to mean scores for all rated criteria as a benchmark.

Data documentation, statistical analysis and figures were performed with Microsoft Excel 2016 (Seattle, WA, USA). Qualitative variables are presented with their frequency distribution. Quantitative variables are expressed as median and range, and in addition mean and standard deviation (SD) or percentage when appropriate. Statistical significance tests for formal requirements were done with χ² and two-sided t-test. Statistical significance was accepted as p<0.05. The interrater reliability for study part I was tested using the Fleiss κ test calculated with R-package irr.¹⁸ The level of agreement was rated as: >0.90, almost perfect; 0.80–0.9, strong; 0.6–0.79, moderate; 0.40–0.59, weak; 0.21–0.39, minimal; 0–0.2, none.¹⁹ For study part II interrater reliability was not tested because analysis was performed across the five enquiries per participant which were rated by alternating three experts.

Ethics approval

Ethics approval was not necessary, as the pharmacists answering the fictitious inquiry did so anonymously. Only the name of the institution was given. No real patient data were used for the fictitious enquiry. For evaluation of recently answered DI enquiries, only anonymised answers to patient-related requests were evaluated retrospectively, and no patient names or details were revealed. Participation was voluntary and assessment was performed anonymously.

RESULTS

Characterisation of participating hospital pharmacies

Overall, 62 hospital pharmacies agreed to participate and received the fictitious enquiry on the test day; of these, 45 (71%) responded in time (study part I). The 17 non-responders sent automatic out-of-office messages or mentioned internal organisational problems afterwards. In addition, 20 of the 45 participants (45%) chose to submit five completed, patient-related enquiries for retrospective evaluation (study part II). Two of these had to be excluded during analysis of the results (see below); thus, 18 hospital pharmacies (40%) took part in both parts of the study.

Detailed characteristics of the hospital pharmacies of both parts of the study are presented in table 2. Institutions of all sizes participated in study part I with 20 (45%) pharmacies employing <5 pharmacists, 15 (33%) employing 6–10, and 10 (22%) employing ≥11 pharmacists. Pharmacies employing ≥11 pharmacists more often operated a specialised DI department (eight (80%) of 10) than pharmacies with 5–10 pharmacists (three (20%) of 15). All pharmacies stated they followed at least one quality assurance measure (table 2).

Table 2.

Characteristics of participating hospital pharmacies study parts I and II.

	Part I n=45	Part II n=18
Pharmacy
Number of pharmacists
Median (range)	6 (1.6–26)	7.3 (1.6–26)
Mean	7.8	10
Specialised DI department (No. (%))	11 (24%)	5 (28%)
Pharmacist responsible for DI per day (No. (%))	5 (11%)	2 (11%)
DI in addition to another routine task (No. (%))	29 (65%)	11 (61%)
Quality measures
Use of a DI documentation system (No. (%))	27 (60%)	12 (67%)
Use of the ADKA-DI documentation database (No. (%))	24 (53%)	9 (50%)
Second look (No. (%))	20 (44%)	9 (50%)
Pharmacists experienced in direct patient care (No. (%))	37 (82%)	17 (94%)
Additional measurements for quality assurance (No. (%))	11 (24%)	4 (22%)
DI experience of answering pharmacist
<1 year (No. (%))	3 (7%)	n.a.
1–3 years (No. (%))	11 (24%)	n.a.
>3 to 5 years (No. (%))	6 (13%)	n.a.
>5 years (No. (%))	25 (55%)	n.a.

ADKA, German Association of Hospital Pharmacists; DI, drug information; n.a., not applicable, questions of part II answered by different pharmacists of the hospital pharmacy.

Hospital pharmacies also participating in study part II tended to employ a higher number of pharmacists, but the distribution concerning DI organisation was comparable to study part I. These participants more often used a DI documentation system, performed a second look, and had pharmacists with ward experience working in the DI department.

Results study part I

Answers to the fictitious enquiry were analysed for content-related and structural requirements. Results of the content-related requirements, essential and optional, are summarised in table 3. All three predefined essential contents were given by 28 participants (62%), while 11 (24%) named two, five (11%) named one, and one participant none. The mean (SD) number of essential contents was 2.47 (0.78). Figure 1 shows the number of essential contents in relation to DI organisation. Of the 11 institutions with a specialised DI department, nine (81%) presented all three essential contents, whereas only 16 of 29 (55%) did so when DI provision had to be performed in addition to daily routine tasks.

Table 3.

Study part I: results of the content-related rating of the fictitious enquiry (n=45; rating 0=no, 1=yes; six rater)

Content-related requirements	Median score	Mean score (SD)	Interrater reliability κ	No. of participants fulfilling the criterion (%)
Essential:
Dexketoprofen contraindicated in renal impairment	1	0.82 (0.35)	0.815	38 (84%)
Dexketoprofen is not a treatment of choice for trigeminal neuralgia	1	0.77 (0.36)	0.673	36 (80%)
Suggestion for proceeding further (eg, reference to guideline trigeminal neuralgia, carbamazepine as drug of first choice or another therapeutic option mentioned)	1	0.91 (0.27)	0.952	41 (91%)
Optional:
High potential for drug interactions of carbamazepine	1	0.78 (0.39)	0.892	36 (80%)
Need for slow titration when starting carbamazepine	1	0.67 (0.41)	0.695	30 (67%)
Carbamazepine as risky drug for older patients (central nervous system adverse effects)	0.5	0.48 (0.41)	0.622	24 (53%)
Oxcarbazepine mentioned as drug with less drug interaction potential	0.17	0.40 (0.45)	0.367	19 (42%)
Possible decline of renal function when combining ACE inhibitor/HCT/NSAID	0.17	0.39 (0.41)	0.66	20 (44%)
Possible adjustment for renal function for suggested drug considered (or if not necessary)	0.33	0.43 (0.40)	0.597	21 (47%)
Additional useful information	0.5	0.42 (0.28)	0.174	23 (51%)
Answer contains irrelevant information	0.17	0.26 (0.25)	0.182	11 (24%)

ACE, angiotensin-converting enzyme; HCT, hydrochlorothiazide; NSAID, non-steroidal anti-inflammatory drug.

Figure 1.

Essential contents in answers on fictitious enquiry in relation to DI organisation (study part I; department DI n=11; pharmacist responsible for DI per day n=5; DI in addition to another routine task n=29). Total number of essential content-related requirements is three. DI, drug information.

Optional content-related requirements were given with much less frequency than essential requirements; nevertheless, this additional useful information was presented in 44–80% of the answers (table 3). All but one participant presented at least one of the predefined optional content-related requirements. Calculated over all answers, a mean (SD) of 3.33 (1.48) additional items of useful information were given. When correlating DI organisation with the number of additional items of useful information given in the answer, no clear dependency was found. Hospital pharmacies with a DI department provided on average 4.7 additional items of useful information, those with a pharmacist responsible for DI per day provided 3, and where pharmacists had to answer DI in addition to routine tasks they provided 3.8.

Regarding structural requirements, mixed results were achieved (table 4) with a mean (SD) number of 7.0 (2.37) fulfilled criteria. Low scores were found for logical deduction of conclusion, recommendation from the presented information, and clear naming of references in terms of traceability and verifiability. Although 18% of the answers were judged to contain unclear or misleading information, in total a high score was attained for absence of unclear or misleading information. Only 58% fulfilled the category “Appropriate length of answer” and a high number (30%) of answers were too short. The subjective overall impression expressed in marks varied widely from 1.5 to 5.3 (median 3). As shown in figure 2, hospital pharmacies with a specialised DI department achieved better results.

Table 4.

Results of structural requirements: part I; part I of hospital pharmacies also participating in part II; and part II

Structural requirements*	Part I (n=45)		Subgroup analysis part I: participants of part II (n=18)		Part II (n=18)
	Median (range)	Mean	Median (range)	Mean	Median (range)	Mean
Answer corresponding to question	1.0 (0–1)	0.72	1.0 (0.17–1)	0.83	1.00 (0.5–1)	0.93
Logical organisation of answer	0.83 (0–1)	0.77	0.92 (0.17–1)	0.79	0.8 (0.2–1)	0.76
Conclusion/recommendation presented	0.50 (0–1)	0.54	1.0 (0–1)	0.67	0.62 (0.22–1)	0.64
Conclusion/recommendation logically deduced of presented information	0.17 (0–1)	0.41	0.58 (0–1)	0.54	0.54 (0.1–1)	0.57
References given	1.00 (0–1)	0.72	1 (0–1)	0.75	0.88 (0.13–1)	0.73
References presented in a way they can be tracked/checked	0.17 (0–1)	0.40	0.5 (0–1)	0.46	0.25 (0–1)	0.36
Correct grammar and spelling	1.00 (0.5–1)	0.91	1 (0.5–1)	0.92	0.92 (0.54–1)	0.89
Absence of unclear or misleading information	0.83 (0–1)	0.75	0.83 (0–0.83)	0.71	0.82 (0–0.82)	0.75
Good readability and understandability	0.83 (0–1)	0.71	0.82 (0–1)	0.73	0.8 (0.27–1)	0.73
Length of answer appropriate	0.83 (0–1.83)	0.71	0.91 (0–1.83)	0.90	0.91 (0.1–1.27)	0.81

*0=no, 1=yes; for length of answer: 0=too short; 1=appropriate; 2=too long.

Figure 2.

Subjective overall rating answers part I: marks (1=excellent to 6=failed; department DI n=11; pharmacist responsible for DI per day n=5; DI in addition to another routine task n=29). DI, drug information.

Interrater reliability was high for essential content-related requirements, but varied substantially for optional content-related (see table 3) and structural requirements (κ 0.072–0.649).

Subgroup analysis: results of study part I of hospital pharmacies participating in both study parts

To test whether the quality of answers to the open, fictitious enquiry were representative of the quality of DI under real-life conditions, we aimed to compare the results of study parts I and II for those hospital pharmacies participating in both parts. Therefore, a subgroup analysis of results from study part I was performed for these participants.

The mean number of essential contents given by participants of part II on the fictitious enquiry was 2.67 and 3.55 for additional useful information. The mean number of structural requirements achieved was 7.44 (for details see table 4, middle column). The median subjective mark, calculated over all responses of the subgroup, was 2.55 (1–4.13; mean 2.5). Overall, hospital pharmacies participating in part II achieved slightly better results concerning content-related and structural requirements in part I of the study in comparison to all participants.

Results study part II

Initially, 20 hospital pharmacies opted for participation by submitting five recently answered patient-specific enquiries for evaluation. On examination, submissions of two participants had to be excluded, since they consisted only of short notes stating a clinical problem and could not be rated.

Of the remaining 90 answers from 18 hospital pharmacies, plausibility (0=not plausible; 1=plausible) was rated as a median of 0.91 (0.53–1; mean 0.87). Out of 11 structural requirements, participants achieved a median of 8.6 (2–11; mean 9). Details of the rating for 10 structural requirements as in study part I are presented in table 4, right column. For the 11th formal requirement, “Does a formal answer exist?”, the median score was 1 (0.47–1; mean 0.9). In comparison to results of part I of these hospital pharmacies, variations can be found for several criteria, but no statistically significant differences were seen (p>0.05 for all criteria). Answers for part II corresponded better to the enquiry (p=0.15). Part II scored worse in naming (p=0.7) and presentation (p=0.08) of references. The length of the answer was judged to be more appropriate in part II (p=0.48). Considering all structural requirements, participants of part I and II achieved comparable results in study part I and II.

Discussion

We tested an open quality assessment procedure using a fictitious enquiry under simulated real-life conditions to evaluate the quality of DI provided by hospital pharmacies. This test was successful in feasibility and acceptance of participants and offers important insights on areas of improvement for DI from hospital pharmacies. Moreover, test results under simulated real-life conditions can be judged to reflect DI in every day life, as the quality achieved in answers to the fictitious enquiry and in recent patient-related enquiries was comparable, as analysed for a subgroup of participants.

Fictitious enquiries have been used as a tool to test the quality of DI in several settings. Norwegian researchers used fictitious enquiries for quality assessment regarding structure, language and time consumption, evaluating responses from seven Scandinavian DI centres.^{9 10} Calis et al evaluated responses to four enquiries placed at 116 US DI centres. Of 79 institutions responding to all enquiries, correct answers were given by 20–90%, depending on the question, and vital patient data were requested in 5–96% for patient-specific enquiries.¹⁶ In a study by Gallo et al, 20 US DI centres were contacted anonymously and responses evaluated by five clinical pharmacists with a maximum achievable score of 100 per answer. Only nine centres answered and responses scored from 23 to 84.²⁰ Beiard et al tested US DI centres with two telephone enquiries. Of 56 centres evaluable for analysis, only 16 gave the correct response to the first question and received the second enquiry. Of these, four gave the correct information linking an adverse effect to a drug, but none offered management recommendations.⁸ Overall, these studies revealed substantial deficits and the need for improvement for DI provided by US DI centres. However, participants of these studies were blinded regarding the specific test enquiries. This condition can hardly be realised for quality assessment of DI of German hospital pharmacies, since they only serve the medical staff of their hospital and any enquirer from outside would be forwarded to other institutions for help. Therefore, in study part I, we chose to test an open quality assessment approach under simulated real-life conditions by announcing the test week without the day of the enquiry and by defining a realistic time frame for the answer. The first measure should rule out hospital pharmacies preparing for the test enquiry by appointing their best expert for this task. The second measure, setting a time frame, forces participants to answer during usual working hours and prevents unusually extended literature searches. This was based on experiences from an open pilot study testing a fictitious enquiry with four hospital pharmacies. In the pilot study, the time frame for the answer was 48 hours and participants used the full period of time allowed, creating very long and unusually referenced results.²¹ Interestingly, a study from Norway found no indication that increasing the time spent on an answer also increased the quality of responses.⁹ Generally, the time required for answering a DI enquiry depends on several factors—for example, the type of question, clear or conflicting results during the literature search, and experience of the performing staff member.^{22 23} Normally, answering the fictitious enquiry of study part I will not take 5 hours of a DI pharmacist’s time, as it was designed to yield an answer available in the literature. The time frame was chosen to reflect real-life conditions in several aspects. First, physicians often expect an answer within a few hours to help with a therapeutic decision. Second, in addition to the time needed for the literature search and formulating an answer, we considered that the email with the test enquiry might not be read immediately. Other DI enquiries might be more pressing or, if DI provision has to be performed in addition to routine daily tasks, these may have to be dealt with first. On the other hand, our measures to simulate real-life conditions led to the exclusion of 28% of initially interested hospital pharmacies, which were not able to answer in time.

To further evaluate whether our approach reflects the quality of DI under real-life conditions, in study part II we aimed to compare the quality of answers to the fictitious enquiry to recently answered patient-related enquiries from the same hospital pharmacy. Unfortunately, only 40% of the participants provided enquiries eligible for evaluation. As described, these participants achieved in general better results in study part I regarding content-related and structural requirements in comparison to all participants. Therefore, we directly compared the quality of the answers in study part I and II for this subgroup. Overall, results for structural requirements were very similar for study part I and II. Plausibility was generally rated good for the patient-related enquiries, but as seen by a wide range of scores, differed substantially for some enquiry answers. This plausibility rating cannot be compared directly to the content-related requirements rated for the fictitious enquiry of study part I. Nevertheless, when looking at the essential content-related requirements of the fictitious enquiry, most participants provided the necessary information, but with some variance in completeness. Altogether, we think that use of a fictitious enquiry under simulated real-life conditions will assess the quality of DI in a satisfactory way, as shown by the results of study part I and II.

In addition, our study provides important insights into areas of improvement for DI provided by German hospital pharmacies. Although the fictitious enquiry of study part I was classified as median complexity,¹⁷ not all hospital pharmacies were able to present the three essential content-related requirements. This should lead to immediate quality improving measures for the hospital pharmacies failing in this point. In addition, we predefined a number of optional contents, which would be helpful for the further treatment of the patient. Mixed results were achieved regarding these additional aspects and although they are not mandatory to answer the initial enquiry, they can help to improve the usefulness of DI and thereby medication safety substantially. Thus, we think inclusion of additional useful information should be part of further education for DI pharmacists from German hospital pharmacies. Also, further education is necessary concerning correct citing of references and logical deduction of a recommendation or conclusion. Indeed, the presentation of specific conclusions and/or advice has been identified as a major quality aspect for written DI responses.¹⁰

Concerning the diverse DI organisation in German hospital pharmacies, institutions with a specialised DI department tended to achieve better results on all aspects of the quality of DI. This result should encourage hospitals to establish their own department for DI wherever possible. Accordingly, the International Pharmaceutical Federation Requirements for DI Centres demand a full time pharmacist during periods of major need, for example during peak periods, for hospital functions.⁵

The main limitation of this study is its open design. We cannot rule out the possibility that participating hospital pharmacies handled the test enquiry in a different way than usual. Nevertheless, our concept of simulated real-life conditions should minimise this effect. In addition, hospital pharmacies participating in part II of the study chose the recently answered enquiries for examination by themselves. Picking the best recent enquiries is a possible bias for our study.

We successfully tested a fictitious enquiry under simulated real-life conditions as a method to evaluate the quality of DI. This approach will be used for future repeated quality measurement of DI as demanded by national and international practice guidelines.

Key messages.

What is already known

• Quality assurance procedures on a regular basis are demanded for institutions providing drug information; however, no established, standardised method has been internationally agreed on.

• The use of a fictitious enquiry has been evaluated in several settings, but, if blinding of the answering pharmacist regarding the test enquiry cannot be realised, this could be a major bias.

What this study adds

• A quality assessment procedure using a fictitious enquiry under simulated real-life conditions can successfully be implemented as an instrument for quality assessment of drug information provided by hospital pharmacies, and sufficiently reflects the quality of drug information under real-life conditions.

• By using this tool for quality assessment, areas for improvement of drug information provided by hospital pharmacies can be identified.

Data availability statement

Data are available upon reasonable request.

Ethics statements

Patient consent for publication

Not required.