Development and validation of a new Prescription Quality Index

Abstract

AIMS

The aims were to develop and validate a new Prescription Quality Index (PQI) for the measurement of prescription quality in chronic diseases.

METHODS

The PQI were developed and validated based on three separate surveys and one pilot study. Criteria were developed based on literature search, discussions and brainstorming sessions. Validity of the criteria was examined using modified Delphi method. Pre-testing was performed on 30 patients suffering from chronic diseases. The modified version was then subjected to reviews by pharmacists and clinicians in two separate surveys. The rater-based PQI with 22 criteria was then piloted in 120 patients with chronic illnesses. Results were analysed using SPSS version 12.0.1

RESULTS

Exploratory principal components analysis revealed multiple factors contributing to prescription quality. Cronbach's α for the entire 22 criteria was 0.60. The average intra-rater and inter-rater reliability showed good to moderate stability (intraclass correlation coefficient 0.76 and 0.52, respectively). The PQI was significantly and negatively correlated with age (correlation coefficient −0.34, P < 0.001), number of drugs in prescriptions (correlation coefficient −0.51, P < 0.001) and number of chronic diseases/conditions (correlation coefficient −0.35, P < 0.001).

CONCLUSIONS

The PQI is a promising new instrument for measuring prescription quality. It has been shown that the PQI is a valid, reliable and responsive tool to measure quality of prescription in chronic diseases.

WHAT IS ALREADY KNOWN ABOUT THIS SUBJECT

• Many studies raise serious questions about the prescribing appropriateness and prescription quality. However, there is a lack of a single measure that will capture all facets of prescription quality.

• Evaluation of prescriptions was usually based on expert judgement of practitioners.

• Definition of prescription quality, reference model, validity and reliability of the measurement tools, and other data such as the number, type and severity of diagnosis of patients were usually insufficient or lacking.

THIS STUDY ADDS

• The Prescription Quality Index (PQI) was developed with a strong structural foundation and was able to capture the clinical, clerical and legal requirement of a prescription.

• Extensive psychometric testing was performed on the PQI and the new tool demonstrated acceptable validity and reliability.

• The PQI has been shown to be a valid, reliable and responsive tool to measure quality of prescriptions in chronic diseases.

Introduction

Prescriptions act as the core communicating medication plans from prescribers to pharmacists, and finally to patients. Components of a complete prescription should include the following information written legibly: date, patient's name, age, weight, registration number, name of medication, dose, route of administration, frequency of administration, duration of treatment, indication, and name and signature of prescriber [1].

A good prescription is one that is rational, evidence-based, clear, complete, and able to improve the health outcomes of the patient treated. Good prescription quality reflects good prescribing process and thus good quality healthcare in general. Prescribing without an acceptable indication, correct dose, frequency, route of administration, schedule or duration of treatment, duplicating therapeutic agents and prescribing drugs without adequate regard to potential interactions or adverse reactions are all forms of inappropriate prescribing [1–4] and contribute to poor-quality prescription.

Many authors have raised serious questions about prescribing appropriateness and prescription quality [5–7]. Ni et al. reported that out of 397 prescriptions screened, 96.7% had missed one or more legal or procedural requirements [5]. Errors of commission involving 8.4% of prescribed drugs were also detected. In a French longitudinal study of 9294 subjects aged ≥65 years, nearly 40% of the participants used at least one potentially inappropriate medication [6]. Similarly, Dhall et al. found that 33% of nursing home residents were receiving at least one potentially inappropriate drug on admission [7]. However, one of the great limitations in measuring the quality of prescriptions is the lack of a method that is sufficiently valid and reliable to allow systematic use in a clinical setting. Several quality measuring tools are available [8–11] but they are not specifically designed to address the multiple problems associated with prescription quality.

Many tools were developed based only on expert judgement of practitioners or consensus [5, 11–14] without information on the psychometric properties of the instruments. Furthermore, these tools are intended for measurement of quality care in general [15], specific disease [13], specific population [16, 17], overall drug use [9], specific areas of drug use [6, 18], or specific drug or groups of drugs [17, 19, 20]. There is a lack of a single measure that will capture all facets of prescription quality. Definition of quality, reference model, validity and reliability of the measurement tools, and other data such as the number, type and severity of diagnosis of patients are usually insufficient or lacking. Therefore, these tools are not applicable to measurement of prescription quality in chronic diseases, especially those with multiple comorbidities.

The World Health Organization has derived indicators to describe key areas of outpatient and inpatient drug use in developing countries [9]. Prescribing indicators include mean number of drugs per encounter, percent of encounters with an injection prescribed and percent of encounters resulting in prescription with antibiotic. These indicators are intended to be objective measures of prescribing behaviour allowing comparison between prescribers or units over time.

The Medication Appropriateness Index (MAI) developed by Hanlon et al. [10] at Duke University Medical Centre (Durham, NC, USA) has been the most widely used instrument to evaluate the appropriateness of medication use in individual patients and has been found to be reliable and valid in a number of clinical settings [21, 22]. Based on the results of a small study of ambulatory elderly patients, the MAI consists of 10 criteria with overall score from ‘0’ to ‘18’ for each medication. The tool is worded in the form of questions, assessing basic appropriateness of drug therapy, as well as interaction potential and cost. This index is intended for elderly patients to provide a valid, reliable and standardized method for risk assessment that can be applied to a range of medication scenarios and clinical conditions in patient care settings. Each criterion is operationally defined and worded in the form of questions that require a rating on a three-point scale. Higher scores indicate less appropriate prescribing. However, the MAI is not designed to measure prescription quality and does not address several important issues, such as adverse drug reactions (ADRs), evidence-based prescribing, compliance, and patient outcomes.

A valid and reliable tool to assess quality of prescription is clearly needed. The ideal tool should be practical, applicable to a broad variety of medications and clinical conditions and can easily be adopted for application in different settings and limited availability of data. Therefore, the objective of this study was to develop and validate Prescription Quality Index (PQI) for the measurement of prescription quality in patients with chronic diseases.

Methods

Development of the PQI

In constructing the PQI, four main steps were involved: item selection, face and content validity, scoring, and validating the PQI. The items or criteria for the PQI should include all important variables that are related and have impact on the prescription quality, taking into consideration the multidimensional and complex nature of drug therapy in different countries. Validation of the newly developed PQI was performed in patients with chronic diseases. This study was approved by the Universiti Sains Malaysia Research and Ethical Committee.

Item selection

Extensive literature reviews were conducted to identify a broad range of items relevant to prescription quality [6, 9–11, 20, 23]. Regulatory requirements and other practical issues related to the topic were also reviewed. All the 10 criteria of the MAI [10] were included in the PQI with the consent of the principal author. Several discussions and brainstorming sessions with other researchers in clinical pharmacy and pharmacology were conducted to select appropriate items for the index.

Face and content validity

The selected items for the PQI and the manual were presented to a group of six multi-specialty experts in medicine (K.M.D.), medicine and pharmacology (A.R.A.R.), psychiatry (H.C.I.) and pharmacy (N.A.A.S., A.M., R.A.) for reviews and comments. This step was intended to test whether the proposed PQI covers the range of meanings that should be included within the concept of prescription quality. Wording changes were made according to the expert panel recommendations.

The first draft PQI consisted of 22 criteria: indication, evidence-based, effectiveness, dosage, correct direction, practical direction, drug–drug interaction, drug–disease interaction, drug–food interaction, ADR, drug duplication, necessity of duplication, duration, cost minimization, Islamic jurisdiction, prescription legality and compliance. In addition, quality of prescription writing such as legibility, medication name, adequate patient's information, diagnosis, and patient's outcomes indicators such as patient's improvement were included. Five-point Likert items were used to assess each criterion. A manual defining each criterion and giving scoring criteria was developed.

Pre-testing was performed on 30 patients suffering from various chronic diseases to test the comprehensiveness, validity, feasibility and practicality of the first draft PQI. Equal numbers of 30 prescriptions were systematically stratified for single, two, three or more diseases. Copies of the written prescriptions were obtained and patient's medical records for all prescriptions were traced for detailed information to aid in the prescription rating. Prescriptions were then rated by the researcher (N.B.H.) using the first draft PQI. Results were analysed and the following problems were identified: wording, scaling and weighting, and difficulties in defining the scales. Improvement was made to produce the second draft PQI. The criterion for legality was dropped since it was too general and aspects for legality were already incorporated in other criteria. Three criteria for adequate prescriber's information, generic prescribing and complication were added (Figure 1).

Figure 1.

Detailed development of the criteria in the Prescription Quality Index

The validity of the modified PQI was further examined using the modified Delphi method [24] in two separate surveys by pharmacists and clinicians working in various states of Malaysia. This method of selecting indicators is reliable and has been shown to have content, construct and predictive validity.

The first survey was conducted on 70 respondents (29 clinicians and 41 pharmacists) using the second draft PQI of 24 criteria. Medical and pharmacy respondents were chosen because they are key decision-makers in drug treatments. Respondents were selected by convenient sampling method and approached face to face in their workplaces. The respondents were asked to comment on the wordings of the criteria, judge the feasibility and practicality, and rate the importance of each criterion in relation to prescription quality. The importance of each criterion was rated according to five-point Likert scales of ‘1’ (unimportant) to ‘5’ (very important). Following the analysis of the survey results, several changes were made. In the third draft PQI, two criteria regarding drug duplication and necessity of duplication were combined since these criteria were intended to measure the same purpose. Two criteria for formulary drug and treated medical conditions were added. Wording changes were made to two criteria derived form the MAI (directions and practicality) based on the comments. The criterion ‘Are the directions correct?’ was rephrased to ‘Are the directions for administration correct?’. Similarly, ‘Are the directions practical?’ was improved to ‘Are the directions for administration practical?’ for better clarity.

The second survey was performed with pharmacists and clinicians working in several states of Malaysia in a variety of clinical settings. Respondents were approached face to face in their workplaces. The nature of the study was explained and each respondent was given the third draft PQI (25 criteria). They were asked to give their rating of importance for each criterion in relation to prescription quality using a Likert scale. Respondents were also encouraged to comment on the wording and contents of the PQI. A total of 120 respondents (57 clinicians and 63 pharmacists) completed the survey forms and with a response rate of 95%.

Qualitative and quantitative analyses were performed. Criteria rated as ‘3’, ‘4’ or ‘5’ by ≥75% of respondents and those required for content validity were to be retained in the PQI. Criteria for drug–food interactions, Islamic jurisdiction (relevant only in a small population of patients and insufficient information) and complication (too broad) were excluded. Furthermore, criteria for cost, diagnosis and treated medical conditions were rephrased. The criterion ‘Is this drug the least expensive alternative compared to others of equal utility?’ was improved to ‘Is this drug the cheapest compared to other alternatives for the same indication?’, since equal utility was difficult to be applied clinically. Another criterion, ‘Is the diagnosis on the prescription fairly written?’, was further improved to ‘Is the diagnosis on the prescription clearly written?’ and the criterion ‘Has all the medical conditions being treated?’ was rephrased to ‘Does the prescription fulfil the patient's requirement for drug therapy?’.

Index scaling and weighting

Considering the complex rationale of medication therapy and different impact of each indicator on patients' outcomes, different weights for different criteria were to be applied based on the results from the second survey. The scales of the final version PQI were assigned three scoring levels (very important, important and least important) based on the response rates (Table 1) of the second survey. The value of ‘0’ represented poor quality rating.

Table 1.

Assignment of scores for the Prescription Quality Index

Category	Definition	Scale	Scale range	Anchor points
Very important	Rated 5 by ≥75% of respondents	0–2–4	0–4	3
Important	Rated 4 or 5 by ≥75% of respondents, but not meeting criteria for very important	0–2	0–2	2
		0–1–2		3*
Least important	Rated 3, 4 or 5 by ≥75% respondents, but not meeting the criteria above	0–1	0–1	2
Rejected	Not meeting any of the criteria above

^*Items that need more refined definitions.

Drug indication and dosage were rated as very important by the responders and given the highest weighted scale of ‘0’ to ‘4’. Fifteen criteria on evidence-based, effectiveness, correct directions, practical directions, drug–drug interactions, drug–disease interactions, ADR, duration, compliance, legibility, prescriber's information, patient's information, medication's name, diagnosis, and patient's improvement were considered as important. Therefore, these criteria were assigned the medium score of ‘0’ to ‘2’. Five criteria on unnecessary duplication, cost, generic prescribing, formulary or essential drug list, and requirement for drug therapy were rated as least important and assigned the lowest score of ‘0’ to ‘1’.

The maximum quality values were weighted according to the degree of importance of each criterion as rated by the responders. Consequently, a final version PQI in question form with weighted scales and a manual for operational definitions was developed (Appendix 1).

Theoretical framework

Donabedian conceptualization of structure, process, and outcome quality model [25] was used as an approach in the development of the PQI (Figure 2). The Donabedian's quality model was selected for this study because it has a broad and long history of applicability. The concepts have been used, expanded, challenged, refined, and proven by the author himself, and by other researchers [26–28]. According to Donabedian, quality is a measure of organization effectiveness as assessed through quality indicators, and categorized as structure, process and outcome variables [25]. These variables are causally linked, with good structure setting the condition for good process, which leads to good outcomes. The most complete quality assessment tool requires measuring all the three categories of variables. However, other factors such as simplicity, practicality, feasibility, manpower, time limitations, and lack of sufficient resources had been taken into consideration in developing the new tool.

Figure 2.

Theoretical framework for the Prescription Quality Index using the Donabedian structure, process and outcome quality model

Pilot study of the PQI in patients with chronic diseases

A pilot study was conducted in a convenient sample of 120 prescriptions from 120 patients with chronic illnesses such as hypertension, diabetes, asthma, migraine, epilepsy, ischaemic heart disease, osteoarthritis, gout, and allergic rhinitis to validate the PQI. Copies of the written prescriptions were obtained from the outpatient pharmacy, Hospital Universiti Sains Malaysia (Kelantan, Malaysia). Using the patients' medical records, the following data were extracted by the researcher: patients' sociodemography, past and current medical illnesses, laboratory results, medication profile, treatment indication or reasons for prescriptions, compliance and other relevant information. The average time required to extract a chart was 20–40 min. The prescription was then rated using the criteria in the PQI and the manual by the researcher (N.B.H.).

Prescriptions may be prescribed as a single drug or multiple drug therapy. For prescriptions consisting of more than one drug, each drug was rated individually (Appendix 1). Similarly, if patients suffered from more than one disease state, each disease state was rated separately. The minimum score was then selected for the PQI summation. Compliance criterion was measured based on physician notes in patient's medical record. When it was not possible to obtain certain data such as cholesterol level or compliance status, criteria were rated as having no information and score of ‘9’ was given.

If a drug was not indicated, criterion 1 should be scored as ‘0’ (not indicated). Subsequently, criterion 2 (dosage), criterion 13 (duration) and criterion 14 (cost minimization) were all scored as ‘0’. The PQI total score was obtained by summing up all the minimum scores for the 22 criteria for all drugs in a prescription. The possible maximum score of the PQI was ‘43’. Prescription with the PQI total score of ≤31 was interpreted as poor quality, 32–33 as medium quality, and 34–43 as high quality.

Although one rater (N.B.H.) was used in this study, intra-rater and inter-rater reliability were still measured to assess stability of the PQI in actual clinical practice. Six raters, including three pharmacists (G.S.H., Z.Z., M.H.A.R.) and three clinicians (A.H.G.R., N.N.I.N.I., S.I.) were recruited. These raters were not involved with the development of the PQI to minimize bias. Two of the raters have postgraduate qualifications and one has basic degree training in each group. The raters were trained on how to use the PQI with the manual. If they needed further information, they were free to use their own references as in their daily clinical practice. Each rater was then given 10 prescriptions from 10 different patients with hypertension, diabetes, epilepsy, hypertension and hyperlipidaemia, and hypertension, asthma and allergic rhinitis to rate. The second rating was performed 1 month later by the same rater on the same prescriptions for intra-rater reliability. For inter-rater reliability, data from the six trained raters were paired and analysed.

Statistical analyses

Descriptive statistics were used to describe the samples. Mean [standard deviation (SD)] or median (interquartile range) were used to describe numerical variables and frequency (%) was used for categorical variables.

To validate the PQI, item analysis, internal consistency, and inter-rater and intra-rater reliabilities were performed. Floor effects (percentage of prescriptions with minimum possible score) and ceiling effects (percentage of prescriptions with maximum possible score) were also assessed.

Item analysis were performed to examine the relationship between the PQI scores and each criterion on the index. Factor analysis [29] was performed to explore common dimensions between the PQI criteria. All criteria were checked for whether each criterion loaded on (was correlated with) the dimension it belonged to, and not any other. If it loaded on the ‘wrong factor’, or on two or more factors, then it was likely that it might be tapping something other than what it was intended, and was either rewritten (if removal of item compromised content validity) or discarded (if removal did not compromise content validity) [30, 31]. Correlations between variables were calculated using Pearson correlations for normally distributed numerical data. Spearman's ρ correlations were used to assess the correlations for skewed data. Categories for the correlation were: absent, <0.2; weak, 0.2–0.34; moderate, 0.35–0.50; and strong, >0.50. Generally, criteria that correlated with the total score >0.20 were considered acceptable and those with lower correlations were considered for review.

Internal consistency was measured using item total correlation and Cronbach's α[32, 33]. These two properties reflect the extent to which items correlate with the total score and how well items measure the same construct. Correlation of criteria should be between 0.2 and 0.8 [31].

The intra-rater and inter-rater reliabilities of the PQI total scores for the six raters and between the possible 15 pairs of the six raters were calculated using intraclass correlation coefficient (ICC) for numerical variables. Inter-rater reliability specifies the extent to which two or more raters applied the PQI in the same manner to the same prescription. High levels of inter-rater and intra-rater reliabilities indicate that the raters conducted the prescription rating in a consistent manner. The ICC indicates how much of the variance actually being measured in scores is due to true differences vs. differences in the way it is measured [34]. Values of scores <0.4 was interpreted as poor, 0.40–0.59 as fair, 0.60–0.74 as good, and values >0.75 as excellent [35, 36].

For the assignment of cut-off points, the PQI total scores were split into four groups on the basis of quartile analysis. Percentile analysis was also performed [37]. The cut-off points for quality assessment were based on the PQI mean value, percentile and quartile analyses, and the nearest possible maximum score that could be obtained if a drug was not indicated.

Results

Characteristics of the patients with chronic diseases in the pilot study

A total of 120 prescriptions from 120 patients with 435 drugs were included in the pilot study (Table 2). Of the 120 patients, 84% were Malays and 98% were married. The mean age of patients was 56 years (range 15–79 years). Most (84%) of the prescriptions were from the Family Medicine Clinic and 16% were from the specialist clinics.

Table 2.

Characteristics of 120 patients with chronic diseases

Variable	n	%	Mean	SD
Gender
Male	60	52.2
Female	55	47.8
Age (years)	120		55.9	9.6
Height (cm)	84		159.5	9.1
Weight (kg)	104		65.3	11.2
Race
Malay	101	84.2
Chinese	19	15.8
Marital status
Married	117	97.5
Single	3	2.5
Number of drugs in the prescriptions	120		3.7	1.8
1 drug	9	7.5
2 drugs	24	20.0
3 drugs	32	26.7
4 drugs	20	16.7
5 drugs	18	15.0
≥6 drugs	17	14.1
Number of diseases or conditions per prescription	120		2.0	0.99
1 disease/condition	43	35.8
2 diseases/conditions	41	34.2
3 diseases/conditions	25	20.8
4 diseases/conditions	10	8.3
5 diseases/conditions	1	0.9
Compliance status
Compliant	23	25.3
Noncompliant	68	74.7

SD, standard deviation.

The number of drugs in the prescriptions ranged from one to 11 with the mean value of 3.6 (SD 1.81). The mean number of chronic medical illnesses in one prescription was 2.04 (SD 0.99). The most common medical conditions were hypertension only (36%), hypertension with hyperlipidaemia (21%), ischaemic heart disease (10%), hypertension with diabetes mellitus (7%), and diabetes mellitus (4%). Other disease states included in the study were asthma, migraine, gout, angina, reflux oesophagitis, allergic rhinitis, sinusitis, osteoarthritis, and epilepsy.

The Prescription Quality Index

The new rater-based PQI with 22 criteria was developed and supported with the manual for detailed operational definitions. The criteria in the PQI were in question form and the range of scores varied from ‘0’ to ‘4’ for very important criteria, ‘0’ to ‘2’ for criteria considered as important, and ‘0’ to ‘1’ for less important criteria. The manual for the PQI included an introduction to the PQI, a listing of the 22 criteria, steps on how to use the PQI, specific instructions with operational definition of the terms, scoring method for each criterion, and the assessment form to be used for the prescription rating (Appendix 1). The PQI can be typically completed in about 10 min or longer, although it may on occasion take longer, depending on the number of drugs in the prescription, and rater's specialty, experience and training.

Psychometric properties of the PQI in patients with chronic diseases

The PQI total scores were normally distributed with the mean value of 31 (SD 5.2). While the PQI score ranged from ‘0’ to ‘43’, there was only one (0.8%) patient who received a minimum score of ‘19’, whereas one (0.8%) patient received a maximal score of ‘41’, indicating the absence of floor or ceiling effects.

Table 3 shows the PQI mean scores and their SDs for each PQI criterion. Two criteria (generic prescribing and diagnosis) were normally distributed, while the other criteria displayed skewed distribution. Four criteria (unnecessary duplication, formulary/essential drug, legibility, and adequate patient information) were severely skewed in their scoring distribution.

Table 3.

Scale properties of the Prescription Quality Index in chronic diseases

No.	Criterion	Weighted scale	Mean	SD
1	Is there an indication for the drug?	0–2–4	2.8	1.5
2	Is the dosage correct?	0–2–4	3.0	1.7
3	Is the medication effective for the condition?	0–1–2	1.8	0.5
4	Is the usage of the drug for the indication supported by evidence?	0–1–2	2.0	0.2
5	Are the directions for administration correct?	0–1–2	0.2	0.5
6	Are the directions for administration practical?	0–1–2	1.8	0.6
7	Are there clinically significant drug–drug interactions?	0–1–2	1.6	0.6
8	Are there clinically significant drug–disease/condition interactions?	0–2	1.8	0.7
9	Does the patient experience any adverse drug reaction?	0–1–2	1.8	0.5
10	Is there unnecessary duplication with other drug(s)?	0–1	1.0	0.1
11	Is the duration of therapy acceptable?	0–1–2	1.3	0.8
12	Is this drug the cheapest compared with other alternatives for the same indication?	0–1	0.2	0.4
13	Is the medication being prescribed by generic name?	0–1	0.6	0.5
14	Is the medication available in the formulary or essential drug list?	0–1	1.0	0.2
15	Does the patient comply with the drug treatment?	0–2	0.6	0.9
16	Is the medication's name on the prescription clearly written?	0–1–2	1.8	0.4
17	Is the prescriber's writing on the prescription legible?	0–1–2	2.0	0.2
18	Is the prescriber's information on the prescription adequate?	0–2	1.5	0.9
19	Is the patient's information on the prescription adequate?	0–1–2	1.0	0.3
20	Is the diagnosis on the prescription clearly written?	0–1–2	0.9	0.5
21	Does the prescription fulfil the patient's requirement for drug therapy?	0–1	1.0	0.2
22	Has the patient's condition(s) improved with treatment?	0–1–2	0.7	0.9

SD, standard deviation.

Exploratory principal components analysis of the PQI total scores revealed an eight-factor solution using the minimum Eigenvalue criteria of ≥1 (Figure 3). These eight factors accounted for 66% of the total variance.

Figure 3.

Scree plot showing the components of the Prescription Quality Index total scores in patients with chronic diseases

The PQI total scores were significantly and negatively correlated with age (correlation coefficient −0.34, P < 0.001), number of drugs in the prescriptions (correlation coefficient −0.51, P < 0.001), and number of chronic diseases/conditions (correlation coefficient −0.35, P < 0.001).

The PQI total scores were strongly correlated with drug indication and drug dosage (Table 4). For the other criteria, there were moderate (six criteria) to weak (10 criteria) correlations. There was no correlation between the PQI total scores and these four criteria: unnecessary duplication, formulary/essential drug, legibility, and adequate patient information. Although these four criteria did not meet the selection criteria, these criteria were still retained in the PQI for content validity, clinical and legal significance.

Table 4.

Reliability of the Prescription Quality Index (PQI) scores in patients with chronic diseases

Criterion	Correlation with the PQI total score	Corrected item total correlation	Cronbach's α if item deleted
1. Indication	0.6b**	0.5	0.5
2. Dosage	0.6b**	0.3	0.6
3. Effectiveness	0.4b**	0.4	0.6
4. Evidence-based	0.3b**	0.3	0.6
5. Correct directions	0.3b**	0.1	0.6
6. Practical directions	0.4b**	0.2	0.6
7. Drug–drug interactions	0.4b**	0.1	0.6
8. Drug–disease/condition interactions	0.4b**	0.4	0.6
9. Adverse drug reaction	0.4b**	0.3	0.6
10. Unnecessary duplication	0.1b	0.0	0.6
11. Duration of therapy	0.4b**	0.4	0.6
12. Cost	0.3b**	0.1	0.6
13. Generic prescribing	0.4a**	0.2	0.6
14. Formulary or essential drug list	0.1b	0.0	0.6
15. Compliance	0.3b**	0.2	0.6
16. Medication's name	0.3b**	0.2	0.6
17. Legibility	0.1b	0.1	0.6
18. Prescriber's information	0.3b**	0.0	0.6
19. Patient's information	0.2b	0.0	0.6
20. Diagnosis	0.4a**	0.2	0.6
21. Requirement for drug therapy	0.2b*	0.2	0.6
22. Patient's improvement	0.2b*	0.1	0.6

^*Correlation significant at 0.01 level (two-tailed).

^**Correlation significant at 0.05 level (two-tailed).

^aPearson correlation.

^bSpearman's ρ.

Cronbach's α for the entire 22 criteria was 0.60. Cronbach's α for each item, if that item was deleted, and the overall Cronbach's α did not change appreciably during the item analysis (Table 4). Thus, all 22 criteria were included in the PQI.

The intra-rater and inter-rater reliabilities of the PQI total scores for the six raters and between the possible 15 pairs of the six raters were calculated using ICC for numerical variables. The average ICC for intra-rater reliability at point one and two by the six raters was 0.76. As for inter-rater reliability, the average ICC was 0.52.

Discussion

The PQI was developed through extensive literature review and varied input from the target population of interest. Peer review and expert judgement for face and content validity are the minimum prerequisites for acceptance of the new tools. The advantage of these approaches is that if the respondents or experts are chosen carefully, they probably represent the most recent thinking in this area of prescription quality. The respondents were not randomly selected, and thus not statistically representative of the clinicians and pharmacists in Malaysia. However, they were carefully selected to provide essential knowledge, experience and expertise, which were extensively accumulated and incorporated in the PQI.

The ability to discriminate different levels of performance depends on the scale of measurement used. The initial basis for scaling and weighting of the PQI was largely judgmental and transparent. They were later supported by statistical results. During the initial stage of the index construction, five-point Likert scales were used for all the criteria. However, after the initial index was piloted in 30 prescriptions, it was found that not all the five-point scales can be defined. Certain definitions did not add meaning to the results and made the index impractical or difficult to be applied in a daily setting. Finally, three-point scales were selected for all the PQI criteria. Hanlon et al. also reported a similar approach in developing the MAI [10].

A good-quality index should be able to discriminate the proportion of good prescriptions to prescriptions with problems. To accomplish this goal, identification of criteria that exert the greatest influence on the PQI was performed and given weights equal to the importance of the criteria in the eyes of the health professionals as a whole. These procedures can significantly increase the predictive ability of the PQI. Therefore, the highest weighted scale of ‘4’ was applied to drug indication and dosage, since these criteria were rated as most important by >75% clinicians and pharmacists. Furthermore, these indicators also showed the highest contribution to the PQI total scores. Hanlon et al. applied slightly different weights to the MAI [10]. Drug indication and effectiveness were given the highest weight of ‘3’. Criteria for dosage, correct direction, drug–drug interaction, and drug–disease interaction were given the weight of ‘2’. The lowest scale of ‘1’ was applied to practical directions, cost, unnecessary duplication, and duration.

The selection of cut-off point is very important when evaluators want to assess whether the prescription of interest is of good or poor quality. For the PQI, the cut-off point of ≤31 for poor-quality prescription was based on the nearest possible maximum score that could be obtained if a drug in a prescription was not indicated. Furthermore, this value also corresponded to the 50th % quartile of the PQI total scores in patients with chronic diseases. The cut-off point of ≥34 for a prescription of high quality was obtained from the upper 75th % quartile score. Setting a cut-off point too low may allow poor-quality prescriptions to pass the quality assessment and cause the PQI to be insensitive. On the other hand, setting the cut-off point too high may unfairly penalise good-quality prescriptions and render the PQI inaccurate.

Item scores should be correlated with the total scale score. For very important indicators such as drug indication and dosage, strong positive correlations with the total PQI scores were observed. Weak to moderate positive correlations with the PQI total scores were obtained except for four criteria (unnecessary duplication formulary/essential drug list, legibility and adequate patient information). Examining the correlations of each criterion may provide valuable information about which criterion most significantly affects the quality of prescriptions.

Prescription quality that is measured by the PQI total score should also be linked to other attributes. If the expected relationship is found, then the measure is valid. However, if no relationship is found, then the fault may be due to the expectation or the new tool. This study has demonstrated that increasing the number of drugs in a prescription, age, and number of diseases inversely correlated with prescription quality. A strong inverse correlation of prescription quality with the number of drugs in the prescriptions was observed. The higher the number of drugs prescribed in a prescription, the lower the prescription quality. This finding was consistent with another study, which reported that inappropriate prescribing was significantly correlated with polypharmacy [38].

A weak inverse correlation between age and prescription quality was observed. This is not surprising, since with increasing age patients tend to suffer more diseases and be at higher risk for complications. Consequently, more drugs are required for treatment with the increase in age [39, 40].

Moderate and negative correlations with number of chronic conditions were also obtained. Increasing numbers of chronic conditions may cause higher number of drugs to be prescribed [39], and thus, lower prescription quality.

Item total correlation measures the extent to which items intercorrelate with one another. The item total correlation ranged from 0.09 (legibility) to 0.54 (indication). Nine criteria (correct directions, drug–drug interactions, unnecessary duplication, cost, formulary or essential drug list, legibility, prescriber's information, patient's information, patient's improvement) displayed an item total correlation value of <0.2. The low correlations are expected since these criteria were not homogeneous and measured different traits. For example, cost would hardly correlate with patient's information. Lower correlation suggests that either reliability of one or the other measure is low, or that they are measuring different phenomena [31].

Reliability addresses the internal consistency of the items and reproducibility of the scores when the tools are applied by the same rater (intra-rater) or different raters (inter-rater) for the rater-based PQI. The PQI was internally consistent with Cronbach's α of 0.60 in chronic diseases. Prescriptions with multiple drugs and different disease states would not be expected to have high internal reliability. The moderate value of Cronbach's α obtained in this study could also be due to multiple factors contributing to prescription quality. Exploratory factor analysis of the PQI identified multiple factors contributed to prescription quality. This was consistent with Coste and Venot, who reported that drug prescribing quality is multifactorial [14].

Four criteria in the PQI (unnecessary duplication, formulary/essential drug, legibility, and adequate patient information) did not meet certain validity and reliability requirements as discussed above. These criteria were purely descriptive in nature and still retained in the PQI for content validity, clinical and legal significance. Furthermore, a study reported that items that are purely descriptive and assessing unnecessary or potentially harmful prescribing do not require validation [17]. In addition, internal consistency of the PQI was not significantly changed by removing these criteria.

Variation from day to day or from one rater to another should be considered by measuring inter-rater and intra-rater reliabilities. These measurements are important when measuring new or unfamiliar devices or when subjective or clinical observations are used. This study showed poor to excellent intra-rater reliability. Four of the raters (raters 1, 3, 5 and 6) scored excellent agreement.

In this study, the moderate value for inter-rater reliability may be due to differences in the clinical experience of the raters, different references used for the prescription rating, or familiarity with the PQI manuals. Raters were allowed to use their routine references for the prescription rating to reflect true situations in daily practice. The result obtained in this study was consistent with other findings [10, 41]. When the MAI was assessed by independent raters not involved with MAI development, inter-rater reliability was moderate [10, 42–44]. Similarly, in drug utilization evaluation assessment, inter-rater reliability was good for explicit, straightforward criteria such as indications for use and critical process indicators. However, low inter-rater reliability for dimensions of complications and clinical outcomes has been observed [41].

Responsiveness assesses the ability of the PQI to detect meaningful clinical change, preferably over a relatively short period of time. When PQI was applied to patients with chronic diseases, the PQI scores displayed normal distribution with no floor or ceiling effects. This will allow discrimination between prescriptions of different quality and patients with various degrees of disease severity.

Practice and requirements in different countries differ to a certain extent. The index represents what is arguably best practice in most countries and easily adaptable to suit various requirements. There are enough generic items in the PQI that can be utilized or customized based on local needs. However, modifications of the PQI may require revalidation and new cut-off points may be needed.

The PQI can be completed in about 10 min or longer, depending on the number of drugs in the prescription, and rater's specialty, experience and training. The utility of the PQI is best appreciated if performed in a longitudinal manner. The PQI can still be used for computer-generated prescription or on-line prescription. With the aid of computer technology, the quality of prescription may be improved, especially for criteria related to prescription writing. Therefore, the PQI is appropriate for use in most clinical settings and research.

Strengths of the study

The PQI was developed with a strong structural foundation and was able to capture the clinical, clerical, and legal requirements of a prescription. Furthermore, the PQI was also subjected to extensive psychometric testing and demonstrated acceptable validity and reliability. The PQI was also able to discriminate between the proportion of good prescriptions that of prescriptions with problems. Thus, the PQI met the standard validity and reliability requirements.

The PQI is specifically developed to be consistent with current scientific knowledge on rational and evidence-based practice, effectiveness, efficacy and safety. This will enable the quality of prescriptions to be measured, analysed and monitored. Therefore, the benefits of interventions can be examined for further improvements in patient care.

Limitations of the study

The study was subject to several limitations that may affect the validity and reliability of the findings. The PQI may not fully capture all criteria involved in prescription quality. The PQI measured main dimensions of prescription quality from the perspectives of clinicians, pharmacists and patients only. However, in real practice, other personnel such as administrative managements, counter staff, nurses, medical assistants, pharmacy assistants and other supporting staff may affect the quality of prescriptions to a certain extent. Organizational factors that may have impact on prescription quality are scheduling and staffing policies, inadequate time and resources for quality management of patient care, absence of dedicated staff, insufficient preventive care, absence of flags on records to identify patients with special needs, lack of continuity of providers, follow-up appointments, and educational programmes. In addition, patients are capable of evaluating the services provided, but they are less capable of evaluating whether appropriate treatments are given for their complaints. However, patients may contribute to prescription quality by providing true and adequate information for correct diagnosis and appropriate treatment and being compliant to their drug therapy. Furthermore, other factors such as dispensing, counselling, monitoring, communication, ratio of doctors per patient, and continuing education are not covered.

The PQI was validated and tested retrospectively and may be subjected to retrospective bias. Patients' medical records were used as the source of information for this study. There are questions concerning the completeness of medical records and whether assessment of quality of care is based on what appears in the record (presumptive evidence) rather than the actual care provided. The records may not contain adequate, consistent and accurate information to serve as a basis for evaluation.

Variations in scores have been observed between the raters in the inter-rater reliability. These variations can be minimized with longer and more intensive training for the raters, especially for those with less clinical experience.

Continuous refinement and improvement of the PQI are certainly needed. Its importance as a new prescription measurement tool and its potential for inducing further research should be further investigated.

The PQI should be further validated in prospective studies and other disease conditions. In prospective studies, data are collected before the events of interest have happened, so data are preserved against forgetting, incomplete data, and wrong interpretation. Prospective studies also accomplished a kind of blinding because information was recorded with the prescribers unaware of the significance it might have in future analyses [45]. However, direct observation of the physician's activities also has its limitations since physicians know that they are being observed. Despite all these limitations, the PQI may provide a valuable and unique tool for future studies.

Conclusions

The PQI is a promising new instrument for measuring quality of prescription. The PQI captured the multidimensional criteria of prescription quality. The PQI incorporates the concept of rational drug therapy, evidence-based approach and other criteria required for prescription quality. The PQI has been shown to be a valid, reliable and responsive tool to measure the quality of prescription in chronic diseases.

Appendix 1

ASSESSMENT FORM FOR THE PRESCRIPTION QUALITY INDEX

To assess the quality of prescription, please answer the following questions and circle the most applicable rating:

Competing interests

There are no competing interests to declare.

We thank Professor Dr Rahmat Awang, Professor Dr Kamaliah Mohd Daud, Noor Aini Abu Samah and Azman Mat for their expert opinions on the development of the Prescription Quality Index. We would also like to thank Associate Professor Dr Aida Hanum Ghulam Rasool, Dr Nik Nur Izah Nik Ibrahim, Dr Suhairi Ibrahim, Dr Gan Siew Hua, Zalina Zahari, Mohd. Hanif Abdul Rahman, Associate Professor Dr Foong Kin, pharmacists and doctors for their contributions to the development and validation of the Prescription Quality Index. This study is supported by the USM short-term grant (304/PPSP/6131271).