Evaluation of a new computerized psychometric test battery: Effects of zolpidem and caffeine

Abstract

Objective:

To evaluate the effects of centrally active drugs using a new indigenously developed automated psychometric test system and compare the results with that obtained using pencil- and paper-based techniques.

Materials and Methods:

The tests were standardized in 24 healthy participants. Reproducibility of the test procedure was evaluated by performing the tests by a single experimenter on two occasions (interday reproducibility). To evaluate the sensitivity of the tests, the effects of zolpidem (5 mg) and caffeine (500 mg) versus placebo were studied in 24 healthy participants in a randomized, double-blind three-way crossover design.

Results:

Psychometric tests were performed at baseline and at 1, 2, and 3 h after administration of study medication. The effects of zolpidem and caffeine on the psychomotor performance were most pronounced 1 h after administration. At this time, a significant impairment of performance in the simple reaction test (SRT), choice discrimination test (CDT), digit symbol substitution test (DSST), digit vigilance test (DVT), and card sorting test (CST) was observed with zolpidem. In contrast, caffeine showed a significant improvement in performance in CDT and DVT only.

Conclusion:

The results suggest that the tests of the computerized system are more sensitive and reliable then the pencil and paper tests in detecting the effects of central acting agents and are suitable for use in clinical areas to conduct studies with patients.

INTRODUCTION

Human psychopharmacology has made a major leap from observation of mere behavioral effects to cognitive changes using very objective sophisticated techniques. Assessment of central nervous system (CNS) effects is very relevant for new drug research, where emphasis is in finding better ways to predict therapeutic and unwanted effects of novel compounds. The effects of drugs on performance have been the subject of much concern, particularly their potential to contribute accidents, whether on road, at work place, or at home. The techniques used to assess psychomotor functions are diverse, often complex, frequently insensitive to drug-induced changes, and inconvenient to replicate.[1]

CNS effects can be classified in terms of effects on different aspects of cognitive and psychomotor functions such as attention, reaction time (RT), visual orientation, perception, vigilance, and motor coordination, as well as on neuropsychological activity of brain. Drugs acting on the CNS will influence more than one of these functions, which offer the possibility to frequently measure drug-induced changes across a range of different CNS activities. Psychometric tests are used in clinical pharmacological studies for quantitative evaluation of CNS effects of drugs during the early phases of drug development.[2] Even though the available standard/validated conventional psychometric tests being used since long, still there exist many disadvantages, viz., time consuming, extensive training in administration and scoring, subjects get bored, chance of data manipulation, and producing a paper trail unmanageable in large-scale studies.

The technological advances in the last 10-20 years have allowed researchers to make more use of computerized tests. Computerized psychometric tests are increasingly used in clinical studies of psychiatric disorders and to evaluate the effects of psychotropic drugs.[3] The primary advantage of a computerized testing includes the reproducibility of testing conditions, randomization of stimuli within the tests, ease of data handling, ease of scoring, and immediate reporting of results to individuals participating in the testing session. Immediate feedback of test results, if properly performed, improves the level of motivation of persons being tested.[4] The purpose of this study is to describe an integrated computer-automated system for assessing CNS effects of drugs and to validate the test system by demonstrating sensitivity to the effects of psychoactive drugs using zolpidem 5 mg and caffeine 500 mg as a reference substances with placebo.

MATERIALS AND METHODS

Instrumentation

The instrumentation consisted of stimulus presentation software compatible with Windows XP operating system (developed using Microsoft's Visual Basics 6.0) with a dedicated custom-built response box with digital pad, digital pen, and personal computer. Instructions, simulation programming and presentation, and response collection were performed with Compaq IBM PC with a 15" monitor. The automated psychometric test battery system consists of a series of individual tasks such as the finger tapping test (FTT), simple reaction test (SRT), choice reaction test (CRT), choice discrimination test (CDT), digit symbol substitution test (DSST), digit vigilance test (DVT), card sorting test (CST), continuous attention test (CAT), numeric working memory test (NWMT), and immediate picture recall test (IPRT). Each test is preceded with instructions and the software allows for recording of demographic data of each subject and reporting test results as separate files. Each test in the battery is individually administrable. A summary of test results can be displayed on the screen or printed out immediately after testing. Minimal training time is necessary for experimenter because they are given the option, through a screen menu, of choosing the tests and the test order to be administered to each subject.

Test selection

Presently, six tests were standardized and validated from the automated psychometric test battery: FTT, SRT, CDT, DSST, DVT, and CST. The results were presented in terms of average RT, total number of clicks attempted, correct attempts, and wrong attempts. The choice of tests was determined by investigator's interest in evaluating vigilance, sensory functions, motor functions, cognitive functions, and memory.

Description of individual tests

FTT

The duration of the test is 10 s, during which the subject has to continuously tap on the “Enter Button” on the response box in quick succession. The test provides information on motor system performance.

SRT

The duration of the test is 60 s. In this test, on the click of a start button, test time will begin and a picture of a boy will appear on the center of the screen for 20 times. Subject has to press the “BOY” symbol button on response box as quickly as possible every time the “BOY” picture appears on the monitor. Each time, the BOY picture will remain on the screen for 1 s and there will be a time gap of 1.5-2.5 s between the appearances of subsequent BOY pictures. This test assesses attention and sensory-motor performance of brain.

CDT

The duration of the test is 60 s. In this test, on the click of a start button, test time will begin and a picture of a boy or girl will appear randomly for 10 times each on the center of the screen. Subject has to press the “BOY” symbol button with right index finger and “GIRL” symbol button with left index finger on the response box as quickly as possible corresponding to the picture. Each time the picture will remain on the screen for 1 s and there will be a time gap of 1.5-2.5 s between the appearances of the subsequent pictures. This test assesses the attention and sensory-motor performance of brain and estimate the psychomotor response speed.

DSST

In this test, the upper panel of the screen will display 1-9 digits with their corresponding target symbol placed over each digit. Subject has to carefully concentrate and remember the corresponding digit for these symbols. On click of start button, the symbols that are shown in the panel will appear randomly one after other on the center of the screen. Subject has to press the corresponding digit as quickly as possible on the response box when the target symbol appears in the center. The total duration of the test is 90 s. This test assesses attention, response speed, central integration, and visuo-motor coordination.

DVT

In this test, a target digit is constantly displayed on the right-hand side of the monitor screen. On click of start button, a series of digits will appear one at a time on the center of the screen at the rate of 105 digits/min. Subject has to press the “Enter Button” on the response box as quickly as possible every time the digit in the series matched the target digit. The task lasts for 60 s and there will be 45 stimulus-target matches. The test assesses alertness and vigilance while placing minimal demands on two other components of attention: Selectivity and capacity.

CST

In this test, the subject was asked to sort a set of 52 cards based on the different colors and shapes using a digital pen. This test assesses sensory, motor, central integrative, and executive functions. Results are presented in terms of average time (sec) taken to complete the sorting and the number of correct and wrong cards.

Study participants

Twenty four healthy male participants aged 20-35 years, took part in the study. Following a full medical history (including smoking habits) and physical examination, which included hematological and biochemical screening, and an electrocardiogram, participants were excluded if there was any evidence of physical illness or drug abuse. Each subject was approached personally and if they agreed to participate, written informed consent was taken after a full explanation of aims, procedures, and risks of the study. The study was approved by the Institutional Ethics Committee and conducted in conformity with the Declaration of Helsinki. The participants were trained on study procedures on at least two occasions prior to the study day to introduce them to the test procedure and to make them familiar with the testing device.

All the subjects abstained from alcohol, nicotine, chocolate, or caffeine-containing beverages, prior to (12 h for caffeine and nicotine; 24 h for alcohol) and during the test day. All the recordings were carried out between 8:00 to 10:00 AM after a light breakfast.

Test procedure

The psychometric performance test battery was administered individually to each participant, taking about 20 min to perform the tests. On the day of the test, the participants were asked to relax and sit comfortably for half an hour before the initiation of test procedures. All the recordings were obtained in a uniformly/adequately lit room, in a noise-free environment with controlled temperature to avoid distraction and increase the comfort level of the subject. An average of three readings was taken for each test with a brief rest of 1-2 min in between. After each trial, subjects received feedback regarding their responses. In order to assess the interday variability and reproducibility of the method, all the tests were performed in 12 participants at the same time in the morning on two occasions with an interval of 3 days by a single experimenter.

Method validation

Twenty four healthy male participants aged 20-40 years took part in the study after they were trained on the psychomotor tests. The study was performed in a double-blind three-way crossover design with subjects randomized to receive either a single oral dose of 5 mg zolpidem or 500 mg caffeine or matching placebo capsule. The treatments were separated by an interval of one week. Subjects arrived at the laboratory following an overnight fast and abstinence of caffeine containing beverages, nicotine, or alcohol confirmed by a questionnaire at the beginning of the session. The procedures as described earlier for performing computerized psychometric tests were carried out between 7:00 and 11:00 AM. All the tests were administered at baseline and then the subject received an oral dose of either zolpidem, caffeine or placebo as identically appearing capsules with 240 ml of drinking water. Further testing was then performed at 1, 2, and 3 h. An average of three readings were taken for each test with a brief rest of 1-2 min in between. Subjects were asked to report any side effects during the study. If there was any side effect, same was noted down in the case report form.

Data analysis

The software records total number of clicks attempted, correct attempts, wrong attempts, and RT for all the tests. In this study, the latency time to respond (RT) result was used as a measure of task performance. Trials in which the participant's RT, that fell beyond ± 2SD considered to be lapses of attention and were eliminated from the analyses. The data analysis comprised of two sessions, one in which zolpidem and placebo were compared at 0, 1, 2, and 3 h and a second in which the treatments were caffeine and placebo at 0, 1, and 3 h time points.

Statistical analysis

All statistical tests were processed using Graphpad Prism software, Version 4 (Graphpad software Inc. Sandiego, California, USA). Mean ± SEM values were calculated for each variable. Demographic details were summarized for all participants using descriptive statistics. The Kolmogorov–Smirnov test was done to assess if data had a normal distribution. Bland–Altman plotting was performed for the assessment of method validity and reproducibility comparing the reaction time obtained in each test for two consecutive days. ANOVA for repeated measures (averaged F) with Bonferroni correction was carried out to detect significant changes in variables over time within each session separately. Pair-wise comparisons between the two treatments (zolpidem vs placebo and caffeine vs placebo) were tested for statistical significance using the paired student's t-test. Statistical significance was at P < 0.05.

RESULTS

The equipment was standardized by performing the tests in 24 healthy male participants. The mean age of the overall sample was 27.21 ± 0.82 years, with a mean BMI of 23.74 ± 0.19 kg/m². Table 1 summarizes the descriptive statistics for the psychometric tests (FTT, SRT, CDT, DSST, DVT, and CST). To study the interday variability, the psychometric tests were recorded on two different days by the same observer. The variability was minimal in both the time periods, and the data were highly reproducible with coefficient of variation (CV) below 10%. There was good reproducibility between the difference among the time periods, which is shown as Bland–Altman plot in Figures 1 and 2 for RT in all the psychometric tests, respectively, and most of the values ranged within a mean ± 2 SD.

Table 1.

Normal values of FTT, SRT, CDT, DSST, DVT, and CST (n:24)

Figure 1.

Bland–Altman plots showing difference between day-1 and day-2 measurements of RT's in FTT, SRT, and CDT

Figure 2.

Bland–Altman plots showing difference between day-1 and day-2 measurements of RT's in DSST, DVT, and CST

Validation results

To confirm the validity of the above method, zolpidem 5 mg and caffeine 500 mg were used as reference study medication in 24 healthy male participants. One subject due to drowsiness and vomiting in zolpidem group and one subject from caffeine group due to gastric irritation were excluded. One subject did not wish to continue the study, so the data of 21 subjects were analyzed, of which participants were of average height and weight for their age (25.90 ± 0.74 years) and their mean BMI was 23.74 ± 0.16 kg/m². There were no significant differences in baseline characteristics between the three sessions in all the psychometric tests.

Effect of zolpidem on RT

Results for the outcome measures are depicted in Tables 2 and 3.

Table 2.

TC, average RT, CC, WC in FTT, SRT, CDT, DSST, DVT, and CST before and after intake of zolpidem

Table 3.

TC, average RT, CC, WC in FTT, SRT, CDT, DSST, DVT, and CST before and after intake of placebo

The results of the study showed that zolpidem significantly increased RT in SRT, CDT, DSST, DVT, and CST. However, no significant effect can be seen with FTT.

ANOVA performed on the RT data revealed significant treatment effects on SRT, CDT, DSST, DVT, and CST {SRT: F (3,20) = 14.0, P < 0.001; CDT: F (3,20) = 17.90, P < 0.001; DSST: F (3,20) = 2.97, P < 0.05; DVT: F (3,20) = 16.31, P < 0.001; CST: F (3,20) = 9.34, P < 0.001}.

Compared to baseline, a significant effect was observed for SRT at 1 h (t (3,20) = 6.19, P < 0.001) and 2 h (t (3,20) = 3.61, P < 0.01); CDT at 1 h (t (3,20) = 6.96, P < 0.001) and 2 h (t (3,20) = 5.12, P < 0.001); DSST at 1 h (t (3,20) = 6.29, P < 0.001) and 2 h (t (3,20) = 2.83, P < 0.05) only. Similarly, a significant effect can be seen at 1 h only in DVT (t (3,20) = 1.79, P < 0.05) and CST (t (3,20) = 4.95, P < 0.001).

Specific time points at which significant zolpidem versus placebo differences occurred were identified by two-tailed paired t-tests. RT increased significantly in SRT at 1 h (t = 4.71, df 20, P < 0.001); CDT at 1 h (t = 4.86, df 20, P < 0.001) and 2 h (t = 3.21, df 20, P < 0.01); DSST at 1 h (t = 5.35, df 20, P < 0.001) and 2 h (t = 2.36, df 20, P < 0.05); DVT at 1 h (t = 2.40, df 20, P < 0.05); and CST at 1 h (t = 2.79, df 20, P < 0.01).

Effect of caffeine on RT

Results for the outcome measures are depicted in Tables 3 and 4.

Table 4.

TC, average RT, CC, and WC in FTT, SRT, CDT, DSST, DVT, and CST before and after intake of caffeine

The results of the study showed that caffeine though decreased RT in SRT, CDT, DSST, DVT, and CST, a significant effect was observed for CDT and DVT only. However, no effect was observed with FTT.

ANOVA performed on the RT data revealed significant treatment effects on CDT and DVT {CDT: F (2,20) = 5.18, P < 0.05; DVT: F (2,20) = 8.97, P < 0.001}.

Compared to baseline a significant effect was observed at 1 h for CDT (t (2,20) = 3.02, P < 0.05) and DVT (t (2,20) = 4.22, P < 0.001). However, no significant effect can be seen at 3 h.

Specific time points at which significant caffeine versus placebo differences occurred were identified by two-tailed paired t-tests. RT decreased significantly at 1 h only in CDT (t = 2.45, df 20, P < 0.05) and DVT (t = 4.46, df 20, P < 0.01), respectively.

DISCUSSION

Psychometric principles have been incorporated into many forms of psychological assessment, but in human psychopharmacology, performance research has often been ignored. Psychomotor performance results from the co-ordination of sensory and motor systems through the integrative and organizational process of brain and the CNS.[5] The processing of sensory information is influenced by personality, memory, and individual motivation, while the overall function of the integrative mechanism is governed by the state of arousal of the CNS.

Psychometric tests are rarely standardized and rarely have any population norms. Thus, control or baseline data must be obtained for comparison each time the tests are used, and this requires some form of study design like repeated measures or within subject design. Effects of drugs on performance have been investigated using a variety of techniques. Assessing the mental performance as processing ability is done using various tests like concept identification, symbol arrangement, DSST, symbol copying, mental arithmetic tests like addition, subtraction, division, etc., Measuring the time taken to complete the task and the number of errors helps in distinguishing the psycho stimulants, which shorten task latency and increase the errors made from drugs that improve performance (shorter task latency without increasing errors).[1] Although the effects of a particular drug on psychomotor performance can be assessed by an individual psychometric test, it is usual to employ a test battery (group of tests each reflecting a different aspect of psychomotor performance) particularly when a new substance with unknown or supposed psychoactive effects is to be investigated.

The purpose of the present study was to standardize and validate indigenously developed automated psychometric test system. The tests evaluated in the present study measure and assess a series of various complex functions of human cognitive and psychomotor performance. The normative values of the various tests (FTT, SRT, CDT, DSST, DVT, and CST) in the test battery that were collected from 24 healthy participants were found to be reliable and accurate. The sensitivity of the computerized psychometric test system to the drug effects may be reduced by concomitant placebo and time effects as well as by interday variability of the end point (RT). There was no significant difference in the RT (FTT, SRT, CDT, DSST, DVT, and CST) for reproducibility reported between the two sessions and values range within mean ± 2SD of the Bland–Altman plot. Validation of the tests (FTT, SRT, CDT, DSST, DVT, and CST) was carried out to assess the effect of zolpidem (5 mg) and caffeine (500 mg) on cognitive and psychomotor performance.

Compared to placebo, zolpidem (5 mg) induces a significant increase in RT in SRT, CDT, DSST, DVT, and CST, a finding consistent with previous studies. Danjou et al.[6] compared the residual effects of zaleplon (10 mg), zolpidem (10 mg), or placebo 2-5 h before awakening. A battery of tests (including Choice reaction test, DSST) were conducted after the subjects morning awakening. Zolpidem's effects were apparent up to 5 h on all the tests after administration, whereas zaleplon showed no residual effect at any time point. Similarly, a comparison of zolpidem (10 or 20 mg), zaleplon (10 or 20 mg), triazolam (0.25 mg), and placebo revealed greater psychomotor impairment with zolpidem 10 mg at 1.25 h compared to other drugs as reported by Hanlon.[7] In a study by Hindmarch et al.,[8] zolpidem (10 mg) or zaleplon (10 or 20 mg) administered at night time at 5, 3, and 1 h before awakening at 8:00 AM showed that zolpidem produced residual effects on DSST and Sternberg memory scanning up to 3 h and 5 h on CRT and delayed free recall of words.

In a crossover study by Troy et al.,[9] zolpidem 10 and 20 mg significantly decreased DSST, immediate and delayed recall at 1.25 and 8.25 h post dose compared to placebo in 24 participants. Otmani et al.[10] reported that zolpidem (10 mg) significantly impaired psychomotor and driving performance at 1 and 4 h post dosing compared to placebo in healthy subjects aged 55 years and older. Mintzer and Griffiths[11] showed that both zolpidem (5, 10, or 20 mg/70 kg) or triazolam (0.12, 0.25, or 0.5 mg/70 kg) produced similar dose-related effects on memory for target information. The two higher doses of zolpidem were associated with significant memory impairment compared to placebo; correct responses in a recognition task were also decreased to a significant extent by the highest dose.

Caffeine intake in normal subjects has long been associated with improved neuropsychological function. Our results are in accordance with the earlier studies,[12,13] which demonstrated association between caffeine intake and enhanced cognitive function. By blocking adenosine receptors, caffeine is thought to enhance neuronal activity throughout the central nervous system. Caffeine increases alertness and improves performance on cognitive vigilance,[14,15] simulated driving tasks, speed of encoding new information in simple and choice reaction tasks,[16] tasks requiring sustained response, dual tasks involving tracking and target detection,[17] and simulated driving tasks.[18]

In the present study, caffeine (500 mg) intake in healthy subjects was associated with decrease in RT on a battery of psychometric tests that encompassed several important areas of cognition: Speed of information processing, attention and working memory, and psychomotor performance. Studies have shown that the most common effects of caffeine are increase in alertness and improvement in reaction time and vigilance. Caffeine showed a significant decrease in reaction time in SRT, CDT, DSST, DVT, and CST compared to placebo. However, no effect was observed in FTT. This might be due to the fact that finger-tapping task involves fast repetitious motor responses on a switch, which involves little cognitive load. However, few studies shown an increase in tapping rate with caffeine,[15,19] which may be due to peripheral rather than central effect, since caffeine is known to increase capacity of muscular work. Previous studies reported a significant reduction in RTs following caffeine administration (dose range 12.5-320 mg),[12,16,17,20,21,22] while few studies found no effect (dose range 32-320 mg).[19,23,24] Few studies[25,26] have demonstrated improvements on simple and choice reaction time with a computerized task that requires participants to tap a button at the sight of target stimuli. Rees, Allen, and Lader[27] demonstrated that participants improved digit symbol substitution, and symbol copying following a 250 mg dose of caffeine. Kaplan and colleagues[28] also demonstrated improvements in DSST with both 250 and 500 mg caffeine doses compared to participants not given caffeine. In addition to reaction time, attention is also one of the most consistently demonstrated testing effects following caffeine intake. Studies[14,29] have shown improvements on a repeated digits vigilance task (detecting targets at irregular intervals) following administration of varying doses of caffeine. Fine et al.[30] found that caffeine (200 mg, n = 20 subjects per intervention arm) significantly increased the number of correct responses and decreased RT's compared to placebo in a visual vigilance task. Lieberman and colleagues[19] reported improvement in correct hits in auditory and visual vigilance tests in normal individuals, when administered with 64 mg of caffeine.

Several methodological factors must be taken in to account when designing studies to assess behavior and cognition because, according to the literature many external variables may interfere the evaluation of results, such as age, gender, education, environmental conditions, use of medication, and co-morbidity among others. Additionally, in patients the disease effects must be considered during evaluation.

CONCLUSION

It can be concluded that the tests of the automated computerized system can quantitatively measure the psychomotor effects of zolpidem and caffeine with good reproducibility in healthy participants. The present low-cost computerized test system provides neuropsychologists with significant new options for test administration and provides an efficient means to obtain both accuracy and speed, require minimal training of administrators, and offers automated scoring. The present model can be used for quantification of new psychoactive compounds in healthy participants and diseased patients before proceeding to expensive clinical trials.