psychmeta: An R Package for Psychometric Meta-Analysis

Abstract

Over the past four decades, psychometric meta-analysis (PMA) has emerged a key way that psychological disciplines build cumulative scientific knowledge. Despite the importance and popularity of PMA, software implementing the method has tended to be closed-source, inflexible, limited in terms of the psychometric corrections available, cumbersome to use for complex analyses, and/or costly. To overcome these limitations, we created the psychmeta R package: a free, open-source, comprehensive program for PMA.

Program Description

The psychmeta package (Dahlke & Wiernik, 2017/2018) includes tools for computing meta-analyses of correlations and Cohen’s d values using bare-bones, individual-correction, and both interactive and Taylor series approximation artifact-distribution methods. The package includes functions to aid in converting a variety of statistical values (e.g., r, d, t, χ², odds ratios, p values) to the r or d metric. psychmeta’s psychometric meta-analysis (PMA) functions support corrections for measurement error and/or range restriction (whether direct or indirect in nature) in one or both variables, as well as numerous other statistical artifacts (Schmidt & Hunter, 2015). psychmeta’s meta-analysis functions allow any number of moderators and can analyze moderators hierarchically or as simple effects. psychmeta also includes numerous features to streamline the process of computing meta-analyses. With a single function call, the meta-analysis routines can synthesize effect sizes for all possible construct pairs contained within a database and create omnibus tables of results from all analyses performed on a database. These routines can also automate the consolidation of dependent samples by computing composite or mean values for effect sizes and statistical artifacts when multiple statistics representing the same effect are reported for a single sample. Compared with existing PMA software, these features can dramatically decrease analysis time for large-scale meta-analysis research. After computing a meta-analysis, the results can be stored as an object and used to construct plots and compute sensitivity analyses (including publication bias analyses, leave-one-out analyses, and bootstrapped meta-analyses), heterogeneity analyses (e.g., Q, τ², variance explained), and meta-regressions.

In addition to computing meta-analyses, the psychmeta package also includes a variety of functions to (a) compute composite effect sizes and statistical artifact values, (b) correct individual effect sizes for statistical artifacts, (c) correct covariance matrices and vectors of means for multivariate range restriction, and (d) compute confidence intervals for converted and corrected effect sizes for reporting in primary research.

psychmeta also includes a suite of functions to enhance research on the development and evaluation of PMA methods. psychmeta can be used to simulate individual samples or meta-analytic databases of correlations and d values with measurement error and/or range-restriction artifacts. The sample-simulation functions allow users to (a) generate Monte Carlo samples of psychometric data or (b) analytically induce statistical artifacts in parameter values. The database-simulation functions allow users to specify parameter distributions in a variety of formats to generate data for meta-analytic simulation studies. These functions can help to make Monte Carlo research on PMA methods more reproducible and easier to conduct. These simulation functions also have pedagogical value in that they provide students a hands-on way to explore the effects of measurement error and sample-selection processes on statistical results.

Availability, Documentation, and Distribution

The psychmeta package (Dahlke & Wiernik, 2017/2018) was written in the R language (R Core Team, 2017) and is compatible with Windows, Mac, and Linux systems. The source code and documentation are freely available from cran.r-project.org/package=psychmeta. The development version of the package is available from github.com/psychmeta/psychmeta.