Quantum Chemistry Common Driver and Databases (QCDB) and Quantum Chemistry Engine (QCEngine): Automation and interoperability among computational chemistry programs

Abstract

Community efforts in the computational molecular sciences (CMS) are evolving toward modular, open, and interoperable interfaces that work with existing community codes to provide more functionality and composability than could be achieved with a single program. The Quantum Chemistry Common Driver and Databases (QCDB) project provides such capability through an application programming interface (API) that facilitates interoperability across multiple quantum chemistry software packages. In tandem with the Molecular Sciences Software Institute and their Quantum Chemistry Archive ecosystem, the unique functionalities of several CMS programs are integrated, including CFOUR, GAMESS, NWChem, OpenMM, Psi4, Qcore, TeraChem, and Turbomole, to provide common computational functions, i.e., energy, gradient, and Hessian computations as well as molecular properties such as atomic charges and vibrational frequency analysis. Both standard users and power users benefit from adopting these APIs as they lower the language barrier of input styles and enable a standard layout of variables and data. These designs allow end-to-end interoperable programming of complex computations and provide best practices options by default.

I. INTRODUCTION

The number of quantum chemistry (QC) programs is continuously increasing, building a rich spectrum of capabilities where varied levels of accuracy, performance, distributed computing, graphics processing units (GPU)-acceleration, or licensing can be obtained. While this is generally beneficial to the end user, the diversity of custom input and output makes it difficult to switch between programs without learning the vagaries of each. Even the simplest research tasks using QC programs require mastering layers of expertise. On the input side, users must know what model chemistry will treat the molecular system of interest with adequate physics in tractable time, as well as pertinent modifications like density-fitting (DF), convergence, and active space, which are all questions of scientific expertise (I-a). [Labels of non-scientific (i.e., beyond I-a) input I-x or output O-x problems enumerated here are referenced by solutions in Sec. II.] They must know the names given by a QC program to the knobs that dial up the model chemistry and modifications, a question of domain-specific-language (DSL) expertise (here, “domain” is the QC software) (I-b). They would benefit from knowing the insider best-practice knobs that select the most efficient algorithms, approximations, and implementations specialized to the model chemistry, a question of program expertise (I-c). They must know the structure of the input specification by which the QC program receives instruction, a question of formatting and DSL expertise (I-d). Last on the input side, they must know the dance of files, environment variables, and commands to launch the job, a question of program operational expertise (I-e).

On the output and analysis side, further skills are required to process the program-specific ASCII or structured data file. Users must know what strings in the output mark the desired result, a matter of DSL expertise (O-a). If the targeted quantity is not explicitly printed but is derivable, they must know the arithmetic or unit conversion, a question of QC expertise (O-b). If individual energies or derivatives are to be combined to create a more sophisticated model chemistry (e.g., basis set extrapolation,^1,2 focal-point methods,^3–5 G3 or HEAT procedures,^6,7 or empirical correction⁸) or for molecular systems decomposition or perturbation [e.g., many-body expansion (MBE), counterpoise procedure,⁹ geometry optimization, or finite difference derivatives], users may be able to use routines built into QC programs (needing DSL expertise) but more generally must script the procedure themselves, requiring QC and programming expertise (O-c). More elaborately, they may want to combine the results with other programs—requiring recognizing and compensating for default knobs that render program results unmixable—a matter of QC and program expertise (I-f). Finally, users may hope that completed calculations can be stored and queried or even reused, matters of database expertise (O-d). Efforts to reduce non-scientific expertise burdens on the user have traditionally aggregated QC methods, geometry optimizers, and sundry procedures into vertically integrated “software silos” that, by increasing the DSL burden, risk locking users into one or a few programs. We pursue reducing the non-scientific expertise burdens on users by restructuring the QC software ecosystem while minimally disrupting longstanding, robust, and debugged computational molecular sciences (CMS) codes.

As a concrete example, in a high-accuracy spectroscopic application (see Sec. III), a user might want to include numerous small corrections, such as electron correlation effects beyond coupled-cluster (CC) through perturbative triples [CCSD(T)],¹⁰ basis set extrapolation, relativistic corrections,¹¹ and Born–Oppenheimer (BO) diagonal corrections.^12,13 The best implementation of each of these terms is not necessarily found in a single QC program. Careful users can evaluate different terms using different programs through a post-processing script to obtain a focal-point energy, but more complex procedures such as geometry optimizations¹⁴ are difficult due to tight coupling in QC programs that generally do not allow arbitrary gradients to be injected into the iterative optimizer.

Finally, in the emerging “data age” of computational chemistry, users increasingly want to treat QC results as a commodity, obtaining them on demand as part of complex workflows or generating datasets of millions of computations to use in force field (FF) parameterization, methodology assessment, machine learning (ML), or other data-driven pipelines. These users must be able to set up, execute, and extract computational results as easily as possible.

To address such challenges, the differing needs of workflows for uniform interaction with CMS codes have been separated into different layers of concern, resulting in the development of QCEngine and Quantum Chemistry Common Driver and Databases (QCDB).¹⁵

• •Consider a new QC practitioner learning which density functional theory (DFT) program best suits the local hardware or accessing the latest ML FF for many molecules. Such users would benefit from a uniform application programming interface (API) to evaluate these diverse capabilities without requiring knowledge of the specifics of each program’s DSL. QCEngine is designed to provide this uniform API and is an I/O runner around individual CMS codes’ core single-point capabilities. QCEngine communicates through a JavaScript Object Notation (JSON) Schema,¹⁶ denoted QCSchema, thus automatically generating program input files from a consistent and simple molecule and method specification.
• •Next, consider the systematic study of dipole moments at different levels of theory from different programs or a FF developer training on the many symmetry-adapted perturbation theory (SAPT) component results over thousands of molecules. These applications would benefit from output layout uniformity and programmatic access to detailed results. QCEngine covers these cases by harvesting binary, structured, or text output into standardized QCSchema fields.
• •Next, consider the maintainers of a CMS code whose users have been making the same formatting and incomplete input mistakes for the past decade and have been petitioning for quality-of-life features that would incur poor complexity-to-benefit ratio if implemented within the native framework and languages. These barriers to research would benefit from a shim layer in an easy and expressive language. QCDB provides a flexible input framework, helpful keyword validation, access to multijob procedures like MBE, and a place (besides documentation) to inject advice like context-dependent defaults.
• •Now, consider a spectroscopist modeling a molecule with a composite method or the QC beginner hoping to avoid learning multiple DSLs. These circumstances would benefit from uniformity of input and results across programs. QCDB compensates for variable defaults and conventions so that multi-program model chemistries can be safely defined and simple methods accessed interchangeably.
• •Finally, consider the experienced QC practitioner who writes inputs from memory and who turns keyword knobs as nimbly as organ stops but who would like to try another optimizer or an MBE procedure or not worry about capitalization and spaces today. This situation would benefit from a light hand in developing the QCSchema translation and common driver API so that existing expertise in direct interaction with CMS codes (DSL for keywords, for example) is applicable to these current projects.

In enabling uniformity at the input, output, and cross-program layers, both QCEngine and QCDB have striven to make their input predictable from customary input and to make customary output available.

Central to the ability of QCArchive¹⁷ and QCDB to provide generic I/O, driver, and database interfaces to CMS codes is a common standard QC data format. Of course, to develop such a standard information exchange format for all QC programs and to encourage its adoption by QC packages is a difficult approach for a single research group, or even a handful of research groups, to successfully prescribe to a broad developer community. However, here the Molecular Sciences Software Institute (MolSSI),¹⁸ funded by the U. S. National Science Foundation, provides a unique opportunity to sponsor community discussions and to advocate for standards. Members of our collaborative team and the codes represented have worked closely with MolSSI on their development of a QCSchema¹⁹ for quantum chemistry information exchange, and we have adopted it for QCEngine and QCDB.

There have been previous efforts to provide a unified interface to set up, drive, and analyze QC computations. For example, Newton-X²⁰ and FMS90^21–23 perform nonadiabatic dynamics computations using any of several QC programs. The Quantum Thermochemistry Calculator (QTC)²⁴ interfaces to a handful of QC programs to provide unified thermochemistry analysis functions independent of the QC data source. Especially tailored to deal with excited state optimizations is Pysisyphus, an external optimizer that localizes stationary points on potential energy surfaces by means of intrinsic reaction coordinate (IRC) integration, chain-of-state optimization, and surface walking for several QC codes through a uniform interface.²⁵ Among more general-purpose programs, Cuby^26,27 is a uniform driver and workflow manager that works with multiple QC and force field tools. Cuby allows the combination of methods across its interfaced programs and provides mixed quantum mechanics/molecular mechanics (QM/MM) and molecular dynamics capabilities. The WebMO project is another that drives several QC programs as backends from a largely unified web portal frontend.²⁸ Another popular tool is the Atomic Simulation Environment (ASE),²⁹ which provides a Python interface to more than 40 QC or force field codes, along with drivers for geometry optimization and transition state searching with the nudged elastic band method and analysis and visualization functions. A recipes collection (ASR)³⁰ supplies further spectroscopy and analysis tools. ASE and ASR are focused on solid-state computations; while molecular computations are also possible, they do not provide the level of detail required for the majority of quantum chemistry workflows. Compared to ASE, QCDB is more focused on high-accuracy quantum chemistry (providing, for example, built-in support for focal-point methods). Newer entrants to the field of computational chemistry workflow tools at the scope of QCArchive (rather than the narrower modular components QCEngine and QCDB discussed here) include AiiDA,^31,32 which at present is materials focused, and ChemShell,³³ which focuses on multiscale simulations. By interfacing with QCArchive, QCDB can also focus on high-throughput quantum chemistry and on creating large databases for force field parameterization and machine-learning purposes. Although not focused on running CMS codes, the cclib^34,35 and HORTON³⁶ projects also have extensive capabilities to regularize output and post-processing.

We describe the modular software built to facilitate interoperability, the community QC codes, and the technical challenges associated with an interoperability project in Sec. II. An example application demonstrating the use of multiple QC codes to perform very high accuracy computations of spectroscopic constants of some diatomic molecules is presented in Sec. III.

II. FEATURES AND DESIGN PHILOSOPHY

Discussed are the present software projects and their place in the CMS ecosystem in Sec. II A, interfaced software providing single-point energies and properties in Sec. II B, interfaced and built-in software providing more complex procedures in Sec. II C, how these are all linked by a common driver in Sec. II D, and further details about implementing interoperability in Sec. II E.

A. QCSchema and the quantum chemistry software ecosystem

The modular software components in our layered approach to QC interoperability and high-throughput computing are shown in Fig. 1. All are open-source projects, and community feedback and contributions through GitHub are welcome (links at Sec. V; QCEngine documentation includes the general process for adding a new QC program). The QCSchema¹⁹ definitions layer is foundational and encodes the community-developed data layouts and model descriptions useable in any language, from C++ to Rust to JavaScript to Fortran. Above that is the QCElemental³⁷ data and models layer that implements QCSchema and imposes a Python language restriction to gain sophisticated validation and feature-rich models. Next is the QCEngine³⁸ execution layer that adapts CMS codes for standardized QCSchema communication and imposes an execution environment restriction to gain easy access to many programs. Last is the QCFractal³⁹ batch execution and database layer that imposes some calculation flexibility restrictions to gain multi-site distributed compute orchestration and provide structured-data storage and querying capabilities. [This layer, beyond the scope of the present work, addresses (O-d).] Together these compose the QCArchive Infrastructure, the Python software stack that backs the MolSSI QCArchive project.^17,40 Enhancing QCEngine is the QCDB⁴¹ interoperability layer that imposes feature-registration and cross-program defaults restrictions to gain input uniformity and multi-program workflows.

FIG. 1.

Modular ecosystem around QCEngine and QCDB. QCEngine is the central, QCSchema-based QC program runner in the QCArchive Infrastructure software stack, while QCDB adds additional interoperability features atop it. User input routes to QC computations are shown as one or more turquoise boxes—“TXT” for a command-line interface, “PyAPI” for an interactive application programming interface in Python, or “JSON” for single-command QCSchema communication through command-line or Python.

QCElemental³⁷ (see Fig. 1) provides data and utilities (like a QCSchema implementation) useable by all QC packages. For data, it exposes NIST Periodic Table and CODATA physical constants through a lightweight API and provides internally consistent unit conversion aided by the external module Pint.⁴² QCElemental supports multiple dataset versions for CODATA and for properties such as covalent and van der Waals radii. Additionally, QCElemental provides a Python reference implementation for the MolSSI QCSchema data layouts, including Molecule (example is given in Snippet 2), job input specification AtomicInput [examples at Figs. 2(b)–2(d)], and job output record AtomicResult. In addition to enforcing the basic key/value data layout inherent to a schema, QCElemental uses the external module Pydantic⁴³ to collocate physics validation, serialization routines, extra helper functions (like Molecule parsing, alignment, and output formatting), and schema generation into a model for the QCSchema. Historically, many QCElemental capabilities were developed for QCDB in Psi4 and then refactored into QCElemental for broader community accessibility free from Psi4 and compiled-language dependence. QCEngine and QCDB use all the QCElemental capabilities mentioned, particularly for QCSchema communication and for uniform treatment of fragmented, ghosted, and mixed-basis molecules across differing QC program features.

SNIPPET 2.

QCSchema MOLECULE from Snippet 1. Translation described at Ref. 97.

FIG. 2.

Degrees of unifying access to quantum chemical calculations illustrated through an open-shell CCSD energy computation. Black text and gray shading are aspects not requiring user knowledge of multiple DSLs. See penultimate paragraph of Sec. II A for discussion.

QCEngine³⁸ provides a uniform execution interface whereby community CMS codes consume QCSchema AtomicInputs and emit AtomicResults via adaptors, called ProgramHarnesses. Depending on the degree of programmatic access a QC package provides, the ProgramHarness may be simple, as for a package that already provides a QCSchema interface; moderate, as for a package that supports a Python API or has serialized output, be it binary, Extensible Markup Language (XML), or JSON; or involved, as for an executable with ASCII I/O; further details may be found in Sec. II E 10. A typical ProgramHarness consists of taking an AtomicInput, translating it into input file(s) and execution conditions, executing it, collecting all useful output, parsing the results into an AtomicResult, and returning it to the user. A ProgramHarness is written to cover analytic single-point computations, namely, energies, gradients, Hessians, and properties, as discussed further in Sec. II B. Adaptors for more complicated actions are classified as ProcedureHarnesses and are discussed in Sec. II C. QCEngine additionally collects runtime data such as elapsed time, the hardware architecture of the host machine, memory consumption of the job, software environment details, and execution provenance (e.g., program, version, and module). As suggested by Fig. 1, adaptors written in QCDB have been migrated to QCEngine so that both projects access more QC codes and share the maintenance and development burden.

QCDB⁴¹ supplements QCEngine’s program and procedure capabilities with interoperability-enhanced ProgramHarnesses and multi-program procedures; furthermore, it links QCEngine calls into an interactive driver interface. From the user’s viewpoint, this layered approach to uniform QC computation is shown in Fig. 2 by an open-shell CCSD single-point energy. Running a QC code directly, as in Fig. 2(a), requires considerable DSL knowledge for method, basis, and keywords, not to mention details of layout and execution; essentially only the geometry (black text) is uniform. By molding the text inputs of Fig. 2(a) into the QCSchema data layout Fig. 2(b), QCEngine unifies the gray-shaded fields but still requires DSL from multiple codes. QCDB imposes more dependencies, like its own basis set library and utilities, to allow uniform basis specification and molecule symmetry as in Fig. 2(c). By imposing keyword registration and precedence logic, QCDB can provide the uniform and single-DSL input of Fig. 2(d). In practice, QCDB harnesses are minimal wrappers around QCEngine harnesses.

By choosing an entry point (software component in Fig. 1) and interface (CLI, Python API, JSON), external projects can satisfy a number of interoperability use cases: convention for data layout (stop after QCSchema), molecule string parsing (stop after QCElemental), uniform CMS execution (stop after QCEngine), tolerant Python interface to single venerable CMS code (QCDB), or multicode workflows (QCDB).

B. Program capabilities

For several community codes or programs [Fig. 3(i); not comprehensive] capable of computing analytic energies, gradients, or Hessians, the authors have written QCSchema adaptors for QCEngine known as ProgramHarnesses [Fig. 3(ii)]. The primary returns can be full scalars or arrays, as for most QC methods, or partial, as for dispersion corrections. So long as program communication fits into the AtomicResult data layout, semi-empirical and molecular mechanics programs can also formulate QCEngine adaptors. A summary of interfaced codes can be seen in Table I. QCDB asserts greater control over codes to assure consistent output values, so its capabilities are centered on CFOUR, GAMESS, NWChem, Psi4, and select partial calculators [Fig. 3(iii)]. Note that output harvesting capabilities (results available programmatically as opposed to text files) may lag behind those for input execution. A test suite that ensures matching values can be extracted from different programs has been established for both QCEngine and QCDB to document differing conventions (e.g., canonicalization for ROHF CC, all-electron vs frozen-core). Uncovered incorrect values or missing properties have been reported to the code developers for further investigation.

FIG. 3.

Layout and access pattern between selected existing and planned (marked by *) community quantum chemistry codes, QCEngine, and QCDB. Community codes (i) in a variety of languages are wrapped in QCSchema input/output by a QCEngine harness (ii) and (iv), which may be light (if the code has an API or structured output) or heavy (if only text output available). The QCDB harnesses (iii) and (v) add unifying and ease-of-use layers atop the QCEngine calls. Whereas analytic energies and derivatives are classified as programs (ii) and (iii) and call QC codes directly, multi-stage and post-processing jobs are written as procedures (iv) and (v) for composability and distributability and call programs in turn. The QCDB driver provides API access to both sets. Labels (a)–(d) correspond to the stages of unified input in Fig. 2.

TABLE I.

Interfaced programs in QCEngine and QCDB. For each, availability of one or more methods for energy (E), gradient (G), and Hessian (H) is shown, as well as collection of properties (e.g., one-electron energy or dipole) and wavefunction quantities (e.g., number of basis functions and orbitals). Symbols are present (), absent (), or inapplicable (). Non-QC programs are not suitable for QCDB. Program I/O is handled primarily through QCSchema API (Q), API (A), structured XML, JSON, binary (S), or text (T).

CMS program	QCEngine					QCDB			Cite	I/O
	E	G	H	Prop.	Wfn	E	G	H
Quantum chemistry
ADCC									44 and 45	A
CFOUR									46	TS
GAMESS									47	T
Molpro									48 and 49	S
MRChem									50 and 51	S
NWChem									52	T
Psi4									53	Q
Q-Chem									54	TS
Qcore									55	S
TeraChem									56 and 57	Q,T
Turbomole									58 and 59	T
Semi-empirical
MOPAC									60	T
xtb									61	Q
Molecular mechanics
OpenMM									62	A
RDKit									63	A
Analytical corrections
DFTD3									8 and 64	T
DFTD4									65 and 66	Q
gCP									67 and 68	T
MP2D									69 and 70	T
Machine learning inference
TorchANI									71–73	A

1. ADCC

The interface to ADCC allows for computations of excited states based on the algebraic-diagrammatic construction (ADC) scheme for the polarization propagator. Several methods are available, including ADC(2), ADC(2)-x, and ADC(3), together with the respective core-valence separation (CVS) and spin-flip variants. For all aforementioned methods, excitation energies and properties are accessible. The interface uses Psi4 to compute the SCF reference state first and then calls adcc via its Python API. A minimum adcc v0.15.1 is required.

2. CFOUR

Many CFOUR features are available to both QCEngine and QCDB, including most ground-state many-body perturbation theory and coupled-cluster energies, gradients, and Hessians: Hartree–Fock, MP2, MP3, MP4, CCSD, CCSD(T) with RHF, UHF, and ROHF references. Excited states are available for running but not parsing. Special features include CC with quadruple excitations through the NCC module, the ability to compute the diagonal Born–Oppenheimer correction using coupled-cluster theory, and, after revision, second-order vibrational perturbation theory (VPT2) (see Sec. II C 6). The interface generates text input and collects mixed text and binary output. A minimum CFOUR v2.0 is required.

3. GAMESS

The GAMESS interface for QCEngine and QCDB provides Hartree–Fock, DFT, MP2, and coupled-cluster methods. Special features include full configuration interaction. In the future, the GAMESS interface will also provide effective fragment potential (EFP) capability through potential file generation (see Sec. II C 7) and running pure EFP calculations on molecular clusters, energy ("gms-efp"). A particular complication for GAMESS is the controlled molecule and custom basis syntax, which led to QCDB feeding only symmetry-unique atoms and their full basis sets into the GAMESS input file. As QCEngine does not have symmetry capabilities, QCEngine-based GAMESS calculations are restricted to C₁. The interface generates text input and collects text output. The harness has been tested with the GAMESS 2017 R1 version.

4. Molpro

Energies and gradients are available in QCEngine from Hartree–Fock, DFT, MP2, CCSD, and CCSD(T) levels of theory, including some local methods. The interface generates text input and collects XML output. A minimum Molpro v2018.1 is required.

5. MRChem

Thanks to a harness to the MRChem software package, quasi-exact energies and selected properties in the multiwavelet, multiresolution basis are available with QCEngine. MRChem provides an efficient implementation for Hartree–Fock and DFT. Electric dipoles, quadrupoles, static and frequency-dependent polarizabilities, magnetizabilities, and NMR shielding constants are available. At variance with GTO-based quantum chemical software packages, the basis used in MRChem is adaptively refined: thanks to the multiwavelet framework, these results are exact to within the user-requested precision.⁷⁴ As a practical consequence, only the method keyword is required to define an input model to MRChem. JSON files are used to handle communication between QCEngine and MRChem. The harness can leverage the hybrid MPI/OpenMP parallelization of MRChem, provided suitable resources are available. A minimum MRChem v1.0.0 is required.

6. NWChem

The NWChem interface for QCEngine and QCDB provides a large selection of the quantum mechanical methods available, including Hartree–Fock, DFT, MP2, and coupled-cluster methods [both the code automatically derived and implemented with the Tensor Contraction Engine⁷⁵ (TCE) and the hand-coded implementations, where available]. Additional calculations available in the TCE include configuration interaction through single, doubles, triples, and quadruples level of theory and MBPT methods through the fourth order. Special features include CCSDTQ energies, excited states through equation of motion (EOM) coupled-cluster energies, and relativistic approximations. The interface generates text input and collects text output. The harness has been tested with NWChem v6.6 and v7.0.

7. Psi4

Essentially, all Psi4 features are available to QCEngine and QCDB, as Psi4 communicates natively in QCSchema (psi4 – qcschema in.json) and QCDB began as the Psi4 driver. These include conventional and density-fitted Hartree–Fock, DFT, MP2, and coupled-cluster methods. Special features are symmetry-adapted perturbation theory, coupled-cluster response properties, density-fitted CCSD(T) gradients, and optimized-orbital MP2, MP2.5, and MP3 energies and gradients. Wavefunction information is returned in QCSchema format. The interface generates JSON (QCSchema) input and collects JSON output. A minimum Psi4 v1.3 is required for QCEngine and v1.4 for QCDB.

8. Q-Chem

Energies, gradients, Hessians, and some properties are available in QCEngine at the SCF (Hartree–Fock and tens of DFT functionals) and MP2 levels (both conventional and density-fitted). The interface generates text input and collects mixed text and binary output. A minimum Q-Chem v5.1 is required.

9. Qcore

Energies, gradients, and Hessians are available in QCEngine from Hartree–Fock, DFT, and extended tight-binding (xTB). Qcore along with Psi4 are the two programs that can return wavefunction information in QCSchema. The interface generates JSON input and collects JSON output. A minimum of Qcore v0.7.1 is required.

10. TeraChem

TeraChem features two modes for driving computations via QCEngine: a standard text interface and a typed Protocol Buffers⁷⁶ interface. The former generates text input and collects text output to provide energies and gradients from Hartree–Fock and DFT levels of theory. A minimum TeraChem v1.5 is required.

TeraChem’s Protocol Buffers (TCPB) server⁵⁷ interface offers a second way to drive computations using QCEngine. It provides energies and gradients from Hartree–Fock and DFT levels of theory, molecular properties including dipoles, charges, and spins, and limited wavefunction data including alpha- and beta-spin orbitals and orbital occupations. The TCPB interface also accelerates calculations by performing GPU initialization routines once at server startup. As a result, subsequent computations can begin instantaneously, thereby providing substantial speed-up for small systems (∼10 heavy atoms) and minor speed-up for medium systems (∼100 atoms).⁷⁷ The TCPB interface requires the installation of an additional Python package tcpb⁷⁸ minimum v0.7.0 to power the QCEngine integration. Subsequent updates to the tcpb package will expand the set of properties and wavefunction data available from TeraChem via QCEngine.

11. Turbomole

Energies, gradients, and Hessians are available in QCEngine for Hartree–Fock, many DFT functionals, and define-fitted MP2, MP3, MP4, and CC2. Turbomole’s interactive define function for processing input proved an extra challenge to integrate with QCSchema. The interface generates interactive text input and collects text output. The harness has been tested with Turbomole v7.3 and v7.4.

12. XTB

The interface uses the Python API of XTB, which provides QCSchema support, to generate JSON (QCSchema) input and collect JSON output. A minimum of XTB v6.3 is required.

13. dftd3 and dftd4

A Python API to Grimme’s dftd3 executable for computing variants of -D2 and -D3 for arbitrary QCSchema Molecule with automatic or custom parameter sets has been available in Psi4 for several years.^8,79,80 This has been adapted as a ProgramHarness for QCEngine and QCDB. The interface generates text input and collects text output. A minimum of dftd3 v3.2.1 is required.

For the separate dftd4 software, the interface uses the Python API, which provides QCSchema support, to generate JSON (QCSchema) input and collect JSON output. A minimum of dftd4 v3.1 is required.

14. GCP

Energies and gradients are available for the geometrical counterpoise correction GCP program developed by Kruse and Grimme that corrects the inter- and intramolecular basis set superposition error (BSSE) in Hartree–Fock and DFT calculations.⁶⁸ It also offers the GCP-part of the “3c” correction used in composite methods like HF-3c or PBEh-3c.⁸¹ The interface generates text input and collects text output. The harness was tested with gCP v2.02.

C. Procedure capabilities

Whenever a quantum chemistry work sequence takes in QC-program-agnostic energies, gradients, Hessians, or properties (i.e., AtomicResults) but requires multiple ones (e.g., a finite difference derivative) or needs additional software [e.g., EFP potentials or symmetry-adapted linear combination (SALC) coordinates] or needs to take action in multiple stages (e.g., a geometry optimizer) or could combine AtomicResults from different programs (e.g., a composite method), it is classified in QCEngine or QCDB as a procedure [see Fig. 3(iv-v)]. Procedures are implemented in a ProcedureHarness to facilitate modularity and address O-c. Because procedures act upon generalized quantities, any code interfaced with QCEngine or QCDB gets all of the applicable procedures “for free.” Together, programs and procedures are elements that can be composed into workflows both simple (e.g., opt + freq + vib) or complex as in Sec. III.

Presently available in QCEngine are the geomeTRIC, PyBerny, and (Python) OptKing geometry optimizers, the first of which has been used extensively (>380k optimizations) by the Open Force Field⁸² community. Presently available or anticipated (*) for QCDB are the Composite, FiniteDifference,* ManyBody, diatomic, and vib routines inherited from the Psi4 recursive driver.¹⁴ The Psi4 OptKing geometry optimizer, written in C++, has been redeveloped in Python as a more versatile tool for future development and with the independence suitable for QCDB, while resp* and CrystaLattE* have been expanded from Psi4 to work with QCDB. Procedures makefp* and vpt2* make use of specially extractable features from GAMESS and CFOUR, respectively, and require installation of the parent code. Similarly, findif retains for the short term a dependence on Psi4. Note that the full capabilities from proven software components that were once or are presently partially or fully interfaced are in the procedure descriptions below. Procedures in QCEngine and QCDB have passed through the proof-of-principle stage and are presently being reworked and expanded into the below forms; current availability is limited.

1. Geometry optimizers

To be used by QCEngine or QCDB, a geometry optimizer must be able to take an input geometry in Cartesian coordinates and to take an arbitrarily sourced gradient and produce a next-candidate geometry displacement rather than be in control of both gradient and geometry-step stages. Regrettably, this eliminates most optimizers embedded in QC programs. Some alternatives are Wang’s geomeTRIC project,^83,84 which uses the TRIC coordinate system to specialize in interfragment and constrained optimizations, King’s OptKing,⁸⁵ which is a conventional IRC- and TS-capable QC optimizer, and Hermann’s PyBerny,⁸⁶ also a QC-focused optimizer. OptKing can apply flexible convergence criteria including those related to energy change and the maximum or root-mean-square of the gradient or displacement, and it has the most common settings for many embedded/native optimizers conveniently accessible as keywords. QCEngine presently has available geomeTRIC, PyBerny, and the Python OptKing, while QCDB only has the original C++ OptKing. After a planned driver update, all three Python optimizers will work with QCEngine and hence with QCDB. All optimizers communicate through schema, in particular, a QCSchema OptimizationInput that contains an ordinary AtomicInput as template for the gradient engine. Optimizations are called through QCEngine using qcng.compute_procedure({“input_molecule”: …, “keywords”: {“program”: “gamess”}, “input_specification”: {“model”: {“method”: “mp2”, “basis”: “6-31G”}}}, “geomeTRIC”) or qcdb.optking(“gms-mp2/6-31G”), where the latter can take as model chemistry any sensible combination of other procedures (i.e., qcdb.optking(“gms-mp2/[23]zapa-nr”, bsse_type=“cp”)).

2. vib: Harmonic vibrational analysis

The harmonic vibrational analysis routine is automatically run after any qcdb.frequency() computation.⁸⁷ Taking in a Hessian matrix, the molecule, basis set information, and optional dipole derivatives, vib() performs the usual solution of whole or partial Hessians into normal modes and frequencies, reduced masses, turning points, and infrared intensities, all returned in schema. Other features include rotation-translation space projection, isotopic substitution analysis, Molden output, and a full thermochemical report incorporating the best features of several QC programs’ vibrational output.

3. FiniteDifference: Derivatives

As QCEngine and QCDB are focused on interfacing QC programs’ analytic quantum chemical methods or unique features, user calls for non-analytic derivatives in QCDB are by default routed through the finite difference procedure.⁸⁷ This procedure (originally from Psi4) performs three- or five-point stencils for gradients and Hessians (full or partial), communicates through schema, and is parallelism-ready. The alternative of letting the internal finite difference of a QC program run and then parsing output files for multiple energies or gradients has been implemented in some cases, but this is not preferred (nor for internal geometry optimization).

4. Composite: Composite method and basis extrapolation treatments

Whenever an additive model chemistry is designated that involves differences of method (i.e., a focal point analysis or “delta” correction), basis [i.e., a complete basis set (CBS) extrapolation], keywords (e.g., all-electron minus frozen-core), or any combination thereof, the Composite procedure can encode it. Here, one can mix QC programs to perform conventional coupled cluster with CFOUR and DF-MP2 with Psi4, for example. Implementing new basis extrapolation formulas is simple, and it works on gradients and Hessians, as well as energies. If a subsidiary method energy can be obtained in the course of a target method, the procedure will recognize and avoid the unnecessary calculation (thus a TQ MP2 correlation energy extrapolation atop a DTQ HF energy will do 3, not 5, jobs). Input specification can be through API, schema, or strings (a user-friendly example is in the final paragraph of Sec. II E 5). All Composite communication is through schema, and the procedure is parallelism-ready.

5. ManyBody: Fragmentation and many-body approaches

All fragmentation and basis set superposition error (BSSE) treatments are collected into the ManyBody wrapper for many-body expansion (MBE) inherited from Psi4. The fragmentation pattern known from the QCSchema Molecule is applied to determine the degree of decomposition into monomers, dimers, etc., up to the full molecule, or the user can set the max_nbody level. Total quantities (energy, gradient, or Hessian) and interaction quantities are accessible through uncounterpoise (noCP), couterpoise (CP), and Valiron–Mayer functional counterpoise (VMFC) schemes.^9,88,89 Geometry optimization with many-body-adapted quantities is also available. The wrapper can act on uniform single-method quantities or apply different model chemistries to each expansion level or interface with Composite or FiniteDifference results or both. All ManyBody communication is through schema, and the procedure is parallelism-ready.

6. vpt2: Anharmonic vibrational analysis

Anharmonic vibrational analysis has long been a feature of CFOUR. It requires a high-quality harmonic frequency procedure as input. It then performs further Hessian computations at geometry displacements along the normal coordinates. These are then combined into a third-order and partial fourth-order potential followed by vibrational analysis. Although many analytic Hessians are available in CFOUR itself, the qcdb.vpt2() procedure focuses on the formulation through analytic gradients, as being suited to distributed computing and generalization to program-generic gradients. Thus, CFOUR is a helper program that, with the QCDB procedure, can perform anharmonic analyses of, for example, CCSD (from CFOUR gradients called through QCDB), DFT (from another QC program’s gradients), or CBS (that produces a generalized gradient). All qcdb.vpt2() communication is through schema, and the procedure is parallelism-ready.

A complication is that the vpt2() procedure is essentially a series of invocations of CFOUR subcommands like xcubic, which expect files in native JOBARC form with energies, dipoles, and gradients. To accommodate this, QCDB uses Python modules to write imitations of the native files in string representations of binary form, which is lossless. Hence, a Psi4 DFT gradient is represented as a JOBARC to pass through the CFOUR mechanisms.

7. makefp: EFP library generation

The two engines for computing EFP interactions, LibEFP^90,91 and GAMESS,⁴⁷ use the same parameter file for storing the EFP potential at a given basis set and monomer geometry. Only GAMESS can generate that file, and the routine has been wrapped by QCDB for access through qcdb.makefp(). The resulting .efp file contents are returned in the QCSchema output and so are available for writing to a personal library or to feed to subsequent qcdb.energy(“gms-efp”) (or “lefp-efp” or “p4-efp”) calls to determine non-covalent interactions between EFP fragments. Certain EFP integrations await expansion of QCSchema Molecule.

8. Diatomic: Spectroscopic constants

The electronic potential analysis for diatomic molecules has long been encoded in Psi4 as a post-processing procedure from a list of electronic energies along the interatomic coordinate. This has been reworked as a procedure and is demonstrated in Sec. III.

9. RESP: Charge fitting

The restrained electrostatic potential (RESP) charge model⁹² is obtained by an iterative fitting of the electrostatic potential emerging from QC calculations on one or several conformers of a molecule to a classical point-charge potential. An existing RESP plugin^93,94 drives the property calculations with Psi4, and this has been expanded to alternately draw from GAMESS using the QCDB API.

10. CrystaLattE: Crystal lattice energies

The process of estimating the lattice energy of a molecular monocrystal via the many-body expansion is encoded in the CrystaLattE software.^95,96 Starting with extracting a subsample from a cif file, the program handles fragmentation into dimers, trimers, etc., identifies unique N-mers, prepares QC inputs, and keeps track of many-body results into final quantities. Although the thousands of component calculations mean that it will only become practical after QCDB upgrades to the distributed driver (see Sec. II D), CrystaLattE is ready to be integrated in serial mode in QCDB.

D. QCDB common driver

The driver component of QCDB [Fig. 3(vi)] is the fairly lightweight coordinator code that (1) facilitates the interactive API of set_molecule, set_keywords, energy ("nwc-b3lyp/6-31g*"), print(variable("b3lyp dipole")) rather than communicating through QCSchema; (2) imposes cross-QC-program suggestions like tightening convergence for higher derivatives or for finite difference; and (3) weaves together procedures and programs so that optimize("mp6") commences finite difference or energy("ccsd/cc-pv[tq]z", bsse_type="vmfc") runs ManyBody, Composite, and program harnesses in the right sequence. The driver is primarily concerned with processing user-friendly input [“User API” in Fig. 3(vi)] into QCSchema as directly as possible and then routing it into a program harness [Fig. 3(iii) for analytic single-points] or through procedures [Fig. 3(v)] on their way to program harnesses (e.g., for Composite, FiniteDifference) or through procedures after program harnesses [e.g., for resp(), vib()]. In order to make good use of the QCDB common driver, a QC program must register capabilities and information. These include the available analytic methods (for appropriate use of finite difference), insider best-practice options from the program’s developers (see Sec. II E 9), and all keywords and their defaults (for flexible and informative keyword validation through Python).

The common driver is based upon the Psi4 v1.0 recursive driver described in Ref. 14 that unifies many complex treatments (e.g., MBE and CBS) into a few user-facing functions that focus on what, not how. After polishing in Psi4 v1.5, a new distributed driver with the same interface but tuned to QCSchema communication and embarrassingly parallel execution through QCArchive Infrastructure will be substituted. See Sec. IV and Fig. 2 of Ref. 53 for details.

E. Technical aspects to interoperability

Details of specifying and running QC computations, particularly arbitrating the expression of QCSchema by QCEngine and QCDB, are collected below. Readers who prefer a software overview should proceed to Sec. III. Symbols like (I-b) mark strategies for overcoming or unifying the expertise barriers to using QC programs enumerated in the initial paragraphs of Sec. I.

1. Memory

User specification of memory resources is managed by QCEngine and is outside the QCSchema. By default, the job is given all of the compute node’s memory (less some buffer). If user-specified, input units are in GiB, e.g., qcdb or qcng.compute(…, local_options={"memory": 10}) (I-b). In either case, the memory quantity is translated into DSL keyword names such as memory_size and mem_unit for CFOUR. Because QCEngine exercises total control over memory, any specification misplaced as a keyword into QCSchema is ignored and overwritten in QCEngine or raises an error if conflicting in QCDB. An exception is cases like NWChem, where aggregated memory is managed by QCEngine but distribution between heap, stack, and global is editable through keywords (e.g., memory__total or memory__stack).

2. Disk

The working directory and execution environment are also governed by QCEngine, and user modifications are outside QCSchema. Each job is run in a quarantined scratch directory created for it and populated by input and any auxiliary files. Execution occurs through Python subprocess (or less often through Python API). Output files and any program-specific files in text or binary format (including the generated input) are collected and returned in QCSchema fields before scratch directory deletion (I-e).

3. Parallelism

The execution flags or environment variables that control CMS program parallelism and their single- or multi-node capabilities are built into their respective QCEngine harnesses. A job gets the full single-node resources (max cores and near-max memory) assigned to it by default; multinode execution (only for NWChem at present) requires explicit specification. Assigning instead an optimal portion of the full resources on the basis of method and memory could be implemented in a harness, but none presently do. User specification of parallelism is managed by QCEngine and is outside QCSchema [e.g., qcdb or qcng.compute(…, local_options={"ncores": 4})] (I-e).

4. Molecule specification

Molecule specification is the most important aspect that QCEngine and QCDB control via QCSchema to the exclusion of a program’s DSL. The QCSchema Molecule can store mass, isotope, charge/multiplicity, fragmentation, ghostedness, and connectivity information (and more), along with the basic element and Cartesian geometry data (I-d). All quantities are stored in amu or Bohr to avoid imprecision from multiple unit conversions through different revisions of physical constants.

Initializing a molecule can occur through a variety of string formats (of Cartesian coordinates) or directly by arrays. Extensive validation and application of physics-based defaults follows such that string Snippet 1 becomes (Ref. ⁹⁷ for details) the schema Snippet 2. In the QCDB API, molecules can additionally be specified via Z-matrix, mixed Cartesian/Z-matrix, and with variable and deferred coordinates. QCSchema Molecule holds almost all data relevant to molecular system specification in QC, including EFP fragments, which are parseable without additional software and are stored in a secondary object. Items that appear in the molecule specification sections of some programs but do not fit in QCSchema Molecule, such as the stars signaling optimizable internal coordinates in CFOUR, reside in an extras section. (EFP and extras are future extensions.)

SNIPPET 1.

A string molecule input with complicating mass number, fragments, and implicit multiplicity.

Like memory or other aspects monopolized by QCSchema, user specification of the molecule in the DSL through keywords (e.g., scf__nopen in NWChem or contrl__icharg in GAMESS) is ignored and overwritten in QCEngine or raises an error if inconsistent in QCDB.

A requirement for combining vector data from multiple jobs is that the data be in a common frame of reference. Although each QC program has a standard internal orientation, these can be different between programs or between input specifications, and not all programs can return quantities in an arbitrary input frame and atom ordering. To smooth over inconsistent capabilities, the input geometry and the output geometry are both collected from output data, and an aligner computes the displacement, rotation matrix, and atom mapping needed to transform between them. Then, any vector results have the appropriate transformations applied so that all results in AtomicResult are in input orientation (O-a). This occurs for both QCEngine and QCDB when the Molecule fields fix_com and fix_orientation are True. (Here, “fix” is used in the “fasten” sense, not the “repair” sense.) When False, QCEngine returns in program native frame, while QCDB returns in Psi4 native frame.

5. Methods

Perhaps the most compelling element of QCSchema is the ability to request methods by a single string rather than piecemeal (e.g., "blyp-d3(bj)", "mp2", "cis" in place of {"method": "blyp","dft_d": "d3_bj"}, {"mplevl": 2}, {"calclevel": "hf","excite": "cis"}), thereby closely tying results to the model section (with subfields method and basis) of the data layout (barring algorithm, space, auxiliary basis set choices). As far as possible, all method specification and no extraneous information are consolidated into the atomicinput.model.method field. This is the primary translation effort of each QCEngine harness, as shown by the uniformity of the field in Fig. 2(b). In calling QCEngine, the user supplies the canonical method name (I-b). There is no compensation for program peculiarities; for example, "b3lyp" returns different answers if submitted to programs that have made a different choice of VWN3 vs VWN5, consistent with the principle that users can translate an input directly into QCSchema.

A complication to this principle is when programs conflate non-method information like algorithm (e.g., rimp2) or alternate code paths (e.g., task tce energy) into the primary method call. To maintain QCSchema integrity for model.method, the project invents top-level keywords like {"qc_module": "tce"} to allow deliberate choice of the TCE over hand-coded CC in NWChem and {"mp2_type": "df"} to instruct DF in GAMESS, NWChem, or Q-Chem. Keyword qc_module can also control choice of VCC/ECC/NCC in CFOUR and DFMP2/DFOCC/DETCI in Psi4, although these also have local knobs cfour_cc_program and psi4_qc_module.

Method specification in QCDB is similar to QCEngine except a compound program-method argument like optimize("nwc-mp2") is used. This difference is historical and endures for ease of specifying composite model chemistries like gradient("p4-mp2/cc-pv[56]Z + d: nwc-ccsd/cc-pv[tq]z + d: c4-ccsdtq/cc-pvdz")⁹⁸ employing Psi4, NWChem, and CFOUR for different stages. Additionally, QCDB tests the major methods to ensure the same string yields the same result (I-f). It also maintains a list of capabilities, so, for example, ROHF CCSD in NWChem can be automatically routed to TCE [see Fig. 2(d)]. User specification of method information in keywords instead of through the model field is overwritten without warning in QCEngine, while in QCDB, contradictory information yields an error.

6. Basis sets

Notwithstanding the curation efforts of the Basis Set Exchange⁹⁹ (BSE), every QC program maintains an internal library of basis sets with uneven upstream (from the basis set developer) updates applied, uneven downstream (by the program owner) specializations applied, and different spellings for accessing a given basis, not to mention different data formats. In QCEngine, only the internal library of a program is used, accessed from the atomicinput.model.basis field. Thus, due to DSL, the same string value directed toward different programs can lead to different results, and different strings can lead to the same results, as in Fig. 2(b). To allow consistency between programs and to reduce user DSL demands, QCDB pulls basis sets from a single library (Psi4’s in .gbs format, which is amply stocked with Pople, Dunning, Peterson, Karlsruhe, and other orbital and fitting basis sets) and performs the translation into the custom per-atom specification and format for each program, including setting spherical or Cartesian for d-shells and higher according to basis set design. In this way, a standard case-insensitive label and a consistent interface to custom and mixed basis sets is available (I-b). Alternatively, QCDB can act like QCEngine to access a program’s internal basis set library through program-specific keywords (e.g., set gamess_basis__gbasis accd vs set basis aug-cc-pvdz). While the Psi4 basis set library is used at present, future work will switch to the new MolSSI BSE.

7. Execution

Apart from CMS programs, QCEngine requires only QCElemental and some common Python packages. It is readily installed by conda install qcengine -c conda-forge or pip install qcengine. Execution occurs through CLI or one-call API with JSON-like input. For example, if AtomicInput specification {…, "model": {"method": "ccsd","basis": "aug-cc-pvdz"}} was in a file, qcengine spec run cfour would run CFOUR and return QCSchema AtomicResult (I-e). If the specification was a dictionary in a Python script, then qcengine.compute(spec, "cfour") produces the same results, as in the “execution” column of Fig. 2(b). QCEngine can be run through a queue manager, but for more than incidental jobs, users should consider the job orchestration capabilities of QCFractal.

QCDB requires only QCEngine and is installed similarly by conda install qcdb -c psi4. Execution modes CLI and one-call API are called analogously, only replacing qcng by qcdb (and ccsd by c4-ccsd) as shown in Figs. 2(c) and 2(d). Additionally, though, QCDB can function through an interactive driver API to reuse molecule and keyword sets and perform more complex sequences. This is shown in Snippet 3 that scans an energy potential and then performs a computation at the optimum distance at a better level of theory. This is analogous to the PsiAPI mode in Psi4. A simplified, plain-text input that gets processed into the API and is analogous to the PSIthon mode of Psi4 will be available after further integration with Psi4; an example is at Snippet 4.

SNIPPET 3.

An interfragment potential energy scan followed by composite energy in QCDB.

SNIPPET 4.

Contradictory input opportunities.

8. Modes

QCDB operates in two modes, which treat keywords, particularly keyword defaults, differently. QCDB supports distinct modes of operation to tailor its capabilities toward driver integration of multiple programs (when unified results are needed) or toward interfacing a single program (when user familiarity is preferred). Most controlling is the driver or unified mode, which endeavors to elicit from different QC programs identical results out of identical input conditions (roughly the combination of method, basis, reference, active space, and integrals treatment) (I-f). Here, the driver imposes QCDB-level defaults such as non-DF algorithms, all-electron spaces, and graduated convergence criteria for energy vs analytic derivative vs finite difference derivative. This mode is required for multi-program procedure runs [e.g., energy("p4-mp2/cc-pv[tq]z + d:c4-ccsd/cc-pvtz")] and is active by default.

Another mode, denoted sandwich since the QCDB pre- and post-processing is less intrusive, is for users focusing on a single QC program who want the driver routines, method mapping [e.g., energy("gms-ccsd(t)",bsse_type="vmfc")], and I/O-wrapping advantages of QCDB but do not want surprise resets of their accustomed defaults. Driver-suggested QCDB-level (e.g., frozen-core), driver-level (e.g., graduated derivative convergence), and best-practices (e.g., module selection) defaults are all turned off. This mode is effectively how QCEngine runs.

Some background facts to illustrate the modes:

• •For the default MP2 algorithm, Psi4 uses DF, while CFOUR, GAMESS, NWChem, and QCDB use CONV.
• •The CFOUR, GAMESS, NWChem, Psi4, and QCDB default HF density convergences are 10⁻⁷, 10⁻⁵, 10⁻⁴, 10⁻⁸, and 10⁻⁸, respectively.
• •For the CCSD energy from CFOUR, the default CC module is VCC, while QCDB best-practice is ECC.
• •The NWChem default task ccsd energy does not run for open-shell, while QCDB uses the CCSD module for RHF and TCE module for ROHF.
• •GAMESS freezes core by default, while CFOUR, NWChem, Psi4, and QCDB correlate all electrons.

In the unified mode, energy("gms-mp2") and energy("p4-mp2") both run all-electron MP2 without DF and with 10⁻⁸ convergence. After setting ROHF, energy("c4-ccsd") runs through ECC, and energy("nwc-ccsd") runs through TCE, again both HF to 10⁻⁸ and all-electron. In contrast, sandwich mode energy("gms-mp2") produces a conventional frozen-core MP2 energy converged to 10⁻⁵, while energy("p4-mp2") produces a DF all-electron value converged to 10⁻⁸. In the ROHF CCSD case, the CFOUR job runs as all-electron through VCC with HF converged to 10⁻⁷, while the NWChem submission declines to run.

9. Keywords

QC programs have hundreds of keywords controlling their operation on matters of substance (e.g., RAS3), strategy (e.g., DIIS), computer science (e.g., INTS_TOLERANCE), and research convenience (e.g., DFT_NEW). The variety in spelling and text arrangement by which the same ideas are communicated to different QC programs is staggering (and a considerable barrier to trying new codes). The necessity to represent any (single-stage, single-program) input file as QCSchema requires mapping rules so that a user familiar with the native DSL can readily translate into the key/value representation of an AtomicInput’s keywords field. The primary guideline is that the right-hand side value must be a simple data quantity in natural Python syntax (e.g., CFOUR’s 3-1-1-0/3-0-1-0 becomes [[3, 1, 1, 0], [3, 0, 1, 0]]), and the left-hand side key is a string that encodes any level of nesting with double-underscore (e.g., GAMESS’s contrl__scftyp or NWChem’s dft__convergence__density). A present/absent keyword (as opposed to a key/value pair) becomes a boolean, such as NWChem scf__rohf. The ProgramHarness handles formatting the keywords field (back) into the input grammar (I-d), including quashing unnecessary case-sensitivity (e.g., Qz2p converts to lowercase for CFOUR, while a filename option passes unchanged). For QCDB, prefixing a keyword by program name targets it toward a particular program; hence, reference becomes cfour_reference or psi4_reference.

The greatest challenge to mapping rules is that some programs have an input structure that blurs module nesting vs keyword name vs keyword value. An extra mapping rule not strictly required by QCEngine is for keywords to be independent and granular such that they are one-to-one with other programs, not overworked like dft__grid={"lebedev": (99, 11), "treutler": True} (insufficiently granular) nor underworked like scf__rhf=False plus scf__uhf=True (insufficiently independent). QCDB uses internal aliasing and mutually exclusive groups to help keyword specification be intuitive for native users.

Making a QCSchema fed to multiple programs produce uniform output is not within the scope of QCEngine. Barriers to accessing multiple QC backends through a single DSL or, more intricately, to compatibly mixing backends include (a) heterogeneous control knobs across QC programs each with its own keyword set and (b) incompatible results due to different defaults yielding slightly different answers. QCDB takes up the task of uniting keywords into a single DSL for a further layer of interoperability. Unlike QCEngine, QCDB registers valid keywords for each QC program and can apply custom validation functions to each. Additionally registered are unified keywords so that, for example, setting REFERENCE is translated into CFOUR_REFERENCE or GAMESS_CONTRL__SCFTYP, as shown in Figs. 2(c) and 2(d) (I-b, I-f). As mentioned above, insisting on granular keywords for the QCSchema representation allows cleaner mapping between QC programs. As mentioned below, QCDB also encodes best-practice keywords to allow shorter inputs, context-dependent defaults, and bridging the developer-user knowledge gap. QCSchema or QCDB API offer ample opportunities for users to submit contradictory input specification, several of which are shown in Snippet 4.

QCDB resolves competing keyword suggestions and requirements by the user, driver, schema, and best practices into a final keyword set that is passed to QCEngine for final formatting. Because of this step, incompatible keywords pass without warning in QCEngine, while in QCDB, contradictory information yields an error.

Codebase authors know best how to run a computation, but they may have conveyed that knowledge only through documentation and forum posts. Due to the unwieldiness of large legacy codebases and the circuity of research (and the burden of backward compatibility), it can happen that a method needs several keywords to express it or that valuable approximations or code-routing do not get turned on by default. Due to its layered Python/C++ structure, Psi4 naturally has a place to express such “best-practice” defaults based on method, basis, system size, etc. The advantage is that simple method + basis inputs yield production-grade results. Thus, QCDB takes advantage of working with codebase authors and the intermediate Python layer to implement best-practice keywords based on available calculation data (I-c). These take the form of routing to the best (or only capable) module for a given method, reference, derivative level, and active space; of supplying sensible defaults such as the number of electrons or roots; of tuning convergence to the derivative and needed precision (analytic vs finite difference) at hand; or of specifying C₁ or highest-Abelian symmetry to modules with symmetry restrictions. Such options can be overridden by the user and can be disabled in sandwich mode (Sec. II E 8). These defaults are themselves subject to change as recommendations evolve, but their state is readily viewed in program inputs.

10. QCVariables

The QC output stream, whether ASCII, binary, or structured, is read immediately after program execution. Scalar and array result quantities, such as PBE TOTAL ENERGY, MP4 CORRELATION ENERGY and PBE TOTAL GRADIENT, CCSD DIPOLE, are extracted and held as significant-figure-preserving floats or NumPy arrays, respectively, and are known collectively as QCVariables (O-a). Extraction uses the most precise available source, whether the standard output stream or available auxiliary files (e.g., CFOUR GRD). The internal geometry is always collected, and any vector results are manipulated in concert with it, as described in Sec. II E 4. For QCEngine, many of the same harvested quantities are directed into QCSchema AtomicResultProperties lists. Results are available programmatically through qcdb.variable("mp2 total energy") or atomicresult.properties.mp2_total_energy in QCDB and QCEngine, respectively.

A mild vexation in QC output files is that they contain different quantities such as total vs correlation energy or opposite-spin vs triplet energy that are interconvertible but not directly comparable. QCVariables enforce the consistency of common QC definitions and encode common combining rules (O-b). They are applied in post-processing to ensure that a maximum of data gets harvested from each run, that exactly the same quantities are collected from each QC program, and that trivially defined methods such as SCS(N)-MP2 and B3LYP-D3(BJ) need not clutter either the QC code or its parsing.

Using binary representations of floats rather than truncated strings from output files is a powerful argument for API integration rather than parsing. Binary representation is essential when dealing with many numbers with slight differences, such as finite differences or MBE sums. Programs with Python APIs (and that use APIs for internal inter-language transfer like between C++ and Python in Psi4) can transfer data with full precision; for QCEngine, these are, for example, adcc, OpenMM, RDKit, TorchANI, dftd4, Psi4, TCPB TeraChem, and xtb. Of these, the last four have implemented QCSchema directly for API access. An intermediate step is to use structured output like XML or JSON from Molpro, MRChem, and Qcore. For certain programs, a combination of reading available binary files (e.g., 99.0 for return energy in Q-Chem and JOBARX/JAINDX for certain QC results and organizational data in CFOUR) and text parsing is employed. Results from other programs are collected solely through text parsing: e.g., dftd3, GAMESS, gCP, MOPAC, mp2D, NWChem, the classic interface to TeraChem, and Turbomole. Although results are collected into QCSchema from QC programs at the greatest accessible precision, in order to maintain that precision among the data transfers and storage of the QCDB and QCArchive Infrastructure ecosystem, the QCElemental implementation of QCSchema (nominally a JSON Schema,¹⁶ which does not handle binary or numpy.ndarray) includes MessagePack¹⁰⁰ serialization.

III. EXAMPLE: DIATOMIC SPECTROSCOPIC CONSTANT FITTING

With contemporary QC software, it is entirely possible to approach the ab initio limit in the description of diatomic molecules.¹⁰¹ Such spectroscopically accurate calculations require extrapolating to the full configuration interaction and complete basis set limits under the non-relativistic Born–Oppenheimer (BO) approximation, followed by usually negligible corrections to account for both relativistic effects and the BO approximation itself. Not only does this type of calculation present a remarkable computational challenge [as it is significantly more expensive than CCSD(T), the usually sufficient target of quantum chemistry], it can also be practically difficult to incorporate multiple corrections and extrapolations into a workflow. While all of the necessary features are present across various QC software packages, no single package implements everything (let alone has the best implementation). Furthermore, enforcing consistent geometries, basis sets, convergence criteria, frozen orbitals, etc. between programs is a cumbersome, often error-prone task. The QCDB driver remedies this problem by providing an easy-to-use Python interface to multiple QC programs.

To showcase this capability of the QCDB driver, the ground states of a few diatomic molecules (BH, HF, and C₂) are optimized at essentially the ab initio limit, and spectroscopic constants are computed and compared to experiment. Previous studies estimating the ab initio limit for the full set of standard spectroscopic constants of these molecules have been reported (see, e.g., Refs. ^102–104). The present study provides improved treatments for some of the small corrections and/or includes more correction terms. Here, we include corrections for electron correlation beyond CCSD(T), basis set effects beyond an already high-quality core-valence quadruple/quintuple-ζ extrapolation, relativistic effects, and the Born–Oppenheimer diagonal correction using four different QC programs through the unified QCDB interface. The effect of each correction is examined separately, as well as the cumulative effect of all corrections. Understanding the cost and importance of each correction is helpful for designing reasonable extrapolations for larger systems.

A spectroscopically accurate model chemistry energy (E_Total) is defined as a base energy (E_Base) with five separate corrections,

$$\begin{array}{ccc}\hfill E_{\text{Total}}& =\hfill & E_{\text{Base}}+\mathrm{\Delta }{E}_{\text{Basis}}+\mathrm{\Delta }{E}_{\text{DBOC}}+\mathrm{\Delta }{E}_{\text{Rel}}\hfill \\ \hfill & \hfill & +\mathrm{\Delta }{E}_{\text{CCSDTQ}}+\mathrm{\Delta }{E}_{\text{FCI}}.\hfill \end{array}$$ (1)

Each energy and the QC program(s) used to obtain it is defined in Table II.

TABLE II.

Composite level of theory for spectroscopic constants and associated QC programs.

Name	Method	Program
E Base	CCSD(T)/cc-pCV[Q5]Z	NWChem
ΔEBasis	MP2/(aug-cc-pCV[56]Z − cc-pCV[Q5]Z)	Psi4
ΔEDBOC	CCSD/cc-pCVDZ	CFOUR
ΔERel	X2C-CCSD(T)/cc-pCVTZ	Psi4
ΔECCSDTQ	[CCSDTQ − CCSD(T)]/cc-pVTZ	CFOUR
ΔEFCI	(FCI − CCSDTQ)/cc-pVDZ	GAMESS/
		CFOUR

The rovibrational spectrum of a diatomic molecule is often expressed with Dunham’s expansion,

$$E_{\nu J}=h\sum _{kl}{Y}_{kl}{\left(\nu +\frac{1}{2}\right)}^k{\left[J(J+1)\right]}^l.$$ (2)

The first few Dunham coefficients correspond to well-studied spectroscopic constants,

$$Y_{10}={\omega }_e,Y_{20}=-{\omega }_e{x}_e,Y_{01}=B_e,Y_{02}=-{\overline{D}}_e,Y_{11}=-{\alpha }_e.$$ (3)

The following truncation of the expansion is used to describe a diatomic:

$$\begin{array}{ll}\hfill E& \approx U(r_e)+h{\omega }_e\left(\nu +\frac{1}{2}\right)+h{B}_{e}J(J+1)-h{\omega }_e{x}_e{\left(\nu +\frac{1}{2}\right)}^2\hfill \\ \hfill & -h{\alpha }_e\left(\nu +\frac{1}{2}\right)J(J+1)-h{\overline{D}}_e{J}^2{(J+1)}^2.\hfill \end{array}$$ (4)

The spectroscopic constants are then describable in terms of the electronic PES U(r) and its derivatives,

$$I_e\equiv \mu r_e^2{B}_e\equiv \frac{h}{8{\pi }^2{I}_e}{\omega }_e\equiv \frac{1}{2\pi }{\left[\frac{U^{\prime \prime }(r_e)}{\mu }\right]}^{1/2},$$ (5)

$${\omega }_e{x}_e\equiv \frac{B_e^2{r}_e^4}{4h{\omega }_e^2}\left[\frac{10B_e{r}_e^2{[U^{\prime \prime \prime }(r_e)]}^2}{3h{\omega }_e^2}-U^{iv}(r_e)\right],$$ (6)

$${\alpha }_e\equiv \frac{2B_e^2}{{\omega }_e}\left[\frac{2B_e{r}_e^3{U}^{\prime \prime \prime }(r_e)}{h{\omega }_e^2}+3\right]{\overline{D}}_e\equiv \frac{4B_e^3}{{\omega }_e^2}.$$ (7)

Note that these are all “equilibrium” constants, i.e., they are with respect to the bottom of the potential well (but with inclusion of the Born–Oppenheimer diagonal correction).

Accessed through the QCDB interface, the Psi4 $\mathtt{d}\mathtt{i}\mathtt{a}\mathtt{t}\mathtt{o}\mathtt{m}\mathtt{i}\mathtt{c}$ procedure fits a set of points [r, E(r)] to this truncation, solving for the spectroscopic constants via a least-squares optimization.¹⁰⁵ This procedure was used in the following way for each diatomic:

• 1.Through the QCDB driver, E_Total was calculated at seven values of r, spaced 0.005 Å apart and centered approximately at the minimum of the PES. The spectroscopic constants were calculated with Psi4, including an approximate r_e.
• 2.This seven-point calculation was repeated using the approximate r_e from the first step as the central point. The spectroscopic constants calculated from these PES points are those tabulated here.

Basis sets with spherical harmonics were used in all calculations, and basis set coefficients were standardized across all programs via QCDB. Electrons in core orbitals were frozen for computations using the cc-pVXZ basis set family, which lack core correlation functions. Energies were converged to at least 10⁻¹⁰ Hartrees in all programs. Even tighter convergence would be beneficial for the numerical differentiation performed in the fitting. Numerical tests suggest that this precision in energy can lead to uncertainties in α_e [proportional to U‴(r_e)] and ω_ex_e [proportional to U^iv(r_e)] as large as 0.0001 and 0.2 cm⁻¹, respectively.

The calculations of all diatomics and spectroscopic constants are presented in Table III, and the results for r_e and ω_e are shown in Fig. 4 for easier analysis. Prior to discussing the chemical and computational implications of these results, it is worthwhile to first note that the corrections for BH closely match those of a previous study¹⁰³ by Temelso et al. (which used a similar but less exact extrapolation). This validates these results from a software perspective: each program must be using correct geometries, basis sets, convergence criteria, etc. The finite-difference nature of the fitting procedure makes close agreement between programs particularly important.

TABLE III.

Comparison between theory and experiment for bond lengths (Å) and spectroscopic constants (cm⁻¹) of three diatomic molecules. All Δ terms correspond to the difference between a value and the base CCSD(T)/cc-pCV[Q5]Z calculation. Experimental values from Ref. 106 (BH), ¹⁰⁷ (HF), and ¹⁰⁸ (C₂). All published experimental uncertainties are smaller than the displayed precision of the spectroscopic values presented here.

Molecule and method	r e	ω e	ω e x e	B e	D e	α e
BH
Base	1.228 90	2371.24	49.4	12.088	0.001 257	0.423
ΔBasis	+0.000 18	−0.44	−0.4	−0.004	−0.000 001	−0.001
ΔDBOC	+0.000 65	−2.33	−0.2	−0.013	−0.000 002	+0.000
ΔRel	−0.000 01	−0.57	+0.1	+0.000	+0.000 001	+0.000
ΔCCSDTQ	+0.000 19	−2.07	+0.1	−0.004	+0.000 001	+0.001
ΔFCI	+0.000 00	+0.00	−0.2	+0.000	+0.000 000	+0.000
ΔTotal	+0.001 01	−5.41	−0.5	−0.020	+0.000 000	+0.000
Total	1.230 00	2365.83	49.0	12.068	0.001 256	0.423
Experiment	1.232 16	2366.72	49.3	12.026	0.001 235	0.422
HF
Base	0.916 54	4147.01	90.5	20.968	0.002 144	0.793
ΔBasis	+0.000 17	−1.79	−0.7	−0.008	−0.000 001	−0.002
ΔDBOC	+0.000 01	+0.32	−0.2	−0.001	−0.000 001	+0.000
ΔRel	+0.000 06	−3.54	−1.3	−0.003	+0.000 003	+0.000
ΔCCSDTQ	+0.000 21	−4.49	+0.1	−0.009	+0.000 002	+0.002
ΔFCI	+0.000 01	−0.19	+0.0	+0.000	+0.000 000	+0.000
ΔTotal	+0.000 47	−9.70	−2.2	−0.021	+0.000 004	+0.000
Total	0.917 00	4137.31	88.3	20.947	0.002 148	0.792
Experiment	0.916 808	4138.32	89.0	20.956	0.002 151	0.798
C2
Base	1.240 39	1873.63	12.6	1.826	0.000 007	0.017
ΔBasis	+0.000 16	−1.01	+0.0	+0.000	+0.000 000	+0.000
ΔDBOC	+0.000 01	+0.09	+0.0	+0.000	+0.000 000	+0.000
ΔRel	−0.000 16	−0.41	+0.1	+0.000	+0.000 000	+0.000
ΔCCSDTQ	+0.001 46	−11.76	+0.8	−0.004	+0.000 000	+0.001
ΔFCI	+0.001 00	−4.58	+0.0	−0.003	+0.000 000	+0.000
ΔTotal	+0.002 48	−17.81	+0.8	−0.007	+0.000 000	+0.001
Total	1.242 87	1855.82	13.4	1.819	0.000 007	0.018
Experiment	1.242 44	1855.01	13.6	1.820	0.000 007	0.018

FIG. 4.

Influence of post-CCSD(T)/CBS corrections on two spectroscopic constants, (a) r_e and (b) ω_e, and three diatomic systems, BH, HF, and C₂. For each system, the right gray bar is the difference between the experimental constant and the constant calculated at the E_Base [CCSD(T)/CBS] level of theory. The left gray bar is the difference in constants calculated at the E_Total and E_Base levels of theories. Within the left gray bar, contributions from each correction are shown as colored bars. Data are from Table III.

The total extrapolation procedure shows remarkable agreement with experiment for bond lengths r_e (within 0.0005 Å) except for BH, off by 0.0022 Å. However, this extrapolation lacks nonadiabatic BO effects, which were found by Martin¹⁰² to be unusually high for BH, ∼0.0025 Å. This is rather close to the overall difference of 0.0022 Å between experiment and our best estimate. Theoretical harmonic frequencies ω_e are in excellent agreement with experiment, off by only 1 cm⁻¹. The rotational constant B_e is also well predicted, within 0.01 cm⁻¹ for HF and C₂ and off by a somewhat larger 0.04 cm⁻¹ for BH. The latter error may be largely due to already-noted non-BO effects, which cause a larger discrepancy in r_e for BH. ω_ex_e is in good agreement with experiment, matching within 0.2–0.4 cm⁻¹ for BH and C₂ but is off by a larger 1.6 cm⁻¹ for HF. It is not clear that the corrections employed here actually improve this constant, and the remaining discrepancy could be due to the numerical precision limitations discussed earlier. ${\overline{D}}_e$ is very well predicted already by the base method, and the various corrections are extremely small. Similarly, α_e appears to not require corrections on top of the base method, each of which changes it by only ±0.002 cm⁻¹ or less. Final values are within 0.005 cm⁻¹ of experiment.

Figure 4 shows that the sum of the small corrections matches experiment very well for r_e and ω_e, except for the bond length of BH, where non-BO effects are important as noted above. All of the small corrections considered can be important for r_e and ω_e, although there is no consistency about their relative importance from one molecule to another. For example, the DBOC is rather important for BH (which has the lightest nuclei), but not for HF and even less so for C₂. Similarly, the FCI correction (beyond CCSDTQ) is negligible for BH and HF but is important for C₂ (worth 0.001 Å and 4.6 cm⁻¹) In total, the corrections for C₂ lower the value of ω_e by a surprisingly large 17.81 cm⁻¹ from the base CCSD(T) value, which is very close to the experimental ω_e (18.61 cm⁻¹ lower than the base). A large majority of this change is due to missing electron correlation: the CCSDTQ correction is responsible for about 12 cm⁻¹ and the FCI correction by about another 5 cm⁻¹. This is presumably due to the much larger degree of electron correlation in C₂, arising from the close near-degeneracy of the [core]$2{\sigma }_g^{2}2{\sigma }_u^{2}1{\pi }_x^{2}1{\pi }_y^2$ and [core]$2{\sigma }_g^{2}1{\pi }_x^{2}1{\pi }_y^{2}3{\sigma }_g^2$ configurations.

IV. SUMMARY AND CONCLUSIONS

Users increasingly desire programmatic (i.e., API: application programming interface) access to QC results, either for their convenience or for incorporation into automated workflows. The interface, volume, and intricacy requirements of that access vary widely across applications and increasingly involve uniform results across QC programs. The QCElemental, QCEngine, and QCDB software modules [the former two being part of the Molecular Sciences Software Institute¹⁸ (MolSSI) QCArchive¹⁷ project] provide a framework to facilitate interoperability among community computational molecular sciences (CMS) programs.

QCArchive and QCDB have been designed to work with emerging tools and standards developed by MolSSI, particularly the QCSchema JSON format for information passing. QCElemental provides implementations and validators around QCSchema objects, while QCEngine provides QCSchema I/O adaptors for CMS codes. In addition to wrapping nearly a dozen QC programs for uniform execution and programmatic access to results, QCEngine interfaces with geomeTRIC and other geometry optimizers that can, in turn, call QCEngine for QC gradients. QCEngine easily expands to additional CMS codes, has parallel execution capabilities through QCFractal, and by definition allows uniform execution, yet it is not in itself a coherent QC driver due to the differing implementations, conventions, defaults, and DSL of QC codes.

The Quantum Chemistry Common Driver and Databases (QCDB) project provides a simple and powerful driver front-end to multiple QC programs, allowing users automatic access to several features formerly requiring specialized scripts or laborious post-processing. These include built-in composite methods, many-body expansion procedures, vibrational analysis, and combinations thereof for not only energies but also gradients, Hessians, and geometry optimizations. By adding the basis set, keywords, and result tools for uniformity and interoperability, QCDB also allows mixing and matching capabilities of multiple quantum chemistry programs within a single computation. These features have been demonstrated with an application computing spectroscopic constants of diatomic molecules at the ab initio limit, including corrections for post-CCSD(T) electron correlation, beyond-cc-pCV[Q5]Z basis set effects, relativistic effects, and the Born–Oppenheimer diagonal correction, combining total energies computed by CFOUR, GAMESS, NWChem, and Psi4.

V. EXTERNAL MATERIAL

Software repositories and documentation are available for QCElemental at https://github.com/MolSSI/QCElemental/ and https://molssi.github.io/QCElemental/, for QCEngine at https://github.com/MolSSI/QCEngine/ and https://molssi.github.io/QCEngine/, for QCDB at https://github.com/qcdb/qcdb/ and https://qcdb.github.io/qcdb/, and for general QCArchive INFRASTRUCTURE at http://docs.qcarchive.molssi.org/. These programs remain in active development. Production computations are under way using many features of the software, and test suites are expected to pass. However, users are encouraged to contact the developers as they venture afield of the verified tests. Many snippets from this work, including an abbreviated diatomic fitting, are demonstrated in the test suite: https://github.com/qcdb/qcdb/blob/master/qcdb/tests/test_manuscript.py.

AUTHOR DECLARATIONS

Conflict of Interest

J.D.C. is a current member of the Scientific Advisory Board of OpenEye Scientific Software, Redesign Science, and Interline Therapeutics and has equity interests in Redesign Science and Interline Therapeutics.

DATA AVAILABILITY

The data that support the findings of this study are available within the article.