Benchmarking crystal structure refinement: A systematic study on Hirshfeld atom refinement

Abstract

This study systematically investigates the influence of various parameters of the wavefunction calculation during Hirshfeld atom refinement (HAR). We aim to address the lack of consensus in the literature and conflicting information on a generally recommended procedure. A set of amino acid test structures, known for their immense biochemical importance and unimpeachable experimental data quality, was employed to ensure reliable results, unbiased by the question of insufficient diffraction data quality. A comprehensive permutation of refinement parameters was conducted to avoid overlooking potential influences, resulting in 2496 structure refinements per amino acid. Applying a solvent model systematically improved refinement results compared to gas-phase calculations. Additionally, it was observed that the pure Hartree–Fock method outperforms all tested density functional theory methods across all structures in this test set of polar-organic molecules. These findings underscore the importance of carefully considering the level of theory applied in HAR and offer an overview of the performance of various methods and parameters.

I. INTRODUCTION

Structural scientists use single-crystal x-ray diffraction (SCXRD) to get precise information about the structural parameters of a substance of interest. Nowadays, they can rely on modern experimental setups, enabling high-quality data acquisition within a significantly reduced timeframe, easily outweighing the data collection potential of SCXRD equipment available to researchers 50 years ago. Still, data analysis is often based on the same model applied for half a century for the structure refinement. The independent atom model (IAM) laid the foundation for the precalculated scattering factors, which are frequently used.¹ It has been shown extensively that the IAM and its assumption of non-interacting atoms are too crude for a modern electronic structure representation.^2–5 The effects the IAM neglects include chemical bonding phenomena, electron lone pairs, weak interactions, and atom or molecular polarization.^6,7 These effects can only be described by applying a non-spherical treatment of the electron density of atoms in a crystal structure. The lower quality of IAM structure models has been demonstrated explicitly for anisotropic displacement parameters (ADPs)⁶ and bond lengths.^8,9 The shortcoming is particularly pronounced for hydrogen atoms, which are classically located at positions that do not agree with accurate neutron data,^10,11 reinforcing the assumption of invisible H-atoms in x-ray experiments. For more complex endeavors than refining non-hydrogen atoms and displacement parameters, it is therefore considered imperative to replace the IAM, which promoted the development of new models over the decades.¹²

Two methodological branches concerning non-spherical refinement have been established in this context. The first branch is the multipole model,¹³ which utilizes an expansion of spherical harmonics to describe the electron density deformation. Its model parameters are usually fitted against high-resolution, high-quality data in a least squares procedure. As this method provided significantly better flexibility and systematic improvement in structure modeling,¹⁴ it has been extended with radial scaling in the Hansen–Coppens formalism,¹⁵ and in the last 30 years, database approaches have been successfully introduced to make it more feasible for routine applications.^16–21 This approach focuses on reconstructing the electron density, the observable quantity of the x-ray diffraction experiment.

The second branch of approaches introducing asphericity in the atomistic crystallographic model can be summarized by the involvement of a quantum mechanical calculation used to augment the model.^12,22 One such method is Hirshfeld atom refinement (HAR).²³ Here, scattering factors are calculated from a stockholder partitioning of a molecular electron density, the so-called Hirshfeld atoms.²⁰ Applying this method requires, thus, a preceding quantum mechanical calculation to obtain the electron density, which is subsequently used to derive unique scattering factors in each structure. HAR was found to be a powerful tool for the refinement of crystal structures²⁴ as shown in various examples,^25–28 including disordered moieties,^5,29 heavy elements,^30–32 and extended periodic structures.^29,33,34 Using HAR in structure refinement has no additional requirement regarding experimental data quality (compared to a final IAM model). However, the quantum chemical calculation might sometimes pose a bigger issue, as the calculation cannot be performed on an unfinished structural model with missing or misassigned atoms or when the charge state of the asymmetric unit is not yet fully resolved. The application of the method, when the quantum chemical calculation is possible, does not entail any inherent disadvantages except for the additional investment of computational time. The model quality is thereby expected to be enhanced when a sufficient accuracy of the quantum chemical calculation is achieved and no inherent issues separating the measured sample and the built model remain, for example, missed substitutional or conformational disorder, systematic errors in the measured data like detector countrate saturation or untreated absorption.

An implementation of the original software framework for HAR, tonto,³⁵ was interfaced with Olex2,³⁶ named HARt,²⁶ offering a restricted set of functionality, making this method available to a broader audience. More recently, NoSpherA2⁵ replaced this interface with a native implementation of HAR inside Olex2. It is designed to be a routine refinement tool as it directly interfaces with the refinement engine olex2.refine,³⁷ allowing the usage of restraints, constraints, disorder, and treatment of special positions, which posed a problem at the time within tonto. The NoSpherA2-HAR is identical to HAR within tonto, in the sense that the initial density calculation during the refinement process relies purely on a single point (SP) calculation of the structure of interest, the calculation of Hirshfeld Atoms based on the pro-molecular atomic densities, and subsequent calculation of atomic scattering factors. Several methodological improvements have since been introduced. A fragmentation approach^38,39 was implemented to refine structures at the scale of proteins in a reasonable time frame by enabling linear scaling, significantly reducing the computation requirements. The absence of an intermolecular environment with neighboring molecules during the density calculation was sometimes considered a drawback for the current implementation of HAR. The placement of cluster charges,²⁶ extremely localized molecular orbitals (ELMO) embedding,⁴⁰ or periodic density functional theory (DFT) in HAR^34,41 can remedy this neglect. These additions provide more accurate structure models in cases where intermolecular interactions dominate the electron density distribution.

However, among all these methods, a significant diversity in the fundamental step of calculating the electron density prevails. This complexity is compounded by the demanding task of choosing the right level of theory, as in all other fields of quantum chemistry.⁴² Since standard software like ORCA⁴³ or Gaussian,⁴⁴ which offer a wide variety of methods, are compatible with NoSpherA2,⁵ the crystallographic users are confronted with the methodological question of a quantum chemist. There are, of course, guidelines for choosing an appropriate level of theory when it comes to optimizing a molecular geometry or calculating reaction energies, but for HAR, those recommendations are not yet tested or necessarily valid. In the context of HAR, comparing crystallographic quality indicators, such as deformation or residual electron density, or the resulting accuracy in hydrogen atom positions, provides a robust assessment of various methods. This has been shown in several examples.^{5,9,23,24,26,32,34,40,45,46} Previous results indicate that DFT is the most recommendable method (for HAR).^5,23,24,40 A frequently utilized functional is the popular B3LYP from the hybrid generalized gradient approximation (GGA) family.^47,48 It is known to perform poorly in other fields of quantum chemistry^49,50 but remains a frequently presented representative of its functional class. The Hartree–Fock (HF) electronic exchange component, intrinsic to the pure HF method and the hybrid class functionals, was demonstrated to benefit the quality of HAR results.⁴⁶ When post-HF methods were investigated in the context of HAR, they showed only minor improvement relative to the tested DFT methods at the cost of a significant increase in computational efforts.⁴⁵

The choice of a basis set has a fundamental role during the HAR procedure. Among the most commonly used basis sets is Ahlrichs' def2 series.⁵¹ Notably, smaller basis sets were observed to perform comparable or even better⁵² than larger ones.⁴⁶ In each attempt, different methods for comparison were employed for only a limited number of crystal structures. Such restrictions in study scope are reasonable since acquiring high-quality datasets is accompanied by substantial requirements in terms of crystal quality, instrumentation, and data collection. Additionally, the chosen target structures must be suitable for the investigated matter. Therefore, the benchmarking of HAR suffers a bias, as it aims to make a general statement about the performance of a method, which should hold for a broad range of structures. However, the limited consideration of the chemical space and the limited number of similar structures in the existing benchmark studies cause a lack of generalizability for the conclusions drawn. Moreover, the observability of trends while comparing levels of theory creates a strong generality requirement on selected tested parameters. For example, a simplified statement that hybrid functionals outperform GGA-class functionals lacks transferability when only B3LYP and PBE were tested. The available benchmark studies do not conclude with a general “best method” for HAR. Since the conflicting results of the available literature do not allow such conclusions, the question arises whether a general statement about a method's performance is possible. To investigate further, it is essential to use a broader range of refinement results from a wide spectrum of levels of theory. In this work, we present a benchmarking study of HAR, which aims to provide a general statement about the performance of a given method when applied to a selection of small organic polar molecules. The methods used for calculating the electron density are focused on isolated molecular-level calculations in this study. The set of test structures analyzed is restricted to the well-defined class of amino acids. Therefore, any statement about the performance of a level of theory should show transferability mostly to similar structures of organic molecules containing polar groups.

II. METHODOLOGY

The benchmark study was conducted on SCXRD datasets of 14 amino acids. Parameters and combinations thereof were chosen based on the availability within the refinement software. The implementation of NoSpherA2⁵ in Olex2³⁶ was used to perform HAR. A complete permutation of the refinement parameters was performed so as not to miss a potential influence by the correlation of inputs on the outcome (see Sec. II B). The sequential refinement was aided by the plugin SISYPHOS.⁵³

A. X-ray data

The SCXRD data recorded for the amino acids were of adequate quality with a resolution of d_min = 0.6 Å or better (see Sec. V). The selection covers two non-essential amino acids, 2-aminoisobutyric acid (Aib) and racemic pidolic acid (racPA). L-arginine, L-asparagine, and L-serine datasets were collected as hydrates (see Fig. 1). The datasets are labeled by the three-letter code of the standard amino acids,⁵⁴ and the L-stereo descriptor and hydrate state are not explicitly stated from hereon. The initial IAM refinement, used as input for SISYPHOS, was performed with olex2.refine,^36,37 and all data were refined with their full resolution. Positions of the H-atoms were refined freely, and the hydrogen displacement parameters were refined isotropically. Anisotropic displacement parameters were employed on all non-hydrogen atoms. The presented data are starting points from which the HARs are performed. From here on, the resulting models of these starting points are referred to as IAM structures.

FIG. 1.

Structures used for this benchmark after initial IAM refinement, ADPs are shown at a 50% probability level.

B. Selection of quantum mechanical methods in HAR

All single-point calculations were performed using the ORCA 5.0 software,⁵⁵ operated by NoSpherA2.⁵ Twelve density functional theory (DFT) functionals and the Hartree–Fock (HF) method were tested, as available within the NoSpherA2 user interface. This selection contains several functionals from each class of DFT methods (see Sec. II C). The tested basis sets include all 16 (non-relativistic) ones available in the NoSpherA2-GUI, covering several examples of the most widespread basis set families (see Sec. II D). The default SP calculation is performed for a free molecule in vacuum. As described above, considering the molecular surroundings in the crystal packing is deemed advantageous to HAR. Instead of explicit modeling of a crystal environment, the easily applicable conductor-like polarizable continuum model (CPCM solvent model)⁵⁶ is tested, in which solvation is treated as a dielectric polarizable continuum surrounding the molecule. A water model is chosen since the crystal packing of the amino acids is dominated by hydrogen bonding. The inclusion of the non-local self-consistent dispersion correction (SCNL) VV10 tests the influence of dispersion interactions on the refinement outcome results.⁵⁷ The Becke grid size,⁵⁸ which accounts for the accuracy of the numerical integration during both single-point calculation and calculation of scattering factors, is varied to test for the influence of this numerical parameter. A systematic permutation of the parameters was performed according to the decision tree in Fig. 2 to investigate any conceivable trends.

FIG. 2.

Flow chart presenting the variation tree and tested parameters for the refinement; following one branch represents one set of options chosen for one refinement. The options cover the choice whether to include a self-consistent non-local (SCNL) dispersion correction after Vydrov and Van Voorhis, a solvation model, the grid density for numerical integration, and the level of theory, including QM method and basis set choice. Three dots represent a repetition of the boxes above for the same path, but are omitted for readability. References for each parameter and further details about the depicted options are listed in Secs. III A–III E.

In combination, all permutation paths yield 2496 refinements, which were performed for each amino acid dataset, yielding 34 944 structural models in total. The sequential processing of these calculations was aided by the SISYPHOS plugin, where an input file controls the settings of each refinement performed successively. Two parameters were chosen to stay constant: The convergence strategy was fixed at “EasyConv,” and the SCF threshold at “NoSpherA2SCF” (NoSpherA2SCF is a selection of ORCA input criteria: Energy convergence at 3 × 10⁻⁵, DIIS error convergence at 1 × 10⁻⁴, the density matrix threshold on 1 × 10⁻⁹, the orbital gradient convergence at 3 × 10⁻⁴ and the orbital rotation angle convergence at 3 × 10⁻⁴). Hydrogen atoms were refined anisotropically and free of positional constraints or restraints.

C. Density functionals

The rating of DFT functionals is often conceptualized with rungs on Jacob's ladder, as proposed by Perdew,⁵⁹ which illustrates the hierarchical nature of functionals in terms of their inherent properties and accuracy. In this work, the low-rung class of local density approximated (LDA) functionals is represented by PWLDA.⁶⁰ The GGA family is represented by BP,⁶¹ BP86,⁶² PBE,⁶³ and BLYP;⁴⁸ the meta-GGA functionals, including higher derivatives of the electron density, tested are TPSS⁶⁴ and R2SCAN.⁶⁵ Hybrid functionals are considered especially suited for molecular systems, represented in this benchmark by PBE0⁶⁶ and B3LYP.^47,48 The meta hybrid functionals M06-2X,⁶⁷ ωB97,⁶⁸ and ωB97X⁶⁸ represent the highest rung tested, closest to chemical accuracy.

D. Basis sets

The 16 basis sets tested comprise several members of different popular families.

• •Pople sets: STO-3G,⁶⁹ 3-21G,⁷⁰ 6-31G(d),⁷¹ 6-31G(d,p),⁷² 6-311G(d,p),⁷³ 6-311++G(2d,2p).⁷⁴
• •Correlation consistent sets of the Dunning family: cc-pVXZ⁷⁵ with X = D, T & Q.
• •Ahlrichs' Karlsruhe sets: def2-SVP, def2-TZVP, def2-TZVPP,⁷⁶ def2-TZVPD, and def2-TZVPPD.⁷⁷
• •jorge-DZP⁷⁸ and jorge-TZP.⁷⁹

E. Quality indicators for quantitative evaluation of refinement results

The performance of each refinement is quantified using two quality indicators. The first accounts for the representation of the electronic structure and combines classical R-values (R₁, wR₂), minimum and maximum residual electron density, and the root mean square (RMS) of the residual electron density map. They are combined into a single value, since they are different aspects of the agreement of the electron density distribution of the diffraction data, either in reciprocal or real space. Their improvement compared to the IAM shows high correlation in most cases (see Fig. S5.1), which allows the combination into a single value. For further details, the comparison of all refinement results of all individual descriptors can be found in the corresponding chapters of the supplementary material.⁹⁹ All individual values should be as low as possible for a refinement that performs better. Those indicators K are combined into a single figure of merit (FOM) of the electron density description that quantifies the improvement relative to the IAM model [Eq. (1)]

$$\text{FOM}=\sum ^{\text{all}\hspace{0.17em}\mathit{K}}\frac{{\mathit{K}}_{\text{IAM}}-{\mathit{K}}_{\text{HAR}}}{{\mathit{K}}_{\text{IAM}}}.$$ (1)

A negative value would indicate a worsening of refinement statistics, while a value of 5.0 would indicate a perfect refinement, with zero discrepancy between the measured data and the HAR model. A superior set of settings for HAR is, therefore, reflected by a higher value of the FOM, which allows a ranking of the parameter sets. Despite their high correlation for HAR (see Fig. S5.1),⁹⁹ the supplementary material contains the visual representations of the discussion with each K separately, replacing the FOM with the individual metric for more in-depth information.

The accuracy of the refined hydrogen atom positions is evaluated to complement the electronic description achieved by the FOM. The corresponding neutron experiments of the amino acids in the literature are used as a reference. The average deviation of the refined X-H distances from the reference neutron data are calculated as the weighted root mean square deviation (wRMSD) [Eq. (2)]. We only consider the improvement of hydrogen atom positions since other atomic positions have been proven not to be influenced as much by the introduction of non-spherical scattering factors^5,80

$$\text{wRMSD}=\sqrt{⟨\sum ^{\text{all}\hspace{0.17em}\mathrm{X}-\mathrm{H}}\hspace{0.17em}\frac{{\left({\mathit{d}}_{\text{neutron}}-{\mathit{d}}_{\text{Xray}}\right)}^2}{{\mathit{\sigma }}_{{\mathit{d}}_{\text{neutron}}}^2+{\mathit{\sigma }}_{{\mathit{d}}_{\text{Xray}}}^2}⟩}.$$ (2)

Employing the wRMSD, the structural agreement between the HAR SCXRD and the neutron diffraction results regarding hydrogen atom positions is quantified. The experimental neutron diffraction data for the amino acids were used from the corresponding studies: Ala,⁸¹ Arg,⁸² Asn,⁸³ Cys,⁸⁴ His,⁸⁵ Glu,⁸⁶ Gly,⁸⁷ Ser,⁸⁸ Thr,⁸⁹ and Tyr.⁹⁰ For structures not subjected to neutron diffraction experiments (including Aib, Asp, Pro, and racPA), generalized X-H bond distances were applied as in Ref. 91. A comparison of the bond lengths from the available neutron structures with the generalized neutron data (Fig. S1.1) shows a close agreement in this regard. The systematic error of the X-H bond introduced by the temperature difference between our x-ray diffraction data and the reported neutron diffraction experiment is considered negligible for this study.²⁴

During the preparation of this manuscript, two articles that highlight the benefits of the general use of NoSpherA2 for small molecule structure refinement⁹² and the benefits of HAR in quantum crystallographic protocols⁹³ were published. Together, they provide recommendations for the application of HAR in quantum crystallographically augmented procedures, further discussion of the experimental requirements and expectations from employing HAR, that would go beyond the scope of this study.

III. RESULTS AND DISCUSSION

Using the FOM [Eq. (1)] and wRMSD [Eq. (2)], each refinement's result is evaluated, allowing the analysis of each parameter's impact on the refinement result. Consequently, the global average of a parameter's influence on the FOM and wRMSD is computed. The respective relative mean values represent the influence on the refinement outcome. For clarification, the four structures for which no neutron diffraction data were available have been positioned at the bottom of Figs. 3 and 4. Additionally, alternative versions of Figs. 5 and 6 are displayed in the supplementary material (Figs. S8.1 and S8.7).⁹⁹ These figures only consider the ten amino acid datasets, which include neutron diffraction data for the calculation of wRMSD. The results are presented in Secs. III A–III E, grouped by the option.

FIG. 3.

FOM (orange) and wRMSD (blue) violin plots for the distributions of refinements using a CPCM model for water or without any treatment of environmental effects (Vacuum) plotted around the global mean (dotted line), the respective means are marked black. The scales are orientated so that improvement compared to the global mean is located at the right-hand side of the mean line.

FIG. 4.

FOM (orange) and wRMSD (blue) violin plots of the distributions using different integration grid settings plotted around the global mean (red dotted line), the respective means of individual distributions are marked in black. The scales are oriented so that improvement compared to the global mean is located at the right-hand side of the mean line.

FIG. 5.

2D-scatterplot of refinement results for all structures using different QM-methods in the FOM^*-wRMSD^* plane. Mean values of the different method clusters are indicated as black x-markers; each method-cluster is colored according to the distance of the mean value to the point (−1,−1); a short distance is indicated with a darker color. Box-and-whiskers plots (right) show the median (white line), and the box frames the interquartile range (IQR). The whiskers are in a range of 1.5 times the IQR; datapoints outside this range are drawn as flier-props.

FIG. 6.

Two-dimensional metric multidimensional scaling (MDS)⁹⁷ output for the employed quantum mechanical methods, according to which the similarities are visualized as pairwise distances. The dimensions which span up the graph do not have an intrinsic meaning and are arbitrarily chosen to preserve the underlying data structure. Stress value for both MDS: 0.011.

A. Influence of the non-local dispersion correction

Listed first in the decision tree (Fig. 2) is whether or not to apply the self-consistent non-local dispersion correction (keyword: SCNL True, or False). The use of a dispersion correction is a standard procedure in quantum chemistry since accounting for the missing contribution of dispersion interactions to the electron density is indispensable for optimizing molecular geometries or calculating reaction energies. The SCNL method in ORCA pairs a nonlocal van der Waals functional with the corresponding exchange-correlation functional of the respective DFT method. A recommended choice is the VV10 correlation model introduced by Vydrov and Van Voorhis.⁵⁷ The influence on the refinements using NoSpherA2 by dispersion correction is visualized in Fig. S2.1. It displays the refinement results of each amino acid dataset separated into two groups, one with the SCNL method applied (1920 refinements per structure, as SCNL is not defined for M06-2X, wB97, PWLDA) and the ones without. The results show that the indicators FOM and wRMSD are closely located on the global mean in all cases with their distributions not being distinguishable. The SCNL method thus shows no significant influence on the refinement outcome. Therefore, the less time-consuming refinement without the SCNL method is recommended for small polar molecules.

B. Influence of a solvent model

Performing the molecular electron density calculation in the gas phase is a simplification, which introduces a systematic error. Several methods to account for the molecular environment were reported for HAR, ranging from periodic DFT calculations³⁴ to explicitly arranging cluster point charges based on the calculated unit.^24,26 This section presents the influence of the electrostatic contribution of a CPCM solvent model⁵⁶ on the refinement outcome, a method not reported to be used before. Considering the application to a solid-state structure, including solvation in the quantum mechanical calculation might appear counterintuitive, but modeling the surroundings even to an imprecise extent may offer an improved structure model compared to the complete neglect of environmental effects. Due to the strong hydrogen bonding network in the crystal packing of the amino acids, a water model was chosen as a suitable CPCM model. The results from these calculations were compared against a solvent-free (isolated molecule in vacuo) calculation.

As presented in Fig. 3, the solvation model exhibits a positive effect for almost all structures. Both averages of the improvement indicators, FOM and wRMSD, shift above the average of all calculations. Glu is the only exception, showing a negative influence of the solvation model regarding the wRMSD. Even though applied in the solid state, the results suggest that using a solvation model is generally beneficial for HAR of small polar organic molecules. The overall beneficial effect probably originates from the hydrogen bonding network in the crystal packing, which a water solvation model imitates by polarizing the bonds within the calculated molecule, and the boundary provided for the negatively charged carboxylate groups, known to otherwise become overproportionally diffuse in vacuo.^94,95

C. Influence of the integration grid

The accuracy of the integration grid is a computational parameter that influences the numerical calculation of the electron density based on spherical, atom centered grids, affecting both the single point calculation in ORCA and the calculation of scattering factors within NoSpherA2. These grids are not evenly spaced with respect to the radius, but grid points are more concentrated in regions of high electron density near the nucleus to capture regions of higher variance more effectively.⁹⁶ The respective options that can be adjusted and tested are the grid settings Low, Normal, and High (Fig. 4).

The resulting distribution highlights that the grid size influences the refinement outcome. In addition to Asn and Asp, the highest FOM values are achieved with the High grid setting, while the mean values of the Low setting are located below the average. A numerical error introduced by the grid accuracy thus impacts the electronic structure representation, and the coarse-meshed grid results in a lower-quality structural model. Since the difference between the Normal and High grid settings is small compared to that between Low and Normal, the numerical stability has mostly been recovered by the Normal grid setting.

The wRMSD of bond lengths is less influenced by the grid size. Still, the Low setting is, surprisingly, above average in most cases, contradicting the above-mentioned trend observed for the electron density description. From this standpoint, the Normal grid setting is recommended to refine amino acids. The Normal setting bears another advantage, as it is significantly less time-consuming for the partitioning than the High setting (see Fig. S7.1), making it a reasonable choice for refinements from a practical standpoint.

D. Influence of the QM-method

The refinement quality indicators underwent a shift for a whole series of refinements by changing each parameter discussed above. The influence of the method and the basis set on the refinement outcome is discussed in this section and Sec. III E. A recalculated version of the electronic FOM and geometric wRMSD values [Eqs. (3) and (4)] will be introduced to allow a better relative evaluation of the effect on the refinement outcome to allow an easier discussion. The values for each refinement i are referenced to the global mean of the respective geometrical or electronic parameter of all refinements. A value of 0.0 for this rescaled metric, therefore, is the average performance. At the same time, both indicators are calculated so that a negative value indicates an improvement compared to the average across all datapoints

$$\text{FO}{\mathrm{M}}_{\mathrm{i}}^{*}=\frac{\text{FO}{\mathrm{M}}_{\text{mean}}-\text{FO}{\mathrm{M}}_{\mathrm{i}}}{\text{FO}{\mathrm{M}}_{\text{mean}}},$$ (3)

$$\text{wRMS}{\mathrm{D}}_{\mathrm{i}}^{*}=\frac{\text{wRMS}{\mathrm{D}}_{\mathrm{i}}-\text{wRMS}{\mathrm{D}}_{\text{mean}}}{\text{wRMS}{\mathrm{D}}_{\text{mean}}}.$$ (4)

All 2496 performed refinements were color-coded using the QM-method for each test structure and plotted in the FOM^*-wRMSD^* plane (Fig. 5). The different performance of the individual quantum mechanical method applied during HAR is observable by the shifted mean values of each subset marked by a cross, the corresponding label for the used method, and the color. In this plot, the best result would be a negative score in either FOM^* or wRMSD^* due to the rescaling of the quality parameters.

The data obtained using Eqs. (3) and (4), shown in Fig. 5, indicate a clear trend after which the methods are rankable. The mean value of the refinements using the HF method has the shortest distance toward the point (−1, −1). It outperforms all other methods regarding the FOM and wRMSD. Functionals with intrinsic HF exchange are positioned at the left side in the ranking by method, including PBE0, the long-range corrected wB97(X), and the meta-hybrid functional M062X. The meta-GGA R2SCAN is the best-performing non-hybrid, surpassing the results of B3LYP. The tested GGA functionals yield surprisingly poor results.

In conclusion, pure HF outperforms all other tested methods, which is, considering the missing account of electron correlation, surprising. Additionally, R2SCAN, a modern meta-GGA without HF exchange, surpasses the popular hybrid functional B3LYP. Ranked in between, PWLDA, considered a rudimentary method unsuitable for modern applications, yields hydrogen positions of comparable quality to functionals of higher rungs on the Jacob's ladder, surpassing those obtained with B3LYP. The amount of HF exchange for modeling strong polar environments is known to impact the results, and comparisons between the HF and hybrid functionals have already shown this result in recent studies.^46,80 However, the results of the complete benchmark show that HF plays an outstanding role as its refinement outcomes make it stand out from all high-level hybrid functionals, not only in individual refinements but also observable as a global average over a broad range of structures and parameter combinations within the tested class of compounds.

The performance of pure HF and how it compares qualitatively the hybrid methods is demonstrated in Fig. 7 by the multidimensional scaling analysis (MDS). The MDS can represent high complexity data's (dis)similarities as distances in a 2D space, allowing an inspection of multidimensional datapoints in only two dimensions. Starting from a distance (or dissimilarity) matrix of the data, MDS seeks points in a low-dimensional space, that best preserve the initial pairwise distances. This enhances the visual interpretation and identification of patterns compared to the original high-dimensional data. Here, MDS was performed on the wRMSD and FOM of all refinements, labeling them by the methods used. The pairwise distance indicates the dissimilarity, so with HF taking a distinct position in both plots, its dissimilarity against the other methods is highly pronounced. The hybrid functionals are located close to each other, with R2SCAN as the only non-hybrid as part of the cluster, especially pronounced in the wRMSD MDS. The observable separation within Fig. 7 between Jacob's ladder's higher and lower rungs indicates the different functional families' disparate behavior and overall performance differences.

FIG. 7.

2D-scatterplot assigning the refinements using different basis sets for all structures to a point in the FOM^*-wRMSD^* plane. Different basis set-cluster mean values are indicated as black x-markers; each cluster is colored according to the distance from the mean value to the point (−1,−1), a short distance indicated by a darker color. Box-and-whiskers plots (right) show the median (white line), the box frames the interquartile range (IQR), and the whiskers are in a range of 1.5 times the IQR for FOM^* (top) and wRMSD^* (bottom); data outside are drawn as flier-props.

The qualitative distinction of pure HF is evident in both the wRMSD and the FOM MDS plot, where it appears noticeably more distant from all other methods. So, based on this benchmark study of amino acids, the HF method emerges as the most favorable choice. However, the generalizability of these findings to a broader chemical context requires further investigation. The R2SCAN functional is a recommended alternative because of its performance and lower computational cost, especially if the system is expected to show strong electron correlation. The suggestion to desist from using B3LYP (and BLYP) should also be highlighted at this point, particularly when the more cost-effective R2SCAN functional is available.

E. Influence of the basis set

Traditionally, the question of choice of a basis set is primarily a question of time and computational resources. What qualitative differences emerge based on the selection of a basis set are analyzed using the same methodology as in the case of a method. Figure 6 is the 2D map of all refinements in the FOM^*-wRMSD^* plane. Most basis sets tested show an average value (black “X”) in both metrics close to each other, indicating similar performance. The expected outliers are the minimal basis sets STO-3G and 3-21G. The refinements with these basis sets are significantly separated regarding wRMSD^* and FOM^*. The 6-31G(d) basis set also exhibits notable deviation from the bulk of refinements concerning the wRMSD^*. Interestingly, the double-zeta basis sets show a superior performance regarding the FOM^*, as indicated by their median values. For the wRMSD^*, on the other hand, the larger basis sets exhibit better results.

Considering the observed deviations between the basis sets (besides the clear outliers), the choice of the basis set is less critical than that of the method. Therefore, a definite ranking according to the performance of the basis sets is difficult due to the slight differences in the FOM^* and wRMSD^* values. Regarding resources and time, even smaller basis sets like def2-SVP are recommendable for HAR. The MDS similarity analysis of the basis sets is presented in Fig. 8.

FIG. 8.

Two-dimensional metric multidimensional scaling (MDS)⁹⁷ output for the quantum basis sets, according to which the similarities are visualized as pairwise distances. The dimensions which span up the grapg do not have an intrinsic meaning and are arbitrarily chosen to preserve the underlying data structure. Stress value for both MDS: 0.033. Note: Using STO-3G did not reach convergence during the HAR procedure for all structures and was therefore excluded from the MDS analysis.

The double-zeta basis sets exhibit proximity within a cluster, while the larger sets demonstrate a distinct separation. The small basis 3-21G is located in a distant position for both the FOM^* and wRMSD^* MDS, indicating its different behavior. The 6-31G(d) basis set is notably segregated from most basis sets regarding wRMSD^*, aligning with its comparatively poor refinement performance observed in Fig. 6.

The clustering of basis sets in the MDS, showing similar performance in Fig. 8, emphasizes the relatively minor influence of the basis set during HAR. Only minute differences in the refinement outcomes are observed for double-zeta and larger basis sets, indicating a limited possibility of spoiling a refinement by choice of a particular basis set.

IV. CONCLUSIONS

This study investigated the impact of various parameters regarding the computational methods on the quality of resulting Hirshfeld atom refinement (HAR) models. Based on conflicting information in the existing literature, a recommendation has lacked a clear consensus. A well-defined set of test structures with sufficient experimental data quality was crucial to address this issue and ensure reliable results systematically. The selection of amino acids represented a diverse chemical range within a defined group and, coupled with enhanced experimental data quality, established a distinct and reliable test set. A comprehensive permutation of refinement parameters was conducted to avoid oversight of potential influences or correlations.

Among these parameters, the SCNL correction did not qualitatively affect the outcomes, while applying a continuum water solvent model proved beneficial for the tested molecules compared to a calculation in vacuo. The integration grid accuracy should be chosen as “Normal,” as it balances refinement outcome and computational effort. The method and basis set exhibited the highest variability and potential influence on refinement outcome. Various DFT methods demonstrated performance proportional to their computational effort, approximately aligning with a Jacob's ladder ranking. Especially the use of the fast R2SCAN functional was beneficial, and it outperformed other, more sophisticated functionals, most notably the popular B3LYP functional.

In contrast, while HF significantly neglects electron exchange, it outperformed all DFT methods across all structures in the test set. This observation can be understood as a hint that another effect overcompensates the missing electron density redistribution into the electronic structure's core during HAR. One possibility is the neglect of the atomic charge by Hirshfeld partitioning, which other partitioning schemes can overcome.^52,98 Future studies could also be extended to explicitly include packing effects or perform periodic calculations to more accurately capture the crystalline surroundings, which may significantly influence the observed trends and provide a more comprehensive assessment of the methods.

Small Pople basis sets were generally less performant than the remaining ones. Larger bases, particularly from the def2 series, demonstrated superior performance with parameter combinations that were beneficial during their benchmark. MDS's qualitatively independent analysis supported the observed trends regarding method and basis set choices.

In summary, based on the obtained results, the following settings are recommended for a HAR on polar organic molecules:

• •Method: R2SCAN
• •basis set of the def2 family: def2-SVP or higher
• •Integration accuracy: Normal
• •Solvation: Water
• •SCF Convergence: NormalSCF or NoSpherA2SCF

These settings do not guarantee the best results, but they have yielded comparable results to computationally more demanding settings while compromising on the required computational effort.

Settings like “High” integration accuracy or extremely sophisticated basis sets like cc-pVQZ or 6–311++G(2d,2p) do not perform better than the recommended settings for polar organic molecules and can, therefore, be avoided. Also, functionals like B3LYP and calculations in vacuo should be exchanged for either more efficient ones or more suitable ones, as shown above.

V. EXPERIMENT

The amino acid crystals were obtained by recrystallizing the commercially obtained bulk materials. Details are described in the supplementary material. The diffraction experiments were conducted on a STOE STADIVARI Eulerian 4-circle diffractometer. All measurements utilized Mo-K_α radiation generated from a Genix microfocus tube, and the collection of intensities used a DECTRIS Pilatus 200 K detector. The experiments were conducted at 100 K, cooled by an Oxford Cryostream 800 with nitrogen gas. The frames were integrated with integrate3d and raw intensities further corrected for absorption by multi-scan absorption correction and scaling with STOE's LANA software. The unmerged data were subsequently employed for structure solution and refinement by olex2.solve and olex2.refine.^36,37 The resulting IAM structure models are the starting points for all HARs as described.

AUTHOR DECLARATIONS

Conflict of Interest

The authors have no conflicts to disclose.

Author Contributions

Daniel Brüx: Conceptualization (equal); Data curation (lead); Investigation (lead); Methodology (equal); Software (equal); Validation (equal); Visualization (lead); Writing – original draft (lead). Florian Meurer: Conceptualization (equal); Software (equal); Writing – review & editing (equal). Florian Kleemiss: Conceptualization (equal); Funding acquisition (lead); Methodology (supporting); Project administration (lead); Supervision (lead); Writing – review & editing (equal).

DATA AVAILABILITY

The data that support the findings of this study are available within the article and its supplementary material.