Skip to main content
NCBI home page
ACS AuthorChoice logoLink to ACS AuthorChoice
. 2026 Jan 13;130(4):1433–1439. doi: 10.1021/acs.jpcc.5c08003

Examination of Replicate Syntheses of Metal Organic Frameworks as a Window into Reproducibility in Materials Chemistry

David S Sholl †,‡,*
PMCID: PMC12862813  PMID: 41635431

Abstract

Replicate experiments are a useful tool in understanding the repeatability of scientific measurements. In 2019, a systematic search for replicate syntheses of a collection of 130 metal–organic frameworks (MOFs) found that 89% of these materials had no reported replicate syntheses apart from the original publications identifying the material ( AgrawalM., et al. Proc. Natl. Acad. Sci. U.S.A. 2020, 117, 877−882 10.1073/pnas.1918484117 ). A potential weakness of that search was that only 5–11 years had elapsed since the original publication of each material. Here, this analysis is extended to all publications 11–17 years after the original publication. Although this extended time period identifies more repeat syntheses, 83% of the materials still have no reported replicate syntheses. We also consider how appropriately selected Density Functional Theory (DFT) calculations can provide corroboration for the experimentally reported crystal structures. By using data from previous high-throughput DFT studies, corroborating evidence from DFT was available for 17% of the 130 structures for which no replicate syntheses are available. In total, approximately 1/3 of the 130 MOFs have data associated with replicate synthesis experiments and/or directly corroborating DFT calculations.

graphic file with name jp5c08003_0003.jpg

graphic file with name jp5c08003_0002.jpg

Introduction

Understanding the reliability and replicability of data reported in the scientific literature is critical to any effort to advance a scientific field. Estimating the reliability of individual reports is a key skill developed by seasoned researchers, since basing future work on results that are uncertain (in the opinion of the researcher) is wasteful. In this mode of research it could be argued that low quality data has limited impact, assuming of course that expert researchers can correctly identify data of this type. This description does not apply, however, to the increasing number of “big data” studies that aim to collect large data sets from published reports and then make predictions using machine learning (ML) methods. In general terms these studies tend to weigh each data point equally, so low quality or erroneous data can significantly compromise ML conclusions if it is prevalent.

Because of the importance of repeatability of scientific data it is useful to move beyond anecdotal experience by systematically assessing the scientific literature. Carefully repeating experiments is perhaps the most powerful way to examine these issues. Well-publicized efforts to reproduce experimental studies in subdisciplines of biomedical science that reported low rates of repeatability , were pivotal in drawing attention to these issues. An impressive recent analysis of reproducibility of more than 1000 claims from 52 years of literature of immunity research in Drosophila identified 45 claims that had not been tested in repeat experiments. A large fraction of these claims were found to be nonreproducible when tested in repeat experiments.

Systematically repeating experiments from previous studies is of course time-consuming and expensive. An alternative approach is to perform coordinated multilaboratory studies in which multiple replicates of selected experiments are used to assess the uncertainties associated with specific measurements. Examples of this approach include generation of reliable data for gas adsorption in reference materials, , comparison of polymer characterization using gel permeation chromatography, testing methods for angle-resolved light scattering, examining reproducibility in X-ray reflectometry measurements, and analysis of systematic uncertainties in measurements of deuterium content in deuterated carbon films. This approach can also be used for computational methods, as in the work of Lejaeghere et al. comparing the performance of a range of codes for performing Density Functional Theory (DFT) calculations.

Multilaboratory studies can reveal sources of significant discrepancies in the reported values of quantities that might otherwise be thought of as highly controlled. In a study involving more than 40 authors, Osterrieth et al. uncovered wide variability in data processing of N2 adsorption isotherm data to determine BET surface areas for porous materials. This work led to development of standardized software tools for this common analysis. In a similar example, a multilaboratory study in 1990 of measurements of extensional viscosity in dilute polymer solutions using samples prepared in a single location gave results varying by 3 orders of magnitude. This outcome spurred development of new and more reliable methods that were later demonstrated in a second multilaboratory study.

Targeted replication of previous experiments and multilaboratory studies are both resource intensive. A third approach that can give information about the reliability of literature data is to search for replicates that already exist in published reports. In highly active subfields of research it is relatively common for replicate experiments to be reported because more than one investigator generates similar ideas. In a series of papers examining single-component adsorption isotherms in metal–organic frameworks (MOFs), 1062 replicate isotherms of this kind were identified. − By applying statistical tests, 16.5% (that is, roughly 1 in 6) of these isotherms were classified as outliers relative to other replicates with the same material at the same temperature. This kind of analysis does not provide information about why some reports are outliers, but knowing that outliers exist can be a useful motivation to understand potential sources of experimental variability. In one example of this kind, Sarswat et al. performed detailed studies of the synthesis of the MOF MUF-16, determining that trace impurities in the synthesis mixture played an important role in stabilizing the adsorption properties of the resulting crystals. Identification of extant replicate experiments has also been useful in considering the reliability of mixture adsorption data in porous materials.

A prerequisite for the replicate adsorption experiments described above is that the porous material of interest has been synthesized more than once. This observation suggests another tool for probing how much is known about the repeatability of literature reports for these materials, namely the likelihood that a material reported as a new material is subsequently synthesized again for any purpose. If a material is reported in the literature once but no later synthesis of the same material is known, it is not possible to draw any firm conclusion about the repeatability of the material’s properties aside from estimations based on consideration of materials that are judged to be similar.

The discussion above motivates interest in understanding how often new materials in the materials chemistry literature are synthesized after the report that describes the material for the first time. In 2019 Agrawal et al. tackled this question for metal–organic frameworks by selecting 130 MOFs that were first reported between 2007 and 2013. These materials were chosen randomly from the CoRE MOF database, a collection of thousands of experimental crystal structures. Agrawal et al. examined all papers from 2018 and earlier that cited the original papers describing each of the 130 MOFs. A key conclusion from this analysis was that 115 of the 130 MOFs (88.5%) had no reported repeat synthesis.

Agrawal et al. also emphasized that a small number of MOF structures exist that have been synthesized in many (sometimes hundreds) of separate experiments. Examples of these structures include HKUST-1 (also known as CuBTC) and ZIF-8. Agrawal et al. noted considerable variation in the reported BET surface areas of these “canonical” MOFs, an observation that in part motivated the later work on reproducibility of analysis of N2 adsorption isotherms by Osterrieth et al. The existence of materials that have been described in many independent reports has important implications for efforts to understand the sources of material variability.

A potential weakness of the work of Agrawal et al. is that it only considered papers relevant to the 130 randomly selected materials published up to 2018. Below, we extend the search for replicate experiments to papers published until 2024, 11–17 years after the original publications selected by Agrawal et al. We also consider if there are situations in which information from computational models, specifically, from DFT calculations, can be used to provide evidence corroborating experimentally reported crystal structures.

Experimental Replicates of MOF Synthesis

For consistency with the work of Agrawal et al., the same 130 MOFs used in that earlier work were considered. These 130 MOFs represent ∼2.7% of the >4700 crystal structures collected in the 2014 version of the CoRE MOF database. Each of these structures is an experimentally reported material that was first reported between 2007 and 2013. Large numbers of additional MOF structures have been reported since 2013, , but for the purposes of this analysis it is useful to only include materials for which many years has passed since their first synthesis.

In the discussion below, repeat syntheses of the 130 MOFs between 2007 and 2018 were identified from the work by Agrawal et al. To search for repeat syntheses between 2019 and 2024, all citations of the original publication reporting each of the 130 MOFs listed by the Web of Science were collected and examined. This procedure gave 1339 publications, adding to the more than 4600 publications examined by Agrawal et al. On average, each original publication had been cited 45.8 times by 2024, with 10.3 of these citations occurring between 2019 and 2024. Of the 1339 publications examined for this work, 24.9% (334 publications) were review articles that did not contain any new data. 5.4% (72 publications) of the total were unavailable to this author and 2 publications had been retracted; these 74 publications were not considered further.

Figure shows that average citations per year for each original paper using citation counts from the Web of Science. Although there is some scatter, the data for two or more years after publication is reasonably well described by the simple function n = 9.42 exp­(−0.164t), where t is the number of years postpublication. Assuming this simple function is valid for all t > 1 predicts that each publication will ultimately receive 47.7 citations on average. For each of the 14 original publications from 2007, this approach estimates that 3.5 citations will appear in the future that were not captured in the analysis presented below. Repeating this calculation for each of the publication dates for the original papers predicts that a total of 820 future citations of the set of 130 publications can be expected. Said differently, the analysis below considered 88% of all citations of the 130 original publications that are expected at any time, past or future. The original analysis by Agrawal, by contrast, covered 68% of all citations for the publications.

1.

1

Open in a new tab

Average number of citations per year as a function of time after publication for the original papers reporting initial synthesis of the 130 selected MOFs. The dashed line is fitted to an exponential decay excluding data for 0 and 1 years.

Table summarizes the number of repeat syntheses that were identified for each of the 130 MOFs considered. Extending the period of analysis to include 2019–2024 significantly increased the number of replicate syntheses reported. By 2024, repeat synthesis had been reported for 22 of the 130 materials (16.9%), 7 more materials than the period before 2019. In the period before 2019 only one material (structure code SAPBIW, also known as Bio-MOF-100) had more than two repeat syntheses. By 2024, 6 of the materials (4.6%) had more than two repeat syntheses, including structure codes IZUMUM, ZEDZAL, HOMZEP (also known as MIL-96), MACHIJ, and KEQJEX. In total, 58 repeat syntheses were reported, with SAPBIW accounting for 18 (31%) of these cases.

1. Number of Repeat Syntheses Reported for the 130 Selected MOFs .

Structure code # repeat syntheses (all years) # repeat syntheses before 2019 # repeat syntheses 2019–2024
SAPBIW 18 6 12
IZUMUM 6 2 4
ZEDZAL 4 2 2
HOMZEP 4 0 4
MACHIJ 3 1 2
KEQJEX 3 1 2
UHISOU 2 2 0
NAYXOC 2 2 0
XUBJAF02 2 1 1
ADODAA 2 0 2
QUQGAL 1 1 0
HAWREE 1 1 0
QEGNOH 1 1 0
UFOFIF 1 1 0
MUVJIX 1 1 0
XUNGUJ 1 1 0
GEDLIM 1 1 0
TOKDON 1 0 1
XOJWEZ 1 0 1
ILITUT 1 0 1
OYUJUO 1 0 1
UVEVUN 1 0 1
Open in a new tab
a

The 108 MOFs without any reported repeat syntheses are not listed.

It is interesting to ask whether repeat syntheses are evenly distributed in time. The results from 2019 to 2024 amount to 42.5% of the time since the original publications (measured in years). 59% (34 of 58) of the repeat syntheses took place during this period. If SAPBIW, which accounted for 31% of the repeat syntheses from all dates, is excluded from this analysis then 65% of the repeat syntheses occurred in 2019–2024. This data suggests that the rate at which repeat syntheses are reported per unit of time increases at dates well after the appearance of the original publication, even though the rate of new citations decreases steadily (see Figure ).

Quantum Chemistry Calculations as Tests of MOF Structures

The discussion above focused entirely on experimental syntheses of materials. Another potential source of information about material properties, if appropriately interpreted, is computational models. The 130 MOFs analyzed above were selected from the CoRE MOF database, a collection of structures that was generated with the specific aim of making these materials readily accessible for computational models. Although hundreds of studies have used the CoRE MOF database to simulate properties of large collections of MOFs, the majority of these studies assumed that each MOF structure was rigid in the structure reported in the CoRE MOF data set. Calculations of this kind have provided many insights into MOF properties, but they cannot be used to draw conclusions about the MOF structures themselves.

Relatively recently, large-scale Density Functional Theory (DFT) calculations have been performed for many MOFs from the CoRE MOF database and other sources. In cases where the structure of the MOFs is energy minimized without constraints on atom positions or unit cell size or shape these DFT calculations can provide a useful supplement to experimental information about MOF structure. Nazarian et al. showed by comparing DFT-optimized structures to MOFs for which high quality single crystal structures were available for solvent-free crystals that the volume of MOF unit cells from DFT is not sensitive to the DFT exchange-correlation functional and that close agreement between DFT and experiment should be expected.

The ODAC23 and ODAC25 data sets of Sriram et al. , each reported DFT-optimized MOF structures for thousands of MOFs, many of which were taken from the CoRE MOF data set. Many of the MOFs in the original CoRE MOF collection were excluded from these DFT calculations because they had very small pores or else had structural problems such as missing atoms or overlapping atoms. The challenges associated with the latter problems have largely been resolved in the most recent release of the CoRE MOF data set by Zhao et al. Other collections of DFT calculations for MOFs have also been reported, but to illustrate the concept of using this data to test MOF structures the analysis below focuses solely on data from the ODAC23 and ODAC25 data sets.

There are DFT-optimized structures in ODAC23 or ODAC25 for 50 of the 130 MOFs described above, including 39 MOFs for which no experimental repeat syntheses have been reported. A simple quantity that compares the DFT-optimized and experimental structures is the unit cell volume, V. There are numerous examples where the two unit cell volumes are in good agreement. The MOF with structure code TIRLIQ, for example, (V exp = 4181 Å3) has V DFT/V exp = 0.982. There are also examples, however, where the experimental and DFT unit cell volumes differ strongly, including DUQSEO (V exp = 1305 Å3) with V DFT/V exp = 0.776. In these two examples no repeat syntheses of the MOFs have been reported, so it is not possible to draw insight from experiments other than the original reported structure. Table S1 lists the unit cell volumes from DFT calculations and experiments.

Of the 50 MOFs for which DFT data was obtained, 29 of them have 0.95 < V DFT/V exp < 1.05. For the remaining 21 cases where V DFT and V exp are very different it may be tempting to think that the original experimental structure is somehow faulty. This conclusion, however, is not correct. Many MOF crystal structures are determined experimentally from crystals that include ordered or disordered solvent species in the MOF’s pores. The CoRE MOF data set, however, explicitly removed solvent molecules to give the “bare” MOF, and examples are known where DFT optimization leads to pore collapse. There are also examples where the processing of experimental crystal structures to produce computation-ready structures introduced unphysical results due to missing atoms, incorrect resolution of experimentally observed disorder, and so on. ,

For the reasons discussed above, it seems prudent to only consider examples with 0.95 < V DFT/V exp < 1.05 as providing partial confirmation from DFT of the originally reported structure. This outcome certainly cannot “prove” that the reported structure is correct, and is perhaps best viewed as being similar to having two independent experiments with matching PXRD patterns. The ODAC23 and ODAC25 data sets provide confirmation of this kind for 29 of the 130 MOFs of interest, including 22 MOFs for which no repeat syntheses have been reported experimentally. Table summarizes the MOFs for which repeat syntheses and/or DFT-optimized structures with 0.95 < V DFT/V exp < 1.05 were found. It is a coincidence that the number of materials for which repeat synthesis has been reported (22/130) and the number of materials for which DFT data exists but no experimental repeats are available (22/130) are identical. It is nevertheless striking that the careful inclusion of data from computation doubles the fraction of materials for which information going beyond a single experimental report is available.

2. MOF Structure Codes from the 130 MOFs Analyzed for Which Experimental Repeat Syntheses and/or DFT Calculations with 0.95 < V DFT/V exp < 1.05 Were Found .

Experimental repeat only Experimental repeat and DFT confirmation DFT confirmation only
XUBJAF02 (2) TOKDON (1) HOMZEP (4) MUVJIX (1) HEXNII TIRLIQ
UFOFIF (1) XOJWEZ (1) QUQGAL (1) IZUMUM (6) KONCIA MOGNAY
UHISOU (2) ILITUT (1) ADODAA (2) GEDLIM (1) CUGVUW LUKLIN
MACHIJ (3) XUNGUJ (1)   MUNPAN RUCGOM
OYUJUO (1) UVEVUN (1)   CURBOH MUTVUT
KEQJEX (3) NAYXOC (2)   QUQGEP UKUBUY
QEGNOH (1) SAPBIW (18)   AXUBOL EBUREA
HAWREE (1) ZEDZAL (4)   IBUYAH NALYEG
    PAMHIW BAXSIE
    HEBKEG PEMRIK
    FEZREJ PETWOC
    SEQTEP
Open in a new tab
a

Numbers in parentheses indicate the number of reported repeat syntheses. Structures are listed in the order reported by Agrawal et al.

The analysis above of experimental repeat synthesis attempted to be comprehensive in the sense that it examined all published papers that cited the original publications. A similar claim cannot be made for the DFT data analyzed here, since this data was drawn from two specific sources in which DFT calculations were reported for large numbers of MOFs. It is possible that additional DFT calculations are available in the literature that would extend the list in Table . We reiterate that only calculations that allow the unit cell size and shape and atom positions to fully relax are useful for this purpose.

We have focused here on data from DFT calculations because unambiguous comparisons between these calculations and high-quality crystal structures have confirmed the accuracy of the calculations. Force fields of various kinds have also been used to relax large numbers of MOF structures, but the precision of these force fields is more uncertain than DFT. As general purpose force fields continue to improve, calculations with these methods could become a useful supplement to the DFT results used here.

Discussion and Conclusion

The main aim of this study was to assess how many MOF materials have been synthesized more than once as reported in the scientific literature. Specifically, 130 MOFs were randomly selected for which 11–17 years of citation data since the original reports was available. Using 5–11 years of citation data for these MOFs, Agrawal et al. found that 89% of the materials had not been synthesized aside from their original report. Extending the analysis with 6 more years of data reduced this fraction slightly, although still 83% of the 130 materials have no reported repeat syntheses. Of the materials for which repeat syntheses have been reported, 12 (9.2%) have exactly one repeat synthesis to date and 10 (7.7%) have two or more repeats. One material (structure code SAPBIW, also known as Bio-MOF-100) accounted for 31% of all the repeat syntheses that were found.

We also explored whether some kinds of computational results could be considered as corroborating experimental syntheses. Specifically, we considered DFT calculations that allow full relaxation of the MOF crystal structure, since agreement between a calculation of this kind and experimental data can be viewed as corroboration of the experimental result. Using DFT data from previous high throughput computational studies , we found that 29 of the 130 MOFs had 0.95 < V DFT/V exp < 1.05. This set of materials included 22 MOFs for which no repeat syntheses have been reported experimentally. Combining the repeat syntheses and DFT corroboration, 45 of the 130 materials (35%) have some evidence of structure or properties in the literature beyond the original report. Said differently, for roughly 2 in 3 of the MOFs considered, the only direct evidence of structure or properties that is available to date is the original report. This observation has important implications for high throughput computational screening studies that aim to find “high performing” materials from large collections of known materials, namely that very little direct information about the repeatability or robustness of many of the screened materials is available.

There are multiple ways in which computational results could be used to provide actionable information that goes beyond the results listed above. In cases where DFT data is available but V DFT/V exp < 0.95 or 1.05 < V DFT/V exp, careful comparisons with the original experimental data could be made to understand whether this discrepancy has a simple physical origin such as missing solvent species in the calculation or else could raise questions about the reliability of the original crystal structure. In cases where DFT data was not available from the specific sources we considered, targeted calculations could be performed. Similarly, other levels of theory such as force field calculations could be considered as alternative sources of information.

The question of how much information a repeat synthesis or corroborating calculation provides about the repeatability of a material’s properties is a subtle one. In many reports that include repeat synthesis the focus is on testing “new” properties or functions of a material, not quantitatively testing previously reported properties. In many examples the only reported data in common between two reports is a powder X-ray diffraction pattern. This data is suitable for qualitative identification of the material, but variations of materials with very similar diffraction patterns can have widely varying performance for properties such as gas adsorption. Nevertheless, the observation that two (or more) groups of researchers have successfully produced crystals following a stated synthesis procedure gives insight into the robustness of the material that simply cannot be known for materials that have only been reported once. Many researchers would consider evidence from experimental repeat syntheses as stronger than purely computational information. As described above, however, thoughtfully including computational data can greatly increase the number of materials for which information on any kind about repeatability is available.

It is interesting to speculate why many of the MOFs analyzed here appeared once in the literature and have not been used in further experiments. The most parsimonious explanation is simply that because the chemical space of available materials is enormous and the chemical research enterprise strongly favors originality most materials did not attract attention simply because of lack of clear motivating interests. It is also possible that some of these materials have been considered but were not used because of perceptions that they would be difficult to make or would not have “interesting” properties. There may also be cases where repeat syntheses were attempted unsuccessfully but not reported. It is not possible from the data examined in this paper to distinguish between these possibilities. Agrawal et al. found circumstantial evidence that suggested many repeat syntheses of MOFs were performed but not reported as part of studies to synthesize new variants of previous materials. The ready availability of Supporting Information in publications means that it would be helpful to the entire research community for “test” experiments of this kind to be reported even when they are not the main thrust of a new paper.

This paper focused on a very specific class of materials, MOFs, because of the availability of materials databases of crystal structures and well-defined structure codes associated with individual materials. It seems likely that the qualitative conclusion that many new chemical species or materials are not synthesized or characterized further beyond their original report also applies to many other subfields within materials chemistry. It would be interesting to perform similar quantitative analysis of experimental replicates for other classes of materials to explore how generalizable these conclusions are.

There has been considerable recent attention in the research community to the potential of using automated or self-driving laboratories to synthesize and characterize materials. We conclude with two suggestions for using facilities of this kind to advance the repeatability of synthesis of materials such as MOFs. First, automated workflows for synthesis should routinely be used to produce multiple independent samples so that data on batch-to-batch variability or consistency can be reported. This choice may not be practical during synthesis campaigns aimed at screening large numbers of potential materials, but should be routine during initial calibration of instrumentation and, just as importantly, for materials selected during screening as “interesting”. Second, a “grand challenge” problem for automated synthesis would be to systematically replicate the reported synthesis of a diverse collection of previously reported materials. To give a specific example, an effort that replicated the synthesis of the 130 MOFs analyzed above would be an extremely impressive showcase for an automated laboratory. Efforts to overcome the difficulties that would surely exist for such a showcase would undoubtedly move the field of automated synthesis forward in useful ways.

Supplementary Material

jp5c08003_si_001.pdf (230KB, pdf)

Acknowledgments

This work was supported by funds from the Oak Ridge National Laboratory LDRD Program. The assistance of Dr. Salah Eddine Boulfelfel in producing the figures is appreciated.

Biography

David S. Sholl is the Executive Vice President for Research at Rice University, where he is also a Professor of Chemical and Biomolecular Engineering and Chemistry. Prior to joining Rice in January 2026, David was the Director of the Transformational Decarbonization Initiative at the Oak Ridge National Laboratory from 2021 to 2025 and the School Chair of Chemical and Biomolecular Engineering at the Georgia Institute of Technology from 2013 to 2021. He received his Ph.D. in Applied Mathematics from the University of Colorado Boulder in 1995.

The Supporting Information is available free of charge at https://pubs.acs.org/doi/10.1021/acs.jpcc.5c08003.

  • Tabulated data comparing experimental and DFT unit cell volumes for all MOFs considered; tabulated data underlying Figure (PDF)

This manuscript has been authored by UT-Battelle, LLC, under contract DE-AC05-00OR22725 with the US Department of Energy (DOE). The publisher acknowledges the US government license to provide public access under the DOE Public Access Plan (http://energy.gov/downloads/doe-public-access-plan).

The author declares no competing financial interest.

References

  1. Prinz F., Schlange T., Asadullah K.. Believe it or not: how much can we rely on published data on potential drug targets? Nat. Rev. Drug Discov. 2011;10:712–U781. doi: 10.1038/nrd3439-c1. [DOI] [PubMed] [Google Scholar]
  2. Begley C. G., Ellis L. M.. Raise standards for preclinical cancer research. Nature. 2012;483:531–533. doi: 10.1038/483531a. [DOI] [PubMed] [Google Scholar]
  3. Lemaitre J., Popelka D., Ribotta B., Westlake H., Chakrabarti S., Xiaoxue L., Hanson M. A., Jiang H., Di Cara F., Kurant E., David F., Lemaitre B.. A retrospective analysis of 400 publications reveals patterns of irreproducibility across an entire life sciences research field. bioRxiv. 2025:1. doi: 10.1101/2025.07.07.663460. [DOI] [Google Scholar]
  4. Nguyen H. G. T., Espinal L., van Zee R. D., Thommes M., Toman B., Hudson M. S. L., Mangano E., Brandani S., Broom D. P., Benham M. J., Cychosz K., Bertier P., Yang F., Krooss B. M., Siegelman R. L., Hakuman M., Nakai K., Ebner A. D., Erden L., Ritter J. A., Moran A., Talu O., Huang Y., Walton K. S., Billemont P., De Weireld G.. A reference high-pressure CO2 adsorption isotherm for ammonium ZSM-5 zeolite: results of an interlaboratory study. Adsorption. 2018;24:703–703. doi: 10.1007/s10450-018-9968-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Nguyen H. G. T., Sims C. M., Toman B., Horn J., van Zee R. D., Thommes M., Ahmad R., Denayer J. F. M., Baron G. V., Napolitano E., Bielewski M., Mangano E., Brandani S., Broom D. P., Benham M. J., Dailly A., Dreisbach F., Edubilli S., Gumma S., Mollmer J., Lange M., Tian M., Mays T. J., Shigeoka T., Yamakita S., Hakuman M., Nakada Y., Nakai K., Hwang J., Pini R., Jiang H., Ebner A. D., Nicholson M. A., Ritter J. A., Farrando-Perez J., Cuadrado-Collados C., Silvestre-Albero J., Tampaxis C., Steriotis T., Rimnacova D., Svabova M., Vorokhta M., Wang H., Bovens E., Heymans N., De Weireld G.. A reference high-pressure CH4 adsorption isotherm for zeolite Y: results of an interlaboratory study. Adsorption. 2020;26:1253–1266. doi: 10.1007/s10450-020-00253-0. [DOI] [Google Scholar]
  6. D’Agnillo L., Soares J. B. P., Penlidis A.. Round-robin experiment in high-temperature gel permeation chromatography. J. Polym. Sci. Pol Phys. 2002;40:905–921. doi: 10.1002/polb.10154. [DOI] [Google Scholar]
  7. Von Finck A., Herffurth T., Duparré A., Schröder S., Lequime M., Zerrad M., Liukaityte S., Amra C., Achour S., Chalony M., Kuperman Q., Cornil Y., Bialek A., Goodman T., Greenwell C., Gur B., Brinkers S., Otter G., Vosteen A., Stover J., Vink R., Deep A., Doyle D.. International round-robin experiment for angle-resolved light scattering measurement. Appl. Opt. 2019;58:6638–6654. doi: 10.1364/AO.58.006638. [DOI] [PubMed] [Google Scholar]
  8. Colombi P., Agnihotri D. K., Asadchikov V. E., Bontempi E., Bowen D. K., Chang C. H., Depero L. E., Farnworth M., Fujimoto T., Gibaud A., Jergel M., Krumrey M., Lafford T. A., Lamperti A., Ma T., Matyi R. J., Meduna M., Milita S., Sakurai K., Shabel’nikov L., Ulyanenkov A., Van der Lee A., Wiemer C.. Reproducibility in X-ray reflectometry: results from the first world-wide round-robin experiment. J. Appl. Crystallogr. 2008;41:143–152. doi: 10.1107/S0021889807051904. [DOI] [Google Scholar]
  9. Behrisch R., Mayer M., Jacob W., Assmann W., Dollinger G., Bergmaier A., Kreissig U., Friedrich M., Sun G. Y., Hildebrandt D., Akbi M., Schneider W., Schleussner D., Knapp W., Edelmann C.. Quantitative analysis of deuterium in a-C:D layers, a Round Robin experiment. J. Nucl. Mater. 2000;281:42–56. doi: 10.1016/S0022-3115(00)00069-6. [DOI] [Google Scholar]
  10. Lejaeghere K., Bihlmayer G., Björkman T., Blaha P., Blügel S., Blum V., Caliste D., Castelli I. E., Clark S. J., Dal Corso A., de Gironcoli S., Deutsch T., Dewhurst J. K., Di Marco I., Draxl C., Dulak M., Eriksson O., Flores-Livas J. A., Garrity K. F., Genovese L., Giannozzi P., Giantomassi M., Goedecker S., Gonze X., Grånäs O., Gross E. K. U., Gulans A., Gygi F., Hamann D. R., Hasnip P. J., Holzwarth N. A. W., Iusan D., Jochym D. B., Jollet F., Jones D., Kresse G., Koepernik K., Küçükbenli E., Kvashnin Y. O., Locht I. L. M., Lubeck S., Marsman M., Marzari N., Nitzsche U., Nordström L., Ozaki T., Paulatto L., Pickard C. J., Poelmans W., Probert M. I. J., Refson K., Richter M., Rignanese G. M., Saha S., Scheffler M., Schlipf M., Schwarz K., Sharma S., Tavazza F., Thunström P., Tkatchenko A., Torrent M., Vanderbilt D., van Setten M. J., Van Speybroeck V., Wills J. M., Yates J. R., Zhang G. X., Cottenier S.. Reproducibility in density functional theory calculations of solids. Science. 2016;351:aad3000. doi: 10.1126/science.aad3000. [DOI] [PubMed] [Google Scholar]
  11. Osterrieth J. W. M., Rampersad J., Madden D., Rampal N., Skoric L., Connolly B., Allendorf M. D., Stavila V., Snider J. L., Ameloot R., Marreiros J., Ania C., Azevedo D., Vilarrasa-Garcia E., Santos B. F., Bu X. H., Chang Z., Bunzen H., Champness N. R., Griffin S. L., Chen B. L., Lin R. B., Coasne B., Cohen S., Moreton J. C., Colón Y. J., Chen L. J., Clowes R., Coudert F. X., Cui Y., Hou B., D’Alessandro D. M., Doheny P. W., Dinca M., Sun C. Y., Doonan C., Huxley M. T., Evans J. D., Falcaro P., Ricco R., Farha O., Idrees K. B., Islamoglu T., Feng P. Y., Yang H. J., Forgan R. S., Bara D., Furukawa S., Sanchez E., Gascon J., Telalovic S., Ghosh S. K., Mukherjee S., Hill M. R., Sadiq M. M., Horcajada P., Salcedo-Abraira P., Kaneko K., Kukobat R., Kenvin J., Keskin S., Kitagawa S., Otake K. I., Lively R. P., DeWitt S. J. A., Llewellyn P., Lotsch B., Emmerling S. T., Putz A. M., Marti-Gastaldo C., Padial N. M., García-Martínez J., Linares N., Maspoch D., Suarez del Pino J. A., Moghadam P., Oktavian R., Morris R. E., Wheatley P. S., Navarro J., Petit C., Danaci D., Rosseinsky M. J., Katsoulidis A. P., Schröder M., Han X., Yang S. H., Serre C., Mouchaham G., Sholl D. S., Thyagarajan R., Siderius D., Snurr R. Q., Goncalves R. B., Telfer S., Lee S. J., Ting V. P., Rowlandson J. L., Uemura T., Liyuka T.. et al. How Reproducible are Surface Areas Calculated from the BET Equation? Adv. Mater. 2022;34:2201502. doi: 10.1002/adma.202201502. [DOI] [PubMed] [Google Scholar]
  12. Sridhar T.. An Overview of the Project-M1. J. Non-Newton Fluid. 1990;35:85–92. doi: 10.1016/0377-0257(90)85039-2. [DOI] [Google Scholar]
  13. Anna S. L., McKinley G. H., Nguyen D. A., Sridhar T., Muller S. J., Huang J., James D. F.. An interlaboratory comparison of measurements from filament-stretching rheometers using common test fluids. J. Rheol. 2001;45:83–114. doi: 10.1122/1.1332388. [DOI] [Google Scholar]
  14. Park J., Howe J. D., Sholl D. S.. How Reproducible Are Isotherm Measurements in Metal-Organic Frameworks? Chem. Mater. 2017;29:10487–10495. doi: 10.1021/acs.chemmater.7b04287. [DOI] [Google Scholar]
  15. Bingel L. W., Chen A., Agrawal M., Sholl D. S.. Experimentally Verified Alcohol Adsorption Isotherms in Nanoporous Materials from Literature Meta-Analysis. J. Chem. Eng. Data. 2020;65:4970–4979. doi: 10.1021/acs.jced.0c00598. [DOI] [Google Scholar]
  16. Bingel L. W., Walton K. S., Sholl D. S.. Experimentally Verified Alkane Adsorption Isotherms in Nanoporous Materials from Literature Meta-Analysis. J. Chem. Eng. Data. 2022;67:1757–1764. doi: 10.1021/acs.jced.1c00967. [DOI] [Google Scholar]
  17. Sarswat A., Bacsa J., Roy A., Marreiros J., Finn M. G., Sholl D. S., Lively R. P.. Investigating the Effect of Trace Levels of Manganese Ions During Solvothermal Synthesis of Massey University Framework-16 on CO Uptake Capacity. Chem. Mater. 2024;36:5378–5387. doi: 10.1021/acs.chemmater.4c00137. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Cai X. Q., Gharagheizi F., Bingel L. W., Shade D., Walton K. S., Sholl D. S.. A Collection of More than 900 Gas Mixture Adsorption Experiments in Porous Materials from Literature Meta-Analysis. Ind. Eng. Chem. Res. 2021;60:639–651. doi: 10.1021/acs.iecr.0c05398. [DOI] [Google Scholar]
  19. Agrawal M., Han R., Herath D., Sholl D. S.. Does repeat synthesis in materials chemistry obey a power law? Proc. Natl. Acad. Sci. U.S.A. 2020;117:877–882. doi: 10.1073/pnas.1918484117. [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Chung Y. G., Camp J., Haranczyk M., Sikora B. J., Bury W., Krungleviciute V., Yildirim T., Farha O. K., Sholl D. S., Snurr R. Q.. Computation-Ready, Experimental Metal-Organic Frameworks: A Tool To Enable High-Throughput Screening of Nanoporous Crystals. Chem. Mater. 2014;26:6185–6192. doi: 10.1021/cm502594j. [DOI] [Google Scholar]
  21. Chung Y. G., Haldoupis E., Bucior B. J., Haranczyk M., Lee S., Zhang H. D., Vogiatzis K. D., Milisavljevic M., Ling S. L., Camp J. S., Slater B., Siepmann J. I., Sholl D. S., Snurr R. Q.. Advances, Updates, and Analytics for the Computation-Ready, Experimental Metal-Organic Framework Database: CoRE MOF 2019. J. Chem. Eng. Data. 2019;64:5985–5998. doi: 10.1021/acs.jced.9b00835. [DOI] [Google Scholar]
  22. Zhao G., Brabson L. M., Chheda S., Huang J., Kim H., Liu K., Mochida K., Pham T. D., Prerna, Terrones G. G., Yoon S., Zoubritzky L., Coudert F.-X., Haranczyk M., Kulik H. J., Moosavi S. M., Sholl D. S., Siepmann J. I., Snurr R. Q., Chung Y. G.. CoRE MOF DB: A curated experimental metal-organic framework database with machine-learned properties for integrated material-process screening. Matter. 2025;8:102140. doi: 10.1016/j.matt.2025.102140. [DOI] [Google Scholar]
  23. An J., Farha O. K., Hupp J. T., Pohl E., Yeh J. I., Rosi N. L.. Metal-adeninate vertices for the construction of an exceptionally porous metal-organic framework. Nat. Commun. 2012;3:604. doi: 10.1038/ncomms1618. [DOI] [PubMed] [Google Scholar]
  24. Pérez-Yáñez S., Beobide G., Castillo O., Cepeda J., Luque A., Aguayo A. T., Román P.. Open-Framework Copper Adeninate Compounds with Three-Dimensional Microchannels Tailored by Aliphatic Monocarboxylic Acids. Inorg. Chem. 2011;50:5330–5332. doi: 10.1021/ic200560b. [DOI] [PubMed] [Google Scholar]
  25. Lin Z. J., Zou R. Q., Xia W., Chen L. J., Wang X. D., Liao F. H., Wang Y. X., Lin J. H., Burrell A. K.. Ultrasensitive sorption behavior of isostructural lanthanide-organic frameworks induced by lanthanide contraction. J. Mater. Chem. 2012;22:21076–21084. doi: 10.1039/c2jm33884j. [DOI] [Google Scholar]
  26. Volkringer C., Loiseau T., Férey G., Morais C. M., Taulelle F., Montouillout V., Massiot D.. Synthesis, crystal structure and Ga solid state NMR of a MOF-type gallium trimesate (MIL-96) with μ-oxo bridged trinuclear units and a hexagonal 18-ring network. Micropor Mesopor Mat. 2007;105:111–117. doi: 10.1016/j.micromeso.2007.05.018. [DOI] [Google Scholar]
  27. Yue Q., Yan L., Zhang J. Y., Gao E. Q.. Novel Functionalized Metal-Organic Framework Based on Unique Hexagonal Prismatic Clusters. Inorg. Chem. 2010;49:8647–8649. doi: 10.1021/ic100558x. [DOI] [PubMed] [Google Scholar]
  28. Stylianou K. C., Rabone J., Chong S. Y., Heck R., Armstrong J., Wiper P. V., Jelfs K. E., Zlatogorsky S., Bacsa J., McLennan A. G., Ireland C. P., Khimyak Y. Z., Thomas K. M., Bradshaw D., Rosseinsky M. J.. Dimensionality Transformation through Paddlewheel Reconfiguration in a Flexible and Porous Zn-Based Metal-Organic Framework. J. Am. Chem. Soc. 2012;134:20466–20478. doi: 10.1021/ja308995t. [DOI] [PubMed] [Google Scholar]
  29. Brabson L. M., Medford A. J., Sholl D. S.. Comparing Classical and Machine Learning Force Fields for Modeling Deformation of Metal-Organic Frameworks Relevant for Direct Air Capture. J. Phys. Chem. C. 2025;129:16811–16825. doi: 10.1021/acs.jpcc.5c04020. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Nazarian D., Ganesh P., Sholl D. S.. Benchmarking density functional theory predictions of framework structures and properties in a chemically diverse test set of metal-organic frameworks. J. Mater. Chem. A. 2015;3:22432–22440. doi: 10.1039/C5TA03864B. [DOI] [Google Scholar]
  31. Sriram A., Choi S., Yu X. H., Brabson L. M., Das A., Ulissi Z., Uyttendaele M., Medford A. J., Sholl D. S.. The Open DAC 2023 Dataset and Challenges for Sorbent Discovery in Direct Air Capture. Acs Central Sci. 2024;10:923–941. doi: 10.1021/acscentsci.3c01629. [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. Sriram A., Brabson L. M., Yu X., Choi S., Abdelmaqsoud K., Moubarak E., de Haan P., Lowe S., Brehmer J., Kitchin J. R., Welling M., Zitnick C. L., Ulissi Z., Medford A. J., Sholl D. S.. The Open DAC 2025 Dataset for Sorbent Discovery in Direct Air Capture. arXiv. 2025:1. doi: 10.48550/arXiv.2508.03162. [DOI] [Google Scholar]
  33. Chen T. Y., Manz T. A.. Identifying misbonded atoms in the 2019 CoRE metal-organic framework database. RSC Adv. 2020;10:26944–26951. doi: 10.1039/D0RA02498H. [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. Coupry D. E., Addicoat M. A., Heine T.. Extension of the Universal Force Field for Metal-Organic Frameworks. J. Chem. Theory Comput. 2016;12:5215–5225. doi: 10.1021/acs.jctc.6b00664. [DOI] [PubMed] [Google Scholar]
  35. Ko T. W., Ong S. P.. Recent advances and outstanding challenges for machine learning interatomic potentials. Nat. Comput. Sci. 2023;3:998–1000. doi: 10.1038/s43588-023-00561-9. [DOI] [PubMed] [Google Scholar]
  36. Llanos E. J., Leal W., Luu D. H., Jost J., Stadler P. F., Restrepo G.. Exploration of the chemical space and its three historical regimes. Proc. Natl. Acad. Sci. U.S.A. 2019;116:12660–12665. doi: 10.1073/pnas.1816039116. [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

jp5c08003_si_001.pdf (230KB, pdf)

Articles from The Journal of Physical Chemistry. C, Nanomaterials and Interfaces are provided here courtesy of American Chemical Society

RESOURCES