Pattern-matching indexing of Laue and monochromatic serial crystallography data for applications in materials science

An algorithm based on the matching of q vector pairs is combined with three-dimensional pattern matching using a nearest-neighbors approach to index Laue and monochromatic serial crystallography data recorded on small-unit-cell samples.

Abstract

Serial crystallography data can be challenging to index, as each frame is processed individually, rather than being processed as a whole like in conventional X-ray single-crystal crystallography. An algorithm has been developed to index still diffraction patterns arising from small-unit-cell samples. The algorithm is based on the matching of reciprocal-lattice vector pairs, as developed for Laue microdiffraction data indexing, combined with three-dimensional pattern matching using a nearest-neighbors approach. As a result, large-bandpass data (e.g. 5–24 keV energy range) and monochromatic data can be processed, the main requirement being prior knowledge of the unit cell. Angles calculated in the vicinity of a few theoretical and experimental reciprocal-lattice vectors are compared, and only vectors with the highest number of common angles are selected as candidates to obtain the orientation matrix. Global matching on the entire pattern is then checked. Four indexing options are available, two for the ranking of the theoretical reciprocal-lattice vectors and two for reducing the number of possible candidates. The algorithm has been used to index several data sets collected under different experimental conditions on a series of model samples. Knowing the crystallographic structure of the sample and using this information to rank the theoretical reflections based on the structure factors helps the indexing of large-bandpass data for the largest-unit-cell samples. For small-bandpass data, shortening the candidate list to determine the orientation matrix should be based on matching pairs of reciprocal-lattice vectors instead of triplet matching.

1. Introduction  

With the emergence of high-energy X-ray free-electron laser (XFEL) sources generating ultra-fast X-ray pulses of high brilliance, serial crystallography has become the method of choice to collect single-crystal X-ray diffraction data (Chapman et al., 2011 ▸; Boutet et al., 2012 ▸). By exposing a crystal to a single monochromatic X-ray pulse, a diffraction pattern is collected before radiation damage occurs. By combining a series of single-shot diffraction patterns obtained from randomly oriented crystals, a complete data set can be retrieved and the structure of complex systems studied, with a focus on macromolecular structural biology applications (Johansson et al., 2017 ▸). The sudden rise of serial crystallography has triggered the development of dedicated analytical tools to analyze the huge number of diffraction patterns collected, index the individual diffraction patterns and reconstruct usable reflection intensities (White et al., 2016 ▸; Hattne et al., 2014 ▸; Kabsch, 2014 ▸; Liu & Spence, 2016 ▸). Owing to the nature of the XFEL beam (each X-ray pulse has its own energy and intensity spectrum), the sample variability (the crystallites exposed to the beam may vary in size and crystallinity) and the measurement strategy (one or several crystals randomly oriented may diffract simultaneously), data processing can be very challenging.

In recent years, new strategies to index such complex data have been proposed, with a focus also on sparse data and smaller-unit-cell samples, in particular making use of prior knowledge of the unit cell (Brewster et al., 2015 ▸; Ginn et al., 2016 ▸; Li et al., 2019 ▸). For example, in order to obtain the crystal orientation matrix of still images with sparse data, Brewster et al. (2015 ▸) used a powder-like diffraction pattern reconstructed from the aggregate of thousands of still images to derive accurate cell information, before using the model powder pattern to assign initial Miller indices to reflections. Li et al. (2019 ▸) developed an auto-indexing algorithm for sparse and small-unit-cell diffraction data, comparing the lengths and angles of paired scattering vectors with referenced values derived from prior knowledge of the unit cell. Finally, dedicated tools have been developed to take into account the non-monochromatic nature of the XFEL beam in the indexing process (Gevorkov et al., 2020 ▸).

While the term ‘serial crystallography’ emerged with XFELs, the concept of collecting single-shot data and processing each frame individually was used at an earlier stage, especially in the case of Laue diffraction. When a crystal is exposed to a broad energy bandpass (a polychromatic pink or white beam), a reasonably large number of reflections can be recorded simultaneously in a single exposure. Because of this, the Laue method is a good alternative to the monochromatic one for in situ time-resolved studies of macromolecules (Moffat & Helliwell, 1989 ▸; Bourgeois et al., 2003 ▸; Yorke et al., 2014 ▸). If combined with a micrometre- or sub-micrometre-sized beam, Laue diffraction can also be used to map crystal orientation and strain in materials (Chen et al., 2016 ▸). In addition, a complete structure characterization only requires a few random orientations to be combined (Cornaby et al., 2010 ▸; Dejoie, McCusker, Baerlocher, Kunz & Tamura, 2013 ▸). The indexing of individual patterns collected using the Laue microdiffraction technique is generally based on a matching process of pairs of reciprocal-lattice vectors (Chung & Ice, 1999 ▸; Tamura, 2014 ▸), which requires prior knowledge of the unit cell. In short, each measured reflection is converted into normalized reciprocal-lattice vectors (they are normalized because their length is not directly accessible in Laue diffraction), and the angle between the vectors is matched with a list of expected angles calculated from the known unit cell. Data processing for structure solution based on a set of Laue reflection integrated intensity measurements tends to be difficult, mainly due to the energy dependence of the various correction factors and the overlap of harmonic reflections (Helliwell et al., 1989 ▸).

The non-monochromatic nature of the XFEL beam attracted our attention a few years ago. With a small energy bandpass (a few percent in ΔE/E, E being the energy of the incident beam), more reflections are in the diffraction condition, and the probability of a reflection intensity being truncated is also reduced. The possibility of measuring more Bragg peaks in a single shot is particularly interesting for samples with small unit cells. As a 4% energy bandpass beam was planned at the Swiss free-electron laser (SwissFEL) (Patterson et al., 2014 ▸), we developed a methodology to simulate such data and implement the data processing appropriate for small-unit-cell samples. An initial indexing of the simulated data was carried out using a Laue microdiffraction approach, showing that such a Laue indexing algorithm could be adapted to index data collected over a smaller energy bandpass (Dejoie, McCusker, Baerlocher, Abela et al., 2013 ▸). A short description of the indexing strategy was published by Dejoie et al. (2015 ▸). The ‘classic’ Laue approach of matching a pair of reciprocal-lattice vectors (Chung & Ice, 1999 ▸; Tamura, 2014 ▸) was combined with a three-dimensional pattern-matching approach based on nearest neighbors initially developed for fast 2D pattern matching of fingerprints (Van Wamelen et al., 2004 ▸), which appeared to be efficient for indexing these 4% bandpass data.

Keeping a similar combined approach, the code has been revised and optimized to index different types of data collected with varying energy bandpass, from Laue (5–24 keV range) to monochromatic. A complete description of the indexing algorithm is presented here, along with the results of indexing tests carried out on five small-unit-cell model samples (cell volume ranging from 722 to 6640 ų). First, angles in the vicinity of selected theoretical and experimental reciprocal-lattice vectors are calculated, and only vectors with the highest number of common angles are kept. The list of candidates from nearest-neighbors matching can be further reduced before checking for a global match. Four indexing strategies are available, depending on whether or not the crystallographic structure is (at least partially) known (this will affect the ranking of theoretical reflections), and on the type of matching process chosen (based on matching a pair or a triplet of reciprocal-lattice vectors). Our objective is to identify the main parameters influencing the indexing process, check the limitations of the indexing algorithm and propose the best indexing strategy depending on the type of sample and the type of diffraction data. We show that knowing the crystallographic structure of the sample helps the indexing of large-bandpass data for the largest-unit-cell samples. On the other hand, for the indexing of small-bandpass data, a matching process based on pairs of reciprocal-lattice vectors should be favored. The current indexing algorithm uses routines written in the XMAS software (Tamura, 2014 ▸), but can be used as a stand-alone program.

2. Sample tests and data acquisition  

The indexing algorithm was tested on X-ray diffraction data collected on five model samples. The chemical compositions of and main crystallographic information for each of the five samples are given in Table 1 ▸. A 30 µm thin section of feldspar (sanidine) (Ackermann et al., 2004 ▸) was provided by Professor H. R. Wenk (UC Berkeley, USA). Hydrated caesium cyano­platinate (CsPt) (Johnson et al., 1977 ▸) and ZSM-5 zeolite crystals (Olson et al., 1981 ▸; van Koningsveld et al., 1987 ▸) were provided by Dr P. Pattison (EPFL Lausanne, Switzerland) and Professor Henri Kessler (Université de Haute-Alsace, Mulhouse, France), respectively. The zirconium phosphate (ZrPOF) (Liu et al., 2009 ▸) and magnesium acetate (MgAc) samples were provided by Dr L. B. McCusker (ETH Zurich, Switzerland). The magnesium acetate structure was refined using single-crystal data collected on the ALS-11.3.1 beamline, and the results agree with the published structure (Scheurell et al., 2015 ▸).

Table 1. Main crystallographic information for the samples of the present study.

Name	Formula	Space group	Volume (Å3)	a (Å)	b (Å)	c (Å)	α (°)	β (°)	γ (°)
Sanidine	KAlSi3O8	C2/m	721.79	8.58320	13.0076	7.1943	90	116.023	90
CsPt	Cs2[Pt(CN)4]·H2O	P65	1619.73	9.791	9.791	19.510	90	90	120
ZrPOF	[(C9H8N)4(H2O)4]·[Zr8P12O40(OH)8F8]		1977.53	10.7567	13.8502	14.8995	109.6	101.1	100.5
ZSM-5	(SiO2)96	Pnma	5343.32	20.022	19.899	13.383	90	90	90
MgAc	Mg5(C2H3O2)8(OH)2	I41/a	6640.68	23.3126	23.3126	11.9855	90	90	90

X-ray single-crystal diffraction data were collected using monochromatic (ΔE/E ≃ 10⁻⁴) and non-monochromatic X-ray incident beams: ΔE/E ≃ 4%, E ≃ 11–17 keV (e.g. ΔE/E ≃ 50%) and E ≃ 5–24 keV. Information on all the data sets collected, varying both the energy range and the experimental setup [setups (1) to (5)], is summarized in Table 2 ▸.

Table 2. Single-crystal data sets collected for the five samples of this study, using five different setups [(1)–(5)].

The number of diffraction patterns per data set, the resolution range (d spacing), the average number of experimental peaks per frame and the number of independent reflections expected in the resolution range are indicated.

	Data set name	Monochromatic	4%	50%	LaueDET-50/60	LaueDET-90
	Energy range (keV)	17.75	16.9–17.7	10–17	5–24	5–24
	Setup No.	(1)	(2)	(3)	(4)	(5)
Sanidine	No. patterns		100	85	80	10
	Resolution range (Å−1)		Inf–0.450	3.599–0.406	5.872–0.312	1.563–0.227
	Average No. peaks per frame		16	67	98	201
	No. independent reflections		4337	5809	12 674	32 313
CsPt	No. patterns	100	100		12
	Resolution range (Å−1)	Inf–0.574	Inf–0.560		5.950–0.315
	Average No. peaks per frame	9	16		353
	No. independent reflections	523	551		2715
ZrPOF	No. patterns	21			17
	Resolution range (Å−1)	Inf–0.510			5.878–0.313
	Average No. peaks per frame	34			170
	No. independent reflections	7767			33 884
ZSM-5	No. patterns	100	100	17	15	19
	Resolution range (Å−1)	Inf–0.552	Inf–0.559	3.471–0.387	4.155–0.287	1.685–0.230
	Average No. peaks per frame	12	26	148	472	517
	No. independent reflections	4604	4427	12 884	31 063	59 358
MgAc	No. patterns			29		37
	Resolution range (Å−1)			3.599–0.406		1.670–0.229
	Average No. peaks per frame			118		213
	No. independent reflections			7150		37 989

Conventional monochromatic data [setup (1)] and 4% bandpass data [ΔE/E ≃ 4%, setup (2)] were collected on the Swiss–Norwegian Beamline (SNBL/BM01A) at the European Synchrotron Radiation Facility (ESRF). To do so, single crystals of sanidine, CsPt, ZSM-5 and ZrPOF were mounted on MiTeGen MicroMeshes. A two-dimensional DECTRIS Pilatus 2M detector was positioned at a distance of 224 mm from the sample. The broad bandpass mode was achieved by collecting a diffraction pattern while the monochromator was scanned over a 4% energy bandpass (average energy 17.34 keV or 0.7153 Å). The shape of the X-ray incident spectrum achieved in this way was extracted using the ‘reverse method’ from the sanidine data (Dejoie et al., 2011 ▸) [Fig. 1 ▸(a)]. The monochromatic data sets and 4% bandpass data sets were collected by rotating single crystals using 0.25 and 1° rotation steps, respectively. Geometry calibration (sample-to-detector distance, normal-incidence position of the detector, tilt angle of the detector) was carried out with the XMAS program (Tamura, 2014 ▸) using an LaB₆ reference powder pattern.

Figure 1.

Incident flux for the four non-monochromatic setups extracted from sanidine single-crystal data using the reverse method (Dejoie et al., 2011 ▸). (a) Setup (2), 4% bandpass. (b) Setup (3), 10–17 keV range (50% bandpass). (c) Setups (4) and (5), 5–24 keV range.

Laue diffraction (E ≃ 11–17 keV and E ≃ 5–24 keV) experiments were conducted on Beamline 12.3.2 at the Advanced Light Source (ALS) of the Lawrence Berkeley National Laboratory. A polychromatic X-ray beam (5–24 keV) was focused down to about 1 × 1 µm using a pair of Kirkpatrick–Baez (KB) mirrors (Tamura et al., 2003 ▸; Kunz et al., 2009 ▸). Laue microdiffraction patterns were collected using a two-dimensional DECTRIS Pilatus 1M X-ray detector. An exposure time of 1 s per pattern was used for all samples. The sample-to-detector distance, the center channel of the detector and the tilt of the detector relative to the sample surface were calibrated using a Laue pattern obtained from a strain-free Si single crystal.

Two different setups were used to collect full-range Laue data (5–24 keV). In the first setup [DET-90, setup (5)], the two-dimensional Pilatus detector was positioned at 90° with respect to the incident beam and the sample at 45° (reflection geometry). The distance from the sample to the center of the detector was ∼141 mm. This configuration is optimized to collect more data in a single shot by favoring the high-resolution (low d-spacing) range and is used in a routine way for strain and stress mapping (Tamura et al., 2003 ▸). Single-shot patterns were collected on single crystals of ZSM-5 and MgAc randomly dispersed on a glass slide, and on the thin section of sanidine. In the second setup [setup (4)], both reflection and transmission geometries were used. First, the two-dimensional detector was placed at 60° (sample-to-detector distance ∼147 mm) and the sample at 15° relative to the incident X-ray beam (reflection geometry, DET-60), and single-crystal data were collected on single crystals of ZSM-5 and MgAc randomly dispersed on a glass slide. For CsPt, ZrPOF and sanidine, data were collected in transmission geometry with the two-dimensional detector positioned at 50° (sample-to-detector distance ∼167 mm, DET-50). The crystals were spread over MiTeGen Micromounts covered with a 10 nm Au layer. Data were collected using the methodology described by Dejoie, McCusker, Baerlocher, Kunz & Tamura (2013 ▸). Both DET-60 and DET-50 setups give access to lower-resolution data. The shape of the X-ray incident spectrum corresponding to setups (4) and (5) can be seen in Fig. 1 ▸(c).

The reduced Laue range [E ≃ 11–17 keV, setup (3)] was achieved in the following way: the high-energy part of the beam was cut by increasing the pitch angle of the vertically focusing KB mirror, and the low-energy part was restricted using the low-energy threshold of the Pilatus detector. As a result, the vertical size of the beam increased to ∼4 µm, the overall intensity of the incident flux decreased by about 20%, and the number of reflections of different orders (harmonics) decreased from 25% (Dejoie, McCusker, Baerlocher, Kunz & Tamura, 2013 ▸) to 5%. The resulting incident flux as a function of energy is shown in Fig. 1 ▸(b). Some remaining intensity can be seen between 17 and 19 keV, with limited impact on data processing. The detector was positioned at 50° relative to the incident X-ray beam, for a sample-to-detector distance of ∼167 mm. Sanidine, ZrPOF and MgAc crystals were spread on MiTeGen Micromounts and measured in transmission geometry. ZSM-5 data were measured using a reflection geometry (sample positioned at 15° relative to the incident beam).

Powder diffraction data were collected on the high-resolution powder diffraction beamline (ID22) at the ESRF (Grenoble, France). The ZrPOF, ZSM-5 and MgAc samples were packed in 0.7 mm-sized borosilicate capillaries and measured using a wavelength of 0.399963 Å over 40° (2θ) at a speed of 2° min⁻¹. Since the ZrPOF sample suffered radiation damage, several fast 2θ scans (15° min⁻¹) were collected on a fresh zone of the sample at 100 K before averaging. Pawley fits were carried out using the TOPAS software (Coelho, 2018 ▸).

3. Description of the indexing algorithm  

The indexing algorithm combines the strategy of matching a pair of reciprocal-lattice vectors, first proposed by Chung & Ice (1999 ▸) for the indexing of Laue microdiffraction data, with a nearest-neighbor ranking approach initially proposed for pattern matching of 2D data such as fingerprint matching (Van Wamelen et al., 2004 ▸). An initial version of the algorithm was written for the indexing of 4% bandpass data (Dejoie, McCusker, Baerlocher, Abela et al., 2013 ▸; Dejoie et al., 2015 ▸). The current version can be used to index several types of serial crystallographic data, from monochromatic to Laue. There are two important aspects in relation to the indexing process: (i) it requires prior knowledge of the unit-cell parameters of the sample; (ii) it does not require knowledge of the energy of the diffraction peaks, which is retrieved during the indexing process. The main steps of the indexing process are described below, and a flowchart is given in Fig. 2 ▸.

Figure 2.

A flowchart for the indexing algorithm.

3.1. Step 1: data pre-processing  

Prior to indexing, experimental patterns were processed (background subtraction, peak search) using the routines developed in the XMAS software (Tamura, 2014 ▸). The positions of the experimental peaks (X _exp, Y _exp) in pixels on the 2D diffraction images were obtained, and the corresponding integrated intensities (I _int) extracted using a simple box method, before being ranked in order of decreasing values. The current version of the indexing algorithm uses a simple text file per frame as input, in which the positions of the experimental peaks in pixels and their integrated intensities for a given frame of interest are listed. The next step is to convert the reflection positions measured on the area detector into lattice vectors of the reciprocal space. The experimental setup (sample-to-detector distance, center channel and tilt of the detector) being known, the experimental peaks are then converted into normalized reciprocal-lattice vectors (q vectors), as defined by the following equation:

with the incident wavevector and the wavevector pointing from the diffracting volume towards the reflection on the detector [Fig. 3 ▸(a)]. The lengths of the q vectors are normalized to unity because the wavelengths of the reflections are not a priori known for non-monochromatic data [Fig. 3 ▸(b)].

Figure 3.

(a) A schematic representation of an experimental setup using a geometry in reflection. (b) Ewald construction for non-monochromatic (Laue) diffraction. Nodes of the reciprocal lattice in the pale-blue zone are in the diffraction condition. Two normalized q vectors, q ₁ and q ₂, corresponding to the h ₁ k ₁ l ₁ and h ₂ k ₂ l ₂ reflections, respectively, are shown. The wavelength of these two reflections, defining the length of the q vectors, is not known a priori, and lies on the solid line crossing the reflections. Note that the reflection h ₁ k ₁ l ₁ has a harmonic reflection (h _1n k _1n l _1n).

From the known unit-cell parameters of the crystal, the theoretical reciprocal-lattice reflections are calculated. Three options for the ranking of these reflections are available. (i) If the structure (i.e. atomic decoration of the unit cell) or part of it is known, structure factors are also calculated, and theoretical reflections are ranked in order of decreasing structure factors. This is the strategy used in the Laue indexing algorithm implemented in the XMAS software (Tamura, 2014 ▸). (ii) In the case of a fully unknown structure (but known unit-cell parameters), theoretical reflections are ranked in order of decreasing d spacing. (iii) A third mode is available in which a user-defined list is used, extended if necessary by calculated theoretical reflections ranked in order of d spacing. For example, the user-defined list can be generated from a powder pattern, after extraction of the strongest intensity reflections using a Le Bail or Pawley fit. The ranking of the theoretical reflections according to the ‘likelihood’ of being found in the actual data set is a way of increasing the speed efficiency of the indexing process, especially for low-symmetry materials, by considerably reducing the number of calculations. Once generated following one of the three options described above, theoretical reflections are converted into normalized q vectors. The number of calculated theoretical q vectors can be restricted, either to a particular threshold d spacing, if the d-spacing ranking strategy has been chosen, or by imposing a minimum structure-factor value in the case of the structure-factor ranking strategy.

3.2. Step 2: nearest-neighbor matching  

The indexing starts at this step. The q vectors corresponding to the brightest (highest integrated intensities) experimental reflections are considered first, and the angles between these selected q vectors and the other experimental q vectors in the first vicinity are calculated. The first vicinity/neighborhood is limited by a maximum threshold angle value. A similar strategy is applied around the q vectors corresponding to a selection of theoretical unique reflections ranked following one of the three options described above [Fig. 4 ▸(a)].

Figure 4.

The indexing algorithm. (a) Step 2, nearest-neighbor matching. The red, green and blue cones represent the limiting maximum threshold angle around the experimental q vectors 1, 2 and 3, respectively. On the experimental side, six, four and four nearest-neighbor angles can be calculated around the q vectors 1, 2, and 3, respectively. On the theoretical side, four q vectors having similar nearest neighbors to the first experimental q vector are shown. The theoretical q vectors m, n, o and p have five, five, three and four similar neighbors (purple arrows) with q _{exp_start}(1), respectively. Additional q vectors (dashed black arrows) may also be present. (b) Step 3, pair matching. If two experimental q vectors form an angle α, then the same applies for theoretical q vectors. Among the four potential candidates with similar nearest neighbors to the first experimental reflection, only two [q _{th_match}(1,1) and q _{th_match}(1,3)] fulfill the requirement. (c) Step 3, triplet matching. Three q vectors are involved in the matching process, and only one theoretical q vector [q _{th_match}(1,1)] fulfills the requirement.

Next, the neighborhoods of the experimental and theoretical q vectors are compared in terms of the number of common angles. At the end of this process, a list of theoretical unique reflections with similar neighborhoods is obtained for each experimental peak, these theoretical reflections being ranked depending on the number of common q-vector angles. This is the ‘nearest-neighbors list’. For example, as can be seen in Fig. 4 ▸(a), the experimental q vector q _{exp_start}(1) (corresponding to the experimental reflection with the highest intensity) has five, five, three and four similar neighbors with theoretical unique reflections m, n, o and p, respectively. These four theoretical matching candidates will then be ranked as m, n, p, o or n, m, p, o.

3.3. Step 3: global matching  

To verify global matching (i.e. reflection matching not limited to the nearest neighbors), a candidate orientation matrix has to be generated. The indexing is considered to be successful if more than a minimum number of experimental peaks are indexed. The orientation matrix is built out of three vectors (or1, or2, or3), with the first vector or1 selected from the nearest-neighbors list, and the vector or2 selected from a non-restricted list of theoretical q vectors.

The theoretical unique q vectors with the highest number of common neighbors obtained from step 2 can be used directly to build the first vector or1. This approach was implemented and tested by Dejoie et al. (2015 ▸). Nevertheless, most of the time these reflections are not the best candidates to build the first vector of the orientation matrix. In particular, when the energy range and/or the volume of the cell increases, the number of calculated reflections increases, and the number of common neighbors becomes a less discriminating parameter. As a result, the indexing time increases. To overcome this, additional conditions were introduced in order to select the first vector or1, and two additional options are currently available.

The first additional option to retrieve candidates for or1 relies on a ‘pair-matching’ selection among the reflections of the nearest-neighbors list. If the angle between the q vectors of two experimental peaks is α, then the angle between the corresponding matching theoretical q vectors has to be α (within the angular resolution). This is illustrated in Fig. 4 ▸(b), where two out of four vectors [q _{th_match}(1,1) and q _{th_match}(1,3)] fulfill the condition. By applying the pair-matching conditions to all of the reflections that are part of the nearest-neighbors list, a reduced list of candidates for the vector or1 is generated.

The second option is an extension of the previous concept, implemented over three q vectors (‘triplet matching’). If an experimental q vector makes an angle α with a second experimental q vector and an angle β with a third one, then the same should apply to the corresponding theoretical q vectors from the nearest-neighbors list, as illustrated in Fig. 4 ▸(c).

In order to determine the second vector of the orientation matrix, or2, a strategy of matching a pair of q vectors is used, over experimental and theoretical q vectors. Candidates are generated by looking for any experimental/theoretical pairs with similar angles (within the angular resolution). In a similar way as for or1, an additional pair-matching restriction in the selection process applies: if the angle between two experimental q vectors is α, then the angle between the two corresponding selected theoretical q vectors should also be α within a given angular resolution.

The third vector or3 of the orientation matrix is deduced from or1 and or2 with only two possibilities, allowing for right-handed and left-handed coordinate systems. From the candidate orientation matrix, the positions of all the expected reflections can be devised and compared with the positions of the experimental peaks on the diffraction pattern. The experimental pattern is indexed if the number of experimental peaks indexed is higher than a defined minimum.

3.4. Step 4: post-calculations  

Once a satisfactory indexing is obtained, the successful orientation matrix is refined using the entire set of indexed reflections, and the number of matching reflections is calculated again. The output is a text file, using either the XMAS indexing file format or the SHELX (Sheldrick, 2008 ▸) format. For each reflection in the diffraction pattern, the Miller indices (h, k, l), the integrated intensity of the experimental peak (I _int) and the corresponding wavelength are given in the output.

3.5. Additional features  

An option for indexing several orientations (several crystal grains) in a single frame is available. This is performed sequentially as implemented in the XMAS software (Tamura, 2014 ▸). A maximum number of orientations to look for per frame is provided, and the algorithm simply loops over the previously described steps 1–4, removing each time the newly indexed peaks from the experimental peak list.

A second option has also been introduced, taking into account possible disorientation of the crystal. This is, for example, the case when the crystal is rotated slightly while an exposure is taken. As a result, additional reflections will be measured. This rotation effect can be specified by introducing a rotation angle. This option will not be discussed further in this paper.

4. Results and discussion  

4.1. Indexing results  

The indexing algorithm is implemented in Fortran90, using existing routines from the XMAS software (Tamura, 2014 ▸). Indexing trials were carried out on a DELL Optiplex 9020 computer equipped with a 3.6 GHz Intel Core i7-4790 processor. The indexing results for the data sets presented in Table 2 ▸ are shown in Table 3 ▸. Four indexing strategies were tested, using either d-spacing (dsp) or structure-factor (strf) ranking of the theoretical reflections (see Section 3.1), and either pair (pm) or triplet (tm) matching when selecting candidates for the or1 vector (see Section 3.3). The percentage of successfully indexed patterns and the average time per pattern processed are given in each case. The complete set of parameters used for the indexing of each data set is given in Table S1 in the supporting information. The set of parameters necessary to obtain a successful indexing may not be unique, and parameter values different from those indicated in this paper may also provide a successful indexing. Moreover, the total indexing time will vary depending on the computing machine used.

Table 3. Indexing results for the different data sets tested.

dsp: d-spacing ranking; strf: structure-factor ranking; pm: pair matching; tm: triplet matching. For each indexing option, the percentage of successfully indexed frames and the average indexing time per pattern (s) are given.

	Data set name	Monochromatic	4%	50%	LaueDET-50/60	LaueDET-90
	Energy range (keV)	17.75	16.9–17.7	10–17	5–24	5–24
	Setup No.	(1)	(2)	(3)	(4)	(5)
Sanidine	dsp-pm		96, 13.2	100, 0.7	100, 0.7	100, 1.6
	dsp-tm		65, 25.1	100, 1.8	100, 1.1	100, 2.0
	strf-pm		99, 12.4	100, 0.7	100, 0.7	100, 1.3
	strf-tm		79, 25.1	100, 1.5	100, 1.4	100, 0.8
CsPt	dsp-pm	87, 3.0	100, 2.8		100, 1.4
	dsp-tm	55, 6.3	93, 6.8		100, 1.6
	strf-pm	85, 3.2	97, 2.8		100, 5.3
	strf-tm	55, 6.2	89, 5.8		100, 5.4
ZrPOF	dsp-pm	75, 38.8/53, 19.4†			76, 432
	dsp-tm	40, 191.1/20, 95.6†			76, 78
	strf-pm	95, 31.3/80, 15.7†			76, 33.5
	strf-tm	40, 178.2/28, 89.1†			82, 24.9
ZSM-5‡	dsp-pm	85, 4.8	98, 8.6		73, 378.5
	dsp-tm	37, 15.5	78, 85.7	65, 301.7	93, 56.7	79, 672.5
	strf-pm	82, 5.2	92, 8.2	100, 17.4	100, 8.4	95, 68.0
	strf-tm	42, 15.2	71, 108.2	100, 3.6	100, 7.2	100, 35.7
MgAc	dsp-pm
	dsp-tm			90, 138.0		57, 550.0
	strf-pm			90, 72.6		65, 271.1
	strf-tm			97, 10.2		86, 108.4

^†For the monochromatic data set of ZrPOF, the algorithm is looking for two orientations in each pattern sequentially. The first two numbers correspond to the percentage of success if at least one orientation per pattern was indexed and the indexing time per pattern (20 in total), and the last two numbers to the percentage of success per orientation indexed and the indexing time per orientation (40 in total).

^‡For ZSM-5, the solutions where the a and b axes are reversed were accepted as correct (the flipped solution can be checked in an additional step, and correct indexing is usually the one where more reflections are indexed).

A key step is to find the successful candidate to build the first vector or1 of the orientation matrix (step 3). An initial selection is done through nearest-neighbor matching (step 2), and a second one at step 3 through pair or triplet matching. In order to have a successful indexing, the first requirement is to have the correct solution as a candidate in the selection list. This candidate should then also be among the first to be checked. Selecting a large number of candidates may increase the indexing success rate, but also the processing time if the solution is ranked too far down the list. The 15 patterns of the ZSM-5 sample collected using setup (4) and indexed using a d-spacing ranking strategy and triplet-matching strategy to obtain or1 candidates (dsp-tm) can be used as an example. For each of the 15 patterns in the data set, the processing time as a function of the ranking of the successful candidate has been plotted in Fig. 5 ▸. One pattern could not be indexed, probably due to the too limited number of possible candidates generated (15). In all the other cases, even if the indexing was successful, a fast indexing could only be achieved with the solution occupying one of the first two positions in the selection list.

Figure 5.

Indexing of the data set collected on ZSM-5 using the DET-60 configuration and the dsp-tm strategy, showing the indexing time as a function of the position of the solution within the candidate list to determine the or1 vector of the orientation matrix.

A few parameters have a strong influence on the ranking of the or1 candidates, and consequently on the indexing time and indexing success. A detailed discussion of these parameters is given in Sections S1 and S2 in the supporting information, using the tests performed on ZSM-5 as examples. The requirements are different depending on whether small-bandpass (monochromatic or 4%) or larger-bandpass (50%, DET-50/60, DET90) data are being indexed. In the latter case, a subset of experimental and theoretical reflections/q vectors should be considered at step 2, when for small-bandpass data it is recommended to exploit as many experimental data as possible. Two main limitations have been identified. On the small-bandpass side, the indexing may fail if not enough data per frame are present (e.g. CsPt monochromatic data, with on average fewer than ten reflections per frame). On the other hand, on the large-bandpass side, the number of required expected reflections may increase drastically, preventing the indexing from being successful in a reasonable amount of time (e.g. ZSM-5 DET-90 data using d-spacing ranking).

4.2. Indexing modes  

Four possible indexing strategies are available, depending on the theoretical reflections’ ranking mode (d-spacing or structure-factor based) and on the selection mode of the first vector or1 of the orientation matrix (pair matching or triplet matching). The efficiency of these four indexing strategies will be discussed next for the various samples and setups.

As the indexing process is based on matching experimental q vectors with theoretical ones, we expected that the ranking of theoretical reflections by structure factor would provide the best results. Indeed, this ranking mode favored the indexing of the data sets of ZrPOF, ZSM-5 and MgAc collected with a large-bandpass beam (50%, DET-50 and DET90) (Table 3 ▸). For the data sets of CsPt and sanidine collected with similar setups, this trend is less clear, and similar or even better results were obtained using d-spacing ranking (Table 3 ▸). Reflection intensities from a single Laue diffraction pattern are usually fully measured (except for the reflections lying at the ends of the energy range), and even when affected by the presence of harmonics (reflections of different orders overlapping) our results show that this does not hinder a good match and the patterns can be indexed. The fact that a d-spacing ranking also provides good results when indexing the sanidine and CsPt data sets is more difficult to interpret. We assume that, because fewer theoretical reflections are expected (smaller cell volume), the matching process converges faster.

The results for data sets collected with a large-bandpass beam show that the choice of using a pair-matching or triplet-matching strategy follows a binary distribution (Table 3 ▸). In order to index the ZrPOF, ZSM-5 and MgAc data sets, the triplet-matching process associated with the structure-factor ranking gives the best results. On the other hand, when indexing sanidine and CsPt, similar results are obtained using either pair-matching or triplet-matching options. By imposing a match between three q vectors, the triplet-matching process provides a higher degree of discrimination between potential candidates to build the first vector of the orientation matrix, which seems to be what is required to successfully index samples with larger cell volumes. Such a degree of discrimination appears less crucial for indexing samples with smaller cells, and both pair-matching and triplet-matching approaches can give acceptable results.

In the case of the indexing of data sets obtained using a smaller-bandpass beam (monochromatic and 4%), both structure-factor and d-spacing ranking can be used, with nevertheless slightly better results with the second option (Table 3 ▸). When using a small-bandpass beam, mainly partial reflection intensities are measured, and this may affect the intensity ranking of the experimental peaks. Consequently, the matching process will not be strongly affected by the chosen ranking strategy of theoretical reflections. On the other hand, the triplet-matching option seems to be much less efficient than the pair-matching one for indexing such small-bandpass data. As previously mentioned, triplet matching requires a match between three experimental/theoretical q vectors, this requirement being more difficult to achieve when fewer reflections/q vectors are available. As fewer data per frame are measured with a small-bandpass beam and with fewer theoretical reflections expected, a pair-matching strategy gives better results.

When searching for the most appropriate indexing strategies, three main tendencies emerge. These main indexing modes are presented in Fig. 6 ▸. For small-bandpass data, ranking by both d spacing and structure factor can be used, combined with a pair-matching approach. For larger-bandpass data there are two main strategies, depending on the dimension of the sample cell volume. In the case of a large cell, combining structure-factor ranking with triplet matching ensured good indexing results. For a smaller cell, any strategy can be used.

Figure 6.

A schematic representation of the most successful indexing modes depending on the volume of the crystallographic cell and the energy bandpass of the beam.

4.3. Alternative ranking of theoretical reflections  

As shown in the previous section, the theoretical reflections’ ranking strategy plays an important role in obtaining a successful indexing. However, the fact that a structure-factor ranking seems to be required to index patterns measured with a large-bandpass beam on large-cell samples (Fig. 6 ▸) is an issue in the case of samples with unknown crystallographic structure. To cope with this, an alternative ranking strategy may be desirable. As mentioned when describing the pre-calculation part (step 1) of the indexing algorithm, a user-defined theoretical reflections list can be imposed, and we have tested the possibility of using the strongest reflection intensities extracted from a powder diffraction pattern. This approach has been tested on three data sets obtained with large-bandpass beams: setup (4) (Laue DET-50) for ZrPOF and setup (5) (Laue DET-90) for both ZSM-5 and MgAc.

A Pawley refinement requires the cell parameters of a particular sample to be known, and this is indeed the case here, as it is also a requirement for the indexing algorithm. The refinements of the powder patterns measured on ZrPOF, ZSM-5 and MgAc are shown in Fig. 7 ▸. The resolution at which the powder diffraction signal vanishes (d _max powder) and the three resolution limits (lowest d spacing, d _min single crystal) of the three relevant single-crystal data sets are indicated. We can see that single-crystal data for ZSM-5 [Fig. 7 ▸(b)] and MgAc [Fig. 7 ▸(c)] mainly cover a high 2θ range (low d spacing), as imposed by the DET-90 configuration. This is also true for ZrPOF [Fig. 7 ▸(a)], even if the DET-50 configuration allows higher d-spacing reflections to be measured. Resolutions of 1 Å (23° 2θ), 0.665 Å (35° 2θ) and 0.888 Å (26° 2θ) have been reached with powder data for ZrPOF, ZSM-5 and MgAc, respectively, which are still far away from the resolutions obtained with single-crystal data (0.313, 0.230 and 0.229 Å, respectively, Table 2 ▸).

Figure 7.

Pawley refinement of (a) ZrPOF (R _p = 4.2%, R _wp = 6.6%, R _exp = 1.4%), (b) ZSM-5 (R _p = 5.9%, R _wp = 9.8%, R _exp = 1.3%) and (c) MgAc (R _p = 4.1%, R _wp = 5.9%, R _exp = 0.7%).

The indexing results using as theoretical reflections the strongest reflection intensities extracted from the refined powder patterns (p_int) are shown in Table 4 ▸. For comparison, results obtained using d-spacing ranking and structure-factor ranking (Table 3 ▸) are also reported. The complete set of parameters to index the three data sets can be found in the supporting information. The number of unique reflections chosen at step 2 of the indexing process corresponds to the number of unique reflections extracted from the powder patterns, ranked in order of decreasing intensity. Following the indications given in Fig. 6 ▸, only the triplet-matching strategy (step 3) has been used. We can see that the indexing using powder diffraction intensities as theoretical reflections gives intermediate results, with a better score than using the d-spacing ranking method, but still not as good as when using the structure-factor ranking method. Using reflections ranked in order of decreasing intensity obtained from a powder pattern or reflections ranked in order of decreasing structure factors calculated from a known structure should give similar results. This is not the case yet, which means that the intensities extracted from the powder patterns may not be fully accurate. We attribute this to the low diffraction signal and the strong overlapping of reflections in the three relevant 2θ ranges (mainly high 2θ), preventing an optimal measure of the integrated intensities. Nevertheless, as we were looking for an improvement in the indexing score when a structure-factor ranking cannot be used, the powder ranking strategy does provide better results.

Table 4. Indexing results using reflection intensities extracted from a powder pattern (dsp: d-spacing ranking; strf: structure-factor ranking; p_int: powder diffraction intensities ranking; tm: triplet matching).

Name	Indexing method	Laue data sets
ZrPOF	dsp-tm	76, 78.0
	p_int-tm	82, 50.8
	strf-tm	82, 24.9
ZSM-5	dsp-tm	79, 672.5
	p_int-tm	84, 50.6
	strf-tm	100, 35.7
MgAc	dsp-tm	57, 550.0
	p_int-tm	76, 169.8
	strf-tm	86, 108.4

4.4. Indexing efficiency  

A good indication of the appropriate indexing method to choose depending on the bandpass of the beam and the volume of the unit cell of a particular sample is given in Fig. 6 ▸. However, using only these two parameters may be somewhat restrictive. In an attempt to assess the results better, the different data sets used in the present study were ranked depending on their ‘complexity’. To do so, we identified three main parameters that may play a significant role: the volume of the crystallographic unit cell (Volume), the number of unique reflections expected in the relevant energy range (Unique refl.) and the number of actual measured reflections per frame. While the first parameter is only related to the dimensions of the crystallographic cell of the sample, the second one is linked to its symmetry and to the experimental setup (e.g. the energy range). The first two parameters can be calculated for a given sample and a particular setup. On the other hand, the third parameter is less predictable, and may fluctuate depending, for instance, on the brilliance of the incident beam, the quality of the crystal or possible radiation damage. As a result, a ‘complexity’ parameter Comp is calculated as follows: Comp = Observed refl./Unique refl./Volume, with Observed refl. being the average number of measured reflections per frame for a particular data set.

The indexing is considered to be successful when a maximum number of frames are indexed in a minimum time. Within the average indexing time per frame (Table 3 ▸), the time taken to index a frame successfully may be much shorter than that for a non-successful indexing. To take that into account, an indexing efficiency coefficient Eff is calculated:

with N _success being the number of patterns indexed successfully, t _success the time taken to index the patterns successfully, N _tot the total number of patterns in the data set and t _tot the total time taken to index the data set. In order to obtain a meaningful comparison between the different data sets, a time normalization t_N is introduced:

with Av_time the average indexing time per frame for a particular data set, given in Table 3 ▸. In this way, an indexing efficiency of 1 (best efficiency) or 0 (worst efficiency) can only be reached for an average indexing time of 0 s per frame or 1000 s per frame, respectively.

Complexity values obtained for each data set and the indexing efficiency coefficients for the different indexing methods are reported in Table 5 ▸. The indexing efficiency as a function of sample complexity is plotted in Fig. 8 ▸. In the case of hopeless indexing (e.g. ZSM-5, DET90, dsp-pm), the efficiency coefficient was set to 0.

Table 5. The efficiency of the four indexing modes (Eff1 to Eff4) in order of decreasing complexity (Comp) of the data sets.

Sample	Data set	Comp (×10−6)	Eff1, dsp-pm	Eff2, dsp-tm	Eff3, strf-pm	Eff4, strf-tm
ZSM5	Monochromatic	0.50	0.758	0.166	0.692	0.243
MgAc	DET-90	0.85	0	0.091	0.260	0.621
ZSM5	4%	1.11	0.946	0.546	0.765	0.469
ZSM5	DET-90	1.63	0	0.206	0.719	0.964
ZSM5	50%	2.16	0	0.143	0.983	0.996
ZrPOF	Monochromatic	2.25	0.566	0.115	0.869	0.141
MgAc	50%	2.48	0	0.546	0.642	0.900
ZrPOF	DET-50	2.53	0.194	0.266	0.416	0.457
ZSM5	DET-60	2.84	0.155	0.746	0.992	0.993
Sanidine	4%	4.95	0.932	0.477	0.973	0.656
Sanidine	DET-90	8.62	0.998	0.998	0.999	0.998
Sanidine	DET-50	10.68	0.999	0.999	0.999	0.999
CsPt	Monochromatic	10.90	0.809	0.370	0.778	0.401
Sanidine	50%	16.01	0.999	0.998	0.999	0.998
CsPt	4%	18.00	0.997	0.865	0.959	0.833
CsPt	DET-50	80.21	0.999	0.998	0.995	0.995

Figure 8.

The efficiency of the indexing methods plotted as a function of the complexity of the data sets (dsp: d-spacing ranking; strf: structure-factor ranking; pm: pair matching; tm: triplet matching).

In Table 5 ▸, the data sets are ranked in order of decreasing complexity, the smallest Comp value corresponding to the highest degree of complexity. With this classification, the data sets collected on ZSM-5 crystals with a monochromatic beam and on MgAc with a large-bandpass beam appear to be the most complex, and the one obtained on CsPt using the DET-50 configuration the least. Indeed, indexing of the ZSM-5 monochromatic data set and of the MgAc DET-90 data set was demanding, with a highest score below 90% (Table 3 ▸) no matter which indexing method was used. On the other hand, the CsPt DET-50 data set is one of those that can be easily indexed using any of the methods chosen (Table 3 ▸). This shows that the complexity of a data set is not correlated with an increase in the bandpass of the beam.

We chose a Comp value of 6 × 10⁻⁶ to separate the data sets into two categories (see the vertical dashed line in Fig. 8 ▸). When looking at the less complex data sets (Comp > 6 × 10⁻⁶), an efficiency coefficient higher than 80% is achieved most of the time, no matter which indexing method is used. The only exception concerns the CsPt monochromatic data set, with a drop in the indexing efficiency when using the triplet-matching method, most probably due to a lack of data per frame (Table 2 ▸), as mentioned earlier. For the most complex data sets (Comp < 6 × 10⁻⁶), the indexing efficiency clearly drops, irrespective of the indexing methods used. One of the most affected is the dsp-tm method, with an efficiency never reaching 80%. On the other hand, the structure-factor ranking methods are the most robust, in agreement with the results shown in Fig. 6 ▸.

5. Conclusions  

The indexing algorithm presented in the current paper has been tested on a series of data sets obtained from five different samples under variable experimental conditions. Four indexing strategies can be used, with the calculated theoretical reflections ranked in order of d spacing or structure factors, and the matching process based on a pair or triplet of q vectors.

The main parameters to tweak and the best indexing mode to choose to obtain a successful indexing differ depending on whether a small bandpass (monochromatic or 4%) or a larger bandpass (50%, DET-50, DET90) is considered. The calculation of a complexity parameter for each data set reveals that the most complex data sets are not simply correlated with a particular bandpass, and the most robust indexing methods are those based on structure-factor ranking

An additional feature has been added, allowing a user-defined theoretical reflection list to be provided. This is particularly useful when only the lattice parameters of a sample are known and a d-spacing ranking strategy has to be used. Using the reflection intensities extracted from a powder pattern shows that the indexing can indeed be improved.

The idea of using crystallographic information coming from powder diffraction to index serial crystallography data has been proposed previously, the powder pattern in that case being directly built from single-crystal data (Brewster et al., 2015 ▸). Combining methods and practices from different communities is always a good approach to solving challenging crystallographic problems.

The indexing program can be downloaded at https://sites.google.com/a/lbl.gov/bl12-3-2/user-resources/.

Funding Statement

This work was funded by U.S. Department of Energy, Office of Science grant DE-AC02-05CH11231.