The use of haptic interfaces and web services in crystallography: an application for a ‘screen to beam’ interface

A haptic interface and related database are used to assess crystallization screening, share results and enable efficient in situ diffraction characterization of the resulting crystals.

Abstract

Haptic interfaces have become common in consumer electronics. They enable easy interaction and information entry without the use of a mouse or keyboard. The work presented here illustrates the application of a haptic interface to crystallization screening in order to provide a natural means for visualizing and selecting results. By linking this to a cloud-based database and web-based application program interface, the same application shifts the approach from ‘point and click’ to ‘touch and share’, where results can be selected, annotated and discussed collaboratively. In the crystallographic application, given a suitable crystallization plate, beamline and robotic end effector, the resulting information can be used to close the loop between screening and X-ray analysis, allowing a direct and efficient ‘screen to beam’ approach. The application is not limited to the area of crystallization screening; ‘touch and share’ can be used by any information-rich scientific analysis and geographically distributed collaboration.

1. Introduction  

Haptic interfaces have been rapidly adopted owing to their intuitive ease of use and convenience. Obvious examples are the screens for mobile phone interfaces or other computing devices where keyboards have been eliminated. This technology, readily adopted in everyday life, can also find use in scientific research, especially in the case of ‘point and click’ interfaces. The haptic interface provides the immediate ‘front end’. An equally powerful and parallel development has been cloud computing technology, where information and processing power can be shared between multiple users. The combination of these two technologies can provide both ease of use in information analysis and a wide area of application in sharing and making use of the analysis.

Point and click interfaces are common in many forms of instrumentation. An image is displayed and a human operator interprets that image, together with any associated data, feeding back results with the motion and click of a mouse and sometimes keyboard data entry. In the field of macromol­ecular X-ray crystallography, the initial images that need to be interpreted are typically from experiments aimed at the crystallization of biological macromolecules. Each is viewed and classified in order to guide and optimize crystallization efforts (Luft, Wolfley & Snell, 2011 ▸). If success occurs in the crystallization step, another application of imaging occurs and the images of mounted crystals are used to position them appropriately with respect to the X-ray beam for diffraction analysis. For limited studies, point and click interfaces are con­venient, but as the number of images involved begins to increase, even these user-friendly interfaces can become burdensome.

At the Hauptman–Woodward Medical Research Institute, the High-Throughput Crystallization Screening Center provides a crystallization screening service sampling 1536 different chemical conditions with 200 nl of the biological sample at each condition. The Center has been in operation for well over a decade, performing screening experiments on over 16 000 samples to date (Luft, Snell & DeTitta, 2011 ▸). Images of each experiment over time are provided to investigators via a secure FTP server. An application called MacroscopeJ is provided for the investigator to read and classify the resulting images. If a potential crystal is identified (or signs that one of the chemical conditions used may be close to that yielding a crystal), the laboratory providing the sample then conducts optimization experiments centered around the screening results. Once optimized, the resulting crystals are used for diffraction studies. On many occasions this research can be collaborative, with several laboratories involved in a single project. With the current system, each person viewing the images requires access to the data supplied by the screening laboratory or shared by the investigator – there is no access to a central repository. This is not ideal, as it introduces an extra step to the analysis and can cause potential communication issues.

X-ray data collection itself has undergone considerable automation developments. Remote data collection is now commonplace (Cohen et al., 2005 ▸) and software interfaces (González et al., 2008 ▸; McPhillips et al., 2002 ▸) have been developed that allow multiple users to conduct an experiment in the same location or at multiple different locations. These interfaces can accept control input from each location and allow multiple teams to follow or even control the process and to communicate, for example, through chat windows. Once crystallization conditions have been optimized and crystal samples prepared and mounted, X-ray screening and data collection are efficient tasks. The pathway between crystal screening and X-ray data collection is not streamlined: optimization is a manual process. When the number of experiments becomes large, minimizing the manual steps can make a large difference in efficiency, in terms of both throughput and, more importantly, output.

In this work, the analysis of crystallization outcome is linked to the subsequent diffraction analysis, facilitating or potentially eliminating an initial optimization step. While in situ data collection is not new (Aller et al., 2015 ▸), a focus on the user interface and process is lacking. Embracing a haptic interface to enable the visualization, classification and notation of experimental crystallization data with a cloud-based database of images allows multiple collaborators to share the information and to fill the missing link between screening and diffraction characterization. Information is passed to the beamline so that the crystallization screening plate can be analyzed in the beam efficiently. A ‘screen to beam’ capability results for our case, but in the broader picture it demonstrates the power of haptic interfaces and web computing to create a sharable scientific (or any visual information) environment. The approach developed is generalizable to any application where visual information may be interpreted by a team and action taken as a result of that interpretation.

2. Application  

Crystallization screening is performed at our Center (Luft, Snell & DeTitta, 2011 ▸) using the microbatch under oil method (Chayen et al., 1992 ▸). Screening occurs in 1536-well Greiner Imp@ct SBS-format polyolefin plates, designed with a thin bottom for low birefringence during polarized imaging. Each experiment in the plate contains 200 nl of a crystallization cocktail and 200 nl of the sample covered by paraffin oil. Before the sample is added, the oil-filled plates are loaded with cocktail, centrifuged and imaged by a video microscope system (as a control), then loaded with sample, centrifuged again and imaged again, first immediately after the sample has been added and then weekly, usually for a period of six weeks. At the week-four point, second-order nonlinear imaging of chiral crystals (SONICC) and two-photon emitted fluorescence (TPEF) imaging also take place (Luft et al., 2015 ▸). The individual images are stored and made available in a compressed .rar format file. The experimental outcomes are determined by observation of the recorded images using an in-house developed application, MacroScopeJ (a Java-based enhancement of the original Macroscope; Luft et al., 2003 ▸). Amongst other things, this displays 96 experimental wells at a time and allows the sample provider to select single experiments and classify the outcomes. For the study described here, the .rar format files were processed and the individual images were made accessible over HTTP to allow viewing from a web browser. The .rar files were converted to a simple comma-separated value (CSV)-based format called an ‘image set’, which contains the name, URL and category of each image.

A haptic based application, called Harrier, was written. This application imports the image-set CSV file and displays the crystallization screening results via a web-based interface. One image at a time is displayed on the screen. The application is configured so that the user can load a defined set of images and scroll through them individually, swiping left on the screen to advance. The application is also configured so that a traditional point and click interface can be used where a haptic display is not present, i.e. so that it is compatible with traditional computing devices. The Harrier application does not store or process images locally and, theoretically, can work with any set of images accessible via HTTP. The image-set CSV file contains the URL to each image and Harrier stores this information in an SQL database. The database schema is described in Fig. 1 ▸. Users can view image sets, select features (crystals) and easily share results with other users. The application is web based, accessing the database of images to select or deselect crystals, and is viewable by multiple people at any one time. The front-end user interface uses a responsive design and is optimized for both desktop and mobile devices. It is written in HTML5/CSS and JavaScript and makes use of the canvas element available in HTML5. The canvas element allows drawing and graphic manipulation via scripting in JavaScript. In Harrier, images are drawn onto a canvas element and the pixel coordinates of the pointer device (finger or mouse) are captured and stored in the back-end database. The back-end web application is written in Python and uses the Flask microframework (Ronacher, 2015 ▸). Harrier supports most SQL databases, including MariaDB, Postgres and Sqlite. By default, Harrier will use Sqlite for the back-end data store. Harrier requires Python and can be installed as a stand-alone application or run on a server.

Figure 1.

Schema for the current implementation of Harrier as a ‘screen to beam’ interface. Note that the schema is customizable to that appropriate for the use of the program. In this case, an image set is accessed and multiple (x, y) positional information can be stored for each image.

The haptic interface within Harrier allows the user to enlarge an image by touching the display with closed fingers and opening those fingers on the display. The degree of magnification is governed by the separation of the fingers. A single finger can be used to pan the image when it is enlarged. The user can select any feature on the image by touching with their finger. This allows closer inspection of images on smaller devices. An option is available to add the X-ray beam-size information and, if this is done, a shadow of that beam footprint is shown centered on the position the user has touched. This position is captured by Harrier as (x, y) pixel coordinates from the top left of the image. To support different degrees of magnification, these (x, y) pixel coordinates are converted into relative coordinates x _r = x/width and y _r = y/height using the width and height of the image at the current magnification level. The relative image (x _r, y _r) coordinates are stored in the database and can be corrected to plate coordinates (P_x, P_y) using the dimensions of the plate and the known well position encoded within the image name, described in the following formulae:

where x _r and y _r are the relative pixel coordinates of the feature from the top left of the image, w and h are the original image width and height, respectively, in pixels, and m _w and m _h are the margin width and margin height, respectively, of the plate from the top left-hand corner, in millimetres. The margin accounts for the border of the plate before the start of the wells. From the plate perspective, n is the well number from which the image was taken, r is the number of wells per row of the plate, z is the spacing between wells in millimetres, and μ is the number of millimetres per pixel and is obtained from the resolution of the camera. The resolution is typically given as micrometres per pixel and is multiplied by 1000 to convert to millimetres per pixel. P_x and P_y are the plate coordinates, in millimetres, of the feature from the top left of the plate. Fig. 2 ▸ shows an example pixel- to plate-coordinate translation. The plate definition is easily changed to one appropriate to the experiment.

Figure 2.

Illustration of the 1536-well crystallization screening plate with the physical x, y distances marked of the separation between wells and the position from the top left-hand corner of the plate (0, 0) to the first well. In this example, n = 1 for imaging the first well, m _w = 11.005 and m _h = 7.865, representing the border of the plate before the wells, r = 48 as there are 48 wells per row, z = 2.25, representing the spacing between each row, w = 632 and h = 504 are the width and height in pixels of the image, respectively, and μ = 0.003, representing the pixels per millimetre. The dashed rectangle indicates the full imaging area, which is cropped to a region of interest (ROI) shown by the solid-line rectangle in the upper right-hand image. The ROI image is 632 × 504 pixels, corresponding to an area of the plate of 2.07 × 1.65 mm, which is roughly 0.003 mm per pixel. In this image, rectangles on the crystals indicate the user-selected beam positions. This is a case where sharing and review of the images proves useful, as the positions could be changed such that the beam would be hitting a single crystal and not a region where potentially two lattices cross.

Once a site of interest has been selected, any other user looking at the data through the same application will see this selection. The site can be deselected by any user or even a note made about the site, e.g. why it is of interest. In the case of selecting crystals for X-ray evaluation, it is unlikely a crystal would be deselected but, for example, in Fig. 2 ▸, where a beam spot has been positioned over two overlapping crystals, the spot could be moved and another one added to sample both crystals separately. Another user who may be more experienced in data collection can deselect and then select new positions for X-ray analysis. In this manner, the ‘point and click’ interface has evolved into a ‘touch and share’ format, a collaborative aspect that has already proved powerful in interfaces such as Blu-Ice (McPhillips et al., 2002 ▸) where users can seamlessly swap control of the experiment.

Harrier provides a representational state transfer (RESTful) web service (Fielding & Taylor, 2000 ▸) application program interface (API) for programmatic access to the plate (P_x, P_y) coordinates. REST (representational state transfer) is a web service architecture style aiming for fast performance, reliability and simplicity. A single file containing plate (P_x, P_y) coordinates of sites of an image is exported in CSV format and used by a mounting robot for X-ray evaluation of the initial crystallization screening result. In this case, the standard SBS format and thin plate bottom ensure ease of handling and minimize absorption of the X-ray beam used, respectively.

3. Experimental  

A potential concern with in situ data collection is the shipping of crystallization plates and how transit may affect the crystals and their positions. Laboratory experiments show that, once set up, the position of the fluid within the plate is maintained in the wells by surface tension. It is very difficult to dislodge the liquid on purpose, and almost impossible to dislodge it accidently. To test the potential of X-ray data collection directly from plates at synchrotron sources, two plates were shipped from the High-Throughput Screening Center in Buffalo, USA, to the Diamond Light Source, UK. Shipping took place using FedEx with the plates contained in plastic bags. No special sealing mechanism was used on the plates. One of these plates contained a putative ATP pyrophosphatase from Pyrococcus furiosus, whose structure had already been determined (PDB code 3rk1; Forouhar et al., 2011 ▸), a novel metabolite proof-reading enzyme that is still under study, and finally a control sample, lysozyme, used in the Crystallization Screening Center to ensure reproducibility of operations.

This was an extreme test of handling and likely to represent a worst-case scenario. The plates were received at Diamond and data collected on beamline I24. The plates were mounted on the standard goniometer fitted with an adapter bracket capable of holding crystallization plates. Data were collected with the beam defocused to 20 × 20 µm and recorded using a Pilatus2.6M detector. The wells were observed with an on-axis viewing system configured to image in line with the X-ray beam. The haptic interface was still under development and not used with this study, the goal being simply to demonstrate that the filled plates were suitable for shipping, crystals could survive a transatlantic journey and crystal positions did not shift during transport. This was indeed the case and showed that X-ray data could be collected from the plates used for screening. It prompted us to test lysozyme protein with the haptic interface software so that we had a large number of wells containing crystals and could evaluate the influence of transport statistically.

A screening plate was set up with chicken egg white lysozyme (HEWL) at 75 mg ml⁻¹ in 20 mM acetate buffer, pH 4.6. Lysozyme is used as one of the regular control experiments in the screening laboratory and is guaranteed to produce well diffracting crystals in a number of the cocktail conditions, thereby allowing us to test the application robustly, rather than worry about sample crystallization and diffracting properties. The lysozyme plate was imaged, as described above, over a period of six weeks. A .rar file from images recorded at two weeks was loaded into the Harrier database and then the application was used to note the positions of crystals to be studied by diffraction.

The crystal positions were exported from the database and sent as a CSV file to beamline X12B at the National Synchrotron Light Source (NSLS) in Brookhaven, New York. The plate itself was sent to NSLS via FedEx. A G-Rob plate-handling robot (le Maire et al., 2011 ▸) was used to position each 1536-well plate in the X-ray beam using the crystal coordinates that accompanied each annotated plate. The coordinates were adjusted to account for a different origin and a slight rotation (0.224°) between the axes of the plate-annotation microscope and the axes of the G-Rob robot. The G-Rob moved each plate from a plate hotel to the X-ray beam, and accessed locations on the plate indicated by the annotated coordinates (adjusted to the G-Rob coordinate frame). An empty plate was used to measure separately the background scatter from the plate and that from the paraffin layer. X-ray data were collected using an ADSC Q4R detector with a crystal-to-plate distance of 140 mm.

Although the plate was not designed for in situ experimentation, it was noted that the X-ray background was comparable to that of other commonly used in situ plates (MiTeGen InSitu-1 and Greiner CrystalQuick X). The G-Rob system has a standard 384-well plate definition, so graphical user interface access to each of the 1536 wells in the plate was achieved by subdividing each of the 96 well fields into a 4 × 4 grid. A custom command-line interface was used to read the CSV file and move to each coordinate on the plate. The command-line interface was also used to determine the vector between the expected coordinate for each annotated crystal and the coordinate observed on the beamline.

While the aim of the experiment was not to show that complete diffraction data sets could be collected from the screening plate, ten X-ray data sets were collected from promising crystals. The images were indexed, integrated and reduced using HKL-2000 (Otwinowski & Minor, 1997 ▸). In all cases, 40° of data were collected from each crystal using 80 images with 0.5° rotation per image and an exposure time of 10 s per degree. On-axis visualization was perceived to be clearer using the Imp@ct plates compared with the CrystalQuick X plates. The Imp@ct plates allowed a sharper image (because there is no adhesive plastic) and a greater rotation angle before the sides of the plate obscured the camera view. The quality of the X-ray data was examined but no structural studies were attempted. This illustrates the power of the application in that any item of interest, crystal, potential crystal or unknown precipitate for example, could be selected and X-ray data collected for X-ray-based characterization and subsequent optimization.

4. Results  

The concerns about transport of samples were alleviated to some extent by the initial shipping experiment from Buffalo, USA, to Diamond, UK. Fig. 3 ▸ shows the images of two wells of interest recorded by the High-Throughput Screening Center in Buffalo. In the first a needle-shaped crystal is present, and in the second a smaller rectangular crystal. An enlarged image is shown of each crystal. This image was recorded from the X-ray point of view imaging down the beam path. The X-ray beam diameter is shown on the image. The crystals had not moved during transport and there was no evidence of leakage from the crystallization screening tray. During the setup process, the crystallization trays are centrifuged once the sample has been added to the precipitant. The experimental drop remains on the bottom of the well in the screening plate and it is likely that crystals adhere to the plate side or surface on growth. This may well explain the robustness during trans­atlantic transport.

Figure 3.

Images of crystallization experiments recorded (a), (c) during the screening experiments and (b), (d) after shipping to the beamline. The crystals showed no movement during shipping and examination on the beamline. The microfocus beamline enabled individual crystals to be examined from the cluster seen in parts (a) and (b). Multiple crystals could be studied in each well without propagation of radiation-damage processes.

It should be noted that, although the original in situ tests at Diamond involved a transatlantic shipment and the work developing the interface involved a shipment within the US, both involved similar stages. Other than time, the shipment process was not quantified in any experimental manner, but both cases involved more than one flight (loading, unloading, some pressure change) and a road-based delivery component. It is likely that similar disturbances would have occurred to the plates in both cases.

The G-Rob plate handler easily retrieved the 1536-well screening plate from the standard plate hotel and returned it after study. No new handling tool was required. There was an offset between the (x, y) position provided by the Harrier program and the positioning of the plate within the beam. This was a systematic difference that, when corrected, enabled all the positions previously defined using Harrier to be easily located in the beam. Note that the offset proved to be variable between successive selection, positioning and retrieval of the plate. This was due to the plate end-effector, which was not designed with absolute precision of plate positioning in mind. This is easily rectified with a dedicated design.

For the lysozyme plate, some 137 of the 1536 crystallization cocktails produced crystals with a well defined morphology that were then annotated with Harrier. Some wells had multiple crystals and 142 crystals from these were selected before screening. Coordinating with available beamtime and configuring the G-Rob to accept coordinates and deal with 1536 wells required several weeks. During this time, 15 of the visually selected crystals disintegrated (these specimens are not included in our evaluation). Crystal degradation over this time frame is not unusual; while fluid is not lost through the paraffin oil, the plates themselves are slightly porous and the wells slowly dehydrate, affecting crystals formed in some conditions. Of the remaining 122 wells containing crystals on the plate, 25 of the crystals were not positioned with sufficient accuracy for automated data collection using a 200 µm X-ray beam size. Visual inspection revealed that approximately half of these 25 crystals had moved during shipping. The remaining crystals were missed by the X-ray beam because of cumulative positioning errors (including a significant 100 µm jitter in the G-Rob plate handler – not specifically designed for the role it was being used for here). One concern, allayed during visualization at the NSLS, was that crystals would move on the y axis due to gravity. This was not the case, mirroring the initial trials at Diamond. Instead, the discrepancies were radially symmetric but bimodal (corresponding to differences in distribution between crystals that became dislodged during transport and those that did not) (see supplementary Fig. S1 in the supporting information). Some 79% of the crystals remained in the same position imaged in the laboratory when compared with the wells imaged on the beamline, and correcting for the robot jitter would raise that percentage to about 90%, arguably an acceptable but still improvable level.

Fig. 4 ▸ shows a screenshot of the Harrier interface in its most basic form. In this case, the beam footprint has been overlaid on two crystals, each of which is large enough that the beam could image two regions. Note that, if a rotation were used, a changing volume would be swept out and a single region would be used. Ten crystals from nine wells were selected for in situ X-ray examination. Fig. 5 ▸ shows those crystals, with the crystallization conditions described in Table 1 ▸. The selection of crystals for study was based on visual inspection without consideration of any systematic sampling of different types of chemical space. Of the conditions selected, both salt and polyethylene glycol as the driving precipitant were present. Crystallization conditions designed for cryogenic preservation are included in the 1536 conditions used for crystallization screening, but these were not among the wells selected. Visual inspection suggested that all crystals were in the common tetragonal crystal system for lysozyme, with the exception of crystal 4 where the potential crystal system was unclear.

Figure 4.

A snapshot of the Harrier screen interface. With the haptic interface there are minimal features visible on the interface. The beam position (shown as two red squares on each crystal) is entered by touching the appropriate point on the screen. The image can be scaled up or down by touching and simultaneously opening or closing two fingers. When enlarged, the image can be panned in any direction with a single finger. The next or previous image is selected by swiping left or right, respectively. Interestingly, minimal instruction is needed for users to discover these functions naturally.

Figure 5.

Crystals used in the lysozyme study. The diameter of the well is 0.9 mm and the X-ray beam footprint is 200 µm. The number refers to the crystal number in the tables of crystallization and X-ray results (Tables 1 and 2, respectively). Crystals 6 and 7 were recorded from the same well. The total volume of protein in each well is 200 nl, with an identical amount of crystallization cocktail.

Table 1. Crystallization conditions for the crystals in the wells studied by in situ X-ray diffraction.

Note that the chemical conditions represent the cocktail that was mixed 1:1 with the protein sample solution. MES is 2-(N-morpholino)ethanesulfonic acid, PEG is polyethylene glycol, HEPES is 2-[4-(2-hydroxyethyl)­piperazin-1-yl]ethanesulfonic acid and PEG MME is PEG monomethyl ether. The molecular weight is given in kilodaltons for both PEG and PEG MME.

Crystal	Crystallization condition	Notes
1	6% v/v Tacsimate (1.8305 M malonic acid, 0.25 M ammonium citrate tribasic, 0.12 M succinic acid, 0.3 M DL-malic acid, 0.4 M sodium acetate trihydrate, 0.5 M sodium formate and 0.16 M ammonium tartrate dibasic) pH 6.0, 0.1 M MES pH 6.0, 25% w/v PEG 4000	Tacsimate is a proprietary reagent developed by Hampton Research (Aliso Viejo, California, USA)
2	0.06 M bis-tris propane, 0.04 M citric acid pH 6.4, 20% w/v PEG 3350
3	0.15 M caesium chloride, 15% w/v PEG 3350	Unbuffered
4	0.4 M ammonium phosphate	Unbuffered
5	1.4 M sodium acetate trihydrate, 1.4 M sodium cacodylate pH 6.5
6	0.2 M ammonium sulfate, 0.1 M sodium cacodylate pH 6.5, 30% w/v PEG 8000
7	0.2 M ammonium sulfate, 0.1 M sodium cacodylate pH 6.5, 30% w/v PEG 8000	Same well as crystal 6
8	2.0 M sodium chloride, 10% w/v PEG 6000
9	0.05 M magnesium chloride hexahydrate, 0.1 M HEPES pH 7.5, 10% w/v PEG MME 550
10	0.2 M L-proline, 0.1 M HEPES pH 7.5, 10% w/v PEG 3350

X-ray analysis of individual crystals was successful. Table 2 ▸ provides information on the data-collection statistics for the ten crystals selected from the nine experimental wells. Interestingly, although different crystallization conditions were used for each well in the plate, the unit-cell parameters were very similar and the samples diffracted to similar resolution [defined as the point where the average I/σ(I) ≃ 2.0]. An indication of the variability of the crystal samples comes from crystals 6 and 7 which, while grown in the same well, diffracted to a slightly different resolution. There are a number of potential reasons for this beyond the crystal properties, but we note that, when harvesting a crystal, the potential for other experimental factors to reduce the overall crystal quality is enhanced. We note that the R _sym values are high for the highest-resolution bin. This is a consequence of a number of factors, including detector limitations, radiation damage at room temperature, and vibrational effects in the implementation of the rotation with the G-Rob system. Each is addressable should the application extend beyond crystal screening and require structural data collection. Lysozyme is structurally well characterized, so no further structural studies were attempted beyond the initial data collection.

Table 2. Diffraction statistics for ten X-ray data sets collected from the crystallization screening plate.

R _sym is defined as ∑(I − 〈I〉)²/∑I ².

Crystal	1	2	3	4	5	6	7	8	9	10
Unit cell a = b, c (Å)	79.27, 37.63	79.44, 37.78	79.43, 37.69	79.29, 37.70	79.26, 37.71	79.25, 37.83	79.26, 37.74	79.46, 37.82	79.26, 37.72	79.27, 37.72
Resolution (Å)	50–1.26 (1.28–1.26)	50–1.31 (1.33–1.31)	50–1.31 (1.33–1.31)	50–1.31 (1.33–1.31)	50–1.50 (1.52–1.50)	50–1.21 (1.23–1.21)	50–1.37 (1.39–1.37)	50–1.34 (1.36–1.34)	50–1.25 (1.27–1.25)	50–1.23 (1.25–1.23)
Unique reflections	29163	25138	25055	23524	15096	32640	20285	21719	28058	29547
Completeness (%)	91	88	87	82	78	90	81	81	85	85
Redundancy	1.1	1.2	1.2	1.2	1.3	1.1	1.2	1.2	1.2	1.2
〈I/σ(I)〉	29 (2)	34 (3)	28 (2)	26 (2)	20 (2)	34 (3)	32 (2)	30 (2)	37 (3)	34 (3)
R sym	0.105 (0.99)	0.091 (0.95)	0.065 (0.69)	0.081 (0.84)	0.100 (0.95)	0.063 (0.66)	0.073 (0.72)	0.076 (0.68)	0.086 (0.83)	0.064 (0.64)

5. Discussion  

The use of a haptic interface when examining and annotating crystallization screening results proved to be a natural mechanism to evaluate those results rapidly. Although we did not measure the performance, qualitatively it was less taxing than using a keyboard for the process and overall seemed significantly faster. A key component proved to be the ability to zoom in on an area of interest and scroll over the drop with simple finger motions on the screen. Our use is not unique in this aspect: the Collaborative Crystallisation Centre (C3) in Australia has developed a mobile phone application called Cinder, a crystal-finding application (manuscript in preparation; https://crystal.csiro.au/en/User-Guide/CNDR.aspx) based loosely on the social tool, Tinder. Users swipe right or left depending on their classification of crystal or not. This combines the haptic interface with the rapid ability of the user to identify the most attractive or interesting crystal results, somewhat similar to Tinder. In our implementation, the ability to mark multiple crystals (or potential crystals) within a well for X-ray evaluation with a finger also proved efficient. Overlapping the experimental beam profile allowed multiple crystals in each well to be selected and subsequent X-ray data collected. By use of an online database, as the images are annotated, all those with access to the database can follow and contribute to the analysis process. Here, all contributors followed the crystal-selection process and, during X-ray screening, the online in situ image could be compared with that from the crystallization step.

We developed the haptic interface application Harrier for the ‘screen to beam’ pathway to characterize initial crystallization screening results in terms of X-ray diffraction. The collection of fairly complete structural data was a bonus from plates not optimized for this application. Utilizing 1536-condition plates on a system not designed for this, the development needed to enable their use, and access to beamtime was not ideal for the experiment. However, the observation that 79% of crystals remained in their initial horizontal imaging position when placed in a vertical orientation on the beamline is very encouraging. Most of the lysozyme crystals formed were larger than typical screening results for more recalcitrant samples. This was a bonus, as they are presumably more sensitive to gravity and acceleration changes during shipping – our results may represent a worst-case scenario. This success rate can be improved by specialized shipping, or by screening close to the particular X-ray facility used. Alternatively, it is also possible to replicate conditions with multiple plates or experimental wells, such that failure becomes acceptable and can be mitigated.

Exposure times at the NSLS were of the order of 5 s for examining each well, with multiple crystals in each well sampled on the timescale of a few minutes or less. With a fast and precise positioning system, a rapid readout detector, focusing, and the brilliance of a third-generation source, e.g. NSLS-II, it is conceivable that images could be collected with a reduction in this time of several orders of magnitude at the increasing number of third-generation synchrotron sources. The gains that can be made are illustrated by the typical parameters for data collection used on I24: 0.05 s exposures with the X-ray beam attenuated 12-fold. The combination of the haptic interface and data-collection parameters such as these enables the possibility of sampling many tens of wells in a few minutes. The current capacity of the Hauptman–Woodward High-Throughput Crystallization Screening Center (one of the largest worldwide) is of the order of a few hundred samples a month. There is a real possibility that X-ray analysis could take place on every sample with minimal impact on available beamtime. The consequent evaluation based on X-ray rather than visual criteria could provide quantitative feedback for crystallization screening and optimization for individual projects, and also generate knowledge for improving crystallization strategies in general.

In situ diffraction at synchrotron sources is not new, and indeed has enjoyed a renaissance in recent years as a tool for the rapid evaluation of crystals (Aller et al., 2015 ▸). Jacquamet et al. (2004 ▸) developed the forerunner of the G-Rob system on a bending-magnet beamline at the ESRF (Grenoble, France), demonstrating that informative diffraction data could be collected in situ. More recently, a crystallization facility adjacent to a beamline allowing automated transfer of plates into the X-ray beam has been developed (Bingel-Erlenmeyer et al., 2011 ▸), while Axford et al. (2012 ▸) have carried out in situ diffraction data collection using microbeams. In the case of microcrystals and microbeams, the challenge imposed by optical refraction, causing a shift in the apparent crystal position, was highlighted. While the standard plates used in this study were not designed with in situ studies in mind, the combination of the flat and thin bottom well wall, growth using the microbatch technique, setup with a centrifugation step, and imaging from the base ensures that any optical errors in positioning are minimized. The small error in positioning due to optical effects was confirmed with successful data collection from ∼20 µm crystals using an X-ray beam of the same size.

In this case, ‘point and share’ has been applied to X-ray crystallography to classify crystallization screening outcomes and enabled the quantitative X-ray characterization of those outcomes. The application of haptic interfaces is wider than crystallization screening. As examples, there are applications in the interpretation of imaging data, e.g. from colonies on plates, or medical applications where the interpretation of a result may need a more expert eye. Outside the realm of purely scientific usage, there are countless applications that could benefit from the technology in engineering, security etc. Harrier can be run as a standalone application on a desktop or in its preferred embodiment, installed on a server for use by multiple users. Any image set that is available via the web can be used, making it easy to integrate with existing data sets. It has built-in support for basic HTTP authentication and can be configured with access controls to restrict editing only to logged-in users. Alternatively, it can be configured to run within a virtual private network, allowing only trusted collaborator access. The web services provided by Harrier allow the export and sharing of the data to third-party applications in an easy to use CSV format.

The source code for Harrier is released under the GPLv3 and is freely available at https://github.com/ubccr/harrier. The code will run on any operating system that supports Python 2.7 or greater. There are no specific hardware requirements and the application will run with minimal resource utilization. Depending on the size of the image library, adequate storage would be necessary to store the raw image files, or they could also be stored remotely on completely separate hardware. Harrier has been tested on an eight-core desktop machine with Intel i7 processors and 8 GB of RAM running Debian Linux version 8.0 (Jessie) and Python version 2.7.9. The source code is provided as a basis for further development in crystallographic applications, and also to extend the concept to any application where visual data are shared, analyzed and acted upon by a group.

6. Conclusion  

Haptic interfaces change human interaction with computing devices. In this application they provide a natural means of scrolling through crystallization screening results, a means of examining each result in detail and the ability to annotate/classify the result. By linking this to positional information readable by beamline robotics, we close the loop from crystallization to X-ray data collection. The ease with which this is accomplished through this kind of interface shows how useful it can be in any setting where visual information is analyzed by multiple people and the results acted upon. We provide a computational framework to build more advanced applications in crystallography and beyond.