Challenges in Small Screening Laboratories: SaaS to the rescue

Abstract

The Miami Project to Cure Paralysis, part of the University of Miami Miller School of Medicine, includes a laboratory devoted to High Content Analysis (HCA) of neurons. The goal of the laboratory is to uncover signalling pathways, genes, compounds, or drugs that can be used to promote nerve growth. HCA permits the quantification of neuronal morphology, including the lengths and numbers of axons. HCA screening of various libraries on primary neurons requires a team-based approach, a variety of process steps and complex manipulations of cells and libraries to obtain meaningful results. HCA itself produces vast amounts of information including images, well-based data and cell-based phenotypic measures. Managing experimental workflow and library data, along with the extensive amount of experimental results is challenging.

For academic laboratories generating large data sets from experiments using thousands of perturbagens, a laboratory information management system (LIMS) is the data tracking solution of choice. With both productivity and efficiency as driving rationales, the Miami Project has equipped its HCA laboratory with a Software As A Service (SAAS) LIMS to ensure the quality of its experiments and workflows. The article discusses this application in detail, and how the system was selected and integrated into the laboratory. The advantages of SaaS are described.

Introduction

As the costs of laboratory automation drop and the flood of information from genomics, proteomics, and metabolomics permits the functional testing of thousands of genes in different experimental systems, the type of discovery science done in smaller laboratories is being fundamentally changed. Similarly, the NIH Roadmap encourages screening of enormous chemical libraries to translate basic science into clinical treatments [1]. To effectively manage these types of projects, which involve teams of scientists and various high throughput instruments, software solutions are needed to track the hundreds of reagents and thousands of perturbagens. Laboratory Information Management Systems (LIMS) have been used in industry for years to track product testing and clinical samples. These systems are now moving into basic and translational research laboratories in academic settings. Unfortunately, academic scientists used to bench work are unlikely to have experience with the purchase and implementation of expensive software solutions. This article will describe the challenges and decision process in one laboratory and why “on-demand” or “software as a service” LIMS provide an attractive option for smaller research groups.

The leaking of High Throughput Screening methods from Big Pharma into small biotech and academic labs

Over the past three decades Big Pharma has driven the development of high throughput screening (HTS) methods to speed the development of new drugs. Liquid handling robots, large scale culturing of cells and sophisticated detection methodologies allow hundreds of thousands of perturbagens to be evaluated in a few days for their ability to alter the activity of enzymes, bind to receptors or alter a variety of cell functions. A technology that is now making large inroads into industry is High Content Analysis (HCA) [2]. HCA typically uses automated microscopes and image analysis to measure scores of features from individual cells, although the basic approach is also being used in tissues and in model organisms such as flies and fish. High Content refers to that fact that many independent features are measured simultaneously. An example is provided in Figure 1. A nerve cell is photographed in 2–4 colors to identify features such as the nucleus, the processes (neurites) and perhaps synapses or some protein of interest. Next the automated image analysis software measures the area and brightness of the nucleus, the number of neurites, the length of the longest neurite, the number of branches from the neurites, etc. This is done over and over for tens of thousands of neurons. The effects of different perturbagens are then compared to controls and to each other for each measured parameter. The fact that it is done at the cell level, rather than averaging the behavior of all the cells in a single well, brings additional richness to the large data sets. Big Pharma uses HCA to study cancer cell lines [3, 4] and to do early stage toxicity studies [5]. The use of cell lines facilitates HTS but suffers from problems of interpretation that always arise with the use of transformed cells [6]. Biotech companies and academic labs are more likely to use primary cells in medium throughput screens.

Figure 1.

High Content Analysis of Neurons. Neurons are imaged and individually analyzed to measure the areas of various cellular compartments, the number of neurites, the length of the neurites and the number of branches of the neurites. The very large data sets are merged across plates and experiments using normalization algorithms that are dependant on controls in each plate. Statistical analysis then allows determination of which perturbations alter various parameters such as neurite length.

Hunting for treatments for paralysis

The Miami Project to Cure Paralysis uses High Content Analysis to screen a variety of libraries to indentify genes or compounds that might enhance nerve regeneration in individuals with spinal cord injury (SCI) or traumatic brain injury. The problem is particularly daunting. First, intrinsic factors inside neurons control their ability to regenerate. Second, environmental factors produced by support cells, especially at injury sites, can prevent even "willing" neurons from regenerating [7]. Our laboratory, the LemBix lab, uses a variety of primary neurons for our experiments; cortical neurons, hippocampal neurons, cerebellar neurons, retinal neurons, spinal cord neurons and sensory neurons located adjacent to the spinal cord. Each cell type must be isolated and prepared in special ways and is grown in custom-made media. In some screens we test Food and Drug Administration (FDA) approved drugs to hopefully speed transition to the clinic by repurposing existing drugs. We also screen compounds with known and unknown activities such as kinase inhibitor libraries and combinatorial compound libraries [8]. Finally, we screen cDNA libraries to alter the expression of specific genes [9–11]. Our motivation for this is to identify individual genes, critical gene families and the signaling networks inside neurons that are essential for nerve regeneration. This could lead the way to novel drug discovery campaigns or gene therapy trials.

In the past we studied neurons using the conventional in vitro approach found in basic science labs: design an experiment, optimize it, do it three times and never do it again. Experiments were typically done by an individual and documented in a lab notebook. The methods used in Good Laboratory Practice/Good Manufacturing Practice (GLP/GMP) environments, such as rigid control of SOPs and mandatory prequalification of all reagents and lots, were either not appropriate or prohibitively expensive. However our HCA experiments involve several individuals, hundreds to thousands of reagents and perturbagens, and are repeated 2–3 times a week. Part of a typical workflow is illustrated in figure 2. A brain region, such as hippocampus, is dissected from a young rat or mouse. Enzymes and pipetting are used to break the tissue into single cells. The cells are mixed with DNA and transfection reagents, and are shocked with electricity (electroporation) to introduce the DNA into the cells. The neurons are distributed to fresh multiwell tissue culture plates and incubated for 2–21 days. The cells are then preserved with chemical fixatives and stained with antibodies and dyes to reveal cell morphology. The plates are then fed through the HCA microscope using a small plate-handling robot.

Figure 2.

Example HCA workflow. Brain tissue is dissociated to single cells, mixed with transfection reagents and plasmid DNA prior to electroporation. Neurons in multiwall plates are incubated to allow differentiation and process growth. The plates are processed to allow visualization in the high content analysis microscope.

HCA experiments are costly. Our experiments using primary neurons, expensive cell attachment proteins, defined media, purified plasmids and a variety of antibodies cost about $100 per plate to prepare. One screen we conducted used 348 plates. If one includes failed experiments and plates used in the optimization process this would amount to more than $50,000 in reagents for the screen of 1100 different genes.

Very quickly it became obvious that our conventional data collection methods were not able to help us identify when and where experimental runs failed. Some information was spread across too many notebooks, and critical parameters from various machines were not being captured at all. Figure 3 shows just a part of the data we needed from our HCA work flow.

Figure 3.

Example of flowchart depicting data to be captured by the LIMS. Each step in the HCA workflow is analyzed to determine the reagents, conditions and manipulations that are associated with. For immunostaining of cells in culture, the specific antibodies used, their concentrations and incubation times are critical variables that affect the quality of the images captured by the HCA microscope.

Hunting for a LIMS

Discussions with our colleagues who had experience in industry helped us realize that electronic laboratory notebooks were probably not the solution we needed. Instead, we needed a Laboratory Information Management System (LIMS) designed to track reagents, processes and tests.

Salespeople for LIMS vendors speak the language of Project Management, a dialect foreign to most basic science laboratories. This was a surprising barrier to communication initially and called for remedial reading on our part. In order even to begin to explain to the vendors what we wanted we had to formally describe our work flows in great detail. Therefore we prepared a user requirement specification (URS) document (18 pages), a Request For Proposal (RFP) spreadsheet, and an appendix with figures (figure 4). This was done in consultation with staff in our research office. Nothing in our previous experience had prepared us for that exercise and, in retrospect, it would have been wise to hire an independent LIMS consultant to help with this important task. Based on the responses from the vendors we selected three for further on site demos and testing.

Figure 4.

Development of User Requirements Specifications: In order to allow effective assessment of the needs of the LIMS a detailed URS and RFP must be prepared. The URS and RFP allow vendors to provide appropriate documents that the team can then evaluate for LIMS selection.

The vast majority of LIMS products use a sample/test model along with a folder hierarchy to navigate the LIMS. Products are differentiated on the richness of software tools they have for representing and manipulating data, such as representations of large chemical libraries, and the ease with which workflows can be built. Due to the complexity of the software and the underlying database (DB) it is most common to either have the vendor build out the initial implementation of the workflows or to have a local full-time LIMS analyst/programmer do the work. This presents an obvious choice of speed to initial release versus cost. Because of the very specialized nature of individual LIMS systems it is likely that a local programmer will have to be trained by the LIMS vendor, especially to build workflows and to integrate instruments.

A major issue that we had not anticipated before launching this effort was the large and costly nature of the infrastructure needed for a LIMS. This includes at least two $10,000 servers (for development and production) and expensive site licenses for commercial database solutions (i.e. Oracle). It also requires that there be a local IT support team who can install and manage the servers and provide a Data Base Analyst (DBA) to maintain the server. A small lab cannot justify or afford a full-time DBA but relying on institutional DBAs in the IT division can result in sluggish response to problems. In fact, the launch of our LIMS was delayed by over 6 months in order to purchase and install the servers, deal with DB licensing issues (very costly) and get the databases operational.

The process of getting departmental approval for the project, developing the URS, testing different products, negotiating with the vendors and then shepherding the purchases through the university took 9 months.

Building the LIMS

After the LIMS vendor was selected, in this case the Thermo Fisher Scientific Nautilus LIMS, a multistep system was used to launch the LIMS. First two analysts from the vendor spent a week in the lab interviewing staff and observing workflows. They then developed a detailed plan for building out various parts of the LIMS. Meanwhile two lab members went for a week of training at the corporate site. Subsequently, one of the vendor LIMS analysts returned to the lab to work with the local programmers to install and configure the LIMS on the two servers. Issues of scheduling with the institutional IT group slowed this process. Over all, this phase took about 9 months, much of the time consumed with local scheduling issues rather than actual programming or implementation.

As lab members tested and started to use the LIMS two issues became apparent. The first was the difference between expectations of users accustomed to the sophisticated graphical user interfaces in today's software for document and image processing and the user interface present in the uncustomized LIMS. To gain entry into the GMP/GLP marketplace a LIMS needs to be "validated" as compliant with various organizational and agency standards such as ISO 9000 and the Food and Drug Administration GLP standard. Once validated, major modifications to the LIMS, such as revising the user interface, are unlikely. The hierarchal folder approach (figure 5A) was criticized as being too cumbersome and slow. It also did not lend itself to obvious mapping of a real world workflow. To resolve this problem, our LIMS analyst built customized data entry forms using tools in the LIMS that exactly corresponded to the data that needed to be captured at specific points in the workflows (figure 5B).

Figure 5.

LIMS on demand data entry. Web browsers use a plug-in to display the LIMS interface. A) A typical LIMS use a "sample and test" metaphor to model laboratory workflows. This is displayed in a folder-based hierarchy as a sample moves through the workflow. B) Customized data entry windows with dropdown windows, pre-populated fields for standard steps and controlled formatting, simplify and speed data entry for lab members.

The second limitation was using a client-server system approach in a university setting where there is a high percentage of computers using Mac, Linux and Unix OS. Rather than use the computers on their desks, some staff would have to move to another workstation to add information to the LIMS. Mac users had the most difficultly adapting to the LIMS, probably because of the extra effort needed.

A tactic we used to gain user acceptance was to provide useful functionalities that are outside the HCA workflows. One major area was stock tracking and freezer searching. All materials entering the lab from commercial sources are entered into the LIMS, using data imported from the purchasing department. This greatly reduces the amount of information that needs to be entered locally. Labels with bar codes are put on the items and they are given to the lab members who have only to enter the items’ locations into the database. Thus a search of the LIMS quickly gives the location of any item, such as an antibody or restriction enzyme. A list of items whose locations are not logged is provided to the Principal Investigator once a week, along with the owner's name. Ouch. Actually, the staff really liked being able to find things quickly. We also log animal use in the LIMS. This facilitates compliance with animal use reporting and provides a convincing trail of veterinary drug use.

SaaS to the Rescue

Recently various vendors began offering LIMS as an "On-demand" or "software as a service" (SaaS) product. In this new model, the vendor or a third party data center hosts the LIMS server. This is likely to be a virtual server instead of a physical device. The LIMS vendor deals with all the DB issues, including user licensing and installing updates to the DB and LIMS itself. A second feature of this model is that the LIMS is delivered over the web, freeing it from the client-server approach.

We were excited to switch to this approach to free us from dependence on the local IT department and to make the LIMS user platform independent. Concerns about the SaaS model for many organizations are mostly about data safety and security. This will need to be assessed on a case-by-case basis. Our concerns were more about speed and ease of conversion and how the SaaS model works in an environment in which the LIMS has to control devices in the laboratory. In our case, once the virtual server was established at the data center, exporting the LIMS from the local server to the data center was trivial, accomplished in a couple of hours by one person from the LIMS vendor working with our LIMS analyst.

In SaaS or On-demand environments, interactions with the local computers are problematic. When providing LIMS functionality over the web it is critical to be able to print reports & barcodes, interact with instruments, & access local files. In general the LemBix implementation went smoothly except for the barcode printing. Barcode printers are not compatible with the standard On-demand Universal Print drivers. This requires the installation of the Barcode printers and fonts on the server in the datacenter to allow for local printing of Barcodes. It took some time to troubleshoot the problems and to ensure that all of the settings and versions of drivers were identical on both the server and the local machines. Once the troubleshooting was complete it was simple to print barcodes and in the future it will be easier to set up barcode printers through On-demand.

Conclusions

Small laboratories doing screening or other projects where there is a complex workflow with several staff members will benefit by using a LIMS to track reagents, stocks and process steps. The initial start-up costs for a conventional LIMS are significant and likely beyond the resources available in typical university labs. These include hardware, local IT support, and DB licenses. Hiring an independent LIMS consultant to assist in development of the URS and vendor selection will facilitate launching a LIMS project. Building the initial workflows will need the participation of the vendor and a LIMS consultant or a highly trained local LIMS analyst/programmer. A SaaS LIMS offers important advantages for smaller labs by eliminating many start-up costs and roadblocks, and offering a more flexible licensing scheme to add or drop users as projects expand and contract. A web-based means of interacting with LIMS eases acceptance and use by lab members by eliminating client-server limitations. Conventional and SaaS based LIMS will need better graphical user interfaces and workflow tools, as well as easier integration of peripheral devices to gain widespread penetration into the “small lab” market, where scientists are accustomed to building or adapting software tools.

Abbreviations

IT

Information Technology

DB

Database

DBA

Database Administrator

URS

User requirements specifications

SaaS

Software as a Service

LIMS

Laboratory Information Management System

HCA

High Content Analysis

GMP

Good Manufacturing Practice

GLP

Good Laboratory Practice

FDA

Food and Drug Administration

RFP

Request For Proposals