Internet of instruments

Abstract

The internet of things is arriving at the laboratory, connecting instruments for remote control and monitoring. Along with Artificial Intelligence Software to analyse data and design experiments, it could fundamentally change the way research is done.

Biologists were among the first who embraced computers and the Internet. In the early 1990s, they used these technologies not only for communication but also for online databases to store, search, and analyze gene and protein sequence data, for protein design, or for protein folding simulation on supercomputers and distributed computer networks. It eventually spawned the field of bioinformatics, which has made an enormous impact on research in the life sciences. Curiously though, biologists have been rather slow to adopt data connectivity in the laboratory itself: while computers are ubiquitous, many instruments and tools from PCR machines to gel imaging or microscopes are still operated manually. Now, the Internet of Things (IoT) is entering biomedical research facilities, hooking up equipment for remote monitoring and data collection, which both saves costs and creates more consistent conditions for experiments. It thus paves the way for artificial intelligence (AI) and machine learning (ML) for further analysis of experimental data and images. Some scientists speculate that once AI and ML become incorporated into research, the potential of IoT could go much further than cutting costs and streamlining laboratory workflows to fundamentally change the way research is done in the life sciences.

While this remains to be seen, there is already huge potential for operating laboratories more effectively and maintaining greater control over experiments, which will itself lead to improved outcomes. Indeed, greater reproducibility has been cited as one crucial benefit of being able to guarantee more consistent conditions within research equipment and the laboratory environment.

Protecting samples

An obvious advantage of IoT is the ability to monitor, for example, refrigeration units to ensure the safety of precious, sometimes irreplaceable samples, which prompted Harvard University to invest in monitoring systems to minimize risk of the freezers failing. The laboratory of Peter Girguis at Harvard's Department of Organismic and Evolutionary Biology studies micro‐organisms from the deep seas that are sampled in expensive expeditions and stored in freezers at temperatures from −20°C down to −80°C. Some of the samples are in practice irreplaceable, according to Girguis, or would cost millions of dollars to replace. The colder freezers at −80°C had already been connected to Harvard's Operations Centre, which automatically raises alerts if the temperature starts to rise. But the −20°C freezers were not compatible—a common problem for many laboratories of being unable to establish universal monitoring systems for all their equipment. This led Harvard University as a whole to employ the services of TetraScience, a start‐up company in Boston, MA, USA, which is dedicated to laboratory connectivity and monitoring. Harvard became the testing ground, and now, 19 of its laboratories use the service, along with 40 external academic and industry clients.

“While working at Harvard, the founders of Tetrascience were awarded a grant through the Office for Sustainability to create a prototype of their technology, which eventually evolved into the equipment they provide today”, said Quentin Gilly, Senior Coordinator for Harvard's Faculty of Arts and Sciences Green Program. “Once the company developed into TetraScience, Harvard became the testing grounds for the early models of their equipment. This is now helping cut costs by providing information directly to the equipment managers and limiting the need for intermediaries”. It also contributes to cost reduction by informing researchers of equipment left on so that it can be turned off remotely. “We can also use this data to record energy use and better understand how energy is used in laboratories”, Gilly added.

… the potential of IoT could go much further than cutting costs and streamlining laboratory workflows to fundamentally change the way research is done in the life sciences.

The Harvard laboratories now have more than 100 instrument and equipment types linked to the Internet, chiefly incubators and freezers, but with scope for centrifuges, balances, pH meters, and high‐performance liquid chromatography (HPLC) systems. In addition to monitoring for faults, the system can track usage and workload and facilitate remote control. The objective is to provide laboratories with a software platform that supports not just equipment but also people and processes, to analyze performance and detect trends that might require action or help to reduce energy consumption, according to the company. The TetraScience platform connects to equipment via a small pocket‐sized Wi‐Fi module, either through an instrument's data port or through an external monitoring sensor. This enables monitoring of temperature, humidity, and levels of carbon dioxide and oxygen, as well as vibration, light intensity, and airflow.

Controlling the environment

Given that a major objective of laboratory connectivity is to protect valuable samples from failure of refrigerators and freezers, and to maintain environmental conditions, the communication network needs to be resilient. For this reason, Elemental Machines, another laboratory monitoring start‐up based in Cambridge, MA, USA, uses cellular communications as a backup in case the Wi‐Fi network fails. Overall, regular and reliable collection of data could help resolve problems beyond immediate equipment failure, commented John Morgan, Director of Marketing at Elemental Machines. “We had a case in which one of our customers saw strange results from his HPLC system. He installed an Element A sensor (one of the firm's products) near the HPLC and found that suspicious results correlated with cold air blowing on the system from his HVAC (heating, ventilation, and air conditioning) system”, Morgan explained. “He was able to correct the situation”.

Given that a major objective of lab connectivity is to protect valuable samples […] the communication network needs to be resilient.

Morgan also highlighted the value of data monitoring for compliance with the growing body of regulations, especially for food and pharmaceutical companies that involve laboratory research. This in turn leads to the larger issue of reproducibility, as ensuring accurate environmental conditions can help to avoid or reduce the need to repeat experiments so as to confirm their findings. The one caveat is that laboratory monitoring should not constrain freedom to vary conditions; a one‐size‐fits‐all approach risks overlooking nuances in data gathering or experimental design. “Good reproducibility is always important but narrowing the parameters unnecessarily may create a culture of inflexibility”, Gilly agreed. “For example, ultra‐low temperature freezers are typically programmed to run at −80°C, however much evidence shows that −70°C works just as well to protect the freezer contents and saves an average of 37% energy per year (https://www.mygreenlab.org/-70-is-the-new--80.html)”.

In most instances though, greater accuracy encourages experimental diversity, because researchers can be more confident of their measurements, commented Steven Niemi, Director of the Office of Animal Resources at Harvard University. “The greatly improved precision in which these and other technologies monitor animals and their immediate environments encourages rather than discourages variation between experiments and researchers, rather than confining them to one‐size‐fits‐all”, he said. “In other words, reducing the uncertainty about husbandry increases the likelihood that one's research data are accurate and reproducible and should enable more intellectual creativity”.

Indeed, monitoring of animal facilities is an application of IoT that could help to cut costs and improve reproducibility. Niemi, who has written extensively on how the IoT can benefit animal research, noted that a typical mouse cage costs US$7.87 per day before any subsequent value can be generated by research. “Apply that total to 120 mouse cages on a double‐sided IVC rack with a capacity of 140–160 cages, and the total invested dollars at risk for cage or rack malfunctions that could torpedo that investment on any given day is around US$1100”, he continued. For a representative rodent facility, maintaining an average 8,000 cages a day, the total sunk research cost then reaches almost US$63,000 on any given day or US$23 million for a whole year. “Yet we remain content with once daily, brief visual glances of animals often difficult to see and rodent houses devoid of automatic and continuous monitoring capabilities to protect that investment as well as ensure animal welfare”, Niemi noted. He does not anticipate that IoT and automation will cost jobs in animal facilities, but rather allow staff to provide better care and supervision, with more consistent conditions and greater scope for detecting and resolving problems.

Another key point is that monitoring reduces the amount of human contact, which is important because intrusion causes distress for rodents in particular. This can be a concern especially for experiments observing behavior, or with animal trials of drugs that have an impact on the central nervous system. Continuous monitoring of laboratory animals also brings potential to generate more experimental data, for example, on activity and sleep patterns, which can add value to experiments and preclinical trials.

“…reducing the uncertainty about husbandry increases the likelihood that one's research data are accurate and reproducible and should enable more intellectual creativity.”

AI‐driven data analysis

This leads further toward the application of AI technologies, especially ML, to derive new insights from experiments and observations, which has already been demonstrated in analyzing visual data from microscopy. The immediate objective here is to overcome some of the inherent disadvantages of current methods for localizing proteins or cellular features using fluorescence microscopy. One popular method is two‐stage antibody labeling. The first step involves a primary antibody that recognizes a target antigen and hence provides the “specificity”. The second step uses a label to seek out the primary antibody and provide light or color that illuminates the target protein under the microscope. Such labels include fluorescent proteins, colored particles, and organic dyes with the purpose of enabling identification and measurability. However, these methods come at the cost of disturbing the specimen under examination. The pre‐embedding procedures to make cellular membranes permeable for antibodies to enter a cell or tissue compromise the cellular ultrastructure: structures can be changed and soluble proteins can even be lost during those incubation steps to establish adequate permeability.

There are also inconsistencies in measurements arising from the use of fluorescence itself, while interference or spectral overlap limits the number of labels that can be used simultaneously to study different proteins or features 1. Use of image processing algorithms can only partly overcome this deficiency, so there has been interest in developing methods that avoid the use of fluorescence altogether. Several US institutions therefore developed a ML approach called in silico labeling that can predict reliably where fluorescent labels would be applied based on images of the either unlabeled fixed or live biological samples 2.

Continuous monitoring of laboratory animals also brings potential to generate more experimental data, for example on activity and sleep patterns …

The researchers designed a model based on deep neural networks, which comprise nodes and links between them and are arranged in multiple layers to allow nested feedback between data and predictions. The neural network was trained on sets of matching images of the same cells, one unlabeled and the other with fluorescent labels, with the objective of predicting the latter from the former. The process was repeated millions of times until the accuracy reached virtually 100%. The software could not just predict where the fluorescent labels belong, but also identify nuclei in cells with high accuracy. Furthermore, it could detect neurons within groups of cells that included astrocytes and immature dividing cells, as well as distinguish an axon from a dendrite. It could also tell whether a cell was dead or alive.

“Our experience has been that computers do a better job discovering nuances in data than humans do, at least for large and complex datasets”, said Steven Finkbeiner from the University of California, San Francisco and a lead author on the study. “In fact, computers can often ‘see’ patterns in data that the human brain can't comprehend. Deep learning is so sensitive that it frequently discovers systematic biases unbeknownst to the investigator that are introduced at the level of data acquisition. It has had a huge impact on the quality control of our systems, for example”.

The researchers’ ambition is now to extend the role of ML to experimental design around the concept of what Finkbeiner calls the “Thinking Microscope”. “My hope with the Thinking Microscope is that it will design experiments on the fly based on the data it is receiving, and hopefully design experiments that are more data‐driven and less constrained by human bias and imagination”, he explained. Finkbeiner conceded though that while deep learning has great potential to exploit data to suggest new avenues of experiment, it is fundamentally limited when it comes to visualizing problems and spotting connections between unrelated topics. “It is unlikely to make critical connections between ostensibly unrelated concepts and ideas”, he said.

Nonetheless, Finkbeiner is optimistic that ML will transform research by deriving new insights from data and accelerate discovery, helping researchers themselves find relevant connections between apparently unrelated concepts. “Whether the specific approach we developed will revolutionize lab science is hard to tell, but I do believe that deep learning will”, he said. “The breadth of the potential applications is astonishing, there are so many possible it is hard to know where to begin. We are working on a project to train networks to diagnose disease in a dish, to create an in‐silico expert pathologist, to stratify patients, and discover results in our preclinical models that predict outcomes in clinical trials. And that isn't considering the work we and others are doing on genomics”.

AI to design experiments

Genomics is indeed an area where ML is starting to make an impact to make sense of the huge amounts of sequence data. One of the biotech companies that develop such tools is Thermo Fisher Scientific, which offers various apps to integrate with a laboratory's genomics platform to combine hundreds of sequencing experiments into a single project so that researchers can analyze large data sets more quickly. One of these apps is for high‐resolution melt (HRM) analysis to identify variation in nucleic acid sequences. It detects small differences in PCR melting through dissociation curves, enabled by high‐brightness, dsDNA‐binding dyes in conjunction with real‐time PCR instrumentation that has precise temperature ramp control as well as advanced data capture and access to software designed specifically for HRM analysis.

More widely, ML looks like it will play a growing role in genomic analysis for which it is ideally suited because it involves finding meaningful patterns in huge data sets. Such patterns might be associated with probabilities of developing certain diseases or susceptibility to given drug treatments. In this way, ML could prove useful in advancing the cause of personalized medicine, which is currently constrained by the logistics and cost of genomic analysis for individuals.

“Our experience has been that computers do a better job discovering nuances in data than humans do, at least for large and complex datasets.”

Machine learning could also accelerate progress in CRISPR‐based gene editing, which requires selecting an appropriate target sequence. The CRISPR system relies on specific RNAs to guide the Cas9 protein to the target nucleic acid. Determining the appropriate guide RNA is a major challenge and can be computationally intensive. Machine learning could significantly reduce the time, cost, and effort necessary to identify that target sequence. Such potential has already attracted some biotech start‐ups, such as London‐based Desktop Genetics, which was founded in 2012. The company developed a software package called DESKGEN AI to determine the biological variables influencing RNA guide sequences, which, unfortunately, do not bind exclusively to their desired target but also to other targets, resulting in cleavage of unintended genomic sites. A certain amount of such off‐target activity can be tolerated but not too much, and a major challenge in RNA guide design is accurate prediction of both on‐ and off‐target effects to maximize on‐target efficacy and sensitivity.

Security concerns

While the benefits of IoT are increasingly appreciated, its use is still being held back by concerns over security. In Thermo Fisher's case, such concerns have helped shape technology choices, according to Sung‐Dae Hong, Vice President and General Manager for Growth, Protection, and Separation Products. “We understand that confidentiality of data and protocols are critical, both to a customer's research and to the publication process”, he said. Security has also loomed large for Consolidated Sterilizer Systems, maker of cloud‐connected steam sterilizers for laboratory equipment. “It's all about risk analysis – what type of data is cloud connected, how is it transmitted, how is it secured and how is it accessed”, commented Arthur Trapotsis, CEO at Consolidated Sterilizer Systems. “We've gone through great lengths to utilize cloud‐technology that protects the data both in transit and at rest”. Users are then allowed access to the data through a web portal that employs the same sort of 2‐factor authentication required by ATMs. The company also offers the ability to isolate the equipment from a facility's internal data network to protect against incoming cyberattacks, as well as provide further defense against theft of data.

While the benefits of IoT are increasingly appreciated, its use is still being held back by concerns over security.

“The industry as a whole, both manufacturers and customers, have been very risk averse and therefore changes to equipment designs and adoption of new technologies have occurred at a much slower rate compared to consumer electronics or manufacturing”, Trapotsis commented. But there are signs of this changing not least because the benefits are becoming firmly established and better understood, even if not all researchers are yet convinced the IoT and AI will revolutionize the way science is done.

EMBO Reports (2018) 19: e47267