An immersive journey to the molecular structure of SARS-CoV-2: Virtual reality in COVID-19

Graphical abstract

Abstract

The era of the explosion of immersive technologies has bumped head-on with the coronavirus disease 2019 (COVID-19) global pandemic caused by the severe acute respiratory syndrome–coronavirus 2 (SARS-CoV-2). The proper understanding of the three-dimensional structures that compose the virus, as well as of those involved in the infection process and in treatments, is expected to contribute to the advance of fundamental and applied research against this pandemic, including basic molecular biology studies and drug design. Virtual reality (VR) is a powerful technology to visualize the biomolecular structures that are currently being identified for SARS-CoV-2 infection, opening possibilities to significant advances in the understanding of the disease-associate mechanisms and thus to boost new therapies and treatments. The present availability of VR for a large variety of practical applications together with the increasingly easiness, quality and economic access of this technology is transforming the way we interact with digital information. Here, we review the software implementations currently available for VR visualization of SARS-CoV-2 molecular structures, covering a range of virtual environments: CAVEs, desktop software, and cell phone applications, all of them combined with head-mounted devices like cardboards, Oculus Rift or the HTC Vive. We aim to impulse and facilitate the use of these emerging technologies in research against COVID-19 trying to increase the knowledge and thus minimizing risks before placing huge amounts of money for the development of potential treatments.

1. Introduction

Severe acute respiratory syndrome–coronavirus 2 (SARS-CoV-2) is a single-stranded positive-sense RNA virus responsible for the coronavirus disease 2019 (COVID-19) global pandemic that has emerged towards the end of 2019 in Wuhan, the capital of the Chinese province of Hubei [1], [2]. Prior to SARS-CoV-2, six coronaviruses were known to cause diseases in humans, four of them causing only mild to moderate symptoms and other two (SARS-CoV and MERS-CoV) provoking severe respiratory syndromes [1]. A large and still growing number of symptoms were ascribed to COVID-19, the most evident shared with other coronavirus infections: fever (87.9%), cough (67.7%), fatigue (38.1%) and, to a less extent, diarrhea (3.7%) and vomiting (5.0%) [3]. However, in contrast with other coronaviruses, COVID-19 frequently exhibits unexpected long-term severe consequences.

Each SARS-CoV-2 virion is a 50–200 nm diameter spherical-like capsule enclosed by a protein-decorated lipid bilayer that contains a single-stranded RNA chain. Some of the proteins encrusted in this membrane are the receptors employed to anchor target cells [4]. Four structural proteins have been reported to form part of coronaviruses, including SARS-CoV-2: S (spike), E (envelope), M (membrane), and N (nucleocapsid) proteins [5]. The protruding part of the membrane embedded S-proteins recognize specific receptors in target cells, thus being key for the different stages of the infection mechanism: recognition, attachment of the virus and membrane fusion. The human angiotension-converting enzyme 2 (ACE2) was identified as the natural receptor of the virus although it has been reported that the nicotinic acetylcholine receptor (nAChR) could also play a significant role [6]. Once the virus enters the cell, an acute infection takes place.

A huge effort has been performed to obtain structural information with atomic resolution of this virus, aiming to understand its infection mechanism and then to develop effective drugs against it. The structures of several COVID-19 proteins are already publicly available and specific sites have been created to facilitate the access to the corresponding entries: see for instance PDBj [7], [8] or the Spanish 3DBIONOTES-COVID19 tool [9]. Both sites contain SARS-CoV-2 protein models from different sources: PDB-Redo, SwissProt, AlphaFold, European Projects and the repository of the Coronavirus Structural Taskforce. Visual and quantitative analysis of these molecules are expected to provide key structural, biological, and pharmacological insights to understand their function that would be hardly accessible by other methods.

The growing implementation of augmented reality (AR), virtual reality (VR) and mixed reality (MR) technologies for many applications in a large variety of fields together with the increasingly easiness, quality and economic access to these technologies is clearly transforming the way we interact with digital information by enriching classical methods of communicating information. Traditional flat pictures can now be easily complemented by adding new dimensions, user interaction, movement and even sound. Although these technologies have existed for decades, it has not been until very recently that they are being successfully exploited, favored by the democratization of the access to the required technologies due to the quick reduction of cost during the last years to and the accelerated enhance of the performance [10]. These technologies are transforming media and entertainment and have already impacted Chemistry and Biology in education, research and dissemination, in particular for molecular visualization, where they have an obvious potential [11]. The use and advantages of using VR during COVID-19 pandemic have been reviewed recently [12]. VR has proved to be useful in a large number of applications such as telemedicine, planning, patient treatment, medical marketing, training & learning, contributing to prevent infections by disseminating information, and even to reduce the face to face interaction of medical doctors with the infected COVID-19 patients. In the present work we will deal with a different and more specific application of VR technologies that is expected to have a large impact in research, education and dissemination of information related with this virus: the immersive visualization of SARS-CoV-2 proteins and receptors involved in their infection mechanism. Molecular structures are naturally tridimensional, flexible and they are rarely isolated. Remarkably, they are typically represented as flat sets of circles (atoms) joined by rigid bars (bonds). Such a representation is not realistic and makes difficult to understand how molecules behave. The ability to properly visualize from different perspectives as well as to manipulate virtual representations of chemical structures is key to understand complex molecules. Thus, VR technologies applied to molecular visualization facilitate rational drug design and, in general, the understanding of molecular mechanisms in life science research. Of course, the same arguments can also be extrapolated to the design of molecules for other (bio)technological purposes [13]. Standard software for the visualization of 3D molecular structures, such as VMD [14] or PyMol [15] significantly improve their comprehension by projecting them onto 2D screens, allowing users to examine them in an artificial 3D environment. However, there is a qualitative difference between this visual approach to three-dimensional structures and the immersive visualization, with no limitation in the translational and rotational perspectives of the observer and even with the possibility to interact with the target system, provided by VR. When applied to COVID-19, VR affords a clear advantage compared to more conventional molecular visualization / interactive technologies. For example, being able to view and interact with the molecules involved in the SARS-CoV-2 infection as three-dimensional objects simplifies the task of modifying them so they fit better to protein binding sites. VR interaction is the more intuitive way to deal with viral proteins, human receptors and possible drugs, realistically sensing their 3D structure and providing a perception that cannot be achieved with more conventional techniques.

Different headsets have been designed to facilitate the natural movement and interaction in such virtual environments, with the quality and performance to cost ratio quickly increasing with time. Immersive environments to visualize molecules have been working for more than 20 years for educational purposes, dissemination, research and structural analysis but the required infrastructure (Cave Automatic Virtual Environments or CAVEs) was accessible only throughout educational or research centers [16], [17]. In these old systems, the images projected on the walls and the floor of confined spaces of different geometries were viewed in three dimensions by using different complementary devices that included headsets and pointer-tracking systems. Molecular dynamics simulations of relatively complex systems could be viewed and analyzed in these environments. Several examples have been reported including drug-protein binding studies [18], [19], comparison between structures of related proteins [20], [21], interactive docking between proteins [22], examination of NMR ensembles and constrains [23], Caffeine computational simulations at quantum–classical levels [24], design and construction of nanotube models [25], and education in the characterization of drug-receptor interactions [26].

More recently, VR technology has been used to visualize specific molecules and even to interact with them in much more portable devices based on smartphones. VR-based learning tools such as NanoOne (by Nanome, http://nanome.ai) or Molecular Zoo have been developed for students to efficiently handle biomomolecules in an affordable way [27]. In the same line, Peppy is a VR tool aimed at facilitating the teaching of the principles of relatively short polypeptides structure to undergraduate classes [28]. From a complementary perspective with educational and research applications, David R. Glowacki et al. developed a framework for interactive molecular dynamics in a multiuser VR environment. This framework allows the synchronized visualization and the cooperative manipulation of complex molecular structures (fullerene, carbon nanotubes, etc) “on the fly” by several users in the same virtual environment [29], [30]. Finally, the MolDRIVE system, developed at the TU Delft [31], allows interactive molecular dynamic simulations with artificial forces applied by the user. This could be useful to reach specific configurations of the studied molecules or molecular systems. MolDRIVE can be used in a variety of VR systems such as workbenches and CUBEs but it is not designed for more affordable devices. The official website of this application (http://graphics.tudelft.nl/Projects/MolDrive) has not been available during the preparation of this manuscript. Although all the previous applications are very interesting to visualize other type of molecules, in this review we will focus just on VR software that allows the visualization of molecular structures involved in the infection caused by SARS-CoV-2.

2. Virtual reality in the molecular visualization of SARS-CoV-2

Next, we will examine different VR implementations that allow the visualization of publicly available molecular structures related to COVID-19. This revision will cover desktop applications, web-based VR implementations and downloadable apps for smartphones.

3. Desktop applications

A number of desktop applications enable VR molecular visualization in combination with high-end VR devices (Oculus Rift, HTC VIVE, and Microsoft Mixed Reality). They can be really helpful for expert/professional users, allowing visualization, analysis and interaction options. Typically, the installation and effective use of these programs are not as straightforward as for applications developed for smartphones since they often require external dependences and some training. Most of them are based on the Unity3D video game engine (https://unity.com/). In general, these applications are properly optimized and quite stable. The most popular desktop applications of this group are:

Molecular Rift [32], [33] is an open-source molecular viewer that allows the interaction with the VR environment throughout hand gestures instead of standard VR controllers. It is based on Unity3D combined with the UnityMol open-source library to visualize molecules in Unity3D [34]. UnityMol [34], [35] is a molecular viewer developed to work with the Unity3D (https://unity.com/) game engine. It incorporates code to identify secondary structure of proteins as well as different visualization formats and specific selections. It is based on the VRTK [36] framework to support HTC Vive, Oculus headsets, and Windows Mixed Reality. UCSF ChimeraX [27], [37] is also based on Unity3D. It is well suited for visualization and analysis of biomolecular structures (even multi-person VR sessions), especially proteins and nucleic acids, using virtual reality headsets such as HTC Vive, Vive Pro, Oculus Rift, Samsung Odyssey and Windows Mixed Reality. However, one of the main limitations of Chimera X to visualize structures from the SARS-CoV-2 is that large molecular structures (more than a few thousand atoms) render too slowly and cause stuttering in the headset [38]. VMD is a classical molecular viewer with a large number of interesting features along with VR capabilities that is compatible with a wide variety of devices including flat screens, 6-degree-of-freedom input gadgets, and haptics accessories [39]. A number of general purpose VR toolkits such as CAVElib, FreeVR, VRPN and VR Juggler are well suited to work with VMD [40], [41], [42], [43]. A freely available software pipeline to transform molecular structures from VMD to 3D objects in the VR environment, has been recently released [44]. More recently, a GPU-accelerated ray tracing engine, so called TachyonL-OptiX, aimed to obtain stereoscopic panoramic and omnidirectional projections compatible with VMD has been developed, allowing the visualization of large molecules with head-mounted displays, as the Oculus Rift or even Google Cardboard compatible [45]. An example can be seen here: https://www.youtube.com/watch?v=QPIyk0V6F7A&list=PL8XC6Vx8S1GBT311VIFLt84iuSRhPuXAB&index=2. Nanosimbox Research, currently inactive, was a commercial platform developed in C# that allows using VR to change molecular aesthetics and snapshot molecular structures [46]. It required a VR-compatible laptop and the HTC Vive virtual reality headset. On the other hand, the open-source BlendMol [47] plugin, written in Python, allows producing high quality images of protein structures from imported PyMol and VMD visualization states using the Blender software (Fig. 1) [48]. Once the structures are imported, third-party plugins can be employed to export the models to VR compatible formats that then can be visualized using, for instance, the BabylonJS engine that uses Javascript libraries to show 3D graphs in a web browser through HTML5. The whole process is not straightforward but allows modifying and decorating the representation of the molecule by using the advanced functions of Blender. RealityConvert [49] can also be used to transform molecular representations to VR and AR compatible format. The process is partially automated throughout a web app but only small molecular representations of less than 200 particles can be processed without additional third-party software: PyMol [15], Blender [48], Open Babel [50] and Molconvert [51], what makes the process non-trivial. AltPDB protein-viewer provides an immersive VR environment to visualize and to analyze complex molecular structures in a collaborative way [27]. This software admits any model from the Protein Data Bank. The source code is available in a public repository (https://github.com/AltPDB) and it can be executed in Windows and iOS through a free AltspaceVR account (https://account.altvr.com), although during the preparation of the manuscript this option was not available. It was developed using Unity3D and it is compatible with HTC Vive, Oculus Rift, and Samsung Odyssey. The highest limitation of this software is that it requires a pre-generated model by using third-party tools. Thus, as in the case of RealityConvert, the process is not trivial. Additionally, this project has not been updated during the last 3 years and the dedicated workspace seems to be inactive (https://account.altvr.com/spaces/altpdb). The Samson Connect module enables VR support in the SAMSON commercial platform for simulation of nanosystems [52], allowing to perform interactive simulations, visualize large scale models containing hundreds of thousands of atoms, design new molecules, build nanotube junctions, etc., all in VR). It is developed in C++ for the Windows Mixed Reality headset. The commercial program Nanome (Nanome Inc.) [53], developed in Unity 2017.4.3f and C#, allows molecular manipulation (e.g., in silico mutagenesis) in addition to molecular visualization. It uses VR to visualize, measure, and analyze drug molecules as 3D structural images. A possible limitation for some applications of this software is that the VR molecular scenes created using the free version are public. The software is compatible with Oculus Rift, HTC VIVE, and Microsoft Mixed Reality. Nanome is currently being used against COVID-19 by enabling scientists to gain insights into the molecular mechanics of the virus. In a recent paper [54], Insilico Medicine (https://insilico.com/) and Nanome applied deep learning methods and VR visualization to design treatments for COVID-19 infection (Fig. 1). They used the protease of COVID-19, required for the replication of the virus, as a target to develop new drugs, and took advantage of the knowledge acquired by artificial intelligence methods to propose potential candidates. Nanome contributes to structure-based design by integrating a 3D virtual reality viewing experience with molecular interaction and manipulation in a collaborative environment, allowing the input from various experts to be incorporated in real-time. The real time collaborative analysis in VR between the Insilico Medicine and the Nanome teams can be seen in at https://bit.ly/ncov-vr. The interaction between the Remdesivir antiviral treatment for COVID-19 [55] and the coronavirus RNA polymerase protein has also been analyzed by using Nanome [56]. Nanome is currently participating in the Covid Moonshot initiative [57] to design inhibitors against the SARS-CoV-2 main protease.

Fig. 1.

Snapshots of VR scenes including protein structures from the influenza virus or SARS-CoV-2, created using BlendMol [47], [58] (downloaded from https://git.durrantlab.pitt.edu/jdurrant/blendmol/-/tree/1_2/examples/web-files/virtual-reality, and carried out in Amaro Lab, UC San Diego) (top) and Nanome [53], [54] (bottom), respectively; Nanome images are obtained from Alex et al., 2020, under the license CC BY-NC-ND 4.0.

4. Web tools VR applications

In parallel to the advances in hardware, algorithms and software for visualization, it is possible to take advantage from internet to create VR scenes with protein and molecular structures, including those involved in SARS-CoV-2. Applications running from a server through a web browser exhibit two enormous advantages: (i) they are compatible with almost any operative system and device without special requirements or complementary software; and (ii) updates are instantaneous accessible to all the users. Using the Web Graphics Library (WebGL) [59], it is now possible to create web-based VR experiences that can be directly accessed with common web browsers. VRmol [60], is a molecular viewer with some analysis tools and a VR visualization mode (see Fig. 2). It is very well documented [61] and it is compatible with HTC, Vive, Oculus Rift, Microsoft Mix Reality and other VR devices that work with WebVR enabled web browsers (Microsoft Edge and Firefox Nightly, for instance). This application can read structures from different databases to facilitate drug design. ProteinVR [62], is a recently released 3D-VR molecular viewer that also works in modern web browsers without additional special requirements, as well as in a variety of electronic devices and VR headsets (see Fig. 2). It takes advantage of BabylonJS, an efficient alternative to Unity3D for this kind of applications (see description above). By using public URLs, collaborative work can also be done with ProteinVR. The code is available free of charge at http://durrantlab.com/protein-vr/, and it can be tested by using the following link: http://durrantlab.com/pvr/. Autodesk Molecular Viewer [63], is also a new web-based molecular viewer with VR capabilities and complementary analysis and edition tools that works in a large variety of devices. It allows visualizing very large molecular systems such as entire viruses. This program can generate input for different VR headsets just by accessing a shared URL. Unfortunately, the authors of this software decided to stop this project in 2018. Another web-based approach for online VR molecular-visualization systems that works also offline is iview [64]. Though a VR functionality is available in the software, we were not able to execute it. Additionally, the server where it is hosted seems to be unstable since we were not able to access during many of our tests. The group of Prof. Ricardo L. Mancera has recently developed a tool for the visualization of molecular dynamics simulations using a stereoscopic 3D Display [65]. Using the Unity game engine, and being developed in C#, this implementation allows a VR visualization of molecular dynamics trajectories carried out with different software (GROMACS, LAMMPS and NAMD) in a HIVE facility. Additionally, it is also possible to run the application on a Windows PC desktop for local demonstration and testing purposes. BioVR is a platform which allows the VR visualization of DNA, RNA and protein structures using Unity3D and C# using the Oculus Rift as headsets [66]. Finally, it is also worth to mention the AR and VR applications developed by EPAM Life Sciences (https://www.epam.com/our-work/videos/vr-ar-molecular-visualization-apps).

Fig. 2.

Snapshots of VR scenes including protein structures created using VRmol [60] (top) and ProteinVR [62] (bottom). The structures correspond to the SARS Spike Glycoprotein - human ACE2 complex (PDB ID: 6CS2) (left) and to the Dimeric DARPin A_angle_R5 complex with EpoR (PDB ID: 6MOJ) (right).

5. Downloadable apps for smartphones

VR applications for smartphones facilitates their use, mainly if they are compatible with cheap headset such as cardboards. On the other hand, this kind of applications are typically very intuitive since they take advantage of the device sensors to move and interact with the VR environment in a natural way. However, we were able to find only two applications able to visualize SARS-CoV-2 proteins. PROteinVR [67] allows the visualization of proteins from the PDB Data Bank directly in VR. It has been developed using the Unity3D engine. With a VR headset (Google Cardboard compatible) and an Android or iPhone cell phone, it is possible dive through the 3D chemical structures of proteins, including those important in COVID-19. Recently, we have developed a new app, Corona VRus Coaster, which allows sliding along the backbone of 22 protein structures from SARS-CoV-2, in its current version, as if they were a roller Coaster, providing unique perspectives of the protein (Fig. 3) [68]. The available structures in the current version are: 7BQY (COVID-19 main protease), 7BV1 (Cryo-EM structure of the apo nsp12-nsp7-nsp8 complex), 7BV2 (nsp12-nsp7-nsp8 complex bound to the template-primer RNA and triphosphate form of Remdesivir), 6WKP (RNA-binding domain of nucleocapsid phosphoprotein from SARS CoV-2), 6WLC (NSP15 Endoribonuclease from SARS CoV-2 in the Complex with Uridine-5′-Monophosphate), 6YB7 (SARS-CoV-2 main protease with unliganded active site), 6YHU (nsp7-nsp8 complex of SARS-CoV-2), 6YI3 (N-terminal RNA-binding domain of the SARS-CoV-2 nucleocapsid phosphoprotein), 6YLA (SARS-CoV-2 receptor binding domain in complex with CR3022 Fab), 6YM0 (SARS-CoV-2 receptor binding domain in complex with CR3022 Fab (crystal form 1), 6YOR (SARS-CoV-2 spike S1 protein in complex with CR3022 Fab), 6M0K (COVID-19 main protease in complex with an inhibitor 11b), 6M71 (SARS-Cov-2 RNA-dependent RNA polymerase in complex with cofactors), 6W9C (papain-like protease of SARS CoV-2), 6W9Q (Peptide-bound SARS-CoV-2 Nsp9 RNA-replicase), 6W37 (SARS-CoV-2 ORF7A encoded accessory protein), 6W41 (SARS-CoV-2 receptor binding domain in complex with human antibody CR3022), 6WCF (ADP ribose phosphatase of NSP3 from SARS-CoV-2 in complex with MES), 6LZE (COVID-19 main protease in complex with an inhibitor 11a), 6WJI (C-terminal Dimerization Domain of Nucleocapsid Phosphoprotein from SARS-CoV-2), 6WIQ (co-factor complex of NSP7 and the C-terminal domain of NSP8 from SARS CoV-2) and 6WEY (SARS-CoV-2 NSP3 Macro X domain). The “PLAY” bottom starts the navigation mode. By clicking on the “PAUSE” bottom the navigation stops in the current residue and the camera may be rotated, allowing a 360° view of the protein from that point. It works for android and iOS devices and it does not require the installation of additional programs or libraries. The application is freely available from http://mduse.com/coronavruscoaster/. Periodic updates will be avialable at the same url address.

Fig. 3.

Snapshots of several SARS-CoV-2 molecular structures visualized with Corona VRus Coaster. This app, developed in our group [68], allows the visualization of several proteins involved in COVID-19, allowing also navigating throughout their backbone as if they were a roller coaster, providing unique perspectives.

6. Summary and outlook

In order to understand and develop effective treatments against SARS-CoV-2, responsible for global pandemic we are currently living, a deep knowledge of the molecular structures involved in the infection process is key. Virtual reality (VR) is a powerful tool for studying biomolecular structures, enabling their visualization in stereoscopic 3D. In this review, we have tried to gather and quickly describe the available VR tools enabling the visualization of the known molecular structures involved in COVID-19. The aim is to improve the understanding of the virus action mechanism as well as to contribute to accelerate the drug discovery process. We have examined the different implementations existent for VR, considering the complexity of the required infrastructure: Cave virtual environments, flat screens combined with specific headsets (Oculus Rift or HTC Vive), and extremely simple solutions such as Google Cardboard. Whereas the desktop applications allow the user to have a much more complete structural analysis, (i. e. Molecular Rift, UnityMol, Nanome, etc) such tools usually require the use of specific and not so affordable hardware. In contrast, web-based implementations are immediately accessible on a broad range of desktops, laptops, and mobile devices, without requiring the installation of any third-party programs or plugins. In turn, these applications typically loose functionality in structural analysis of the biomolecular structures (i. e. ProteinVR, VRmol, etc). Smartphone apps are still scarce (PROTeinVR and Corona VRus Coaster) but its development is expected to advance in giant steps at the short term. VR has been touted as a drug development tool for two decades but dismissed by some scientists as little more than a toy for looking at molecules and proteins, not a genuine research or education tool. However, the maturation of VR together with the increasingly easiness, quality and economic access of this technology is transforming the scope this technology can have in drug design and understanding of diseases. Despite the fact that further development is encouraged both in the number and in the usability of new VR applications that allow the visualization and manipulation of large biomolecules, the existing tools already urge their introduction into the conventional drug development processes addressed to COVID-19.

CRediT authorship contribution statement

Martín Calvelo: Investigation, Writing - review & editing. Ángel Piñeiro: Investigation, Writing - review & editing. Rebeca Garcia-Fandino: Conceptualization, Investigation, Writing - review & editing, Supervision.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.