Viv: multiscale visualization of high-resolution multiplexed bioimaging data on the web

To the Editor

Advances in highly multiplexed imaging have enabled the comprehensive profiling of complex tissues in healthy and diseased states, facilitating the study of fundamental biology and human disease at spatially resolved, subcellular resolution^1,2. Although the rapid innovation of biological imaging brings significant scientific value, the proliferation of technologies without unification of interoperable standards has created challenges that limit the analysis and sharing of results. The adoption of community-designed next-generation file formats (NGFF) is a proposed solution to promote bioimaging interoperability at scale³. Here we introduce Viv (https://github.com/hms-dbmi/viv), an open-source bioimaging visualization library that supports OME-TIFF⁴ and OME-NGFF³ directly on the web. Viv addresses a critical limitation of most web-based bioimaging viewers by removing a dependency on server-side rendering, offering a flexible toolkit for browsing multi-terabyte datasets on both mobile and desktop devices—without software installation.

Viv functions more similarly to popular desktop bioimaging applications than to corresponding web alternatives (Supplementary Note 1). Most web viewers require the pre-translation of large binary data files into rendered images (PNG or JPEG) for display in a browser client. Two existing approaches perform this step (server-side rendering) but differ with regard to when rendering occurs and how much of the binary data is transformed at one time (Fig. 1). The offline option performs all rendering before an application is deployed to users, meaning that all channel groupings and data transformations are fixed and cannot be adjusted via the user interface. The online option supports on-demand, user-defined rendering but introduces latency when exploring data transformations and requires active maintenance of complex server infrastructure. Neither approach provides a flexible solution to directly view datasets saved in open formats from large-scale public data repositories, and the transient data representation introduced by server-side rendering inhibits interoperability with other visualization and analysis software.

Fig. 1 ∣. Overview of data flow and rendering approaches for web-based bioimage data visualization and Viv features.

DT indicates the location where data is transformed into an image. The right column displays a subset of Viv’s flexible client-side rendering, including multiscale 2D pyramids, magic lenses and 3D volumes via raycasting.

Viv implements purely client-side rendering to decouple the browser from the server and still offer the flexibility of on-demand multichannel rendering. Existing web viewers also leverage graphical processing unit (GPU)-accelerated rendering but are generally tailored toward single-channel volumetric datasets and, crucially, lack the ability to reuse and compose features for existing or novel applications^5,6. In contrast, Viv’s modularity allows core functionality to be repurposed and extended. The library consists of two primary components: (i) data-loading modules for OME-TIFF and OME-NGFF and (ii) configurable GPU programs that render full-bit-depth primary data on user devices.

The data-loading component of Viv is responsible for fetching and decoding atomic ‘chunks’ of compressed channel data from binary files via HTTP. We targeted both OME-TIFF and OME-NGFF to suit the diverse needs of the bioimaging community and promote the use of interoperable open standards. Although OME-TIFF is more ubiquitous, its binary layout and metadata model are too restrictive to efficiently represent large volumes and datasets with high dimensionality. OME-NGFF is designed to address these limitations and is natively accessible from the cloud due to the underlying multidimensional Zarr format (https://doi.org/10.5281/zenodo.3773450). Zarr provides direct access to individual chunks, whereas TIFF requires seeking to access chunks from different planes, introducing long latencies when reading datasets with large c, t or z dimensions³. Similar to approaches popularized in genomics⁷, we propose indexing OME-TIFF to improve random chunk retrieval times. Our method, Indexed OME-TIFF, improves read efficiency and is implemented as an optional extension to OME-TIFF (https://doi.org/10.6084/m9.figshare.19416344).

Viv’s image layers coordinate fetching data chunks and GPU-accelerated rendering. All rendering occurs on the client GPU, allowing continuous and immediate updates of properties such as color mapping, opacity, channel visibility and affine coordinate transformation without additional data transfer. Library users can modify rendering with custom WebGL shaders, a feature we used to implement ‘magic lenses’⁸ that apply local data transformations: for example, rescaling brightness or filtering specific channels.

Because all rendering is moved to the browser, Viv does not rely on a server like previous web viewers for highly multiplex and multiscale images, making it flexible and capable of being embedded within a variety of applications (https://doi.org/10.6084/m9.figshare.19416401). We integrated Viv into Jupyter Notebooks (https://github.com/hms-dbmi/vizarr) via Imjoy⁹ to enable a remote human-in-the-loop multimodal image registration workflow and direct browsing of OME-NGFF images. Viv also underlies the imaging component of the Vitessce single-cell visualization framework¹⁰, demonstrating its ability to integrate with and power other modular tools that include imaging modalities. Finally, we developed Avivator (http://avivator.gehlenborglab.org), a stand-alone image viewer that showcases Viv’s rich feature set.

With its extensibility and minimal deployment requirements, Viv presents a novel toolkit with which to build a wide range of reusable bioimaging applications. It is best suited for displaying 2D visualizations of highly multiplex and multiscale datasets, although 3D visualizations via raycasting are also supported. We do not expect Viv to replace existing web-based viewers but instead to shepherd a new generation of web-based visualization and analysis tools to complement desktop software also built around interoperable standards.

Code availability

Viv is open source and available under an MIT license at https://github.com/hms-dbmi/viv and at https://www.npmjs.com/package/@hms-dbmi/viv.