\section*{Introduction}
\label{Sec:Intro}
Imaging over large areas of tissue is often necessary in clinical and research settings. However, high-resolution microscopy is limited to a small field of view in a single image acquisition. To extend the field of view, mosaicking techniques – acquiring a sequence of adjacent images and stitching or blending them together at the boundaries – have been developed \cite{Tomasz,Loewke,Vercauteren,Piccinini,Sun,Hsu,Schroeder,Legesse}.  Approaches for mosaicking of a two-dimensional matrix of individual still images have been developed for bench-top work on tissue specimens in vitro, and ex vivo \cite{Piccinini,Chow,Saldua,Yang,Gao,Tao,Giacomelli,Gareau,SanjeeJBO2011}. In bench-top settings, still images are typically acquired in a highly controlled manner along a specific predefined raster path. However, mosaicking for the newer, increasingly powerful and widespread implementation of video-imaging on living tissue in vivo is significantly more challenging. In this situation, video-images are acquired in a relatively uncontrolled manner and along an ad-hoc arbitrarily free-form, non-rastered path on living humans and animals. Approaches for mosaicking of a sequence of images from a video are called ``video-mosaicking" \cite{Szeliski1996}.

Three problems commonly occur when video-mosaicking in vivo are (i) deformations - i.e. warping or distortion in images - due to physical changes (angle, pressure) at the microscope lens-to-tissue contact, (ii) abrupt motion artifacts - i.e. “jumps” or discontinuities between consecutive frames that are caused by sudden changes in depth or topography of tissue motion and/or manual and variable operator control of the microscope during imaging, and iii) optical sectioning and resolution may vary between consecutive images due to scattering and aberrations induced by changes in depth and morphology. In the presence of these difficulties, spatially overlapping images need to be “combined” to produce a video-mosaic.

Although these challenges, and our solutions presented in this paper are generic for any setting in which large field-of-view video-mosaics are needed from deep, high-resolution microscopic imaging in living tissue, we are, in this paper, specifically focused on the use of video-mosaicking for reflectance confocal microscopy (RCM) imaging in human skin in-vivo. RCM imaging non-invasively acquires nuclear- and cellular-level detail with thin optical sectioning ($1 - 3 \mu m$) and high resolution ($0.5 - 1 \mu m$) \cite{RCM_Atlas,LSM:LSM22600}

Several clinical studies have consistently reported RCM imaging to be capable of diagnosing melanocytic and non-melanocytic cancers with high sensitivity and specificity \cite{LSM:LSM22600,Stevenson,Xiong,Kadouch}.  Recently, the Centers for Medicare and Medicaid Service in the USA granted current procedural terminology (CPT) codes, which means that RCM imaging is now a billable and reimbursable procedure\cite{CPT16}. The imaging is advancing toward routine implementation for guiding diagnosis as it has shown to increase diagnostic accuracy while maintaining a good specificity (and reducing the rate of biopsies of benign lesions per detected malignancy) \cite{NumBiop-Pellacani,NumBiop-Alarcon,Ferrari2014,Stanganelli2015,Lovatto2015}.  Meanwhile, with the advent of a new generation of handheld confocal microscopes, acquisition of videos instead of only still images is becoming increasingly useful \cite{kkoseBJD2014,EileenJBO, HandheldClinicRabinovitz14} to consistently and rapidly display tumor morphology and surrounding normal tissue over much larger areas of skin (from $2mm$ \;X\; $2mm$ up to $10mm$ \;X\; $10mm$).

In such settings, the special challenges are that the microscope is manually controlled and focused at varying depths ($50~to~200 \mu m$) in living tissue, resulting in varying resolution, exaggerated motion and distortion between images in any video. Moreover, the regions to be imaged differ greatly from subject to subject, skin site to skin site, and lesion to lesion, so operator flexibility in acquisition is necessary. Although ``playback” of videos can be useful, the ability to spatially ``map” the imaged region and display the entire field of view at once, by constructing a mosaic, allows much more ``natural” analysis by trained clinicians or (semi-)automated algorithms \cite{kkoseBJD2014}. Reading and interpreting a mosaic more closely mimics the standard procedure for examining pathology slides, which usually starts by looking at 10 mm x 10 mm areas of a tissue section with 2X magnification. Thus, there is a need in the clinic for video-mosaicking of RCM images, toward wider acceptance and adoption for routine use.

In this paper, we present a new algorithm for video-mosaicking of subsurface confocal microscopic images, which addresses all of the expected challenges. Earlier video-mosaicking techniques, for both confocal and non-confocal endoscopy, either address only a subset of these problems, or attempt to avoid some of them by limiting the freedom of motion during acquisition \cite{Tomasz, Loewke,Vercauteren}. Video-mosaicking was demonstrated for imaging of oral and cervical tissue \cite{Tomasz}. However, the imaging depth was close to the surface of tissue, so tilt and loss of resolution were not significant. Similarly, another algorithm was reported on human skin and mouse brain tissue \cite{Loewke}, but, again, no attempt was made to handle the warp or deformation that can happen when imaging at greater depth \cite{Loewke}. In another study on human and mouse colon tissue \cite{Vercauteren}, the algorithm models both rigid and non-rigid deformations in a linear fashion. However, the algorithm does not consider abrupt motion artifacts between frames and, furthermore, can enhance blurring at image boundaries through the use of blending to produce the final mosaic. 

We address all these challenges by (i) directly modeling deformations, (ii) automatically detecting abrupt motion artifacts (discontinuities) in the image sequence, and (iii) robustly stitching the registered frames with an approach that combines overlapped images flexibly, in a data-driven fashion, without blurring or loss of resolution. Our algorithm is based in part on adaptation of methods from computational photography \cite{Hartley2004}.  In particular, we describe the use of keypoint matching to design affine transformations to allow warping among consecutive video frames (images) during registration. The properties of the resulting affine transformations are also used to detect abrupt motion artifacts between frames. Finally, we developed an approach, based on a graph-cuts based model \cite{Boykov2011}, to stitch overlapping frames together along arbitrary (non-rectilinear) boundaries, driven by the image data itself, without the loss of resolution that, otherwise, would occur from blurring or other blending techniques, and without imposing arbitrary pre-defined stitching boundaries.  Herein, we describe our method in detail, and then present illustrative video-mosaicking results on human skin lesions in-vivo. The advantages and limitations of our algorithm are discussed, as well as the possible future directions for both research and clinical applications.



