Document Actions

Introduction

Virtual microscopy is a recent technological innovation involving the merging of digital technologies with conventional microscopy that enables online viewing and interactive navigation through submicron-resolution digital images of glass microscope slides, dubbed "virtual" slides, exactly as if viewing the sections through a microscope ( Ferreira, Moon, Humphries, Sussman, Saltz, Miller & Demarzo 1997 ; Afework 1998 ; Felten, Strauss, Okada & Marchevsky 1999 ; Romer & Suster 2003 ).

The advantages of virtual slides over conventional glass microscope slides are many, and include facilitation of data sharing over the internet, ease and speed of slide navigation, and tight integration into database management systems. For purposes of brain mapping, current virtual microscopy scanners achieve resolutions of digitized slides of more than 100,000 dpi, enabling the generation of brain atlases at microscopic resolutions that exceed, by several orders of magnitude, the resolutions obtained through other digital scanning technologies, MRI, or in conventional print media.

At BrainMaps.org, we have implemented an interactive internet-accessible submicron-resolution digital brain atlas that utilizes over 20 million megapixels of virtual microscopic slides corresponding to serial sections of primate and non-primate brains. In this report, we describe how the atlas has been created, review miscellaneous software tools for accessing the atlas for purposes of visualization and analysis, and finally, discuss the utility of the brain atlas for teaching neuroanatomy.

Tool Description

BrainMaps.org consists largely of annotated, internet-accessible virtual slides and methods for navigating them and extracting data from them through a distributed server-client architecture. The infrastructure, consisting of web servers, relational databases, RAID arrays, and server-side scripts, has been developed ( Mikula, Trotts, Stone, & Jones, 2007 ) and tested over the past two years utilizing over 50 terabytes of virtual slides of largely histochemically-stained whole-brain sub-micron mouse and non-human primate brain material, but also including miscellaneous immunocytochemical, blockface, and in situ hybridization processed brain section series from myriad mammalian and avian species.

Multiresolution Images and Virtual Slides

The virtual slides at BrainMaps.org range from 100 megapixels, for rodent brain slices, to more than 10 gigapixels, for macaque brain slices. When working with massive images such as these, it is generally preferable and often necessary to break the image down into smaller pieces and to organize it as a multiresolution image. By 'multiresolution image', we mean that each image consists of a hierarchical image pyramid composed of small image tiles (Figure 1). The multiresolution images we employ are divided into JPG image tiles, of maximum size equal to 256 x 256 pixels, organized as quad-trees (Figure 1). A quad-tree models a two-dimensional region by recursively dividing it into quadrants. By doing this, it is possible to develop software for rapidly accessing just the portions of the multiresolution image that we are interested in, over the internet for example, and thereby bypassing the need to load the entire image into computer memory (RAM), which is oftentimes not technically possible.

Figure 1:Individual virtual slides are represented as multiresolution image pyramids composed of small image tiles, with a maximum size of 256 x 256 pixels. This allows for rapid online navigation because only the image tiles that are currently being viewed need to be downloaded from the Brainmaps server.

To date, more than 5000 virtual slides of primate and nonprimate brains, totaling over 16,033 gigapixels (or 48.1 terabytes, uncompressed) (Figure 2), and scanned in at 0.46 microns per pixel, have been uploaded and are directly accessible from BrainMaps.org.

Figure 2: Total quantity of data scanned in, shown here as the distribution of image sizes at BrainMaps.org as of 06-18-2007 (a). The total size of the brain images is 16,435,723 MegaPixels, or 49.31 TeraBytes. The total number of images is 5264, with an average size of 3122.29 MegaPixels/image, or 9.37 GigaBytes/image. Distribution of datasets shown as a function of species (b). The distribution of image pixels shown as a function of species (c) shows that most of the image data online corresponds to the non-human primates, Chlorocebus aethiops, Macaca mulatta, and Macaca fascicularis.

Online Navigation and Graphical User Interfaces

Users access virtual slides from the BrainMaps server either through a web browser, usually through an AJAX (i.e., javascript and dHTML) graphical user interface (GUI) utilizing a MySQL backend, or a custom-coded desktop application, for example, using C and OpenGL (see StackVis below). We have also coded browser-based GUI's in Flash and Java, in addition to AJAX.

Figure 3 demonstrates navigation, using actual screenshots from a web browser using the AJAX GUI, through a Macaca mulatta dataset of Nissl-stained virtual slides at BrainMaps.org. The capability to interactively zoom in/out of and pan within individual virtual slides, and to rapidly jump to adjacent virtual slides, enables rapid navigation of virtual slides within a given dataset. Database integration permits both querying across datasets for various subsets that satisfy search query constraints and also for simple browsing of all available datasets, organized according to species or other criteria.

Figure 3: An example of navigation through virtual slides at BrainMaps.org using a Macaca mulatta Nissl dataset. All images are actual screenshots from a web browser and are what a visitor to BrainMaps.org would see. (a) An array of virtual slides for the dataset, shown as clickable thumbnails that, when clicked on, launch a new browser window allowing navigation through the high-resolution image (b). The image in (b) is 104,640 x 84,144 pixels and 24 gigabytes in size. The thumbnail in the lower right is for navigation purposes. Shown also are overlying labels of brain areas that may be toggled on and off. (c) Zooming in on the slide in (b). The red box in (b) corresponds to (c). (d) Zooming in to full resolution in (c), showing details of individual neurons in the cingulate cortex. The red box in (c) corresponds to (d).

Advanced Techniques

Additional progress at BrainMaps.org has been made on three fronts: 1) interactive visualization, 2) data-mining, and 3) informatics (Figure 4).

Figure 4: Three avenues for extending BrainMaps.org functionality. Once virtual slides become internet-accessible, additional development in informatics, interactive visualization and data-mining is possible.

Interactive 3D Visualization

How we interact with data is very important. Towards this end, we have developed a desktop application that enables complete interactive visualization of virtual microscopy images and image stacks from BrainMaps.org in both 2D and 3D ( Trotts, Mikula, & Jones, 2007 ). An example is shown in Figure 5.

Figure 5: Example of interactive 3D visualization of multiresolution image stacks using the desktop application, StackVis. The dataset corresponds to a coronally-sectioned, Nissl-stained Chlorocebus aethiops brain.

Data-Mining

Data-mining includes image analysis of virtual slides and finding patterns in image data. Some examples include cell counts, stereological analysis, cell size distributions, granulometry, neuron morphology analysis, and quantitative architectonics. We have made progress in writing programs and scripts for performing different types of image analyses, including image variance and granulometry (for determining cell size distributions) on virtual slide data obtained remotely from BrainMaps.org.

Informatics

Informatics has three components: 1) Organization of raw image data (virtual slides) into a logical system that facilitates data navigation, retrieval, and querying, 2) integration of diverse types of data into the image data (i.e., EM, MRI, in situ), and 3) organization of diverse types of data that are integrated into the image data. We have developed a relational database for informatics that enables organization of virtual slides into datasets and classification by species, staining method, and other criteria, and that also allows for associating and integrating additional types of data into the virtual slide in a manner similar to GIS. Additional progress in neuroinformatics has been made with the Java annotator (Figure 6), which enables users to create, view, and query annotations of the images on the BrainMaps.org web site.

Figure 6: An Annotation links a geometric ROI (shown here as a closed Bezier curve delimiting the boundaries of LGN layer 6) on an Image to one of a set of Concepts. The annotation is shown with three properties associated with it. These properties are all of type image (because they are the result of scanning documents) but the original data could have been stored as time series data, enabling data sharing among researchers.

Educational Use

BrainMaps.org, functioning as a virtual microscopy web service, is ideally-suited for teaching neuroanatomy in a classroom setting that has computers with internet connectivity. For example, medical students at SUNY, as part of their neuroscience curriculum, use BrainMaps in conjunction with a worksheet tutorial ( http://ect.downstate.edu/cortex_exercise/ ) to learn about neuroanatomy.

Myriad educational uses can be devised along similar lines, including purely exploratory learning. For example, clicking on a tree symbol in the upper right of the virtual slide viewer will open a labeling hierarchy, shown in Figure 7, which enables navigation through the dataset using the labeling hierarchy.

Figure 7: Hierarchical navigation through virtual slide datasets. Clicking on a tree symbol in the upper right of the virtual slide viewer will open a labeling hierarchy, shown above, which enables navigation through the dataset using the labeling hierarchy.

In addition, BrainMaps.org virtual slide annotations and labels feature complete integration with the research literature through its "parse pubmed" option, shown in Figure 8. The "parse pubmed" option appears when a label of a brain area or brain object is clicked on, and if the "parse pubmed" option is selected, a window will appear with parsed Pubmed results for the given brain area or brain object.

Figure 8: The virtual slide viewer features complete integration with the research literature through its "parse pubmed" option, shown here. The "parse pubmed" option appears when a label of a brain area or brain object is clicked on, and if the "parse pubmed" option is selected, a window will appear with parsed Pubmed results for the given brain area or brain object. Note that this window may be closed using the "x" in the upper right.

Summary

We have described and implemented a method for the generation of virtual slides and for their incorporation into an online, interactive, multiresolution brain atlas at BrainMaps.org. We have demonstrated that internet-accessible virtual microscopes and interactive multiresolution histochemical and immunocytochemical brain atlases can be developed using existing computer technologies that facilitate data sharing, rapid navigation, data visualization, and analysis. By allowing for interactive visualization of completely digitized brains at submicron resolution, our online tools will be useful for research, including large-scale histochemical, gene expression, and eventually stereological analyses, and also for didactic purposes, such as the teaching of neuroanatomy.

Literature references

Afework A (1998). Digital Dynamic Telepathology: The Virtual Microscope. University of Maryland.

Felten C, Strauss J, Okada D & Marchevsky A (1999). Virtual microscopy: high resolution digital photomicrography as a tool for light microscopy simulation. Hum Pathol, 30 (4), 477-83.

Ferreira R, Moon B, Humphries J, Sussman A, Saltz J, Miller R & Demarzo A (1997). The Virtual Microscope. Proceedings of the 1997 AMIA Annual Fall Symposium, 449-453.

Mikula S, Trotts I, Stone JM & Jones EG (2007). Internet-Enabled High-Resolution Brain Mapping and Virtual Microscopy. NeuroImage, 35 (1), 9-15.

Romer D & Suster S (2003). Use of virtual microscopy for didactic live-audience presentation in anatomic pathology. Ann Diagn Pathol, 7 (1), 67-72.

Trotts I, Mikula S & Jones EG (2007). Interactive Visualization of Multiresolution Image Stacks in 3D. NeuroImage, 35 (3), 1038-43.

License

Any party may pass on this Work by electronic means and make it available for download under the terms and conditions of the Digital Peer Publishing License. The text of the license may be accessed and retrieved via Internet at http://www.dipp.nrw.de/lizenzen/dppl/dppl/DPPL_v2_en_06-2004.html