The volume rendered brain atlas (VRBA) is a standalone application designed to facilitate learning and understanding of the three-dimensional structure and function of human brain anatomy. The application consists of segmented anatomical data and co-registered T1 magnetic resonance (MR) data that is freely available online ( IBASPM , Alemán-Gómez et al. 2006 , Figure 1). A T1-weighted MRI measures the longitudinal relaxation time of protons in the applied magnetic field after a radiofrequency pulse. Typically, T1 images display white matter as white, gray matter as gray and cerebrospinal fluid as black. The original segmented data consists of an 8 bit raw volume data with 116 segmented regions of the brain (the complete list is available online ). Each anatomical structure is coded by a particular bit value in the data. In this scenario, each region is effectively a different grayscale value in the data set (see figure 1a). Color and opacity transfer functions are used by the VRBA to recolor and add transparency to the grayscale pixels allowing the user to see specific structures as different colors, and hide others. The interface is designed to give the user control over the opacity of each of the different pixel values, or anatomical structures. The color map applied to the data effectively replaces the ramped grayscale values with a color spectrum, increasing the contrast between adjacent structures. A number of sliders represent the opacity of the different pixel values, labeled as anatomical structures. Co-registered T1 magnetic resonance data is also overlayed onto the anatomical volume to correlate the segmented structures with a more familiar view of the brain. T1 display parameters are also tunable with four different sliders that modify the opacity of 4 predefined transfer functions. The predefined transfer functions loosely display the outer surface of the head, white matter, gray matter, and the last displays all non-zero pixels in the T1. Sliders are used to modify the opacity of each T1 representation and a clipping function is available to remove portions of the T1 and visualize internal structures. Basic descriptions of many of the anatomical regions are incorporated into the software, and appear in a text region when the slider for a particular region is moved to the fully “on” position. The user has the option to hear a synthesized text-to-speech (TTS) dictation of a description of the anatomical region that uses the TTS engine built into the Windows operating system. Stereoscopic rendering capabilities are also incorporated into the software including interlaced, anaglyph and active stereo modes. This is particularly useful when analyzing 3 dimensional relationships between the different nuclei, circuits, and their position within the brain. The VRBA was developed at the Department of Biological Sciences at Kent State University for use with the stereoscopic 3d resources on campus, and is available for free download through the university website.

1a 1b

Figure 1a-b: These images display a single plane from both the segmented (a) and T1 (b) data. In the segmented image, a number of different grayscale values are visible that represent specific anatomical regions. The corresponding T1 image gives a frame of reference from a more familiar view of the brain. Each image is from the same plane of the brain.

The VRBA was coded using Tcl/Tk for the user interface and uses the Visualization Toolkit (VTK) for 3d rendering. VTK was selected for 3 reasons: 1) VTK provides a high level programming interface to OpenGL for rapid development; 2) The toolkit natively supports stereoscopic rendering, which was a prerequisite for the VRBA, 3) Data input routines are available for raw data sets. A second prerequisite for this brain atlas was the use of volume rendering to display the data. Other brain atlases, i.e. the Secret Life of the Brain , have been created that utilize isosurfaces to demarcate anatomical regions. There are a number of benefits to the use of isosurfaces for data display, however they fail to adequately portray this particular data for a number of reasons. Isosurfaces consist of a surface with a hollow core and do not render inner portions of the data within the surface. Brain regions are relatively well defined, but many have fuzzy borders and are ill represented by precise lines. Anatomical regions also occupy volumes, rather than surfaces and as such are more realistically defined by volume rendering methods. Isosurfaces also have to be computed at runtime, or predefined in files/processes and loaded prior to the visualization. This is undesirable for this dataset since the resolution of the data is such that significant artifacts would be created in the surface construction routine (i.e. contoured lines from individual scan slices) and need to be smoothed and processed for acceptable rendering, increasing runtime computation. Also, the atlas consists of 68 different structures, which is a large number of isosurfaces to both compute and display. For these reasons volume rendering was the chosen method to display the segmented and MRI data.

VTK offers a variety of volume rendering algorithms including raycasting, 3d texture mapping, 2d texture mapping, and fixed point volume rendering. The first choice was to use 3d texture mapping due to its rendering speed and quality. However, a few drawbacks of the toolkit were encountered during the coding process: most notably with the use of 3d texture mapping. When using 3d texture mapping, VTK resamples the data and consequently performs the rendering algorithm. For the T1 dataset, resampling has minimal adverse effects and gives acceptable results. However, for the segmented data, the resampling process generates numerous artifacts, causing the software to inaccurately display the anatomical regions (halos were created around the entire periphery of the brain, as well as other regions). The nature of the data is extremely specific, and relies on very minor differences between adjacent pixels. Consequently, any visualization artifacts are both undesirable and intolerable. After some testing, it was apparent that the 2d texture mapping algorithm did not resample the data and yielded the most precise rendering results. The caveat of using the more precise technique was reduced application speed. To counteract this effect the segmentation data was downsized in both the x and y dimensions by a factor of 2, which still preserved the clarity of the anatomical regions while increasing performance to an interactive level. A second problem was encountered when trying to overlay the MRI and segmented data.

Initially, the addition of both volumes to the visualization pipeline caused the T1 data to completely occlude the segmentation data, even if the T1 had a lesser opacity factor (or vice versa if the segmented volume was added first to the pipeline). This was rectified by the generation of two rendering layers, adding both volumes to both layers and allowing VTK to composite the layers into a single viewport. This technique seems to work well with multi-channel visualizations, and allows any given pixel to be dependant on the summation of all underlying data from both rendering layers. Effectively, this means that if an anatomical region has a greater opacity value than the overlying MRI, it is visible inside the MRI (Figure 2a), otherwise it is occluded by the T1 (Figure 2b).

2a 2b

Figures 2a-b: These figures represent the use of opacity and its effect on the visualization pipeline. In (a) the MRI has an opacity factor less than that of the anatomical regions (hippocampus, vermis and basal ganglia), consequently, the structures are visible. In (b) the T1 scalar opacity value is greater than the anatomical volume, and occludes the internal structures.

To have precise control over the volume rendering of the segmented data set, it was necessary to generate explicit opacity and color transfer functions and apply it to the volume. The data consists of 116 levels of grey that represent different anatomical regions in the brain. An opacity table was generated with 62 variables representing the opacity values of the different anatomical structures (bilateral structures were paired to reduce the number of sliders in the interface). A color ramp was also created including all colors of the visible spectrum changing from values 1 to 116. This created a situation where the 116 grey levels were now coded to different colors in the rainbow, allowing greater contrast between adjacent structures. At initial runtime, all opacity values are 0 except values at position 37 and 38 in the opacity function. This means that only pixels having a value of 37 and 38 are visible, and all other pixels are completely transparent. In this atlas, these pixel values define the left and right hippocampus (figure 3). The interface is setup as a number of sliders that represent the different opacity values 1-116, allowing the user to move the sliders and control the opacity of individual anatomical structures. It was decided to allow a maximum opacity value of .3 for all segmented regions because higher values cause individual scanlines to become visible in the data, while the transparency still allows the viewer to see internal regions.

Figure 3: When the atlas is started only segmented atlas pixel values of 37-38, this encodes for the left and right hippocampus in the original data. The T1 MRI’s transfer functions (opacity and color) are set to display values corresponding to parts of the skull and the grey matter.

The MRI data is displayed as a grey-scale volume (pixel colors ramp from black to white) so as not to interfere with the color rendering of the atlas data. Four opacity functions were created that loosely represent: 1) grey matter 2) All non-black pixels in the data 3) skull and surface of the head, and 4) white matter (Figure 4). The scalar opacity of each T1 representation can also be manipulated to help the user fine-tune the visualization parameters.

Figure 4: These images display the four opacity functions designed to highlight different regions of the MRI data. From left to right the functions are loosely designed to highlight 1) Grey matter 2) All non black pixels 3) Skull and surface of the head 4) White matter. The different display parameters allow the user to relate the segmented atlas data to anatomical landmarks and features on a T1 MRI.

In the context of the VRBA, the usefulness of T1 opacity values that fully occlude the underlying segmentation data is negligible. For this reason, a clipping function is included in the atlas to allow the user to remove portions of the T1 and view the underlying segmented anatomy, while regions of the T1 remain to orient the viewer. VTK supports a clipping “box widget” that gives the user simple access to this capability, and interactively remove portions of the data (Figure 5).

5a 5b

Figure 5: a. In this figure a portion of the T1 data has been removed using the clipping widget. The VRBA is displaying the frontal cortex with the cerebellum; the MRI data is mostly opaque. This demonstrates the usefulness of removing T1 data to unmask regions of the atlas but still giving the user hints about interbrain location. b. VTK includes a “box widget” that gives the user an easy interface to remove portions of the T1.

The VRBA consists of an array of sliders and buttons on the left, a rendering panel that displays the visualization on the right, a text box displaying a brief description of the anatomical region and a number of function buttons on the lower right (Figure 6).

Figure 6: This image displays the general layout of the VRBA, consisting of an array of sliders on the left that control the opacity of each anatomical region, a rendering panel displaying the current view, and a series of buttons that activate particular functions of the atlas.

The sliders are labeled with the individual structures that they manipulate. Each slider represents the opacity of the structure defined by the label, with the right being highest opacity, the left fully transparent. When the sliders are moved all the way to the right (maximum opacity), the text box displays a description of the anatomical region, and appears on the visualization on the right. The description is read aloud by the TTS engine, which is triggered only when the opacity is at a maximum. The voice can be paused, stopped and volume changed with the slider and buttons below the “VOICE” label. Stereoscopic rendering modes can be activated with the buttons under the STEREO OPTIONS tag, which include interlaced, active, anaglyph or off. The active mode is for hardware quad-buffered stereo ready systems only, while interlaced and anaglyph methods should be available for all GPUs. The T1 and anatomical data can be manipulated on-the-fly with the mouse: left button rotates, right button scales, and the middle button moves the object. If a middle scroll button is available it also scales the object. The clipping button activates the clipping widget, which consists of a wireframe box with balls on each side. The user can grab the balls on the side of the box (by left-clicking on the ball) and slide the plane towards the volume to remove T1 data perpendicularly. In this scenario the clipping is restricted to a single plane (i.e, axial, coronal or transverse). If the box is clicked anywhere else with one of the three mouse buttons, it responds to movements in the same way that the volume does. The only way to rotate or translate the volume when clipping is enabled is to grab outside of the wireframe box area. The RESET button resets all clipping planes, and places the object at the origin in its initial state. The snapshot button prompts the user for a filename and creates a portable network graphics file (.png) stereoscopic snapshot of the current view in the visualization panel. The snapshot function generates 2 images representative of the left and right stereoscopic views (figure 7).

Figure 7: This is an example of the snapshot function in the software. The application generates 2 images that represent the right and left stereoscopic views of the visualization. In this particular image, the 2 snapshots have been placed side-by-side to allow the user to use the cross-eyed viewing method to passively view the image in 3d. Stare at the point between the two images and slowly cross your eyes; the image in the center will be in stereoscopic 3d. In this image the enitre brain atlas is visible, and a portion of the T1 clipped.

A number of useful interactive brain atlases have been created, and are freely available online, for download or use ( The Whole Brain Atlas , The Allen Brain Atlas , The Human Brain Atlas , The Secret Life of the Brain ). The VRBA contains a combination of features that makes this atlas different; 1) The data is volume rendered in real time 2) Stereoscopic projection is directly supported in the atlas, 3) A co-registered MRI is simultaneously rendered, and interactively tunable 4) Text-to-speech translation is incorporated into the software giving verbal auditory descriptions of the anatomical regions. Volume rendering provides a way to visualize data voxels as volumetric structures rather than hollow surfaces. This visualization method is well suited for displaying the current segmentation data that would not be well represented by a single hollow surface. A surface rendered model that without excessive smoothing would be artifact ridden due to the low resolution segmentation data. The usefullness of the stereoscopic imaging mode is immediately apparent as the brain volumes appear to float in front of the screen allowing 3 dimensional spatial relationships to be clearly evident. Utilizing both the T1 MRI and anatomy data users can correlate the segmented structures with more familiar views of the brain, and easily spatially localize them within the skull. The text to speech feature allows the user to concentrate on manipulating and viewing the atlas rather than reading the description, while still absorbing the information. Nevertheless, this feature can be disabled by simply muting the volume. Descriptive explanations of the different anatomical regions were acquired from wikipedia and edited to suit the atlas. The reason for using this source for the verbage was an attempt to make the information accessible to those without any prior knowledge of brain anatomy/function while still providing some detailed insights. Consequently, the atlas should be of use to a wide variety of users interested in the anatomy and function of the nervous system. However, should a user want to modify/add on to the definitions, they are contained in the sound directory as a number of text files. These can be editted at will and should not affect the functionality of the brain atlas.

Drawbacks of the VRBA include the relatively high system requirements, limited direct operating system support and the relatively low resolution segmentation data. More specifically, the graphics and main processing unit on the available PC dictate the ability to render, and the visualization speed. However, most PC’s less than four years old should run the program adequately fast and bug-free and future hardware should also have no problem rendering the application. The software was created on a windows XP machine and reportedly runs on Vista as well as linux under the WINE emulation layer. As of the creation of this document Mac OS x has not been tested, but WINE is available for Mac so the software is potentially compatibile. The generated image quality and accuracy in the VRBA is dependant upon the orginal data. As such, the relatively low resolution of the T1 and especially the downsampled anatomical data affect the visualization output. The segmentation data is most limiting, and to more accurately display T1 anatomy it would be optimal to bypass any downsampling and render the segmentation data directly. Downsampling can cause small structures at or close to the resolution limit of the original data (ie. Structures that contain single pixel wide regions) to disappear. Even without downsampling, the data is low resolution and pixels are still clearly visible in figure 1a. The optimal solution would be to generate segmentation data from a higher resolution MRI, this would reduce large pixel artifacts in the data. By reducing the size of pixels in the data, finer resolution, and more accurate anatomical representations could be achieved. For compatability and rendering speed reasons this was impractical in the current version of the VRBA, however, as processor and GPU speeds increase further versions of the VRBA will be released utilizing non down-sampled higher resolution datasets.

In summary, the volume rendered brain atlas incorporates realtime rendering with an intuitive interface and almost no learning curve. Both structure and function are incorporated into the software and provide a multimedia educational experience that could be of use to a wide audience with varying degrees of prior knowledge of the human brain.

Alemán-Gómez Y, Melie-García L, Valdés-Hernandez P (2006). IBASPM: Toolbox for automatic parcellation of brain structures. Presented at the 12th Annual Meeting of the Organization for Human Brain Mapping, June 11-15, 2006, Florence, Italy. Available on CD-Rom in NeuroImage, Vol. 27, No.1.

Individual Brain Atlases using Statistical Parametric Mapping Software (IBASPM) http://www.thomaskoenig.ch/Lester/ibaspm.htm - Website providing functions and atlas for statistical parametric mapping plugins for Matlab.

OpenGL – http://www.opengl.org - The industry standard software 3D graphics library OpenGL homepage.

The Allen Brain Atlas – http://www.brain-map.org/mouse/atlas/ARA/Sagittal/browser.html - High-resolution 8-week old mouse brain atlas with organized taxonomy of brain structures.

The Human Brain Atlas – http://www.msu.edu/%7ebrains/brains/human/index.html - A human brain atlas consisiting of MRI, 3D renderings. and histology data displaying cells and fibers.

The Secret Life of the Brain – http://www.pbs.org/wnet/brain/3d/ - A surface rendered 3D brain anatomy atlas explorable by area or function.

The Whole Brain Atlas – http://www.med.harvard.edu/AANLIB/home.html - A number of human brain atlases from healthy and disease states, including multi-timepoint brain MRI’s.

Tcl/Tk Homepage – http://www.tcl.tk – Tcl (Tool Command Language) programming website providing information and documentation about the language.

VTK homepage – http://www.vtk.org - Visualization Toolkit information and programming example pages.

VRBA Homepage – http://bioweb.biology.kent.edu/FacultyPages/Clements/vrba.html - The Volume Rendered Brain Atlas homepage providing links to download the software, as well as a list of the segmented structures.

Wikipedia Homepage – http://en.wikipedia.org/ Free online content encyclopedia providing articles and information on a variety of subjects.

WINE homepage – http://www.winehq.org/ - The open source windows compatability layer for linux allowing unmodified windows programs to run on x86-based Unix machines.