A high-level 3D visualization API for Java and ImageJ.
Schmid B, Schindelin J, Cardona A, Longair M, Heisenberg M. · 2010
Read A high-level 3D visualization API for Java and ImageJ. by Schmid B, Schindelin J, Cardona A, Longair M, Heisenberg M. (2010) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/europmc-pmc2896381.txt.
Introduction
Schmid and colleagues built in 2010 a three-dimensional visualization toolkit that plugs into ImageJ, the free image software biologists already use. Modern microscopes and scanners produce 3D stacks that flat viewers handle badly, and writing custom 3D code is slow and specialist work. Their Java framework hides the graphics machinery behind simple high-level calls.
Plug-in authors gain volume rendering, direct editing of 3D volumes, surface extraction from labeled structures, and annotation tools with little extra code. Hardware acceleration keeps big confocal and electron microscopy datasets interactive. Because it rides on ImageJ's vast community library, a lab can move from raw stack to measured 3D structure in one environment. The authors released the source openly so other groups could extend it.
Catalog record
| CREATOR | Schmid B, Schindelin J, Cardona A, Longair M, Heisenberg M. |
|---|---|
| YEAR | 2010 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 29 KB |
| LICENSE | CC BY 4.0 |
| RIGHTS | CC BY |
| SOURCE | Europe PMC |
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
A high-level 3D visualization API for Java and ImageJ By Benjamin Schmid, Johannes Schindelin, Albert Cardona, Mark Longair, Martin Heisenberg Published in: BMC Bioinformatics DOI: 10.1186/1471-2105-11-274 ABSTRACT Abstract Background Current imaging methods such as Magnetic Resonance Imaging (MRI), Confocal microscopy, Electron Microscopy (EM) or Selective Plane Illumination Microscopy (SPIM) yield three-dimensional (3D) data sets in need of appropriate computational methods for their analysis. The reconstruction, segmentation and registration are best approached from the 3D representation of the data set. Results Here we present a platform-independent framework based on Java and Java 3D for accelerated rendering of biological images. Our framework is seamlessly integrated into ImageJ, a free image processing package with a vast collection of community-developed biological image analysis tools. Our framework enriches the ImageJ software libraries with methods that greatly reduce the complexity of developing image analysis tools in an interactive 3D visualization environment. In particular, we provide high-level access to volume rendering, volume editing, surface extraction, and image annotation. The ability to rely on a library that removes the low-level details enables concentrating software development efforts on the algorithm implementation parts. Conclusions Our framework enables biomedical image software development to be built with 3D visualization capabilities with very little effort. We offer the source code and convenient binary packages along with extensive documentation at http://3dviewer.neurofly.de. BACKGROUND Life sciences are experiencing an increasing demand for scientific image processing. Images are the primary data of developmental and cell biology. The number of images is exploding with the availability of high-throughput and high-resolution technologies. The acquisition of large three-dimensional (3D) data sets, often as time series (4D), has become the new standard. The first step in the analysis of biological image data is its visual inspection. In addition to the general requirement for visualization, the unique characteristics of each data set may demand specialized analysis. The development of novel analytical tools is greatly facilitated by the existence of well-documented software libraries. These libraries must provide (1) means to load and save any of the large diversity of image file formats; (2) implementations for computer vision algorithms; and (3) graphical user interfaces for data access by a human operator. We have identified a lack of accessible 3D/4D visualization software libraries for biological image processing. Numerous image processing packages exist, either commercial (Amira, Visage Imaging; MeVisLab, Mevis; Imaris, BitPlane; Volocity, PerkinElmer) or open source (VOXX, [1]; VTK and VTK-based applications such as Slicer3D, BioImageXD, and V3D [2]; UCSF Chimera [3]; VolumeJ [4] and Volume Viewer [5]). These packages offer excellent solutions for the specific problems they were designed to solve. While end-users benefit from well-documented, special-purpose commercial applications, the development of custom analytical tools is better handled by open source packages. The application programming interfaces of existing packages range from the non-existent for most closed commercial solutions, to the very detailed and comprehensive open source VTK environment. We have created a software library for 3D/4D visualization, with functions for surface extraction, volume rendering and interactive volume editing. Our library removes all the complexity of creating and interacting with image volumes and meshes in a 3D environment. We have designed our library to enrich the core functionality of ImageJ (and its descendant Fiji [6]), an open source image processing application. Via ImageJ, our library has access to hundreds of biological image file formats. Over the years, the scientific community has contributed a very large number of ImageJ extensions, known as plugins, which provide readily accessible implementations of numerous computer vision algorithms. With our library, we empower the ImageJ scientific community to rapidly implement custom analytical tools for 3D/4D data sets, with a minimal investment of time and resources in handling the complex details of a hardware-accelerated 3D environment. This reduction in the difficulty of visualizing 3D information commoditizes the usage of a 3D scene. For example, our library enables software developers to visually assess the correctness of individual algorithmic steps, such as the 3D shape of a mesh deformation. The simplicity of our library is in stark contrast to existing libraries such as VTK, which require detailed knowledge of the underlying data structures. An example application in need of effective 3D visualization is the image volume reconstruction from electron tomography data as provided by TomoJ [7], an ImageJ plugin. Our library complements TomoJ, enabling convenient 3D analysis of the results without the need of external software such as Chimera [3]. Many other plugins for image registration or object segmentation are similarly using our library for integrated and interactive 3D visualization. IMPLEMENTATION Our software library implements the concept of three-dimensional scene ("3D scene") for the interactive visualization of biomedical image data (Figure 1). The 3D scene is a virtual 3D space in which any number of entities are hosted. These entities are volume renderings, isosurfaces and orthoslice sets. Volume renderings are representations of voxel data viewed from arbitrary angles, with transparency determined by the intensity values. Isosurfaces are meshes of triangles described by a list of vertices. Orthoslices are orthogonal planes that cut through an image volume. Each of these entities, or objects, has a number of properties, such as transparency, color, and a unique name. The 3D scene that hosts all objects has the following properties: a zooming level, an origin of viewer coordinates, and a scene background. [Figure: Three display modes supported by the framework. An example image volume containing an adult Drosophila brain, shown (a) as a volume rendering, (b) as orthoslices and (c) as an isosurface of its external contour.] For efficient rendering in computer graphics, we chose Java 3D: a low-level hardware-accelerated software library. Java 3D has the further advantage of being implemented for Java, thus enabling Java applications like ImageJ to interoperate with the graphics card of a computer, via either OpenGL or DirectX low-level native layer. Java 3D provides a fine-grained representation of a virtual scene as a directed acyclic graph [8]. Operations on any node of the graph affect its entire subtree. In practice, this means that, for example, zooming in and out of a scene is expressed as a scaling transformation in a low-order node. High-order nodes encapsulate images and meshes. A key feature of our library is to substantially simplify the usage of Java 3D. We define our 3D scene in terms of Java 3D nodes (Figure 2). We provide straightforward means to instantiate a new interactive 3D scene and to add objects to it. In the following, we describe the structure of the scene graph as implemented in our library. [Figure: Scene graph structure of the framework components. The Java 3D library offers a set of classes that are organized in a directed acyclic graph. The relationships of the nodes in the graph determine how they are rendered in the 3D scene. Our framework, built on top of Java 3D, consists of a chain of global TransformGroup (TG) nodes that represent the view's zoom, rotation and panning (top), and which affect the rendering of all subtree nodes. Subtree nodes contain the image volumes. Other nodes include BranchGroup (BG) nodes, capable of holding several subtrees (each representing the internal state and data of an image volume or mesh; lower right). Switch (S) nodes are immediate parents of data nodes, and are used to toggle their