Frog3D: from medical imaging to biomechanical models

This project is part of a research collaboration funded by CIHR (Canada) and NSF (USA) through the CRCNS program.

Drexel University: Simon Giszter (PI), Arun Ramakrishnan

Northern Arizona University: Kiisa Nishikawa (PI), Ted Uyeno, Jenna Monroy

University of British Columbia: Dinesh K. Pai (PI), Benjamin Gilles, Sang Hoon Yeo, Shinjiro Sueda

To understand motor primitives better, we are building a detailed functional model of the frog musculoskeletal system. The ultimate goal would be to reproduce body movements through accurate 3D physical simulation that takes as inputs muscle actuation patterns and proprioceptive feedbacks. To begin both physical modeling and virtual simulations we must have a solid anatomical understanding of both rigid and soft tissues of the frog.

Our first case study is the investigation of the mechanics of the frog jaw. There are relatively few mechanical elements responsible for the rapid ballistic movements of the frog tongue during feeding. This makes testing their function a feasible endeavor.

To build a 3D anatomical model of the frog, several modalities are considered to image bone, muscle and connective tissues with different levels of contrast and resolutions. X-ray Computed Tomography can achieve a good bone-to-soft tissues contrast and is therefore employed to construct an accurate 3D model of the skeleton. We use Cryoplane microscopy (high resolution pictures of frozen slices) with different staining techniques to acquire some muscle features such as the spindle distribution or sarcomere length. To couple multimodal data and incorporate complementary information into a model, we are exploring 3D registration techniques based on deformable models. Subsequently, our model is functionalized by adding an underlying kinematical model to parameterize joints, and muscle strands to simulate muscle function.

Gross anatomy

The traditional technique in gross dissection first consists in removing the skin from a freshly killed specimen. starting with the superficial layers we observe the locations and orientations of connective tissue structures, muscles, and bones. Because these structures are composed of fibers, the orientations of which impact their overall mechanical properties, we often use histology to identify their trajectories within the structures of interest. As we peel away layers of structures we note the three-dimensional relationships between them and identify important features such as possible directions of force production and effects of changes in muscle shape during contraction, where muscles are attached to bones, where bones contact each other to form pivots, and where energy may be stored elastically in muscle, tendons or beam-shaped bones.

Preserved leopard frog specimen

The results of our morphological investigations show that the rigid elements of the head can be most simply modeled as the skull and the left and right halves of the mandible. Between these rigid bodies there are three joints. The lower jaw swings downwards relative to the skull as the mouth opens because of the left and right rotational joints that connect the skull to the mandibles. Additionally, there is a flexible connection at the chin where the left and right jaw bones are attached.

There are also a number of important soft tissue elements that are divided into muscles responsible for opening and for closing the mouth. There are two sets of muscles that create force to open the jaws; the powerful depressor mandibulae and the somewhat smaller masseters. They are symmetrical on the left and right sides. There are also two symmetrical sets of jaw closing muscles. We describe them collectively as the inner and outer levator mandibulae.

Computed Tomography (CT)

Computed tomography is a standard medical imaging technique that generates a 3D image from X-ray projections. We scanned a bullfrog in a natural pose using regular CT acquisition. The purpose was to identify a rest pose which is important for modeling the joints. Our dataset consists in a 512´512´302 grayscale 3D image of resolution 350x350x625 microns.

Bones can be delineated in CT images using standard segmentation tools such as thresholding and morphological operations, although a complete separation of the bones can only be achieved through a meticulous manual contouring. Still, the resolution was not high enough to accurately model small bones such as the phalanxes. From the segmented images, 3D surface meshes are constructed by following a standard isosurfacing pipeline (marching cubes + smoothing + decimation).

Micro CT

Micro CT can do imaging with a much higher resolution than regular CT. The drawback is however the smaller field of view and longer scanning time. Frogs were first euthanized and stabilized in ethanol. We scanned them dry and wrapped the specimens in paper to reduce motion artefacts. Scanning was done at the UBC hospital facility (Centre for Hip Health and Mobility) on a Viva CT 40 scanner. The size of the final 3D image was 1024x1024x2222 and the resolution 45x45x45 microns.

Histology

Cryoplane microscopy is also considered to measure important muscle functional features such as the densities of the muscle spindle, the sarcomere lengths and the nerves. This consists in taking high resolution pictures of frozen slices. Different staining techniques are investigated to enhance specific tissues. This study is performed by the Simon F Giszter group at Drexel University (Arun Ramakrishnan). A first dataset was acquired with a 30x30x30 microns resolution and a size of 2920x4384x1104 voxels.

Sample cryoplane image (coronal plane)

Model registration

To combine information from the different datasets and different frogs, 3D registration is necessary. Registration is currently investigated by the Dinesh K. Pai group at UBC (Benjamin Gilles). The principle is to deform a source model and adapt the skeleton pose to minimize its distance to a target model.

Regular and micro CT are complementary: the first allows for low resolution scanning of large regions of interest while the second allows for the opposite. As shown in this picture, we can transfer the geometric details of the high resolution models to the model in the rest pose.

We are also doing registration with cryoplane images as shown here:

Registered Head model with cryoplane data

References:

- B. Gilles, D.K. Pai, “Fast Musculoskeletal Registration based on Shape Matching”, International conference on medical image computing and computer assisted intervention (MICCAI’08), pp. 822-829, 2008. pdf

- B. Gilles, L. Revéret, D.K. Pai, “Creating and animating subject-specific anatomical models”, Computer Graphics Forum (major revision).

Here is a Quicktime VR movie of the final high resolution skeleton model of a bullfrog:

Kinematical skeleton construction

To animate our skeleton model, we need to parameterize the different joints by defining their center, their natural axis of rotation, and their degrees of freedom. Joints are linked by bones segments to define a tree structure called the kinematical or animation skeleton. We manually defined it on one model. Thanks to our registration techniques, we are able to map this skeleton to other specimens.

Physical simulation

In order to create the most realistic computer simulation of the frog feeding mechanism, we use our soft tissue morphological descriptions to draw accurate representative muscle fiber trajectories (or strands) on the three-dimensional computer model.

Reference:

- S. Sueda, A. Kaufman, and D. K. Pai. 2008. "Musculotendon Simulation for Hand Animation." In ACM Transactions on Graphics (Proceedings of SIGGRAPH 2008), 83:1-83:8. pdf

A fully articulated skull and mandible with strands representing the jaw opening and closing muscles.

These strands may be given physical parameters such as a thickness, volume, length before and after contraction, elasticity, force and timing of contraction, and may also be constrained in space, such as being required to bend along a curve. In short, these strands may be given the characteristics to accurately behave as they do in its living biological counterpart. In fact, the values of these parameters are measured from frame-by-frame kinematic analyses of video taken from freely behaving frogs eating prey items. Here, we use multiple synchronized high speed cameras to fully capture the fast three-dimensional motion.