Diffeomorphic (Non-Linear) Image Registration

Set up for diffeomorphic registration by selecting the Diffeomorphic (Non-Linear) tab:

Selecting diffeomorphic registration

Diffeomorphic registration is achieved by modelling the "flow" of one image (the moving image) into the fixed image using a pair of vector fields. The first field moves the moving image towards the mid-point, while the second field moves from the mid-point to the fixed image. The vector fields are defined at a discrete set of points in a grid across the field-of-view, and the fineness of the grid determines the level of anatomical detail that can be achieved in the registration.

Affine Pre-Registration

Normally, before images are diffeomorphically registered, they should already be in close correspondence so that the non-linear part of the registration is only concerned with the local distortions required to bring anatomical structures into register. This is done using an affine pre-registration in one of two ways:

  • Select to preform an affine registration prior to diffeomorphic registration This will perform an affine registration to the fixed image before automatically moving on to the diffeomorphic registration.

    The affine registration uses normalised correlation as the cost function and all possible degrees of freedom for inter-subject registration. It uses the RMS difference cost function and a rigid-body transform for intra-subject registration.

  • Select to apply an affine registration prior to diffeomorphic registration You may already have performed an affine registration and saved the resulting transform to disk. Selecting this option will allow you to apply that transform to the image to be registered, before moving on to the diffeomorphic registration. When selected, you can choose the transform file (with extension .rtp) that was produced by the affine registration:

    Selecting the affine transform file for pre-registration

    This can be useful if you find yourself repeatedly performing a diffeomorphic registration (perhaps to optimise other settings). You will save a lot of time because the affine registration will not need to be repeated.

    You may also find it useful if you want to perform the affine registration in a specific controlled way, perhaps removing some of the degrees of freedom, or using a different cost function.

Of course, if your images are already in close overall correspondence, you don't need to use either of these options.

Regularisation

Non-linear registration is an ill-conditioned problem - there is a vast array of non-linear transforms that could map the moving image onto the fixed image. In order to give a "plausible" transform, it is necessary to apply some constraints on the degrees of freedom of the deformation. This is done using "regularisation", which imposes penalties on sharply-varying deformation fields. There are four types of penalty:

Regularisation settings for diffeomorphic registration

  • Bending energy. This penalises sharply curving deformations.
  • Membrane energy. This penalises the volume-changing distortions.
  • Linear elasticity. This is a measure of tissue element length changes, but doesn't penalise rotations.
  • Divergence. This is a measure of the rate at which tissue volumes expand or contract.

There are currently no hard rules about which regularisation penalties to apply. You can apply any combination of penalties - setting a higher value for any of the penalties will result in a smoother deformation, but one which is less able to conform to very small detailed differences in the images.

The differences in the deformations for the different regularisation penalties are illustrated below. The image on the left is the fixed image registration target. The circular image on the right in the moving image which is to be registered to the target.

Diffeomorphic registration fixed and moving images.
The club-shaped registration target The annular ring moving image to be registered
Fixed image target Moving image

The diffeomorphic registration was performed using the maximum allowed setting (10.0) for each of the regularisation penalties in turn, with all other penalties set to zero. First, the results of the registration:

Results of the diffeomorphic registration for the different regularisation penalties.
Diffeomorphic registration regularised by bending energy Diffeomorphic registration regularised by membrane energy Diffeomorphic registration regularised by linear elasticity Diffeomorphic registration regularised by divergence
Bending Energy=10 Membrane Energy=10 Linear Elasticity=10 Divergence=10

The resulting registered images are, qualitatively, very similar, although some such as the ones for divergence and bending energy conform slightly more precisely to the registration target than others with this setting for the penalty. However, when we look at the deformations that took place to achieve the registration, the results are very different. The Jacobian matrix for each pixel characterises the deformation, and the determinant of the Jacobian (or the 'Jacobian determinant' for short) shows how much each pixel was stretched or compressed during the deformation. A Jacobian determinant higher than 1.0 signifies expansion; a Jacobian determinant lower than 1.0 signifies compression.

Jacobian determinant images for the deformation for the different regularisation penalties.
Jacobian determinant of the registration regularised by bending energy Jacobian determinant of the registration regularised by membrane energy Jacobian determinant of the registration regularised by linear elasticity Jacobian determinant of the registration regularised by divergence
Bending Energy Membrane Energy Linear Elasticity Divergence

As can be seen, the deformations that achieve the registration are very different and reflect the different penalties. For example, a high divergence penalises voxel expansion and contraction, so it is no surprise that the Jacobian determinants have a smaller range for this penalty (voxels that to not change volume have a Jacobian determinant of 1.0).

There is currently no consensus in the literature about what is the best regularisation penalty to apply when registering medical images. Indeed, the best penalty is likely to vary according to the application (for example intra-subject longitudinal registration vs. inter-subject registration). You will need to experiment to achieve satisfactory results.

Fidelity Settings

There are several settings that affect how closely the moving image can be deformed to the fixed image, and the speed of processing.

Diffeomorphic registration fidelity settings

  • Intra-subject registration. When selected, and affine pre-registration is also selected, then the affine transform is restricted to a rigid-body registration rather than full affine. This makes the affine pre-registration faster, and it should be more reliable.

  • Number of integration steps. The deformation is modelled using vector fields, with pixels moving through the fields from the moving image to their position in the fixed image. The movement is achieved by integration of the vectors, and setting a high number of integration steps should lead to greater fidelity of registration. The number of integration steps must be a multiple of 2, and in practice, you can set this as low as 4 or even 2 without greatly impacting on the registration results. This leads to slightly faster processing.

  • Max. number of iterations at each resolution. Diffeomorphic registration is an iterative process. This sets the maximum number of iterations. If the registration does not convergence before this number of iterations, then iteration is stopped anyhow. A setting of 150-300 is normally sufficient so that convergence is always achieved.

  • Max. number of samples at in any dimension. Regularisation is achieved using a Fourier transform technique, and therefore the number of samples of the deforming vector fields must be a power of 2. This sets the maximum number of samples of the vector fields, and alters the level of detail that can be achieved when deforming the anatomy.

For intra-subject longitudinal registration, it is likely that atrophy can be modelled using a low number, such as 32. For inter-subject registration where there are detailed differences in anatomy, a setting of 128 or even 256 may be most appropriate. However, this setting has the biggest influence on the computer memory requirements and speed of processing, with a setting of 256 taking an inordinately long time.

This setting affects all dimension of the image. For example if you set the value to 64, and are registering an image that has a 256×256×44 matrix, then the vector fields will have 64×64×32 samples. None of the dimensions is up-sampled, and the each is downsampled to either the maximum value, or to the next-lower power of 2.

Effect of the number of samples on the registration fidelity.
Registration with a resolution of 32x32 Registration with a resolution of 64x64 Registration with a resolution of 128x128 Registration with a resolution of 256x256 The registration target
32 64 128 256 Target

The images above show a detail of one of the corners in the registration target image, and the registered image with increasing maximum number of samples from 32 to 256. Although the highest resolution gives the best conformation to the target, a surprising good registration can be achieved even with a much lower resolution of the vector fields.

Cost Function

Diffeomorphic registration only really works well for registering images with similar contrast. For example, two images of the same subject acquired using the same pulse sequence at different times; or registering a patient image with an atlas of the same approximate contrast. You can choose:

  • RMS difference. The root mean square difference between the fixed image and registered image is minimised. This option works best when the two images are collected with the same modality, and have very similar contrast and intensity ranges.
  • Normalised correlation. This is the cross-correlation of the fixed image and registered image, normalised by dividing by the RMS intensities of both images. This is the normal cost function to select for practical medical image registration.

Output Settings

The following affect the final outputs created by the diffeomorphic registration:

Diffeomorphic registration output settings

  • Perform bias correction. When selected, after the registration is complete, the image intensity non-uniformity (bias) of the registered image is corrected so that the intensity variations match those of the fixed image. If creating half-way images, the intensity variations are matched in both half-way images.

  • Rescale intensities. When selected, before registration is started, the intensities of the image to register are matched to those of the fixed image using a histogram matching technique. This may help to improve the evaluation of the cost function, particularly for the RMS difference measure when images have widely-varying intensity ranges.

  • Create half-way co-registered image. When selected, registered versions of both the moving image and the fixed image are created at the mid-point between the two. The names of the two half-way images will have the prefix "hw" prepended to their original names.

  • Write Jacobian determinant image. This will create an image of the Jacobian determinant of the deformation to be written to disk with the suffix "_Det" appended to the moving image's original file name.

  • Write deforming vector fields. If you want to apply a deformation found for one registration to another image, you will need to select this option. The vector fields that define the deformation are then saved to disk and can be applied to another image. See Apply a pre-calculated deformation. The vectors fields image is written with the suffix "_V" appended to the moving image's original file name.

  • Create a movie showing the deformation. This creates a movie image showing the transition of the moving image from its original shape into the deformed shape when registered to the fixed image. This is useful for visualising the registration. When selected, you need to also set the number of frames in the movie - setting a higher value will result in a smoother movie. The movie image is written with the suffix "_Movie" appended to the moving image's original file name. One such movie can be seen below for the deformation of the synthetic image above, playing in yo-yo mode.

Movie showing the deformation of the moving image to the fixed image (Animated: View cine on-line version)

Other Settings

  • Fixed image mask. The standard masking options are available to select pixels in the fixed image that are used to compute the cost function. This can be useful for removing background noise, or removing portions of the image where alignment is not important (such as soft tissue, when registering brains).
  • register_multi_resolution If this option is selected, the number of samples of the vector fields is increased from 32 by factors of 2 until the "maximum number of samples in any dimension" is reached. This option is strongly recommended unless you know that the two images already have good correspondence of anatomical features. It helps to prevent getting stuck in a false minimum, and the low-resolution estimates are in any case quick to compute.
  • Fractional tolerance. The fractional tolerance for convergence: when fractional changes in the cost function are less than the tolerance, then the iterative registration will be deemed to have converged. If registration is taking inordinately long, try increasing this value. If registration stops before the images are accurately registered, try reducing this value.
  • Final interpolation. Select the type of interpolation to be used when producing the (registered) output images. Choosing Sinc will result in improved image quality, but at the expense of (very) slightly longer processing time for this last processing step. Resampling a large image with sinc interpolation may take a few seconds.

Select the fixed image by clicking on the open icon, or by typing in the folder (directory) and file name of the image, or by pressing the right mouse button and selecting from the menu of recently-used images.

Selecting the fixed image

Next, select the images that you want to register to the fixed image by clicking on the select_images_button button. This will bring up a file chooser. Select one or more images to register in the file chooser, and press the Select button. You can add more images to this list by pressing the add_images_button button. Ensure that the images are in the order in which you want them to be processed. If they are not in the right order, select one or more images in the displayed list to move, and the click either the move_up_button button or move_down_button button to get them in the right order.

You can remove all the images from the list by clicking the remove_all_button button, or remove just some of the images by first selecting them, and then clicking the remove_selected_button button.

Now click on the Button to do the registration button to perform the registration. Registration will take several minutes for each of the images, during which time various progress indicators will pop up, allowing you to cancel the registration at any stage. If registration completes, the registered images will be produced.

The registered images have the same numbers of rows, columns and slices as the fixed image, but will have pixel intensity values taken from the registered images after they have been re-aligned to match the position of the fixed image. The registered image are automatically given a name the same as the input images, but with the letter "r" prepended to the name. For example, if you chose an image to be registered with a name MyImage, the registered image will be given the name rMyImage.

Viewing the Result

If you want to check on the quality of the registration, the best way is to load the fixed image into Jim's main display window. Then, load the registered image as an overlay image. You can automatically get Jim to load the registered image to the overlay by selecting the Load result button load_save. With this selected, as each registered image is produced, it is overlaid onto the image in Jim's main display.
Note: you must have the fixed image loaded into Jim's main display for this feature to work.