home | news | snapshots | manual | how to | tutorials | scripting

How to ...

Click on "For plugin writers" to see Java code that does the exact same thing.

  1. How to Install TrakEM2.
  2. What to do if TrakEM2 doesn't work for you.
  3. How to create a new project.
  4. How to import a stack
  5. How to register slices in a stack.
  6. How to add an image
  7. How to snap an image onto another one.
  8. How to stitch several images together.
  9. How to register simultaneously multiple images in multiple layers.
  10. Feature extraction: the effect of modifying the SIFT parameter "maximum image size"
  11. SIFT parameters explained.
  12. Minitutorial: create new project, add 3 images, register them, save the project and close it.
  13. How to install and run from TrakEM2 source code with Fiji
  14. How to update TrakEM2 within a Fiji repository

Click on "For plugin writers" to see Java code that does the exact same thing.

See also TrakEM2 Scripting wiki page, and the video tutorials

How to install TrakEM2

Just download Fiji.

If you are a software developer follow these instructions on how to download and build Fiji from source including TrakEM2. See also these general instructions on how to develop software with or for Fiji.

What to do if TrakEM2 doesn't work for you.

Step 1: setup your system to be able to see the error messages.

For Windows XP only: be sure to run ImageJ with java.exe and NOT javaw.exe, as explained in the Windows installation how to above.

For MacOSX only: open the /Applications/Utilities/Console.app, there is where all error messages are to be found.

For Linux and *nix only: launch ImageJ from an open terminal and not from a system menu, so that the associated terminal will remain open even in the event of an ImageJ crash. Otherwise you'll never see the error messages.

Step 2: gather as much information as possible on your system.

The easiest is to open ImageJ and then open the menu Plugins - ImageJ - Utilities - ImageJ Properties. This will open a text file; please save it and send it along.

Step 3: email all the error messages and your system information to Albert Cardona at acardona at ini phys ethz ch.

How to create a new project

Step 0: Install ImageJ with TrakEM2, and launch ImageJ.

Step 1: Select from the menus Plugins / TrakEM2 / New Project (blank).

NOTE: in Fiji, go to File / New / TrakEM2 Project (blank).

Step 2: You'll be prompted for a storage folder. TrakEM2 will store its own files in this folder.

I strongly recommend that the selected storage folder also contains the images you are going to use, and that you save the TrakEM2 project within this folder as well. These practices ensure encapsulation: file paths will be relative, and thus the folder may be moved to other locations or computers without incurring in "file not found" situations (think of Powerpoint or Indesign or Inkscape not finding image files or videos because those have been moved to different folders.) You can fix those "file not found" situations by editing the file paths directly in the project .xml file.

// a new, empty project
Project p = Project.newFSProject("blank", null, "/path/to/storage_folder/");
// or a project from an existing DTD or XML file as template
Project p = Project.newFSProject(null, DTDParser.extractTemplate("/path/to/a_template.dtd")[0], "/path/to/storage_folder/");

How to import a stack

Step 0: Create a new TrakEM2 project.

Step 1: Drag and drop any image file that ImageJ can read into an open TrakEM2 display.

Extra layers will be created automatically for you.

Project p = ControlWindow.getActive(); // or: p = Display.getFront().getProject();
                                       // or new: p = Project.newFSProject("blank");
String filepath = "/path/to/image.tif";
ImagePlus imp = new Opener().openImage(filepath);
p.getLoader().importStack(first_layer, imageplus, true, filepath);

How to register slices in a stack

Step 1: Import the stack into TrakEM2.

Step 2: if the stack is not selected, select it by clicking on its panel ("Patches" tab, scroll to find it and then click on it - the panel turns blue).

Step 2: Select the Align tool (the double planes tool; put mouse over tools to read their name in ImageJ status bar).

Step 3: right-click on the stack and select "Align stack slices".

While the alignment executes, you can use the hand tool and the glass tool to navigate it. all other input is blocked until the task finishes.

// if the Patch object wrapping the stack is selected, then:
Patch patch = (Patch)Display.getFront().getActive();
Thread task = AlignTask.registerStackSlices(patch);
// wait for asynchronous task to complete
try { if (null != task) task.join(); } catch (Exception e) { e.printStackTrace(); }

How to add an image

Step 0: Create a new TrakEM2 project.

Step 1: Drag and drop an image from a directory into the canvas.

Alternatively, right-click on the canvas and select "Import / Import image..."

Layer layer = Display.getFront().getLayer();
// or:
// LayerSet ls = ControlWindow.getActive().getRootLayerSet();
// double z_coord = 25.5;
// double thickness = 20.0;
// Layer layer = ls.getLayer(z_coord, thickness, true); // will create one new if not there

String filepath = "/path/to/image.png";
ImagePlus imp = new Opener().openImage(filepath);
double x = 300;
double y = 500;
Patch patch = new Patch(layer.getProject(), x, y, imp);
patch.getProject().getLoader().addedPatchFrom(filepath, patch);
// optional: enlarge/shrink the canvas dimensions to fit the new image

How to snap an image onto another one.

Step 1: select an image by clicking on it with the Select tool (black arrow).

Step 2: drag the image until it overlays any other number of images.

Step 3: right-click on the image and select "Snap". The image will be snapped onto the best matching image from those that it overlaps with.

Display display = Display.getFront();
// get a layer from which to grab patches (class Patch wraps an image)
Layer layer = display.getLayer();
// or:
// LayerSet ls = ControlWindow.getActive().getRootLayerSet();
// Layer layer = ls.getLayer(23.4); // by z coord
// Layer layer = ls.getLayer(10); // by index: the 10th layer
// Layer layer = ls.getLayer(122345); // by id (a long)

// There are several ways to get patches:
Patch image1 = layer.get(1234); // by id
Patch image2 = (Patch)layer.getDisplayables(Patch.class).get(0); // by z-index within the layer
// or create it new (see howto above)

// snap image1 onto the image with which it overlaps the most:
Thread task = AlignTask.snap(image1, AlignTask.p_snap, list);
try { if (null != task) task.join(); } catch (Exception e) { e.printStackTrace(); }

// optional: enlarge/shrink the canvas dimensions to fit the new image

How to stitch several images together.

Step 1: select several images by shift-clicking them with the Select tool (black arrow)

Step 2: right-click and choose "Montage". A best attempt will be made to match them all to all and figure out the best solution.

Here is an example of the process and result of stitching several images together. The underlying algorithm is the Scale Invariant Feature Transform (Lowe 2004) as implemented by Stephan Saalfed.

Display display = Display.getFront();
// get a layer from which to grab patches (class Patch wraps an image)
Layer layer = display.getLayer();

// grab all patches from the layer
ArrayList<Patch> all = (ArrayList<Patch>) layer.getDisplayables(Patch.class);

// the "nail" image won't move
Patch nail = all.get(0);
ArrayList<Patch> nailed = new ArrayList<Patch>();

// start registration
Thread task = AlignTask.alignPatchesTask( all, nailed );
try { if (null != task) task.join(); } catch (Exception e) { e.printStackTrace(); }

// optional: resize display to fit all images in it

How to register simultaneously multiple images in multiple layers.

Step 1: import any number of images, many per layer or just one, onto many layers.

Step 2: with the "Align Tool", right-click on the display and select "Align multi-layer mosaic".

Layer layer = Display.getFront().getLayer();

// With a dialog:
Thread t = AlignTask.alignMultiLayerMosaicTask(layer);

// Without a dialog:
LayerSet layer_set = Display.getFront().getLayerSet();
List<Layer> some_layers = layer_set.getLayers().subList(3, 6); // includes layers 3,4,5
Align.Param cp = Align.param.clone(); // for connecting tile pairs
Align.ParamOptimize p = Align.paramOptimize.clone(); // for extracting features
Align.ParamOptimize pcp = p.clone(); // for optimizing across layers
pcp.desiredModelIndex = 1; // 0=translation, 1=rigid, 2=similarity, 3=affine

boolean tilesAreInPlace = false; // if tiles are already overlapping with the proper ones within a layer, from a manual registration for example.
boolean largestGraphOnly = false; // consider only the largest graph of connected tiles
boolean hideDisconnectedTiles = false; // call setVisible(false) on Patch instances that do not belong to the largest graph
boolean deleteDisconnectedTiles = false; // remove Patch instances that do not belong to the largest graph
AlignTask.alignMultiLayerMosaicTask(some_layers, cp, p, pcp, tilesAreInPlace, largestGraphOnly, hideDisconnectedTiles, deleteDisconnectedTiles);

Feature extraction: the effect of modifying the SIFT parameter "maximum image size"

We are aligning TEM tiles acquired by Leginon. Both images overlap around 10 % in width. Despite the electron beam elastically deforming the ultrathin section at each image acquisition due to heat, we expect very little differences within the overlapping region.

On the left, max_size = 256. Note how only large features are found. The finer details of the image have been merged away when starting the image pyramid at 1/8th of the original image tile size of 2048x2048 pixels. This is the equivalent of looking at an object from very far: only broad features are visible.

On the right, max_size = 1024. Now both large and small features are found. We could find even more smaller features by starting at the original size, 2048.

Both images show only the inlier features: those identified by RANSAC as belonging to the best possible transformation model. All other identified features are not depicted, for clarity.


[click images to enlarge]

Images were generated with the Sift_Matching.java plugin, which requires TrakEM2 to be installed.

The resulting affine transforms that describe the registration:

max_size = 256

 0.99999764769672,   0.002169009227177, 1840.5052774712833
-0.002169009227177, 0.99999764769672,     33.38239346132822
max_size = 1024

0.999979670244666, -0.006376448648657, 1846.9495819316148
0.006376448648657,  0.999979670244666,   38.051533861723584

Notice how there are small, noticeable differences in the translation (right columns): when scaling down, some information is lost.

For intra-slice registration, that is, to register neighboring tiles which show part of the exact same object (even if slightly deformed), one can go up with the max_size as much as desired. The only potential problem is much longer processing time.

For inter-slice registration, that is, to register images corresponding to consecutive sections of the same object, it is far more desirable to use a low max_size value. The reason: we want to find only large features, which correspond to objects large enough that their profile does not change significantly across the slice thickness. Any features smaller than the slice thickness may be totally bogus, and potentially lead to the creation of an incorrect transformation model.

SIFT parameters explained

Minitutorial: create new project, add 3 images, register them to make a montage, save the project, make a snapshot of the montage and close the project.

Step 0: Install ImageJ with TrakEM2, and launch ImageJ. Or do it from Fiji.

Step 1: Go to Plugins - TrakEM2 - New Project (blank) (or to File - New - TrakEM2 Project (blank) in Fiji).

Step 2: Select a storage folder when prompted.

Step 3: Drag and drop three images from a system folder into the opened TrakEM2 display.
ImageJ status bar will report on import progress. Scaled down versions of the images are generated and stored in a new trakem2.mipmaps/ folder in your selected storage folder.

If you drag and drop a folder of images instead, you will be prompted for options on how to automatically position all the images -- but this is beyond the scope of this minitutorial. See the manual at "Importing a sequence of images as a grid or montage". You can also import from a text file that specifies file paths, X, Y, and Z coordinates.

Step 4: In ImageJ toolbar, choose the Select Tool (black arrow).

Step 5: Click on an image inside the display and drag it away to reveal the other image underneath; do the same again to reveal the third image.

Step 6: For a faster montage, drag each image onto another so that they all grossly overlap as you know they should.
Then click on an image with the Select Tool, then shift+click on another, and finally shift+click on the third, so that all three are selected. Notice how the first two are framed in pink and the last in white. The last selected image is the active image, and will not be moved from position. The other two will be registered pivoting around the active image.

Additional commands and capabilities:
Deselect images by shift+click on it to make it active, then again shift+click to deselect.
Push ESC key to deselect all.
Select any number of images by choosing a ROI tool such as Rectangular Selections tool, and then right-click and choose Selection - Select all under ROI.
You can also lock any image or images by selecting it, then right-click and choose Lock (or Unlock). Locked images won't be affected by the montaging.

Step 7: Right-click and select Montage. If your images are big, choose a maximum image size of 512 and a scale proportional to that (for example, for 2048x2048 images, use scale 25%.)
Push OK and wait -- observe ImageJ status bar for progress reporting (and also the terminal output, or Console.app in a MacOSX.)
For a faster montage, be sure to check the box that says "Tiles are roughly registered" so that registration will be performed on overlapping tiles only (as opposed to all to all.)

If unhappy with the results, push shift+z to undo the transformations, and repeat steps 6 and 7 increasing the maximum image size parameter (and the scale parameter accordingly.) The largest maximum image size that makes sense is the largest length of a side of an image (i.e. 2048 for a 2048x2048 image), and scale 100%.

Step 8: To correct uneven brightness and contrast, right-click and explore the Adjust submenu. You can set a min and max for all selected images, automatically enhance each independently or all in a coordinated fashion.
There is no undo, but the original images are not modified in any way -- only the scaled mipmaps.

Step 9: Right-click and choose Project - Save, or push 's' while having the display window in focus. You can do this as many times as you please.

Step 10: Take a snapshot of the registered montage. Select the Rectangular Selection tool from ImageJ toolbar, draw a rectangular ROI and then right-click and choose Display - Make flat image.
Enter a scale value (1.0 equals 100%; use 0.25 for 25%, etc.), and 8-bit or RGB snapshot.
A new image will open within ImageJ, which you can save with standard File - Save menu.

Step 11: Once saved, go to the TrakEM2 window (with the three colored panels) and close it. Done!

// Birth, life and death of a TrakEM2 project
// ------------------------------------------
// Create a project, add images to its default layer,
// register and adjust the images,
// export a snapshot of the result,
// and save and close the project.

// 0 - Disable GUI if headless desired

// 1 - Create a new, empty project
Project project = Project.newFSProject("blank", null, "/path/to/storage_folder/");
// or a project from an existing DTD or XML file as template
// Project project = Project.newFSProject(null, DTDParser.extractTemplate("/path/to/a_template.dtd")[0], "/path/to/storage_folder/");

// 2 - Obtain LayerSet and Layer pointers
LayerSet ls = project.getRootLayerSet();
Layer layer = ls.getLayer(0); // there is one layer by default in a new Project.

// 3 - Adding images to a Layer
final Opener opener = new Opener();
double x = 100; // initial coordinates
double y = 300;
ImagePlus imp1 = opener.openImage("/path/to/image1.tif");
Patch patch1 = project.getLoader().addNewImage(imp1, x, y);
// repeat for other 2 images
// ...

// Optionally, you can set the transform as well for each Patch:
AffineTransform at = new AffineTransform();
affine.translate(100, 300);
affine.scale(0.5, 0.5);
patch1.setAffineTransform(affine); // will copy its values,
           // not retain the AffineTransform object, so you can reuse it.

// 4 - Update displays (optional, it's been done already at layer.add)
//    You can use layer.addSilently(patch1) to avoid updating displays
//    but such method is dangerous: no proper stack index ordering
//    for different kinds of Displayable objects (such as text DLabel,
//    which should be always on top, i.e. at the end of the layer list.)

// 5 - Register/Montage images
List<Patch> patches = new List<Patch>();
// ... add all other images to the List
// ...

List<Patch> fixedPatches = new List<Patch>();

Thread task = AlignTask.alignPatchesTask( patches, fixedPatches );
if (null != task) {
    try { task.join(); } catch (Exception e) { e.printStackTrace(); }

// 6 - Adjust brightness and contrast
ArrayList<Patch> al = new ArrayList<Patch>();
// ... add all other images 
// WARNING: must all be of the same dimensions.
//          Will refuse to work otherwise.
// Strategy A: set same min and max for all images,
//   but shifting the histogram's peak to compensate a bit.
double min = 15600; // for 16-bit TEM images
double max = 24700;
Thread task = project.getLoader().setMinAndMax(al, min, max);
if (null != task) {
    try { task.join(); } catch (Exception e) { e.printStackTrace(); }
// Strategy B: order all tiles by the stdDev of their histograms, and then
// use the central 50% to obtain average values to apply to all tiles.
Thread task2 = project.getLoader().homogenizeContrast(al, null);
if (null != task2) {
    try { task2.join(); } catch (Exception e) { e.printStackTrace(); }

// 7 - Update displays if any:

// 8 - Save the project
// It is recommended to store .xml inside the storage folder, so that
// then image file paths are relative and thus the whole folder
// can be moved between different computers trivially:
project.saveAs(project.getLoader().getStorageFolder() + "the_name.xml", true); // overwriting any existing xml file with that name

// 9 - Take a snapshot of the registered images
Rectangle box = layer.getMinimalBoundingBox(Patch.class); // or any other ROI you like
double scale = 1.0;
int type = ImagePlus.GRAY8; // or .COLOR_RGB
int c_alphas = 0xffffffff; // channel opacities, applies to RGB images only
boolean quality = true; // use max possible rendering quality
ArrayList list = null; // null means all. You can define a subset of Displayable objects
                       //  of that layer (such as Patch objects) to be rendered instead.
ImagePlus snapshot = project.getLoader().getFlatImage(layer, box, scale, c_alphas, type, Patch.class, list, quality);
new ij.io.FileSaver(snapshot).saveAsTiff("/path/to/image.tif");

// 10 - Close the project
// If GUI was disabled, be nice and reenable it:

// Done!

How to install and run from TrakEM source code with Fiji

This How To is meant for developers. To install TrakEM2 with Fiji, all ready to go, just download Fiji.

Step 1: Install Fiji with git. Open a shell and type:

$ git clone ssh://contrib@pacific.mpi-cbg.de/srv/git/fiji.git/
See also help with git and fiji.

Step 2: Initialize a java submodule. Be sure to select the adequate java module for your operating system listed under java/ folder! Includes java3d and is tested to work well with all Fiji components. For MacOSX, get the macosx-java3d only (java is already installed; see the "Utilities - Java Options" to change from 1.5.0 to 1.6.0, for example):

$ cd fiji
$ git submodule update --init java/linux-amd64
Note the init step needs to be done ONLY the very first time.

Step 3: Initialize the submodules for TrakEM2, mpicbg (the image registration library) and ImageJA:

$ cd fiji
$ git submodule update --init TrakEM2 mpicbg ImageJA

Step 4: generate all binary files:

$ ./Build.sh

Step 5: launch Fiji:

$ ./fiji

How to update TrakEM from source code within a Fiji repository

This How To is meant for developers. To update TrakEM2 from within Fiji, just go to menu "Help - Update Fiji".

Step 1: Open a shell and go to Fiji's root directory:

$ cd fiji/

Step 2: Update TrakEM2 git repository:

$ cd TrakEM2/
$ git checkout master
$ git pull
$ cd ..
$ ./Build.sh
You can inspect TrakEM2's repository state with git log and git branch. The git pull command above updates the repository to the latest. The git checkout master above serves the purpose of updating TrakEM2 to the head of the repository, up from the latest submodule commit in fiji (most likely a "(no branch)" situation.)
Note that the ./Build.sh command is issued within the fiji directory, not the TrakEM2 subdirectory.

Step 3: launch fiji with a more appropriate heavy-duty command such as (assuming the computer has more than 10 Gb of RAM):

$ ./fiji -Xms10g -Xmx10g -Xincgc -XX:MaxPermSize=512m -XX:PermSize=512m \
-XX:NewRatio=5 -XX:CMSTriggerRatio=50 -XX:+UseCompressedOops --
See the TrakEM2 wiki page for explanations on the above parameters.
The double dash '--' separates Java virtual machine arguments from ImageJ's arguments, parsed and set properly via the fiji launcher.
Try ./fiji --dry-run to see the JVM instantiation command that fiji uses.

Step 4: Within fiji (Fiji Is Just ImageJ -- batteries included), launch TrakEM2 normally by opening project .xml files, or by going to "File - New - TrakEM2 Project" menu entry.

Last updated: 2010-08-26