↑ Return to Tutorials

Area-based (direct) alignment

Area-based alignment methods, a.k.a. direct methods [3], rely on the following brightness constancy assumption:
I_1(x,y)\sim I_2(T(x,y))
where T(x,y) warps the coordinates of I_2‘s plane. Two methods that are based on this assumption is the Lucas-Kanade algorithm [1] and the Enhanced Correlation Coefficient (ECC) algorithm [2] (see [2] for their differences). In short, both optimize a criterion to find the optimum warp T; the sum of squared differences [1] and the correlation coefficient [2] respectively, between I_1(x,y) and I_2(T(x,y)). IAT includes the forwards additive and inverse compositional version of these algorithms (see the functions iat_LucasKanade, iat_ecc, iat_LucasKanaceIC, iat_eccIC). See [3] for a detailed discussion on this topic (especially Chapters 8,9). In this tutorial, forwards additive version are discussed.

Matlab examples

Let us load the following pair of images that are related with a Euclidean transform:

   
Template Image (to be warped)

 

>> tmp = imread('BabbonTemplate.png');
>> img = imread('BabbonImage.png');

The parameters of both Lucas-Kanade and ECC are: number of iterations, number of levels for multi-resoluiton, initial warp, type of transformation. When parameters are not defined, default values are used (see the documentation).

 

>> par.transform = 'euclidean'; 
>> par.levels = 2;
>> par.iterations = 20; %iterations per level

We choose a two-level execution here that helps the convergence. Notice that for a single-level alignment of the above images, ECC needs more than 70 iterations to converge and Lucas-Kanade even more.

In order to call the algorithms, use the following syntax:

>> LKWarp = iat_LucasKanade(img, tmp, par); 
>> ECCWarp = iat_ecc(img, tmp, par);

In what follows, we use ECCWarp to visualise the results, while one can use LKWarp as well.

The above execution returns the warp that must be applied to the image in order to get a profile registered with the template. In order to obtain the warped image, use the function iat_inverse_warping:

>> [M,N] = size(tmp);
>> [wimECC, suportECC] = iat_inverse_warping(img, ECCWarp, par.trnasform, 1:N, 1:M);

Notice that the template grid [1:M,1:N] is projected through ECCwarp into the plane of img, and the mapped area is warped back to obtain an MxN warped image. supportECC is a binary mask that marks the area being warped.

As long as there is no appearance variation, we can visualise the error between the template and the warped image. For example:

>> [~, grayerrorECC] = iat_error2gray(tmp,wimECC,suportECC);
>> figure; imshow(uint8(wimECC)); figure;imshow(tmp); figure; imshow(grayerrorECC);

 

   
Template Warped image Alignment error
(more white = lower error)

One can easily draws the area of img that corresponds to tmp:

>> iat_warping_area(img, WarpECC, 1:N, 1:M);

The above command should popup the following image

 
Image with warping area marked

The results of Lucas-Kanade algorithm are similar for this image pair. Strictly speaking, the algorithms behave differently, in particular when appearance variation exists, images are noisy etc. [2]. For example, the user can experiment by using the template BaboonTemplateIntensityChange.png which is photometrically distorted.

The inverse-compositional schemes of ECC and Lucas-Kanade algorithms as well as the dual inverse compositional (DIC) algorithm can be used in a similar way.

Demos – Matlab scripts

IAT provides Matlab scripts (demos) that automatically run alignment and display results.

Relevant IAT demos: areaBasedAlignmentDemo1, areaBasedAlignmentDemo2areaBasedAlignmentDemo3, areaBasedAlignmentDemo4 (it uses DIC algorithm)

References

[1] S. Baker, I. Matthews, “Lucas-Kanade 20 years on: A unifying framework Part I”, IJCV, vol.56, no. 3, 2004

[2] G.D.Evangelidis, E.Z.Psarakis, “Parametric Image Alignment using Enhanced Correlation Coefficient”, IEEE Trans. on PAMI, vol.30, no.10, 2008

[3] Richard Szeliski, “Computer Vision, Algorithms and Applications”, Springer, 2010