Area-based alignment methods, a.k.a. direct methods [3], rely on the following brightness constancy assumption:

where warps the coordinates of ‘s plane. Two methods that are based on this assumption is the Lucas-Kanade algorithm [1] and the Enhanced Correlation Coefficient (ECC) algorithm [2] (see [2] for their differences). In short, both optimize a criterion to find the optimum warp ; the sum of squared differences [1] and the correlation coefficient [2] respectively, between and . IAT includes the forwards additive and inverse compositional version of these algorithms (see the functions iat_LucasKanade, iat_ecc, iat_LucasKanaceIC, iat_eccIC). See [3] for a detailed discussion on this topic (especially Chapters 8,9). In this tutorial, forwards additive version are discussed.

## Matlab examples

Let us load the following pair of images that are related with a Euclidean transform:

Template | Image (to be warped) |

>> tmp = imread('BabbonTemplate.png'); >> img = imread('BabbonImage.png'); |

The parameters of both Lucas-Kanade and ECC are: number of iterations, number of levels for multi-resoluiton, initial warp, type of transformation. When parameters are not defined, default values are used (see the documentation).

>> par.transform = 'euclidean'; >> par.levels = 2; >> par.iterations = 20; %iterations per level |

We choose a two-level execution here that helps the convergence. Notice that for a single-level alignment of the above images, ECC needs more than 70 iterations to converge and Lucas-Kanade even more.

In order to call the algorithms, use the following syntax:

>> LKWarp = iat_LucasKanade(img, tmp, par); >> ECCWarp = iat_ecc(img, tmp, par); |

In what follows, we use `ECCWarp`

to visualise the results, while one can use `LKWarp`

as well.

The above execution returns the warp that must be applied to the image in order to get a profile registered with the template. In order to obtain the warped image, use the function iat_inverse_warping:

>> [M,N] = size(tmp); >> [wimECC, suportECC] = iat_inverse_warping(img, ECCWarp, par.trnasform, 1:N, 1:M); |

Notice that the template grid `[1:M,1:N]`

is projected through `ECCwarp`

into the plane of `img`

, and the mapped area is warped back to obtain an `MxN`

warped image. `supportECC`

is a binary mask that marks the area being warped.

As long as there is no appearance variation, we can visualise the error between the template and the warped image. For example:

>> [~, grayerrorECC] = iat_error2gray(tmp,wimECC,suportECC); >> figure; imshow(uint8(wimECC)); figure;imshow(tmp); figure; imshow(grayerrorECC); |

Template | Warped image | Alignment error (more white = lower error) |

One can easily draws the area of `img`

that corresponds to `tmp`

:

>> iat_warping_area(img, WarpECC, 1:N, 1:M); |

The above command should popup the following image

Image with warping area marked |

The results of Lucas-Kanade algorithm are similar for this image pair. Strictly speaking, the algorithms behave differently, in particular when appearance variation exists, images are noisy etc. [2]. For example, the user can experiment by using the template `BaboonTemplateIntensityChange.png`

which is photometrically distorted.

The inverse-compositional schemes of ECC and Lucas-Kanade algorithms as well as the dual inverse compositional (DIC) algorithm can be used in a similar way.

## Demos – Matlab scripts

IAT provides Matlab scripts (demos) that automatically run alignment and display results.

Relevant IAT demos: areaBasedAlignmentDemo1, areaBasedAlignmentDemo2, areaBasedAlignmentDemo3, areaBasedAlignmentDemo4 (it uses DIC algorithm)

## References

[1] S. Baker, I. Matthews, “Lucas-Kanade 20 years on: A unifying framework Part I”, IJCV, vol.56, no. 3, 2004

[2] G.D.Evangelidis, E.Z.Psarakis, “Parametric Image Alignment using Enhanced Correlation Coefficient”, IEEE Trans. on PAMI, vol.30, no.10, 2008

[3] Richard Szeliski, “Computer Vision, Algorithms and Applications”, Springer, 2010