# Area-based (direct) alignment

Area-based alignment methods, a.k.a. direct methods [3], rely on the following brightness constancy assumption:
$I_1(x,y)\sim I_2(T(x,y))$
where $T(x,y)$ warps the coordinates of $I_2$‘s plane. Two methods that are based on this assumption is the Lucas-Kanade algorithm [1] and the Enhanced Correlation Coefficient (ECC) algorithm [2] (see [2] for their differences). In short, both optimize a criterion to find the optimum warp $T$; the sum of squared differences [1] and the correlation coefficient [2] respectively, between $I_1(x,y)$ and $I_2(T(x,y))$. IAT includes the forwards additive and inverse compositional version of these algorithms (see the functions iat_LucasKanade, iat_ecc, iat_LucasKanaceIC, iat_eccIC). See [3] for a detailed discussion on this topic (especially Chapters 8,9). In this tutorial, forwards additive version are discussed.

## Matlab examples

Let us load the following pair of images that are related with a Euclidean transform:

 Template Image (to be warped)

 ```>> tmp = imread('BabbonTemplate.png'); >> img = imread('BabbonImage.png');```

The parameters of both Lucas-Kanade and ECC are: number of iterations, number of levels for multi-resoluiton, initial warp, type of transformation. When parameters are not defined, default values are used (see the documentation).

 ```>> par.transform = 'euclidean'; >> par.levels = 2; >> par.iterations = 20; %iterations per level```

We choose a two-level execution here that helps the convergence. Notice that for a single-level alignment of the above images, ECC needs more than 70 iterations to converge and Lucas-Kanade even more.

In order to call the algorithms, use the following syntax:

 ```>> LKWarp = iat_LucasKanade(img, tmp, par);  >> ECCWarp = iat_ecc(img, tmp, par);```

In what follows, we use `ECCWarp` to visualise the results, while one can use `LKWarp` as well.

The above execution returns the warp that must be applied to the image in order to get a profile registered with the template. In order to obtain the warped image, use the function iat_inverse_warping:

 ```>> [M,N] = size(tmp); >> [wimECC, suportECC] = iat_inverse_warping(img, ECCWarp, par.trnasform, 1:N, 1:M);```

Notice that the template grid `[1:M,1:N]` is projected through `ECCwarp` into the plane of `img`, and the mapped area is warped back to obtain an `MxN` warped image. `supportECC` is a binary mask that marks the area being warped.

As long as there is no appearance variation, we can visualise the error between the template and the warped image. For example:

 ```>> [~, grayerrorECC] = iat_error2gray(tmp,wimECC,suportECC); >> figure; imshow(uint8(wimECC)); figure;imshow(tmp); figure; imshow(grayerrorECC);```

 Template Warped image Alignment error (more white = lower error)

One can easily draws the area of `img` that corresponds to `tmp`:

 `>> iat_warping_area(img, WarpECC, 1:N, 1:M);`

The above command should popup the following image

 Image with warping area marked

The results of Lucas-Kanade algorithm are similar for this image pair. Strictly speaking, the algorithms behave differently, in particular when appearance variation exists, images are noisy etc. [2]. For example, the user can experiment by using the template `BaboonTemplateIntensityChange.png` which is photometrically distorted.

The inverse-compositional schemes of ECC and Lucas-Kanade algorithms as well as the dual inverse compositional (DIC) algorithm can be used in a similar way.

## Demos – Matlab scripts

IAT provides Matlab scripts (demos) that automatically run alignment and display results.

Relevant IAT demos: areaBasedAlignmentDemo1, areaBasedAlignmentDemo2areaBasedAlignmentDemo3, areaBasedAlignmentDemo4 (it uses DIC algorithm)

## References

[1] S. Baker, I. Matthews, “Lucas-Kanade 20 years on: A unifying framework Part I”, IJCV, vol.56, no. 3, 2004

[2] G.D.Evangelidis, E.Z.Psarakis, “Parametric Image Alignment using Enhanced Correlation Coefficient”, IEEE Trans. on PAMI, vol.30, no.10, 2008

[3] Richard Szeliski, “Computer Vision, Algorithms and Applications”, Springer, 2010