Automatic Panoramic Image Merging

Paul Haeberli and Eyal Ofek

If you've every tried to make a panorama from a series of photographs, you may have noticed that it's very hard to make the pictures overlap perfectly. This is due to the fact that the film plane gets tilted as the camera is pointed in different directions. Unfortunately, it's not enough to translate, rotate or scale the images to make them merge. You must apply a projective transformation in order for the images to seamlessly overlap.

As an example, let's look at two individual pictures taken in my office:

Using a projective warp, these two pictures can be made to merge perfectly. The edges of the images are blurred to help the pictures merge together. This was done by having a program first find the x and y offset between the two images that has the smallest RMS difference in brightness. Then the image are made to merge into each other using an extension of a technique described by [Irani and Peleg 91]. This gradient following technique determines the projective transformation needed to make the images overlap perfectly.

Some researchers are doing some work with another alignment method [Mann and Picard 94].

It's possible to build up a composite image from a collection of independent images by finding all overlapping areas and warping the images to fit. Here's a composite image that was generated by combining 13 source images:

This image was created without any user interaction. The only input was the set of 13 source images and about 30 minutes of CPU time. To do this, first the program warped and correlated each image to all the remaining images. For each pair of images, a difference value was found to describe the quality of the overlap. A directed graph was then created where each node represented one image, and the distance values were used to create bidirectional edges between the nodes. Finally, the graph was analyzed and the best overlaps were used to create the final composite image.

Inaccuracies in this composite are due to lens distortion in the video camera.

This method can be used to acquire very high resolution images with an inexpensive camcorder. It could also be used to create complete cubical environment maps from videotape.

This software is still under development.

References