Fun with Autostitch : Panorama Embedding

20 May 2006

Autostitch is fun for building panoramas. After playing with it a bit I got thinking. If you can build a panorama automatically, then you can "embed" any image taken from the same position into the panorama. This could be used creatively to build montages, or even panoramic video.

Aim

To see if we can "embed" images of a subject into a background panorama of the surroundings. First, we'll make the panorama. Then, we'll image the subject at different locations and from slightly different view-points to see how well the matching works.

Step 1 : Make a Panorama

First, I took a series of images with a digital camera to produce a panoramic background image of my lounge room. The panorama is generated from 30 separate overlapping images. With blending, it looks like this:

background panorama generated using Autostitch

To get good matching, you need at least 30% overlap between images..

Step 2: Take images in various positions, then stitch against background

I moved Dorothy the Dinosaur around the room, photographing her under different conditions. I varied the focal length (zoom) and orientation from the original panorama view-point. Where the variation is small, we expect the test frame to map into the panorama. I used AutoStitch to combine the images, turning off blending in the output images. This means that the images are simply tiled in the output according to processing order. By adjusting the image order, you can usually make the target image appear on top. Unfortunately, the order seems to be arbitrary but reordering the images does make a difference.

Results

These images show a shot which is taken from different places in the room (left), and the result of stitching this shot with the frames for the background panorama (right). These images are scaled down to fit the page. Click on either image to see the full-sized panorama (1400x270 pixels).

Shots are tagged "OK" or "Fail" according to whether the shot matches the background. If you can see the subject in the panorama, it worked OK. 9 out of 11 (80%) of the shots matched well.

subject embedded into panorama using Autostitch shot1 - OK
subject embedded into panorama using Autostitch shot2 - OK
A long shot, and close shot in the blue chair work fine.

subject embedded into panorama using Autostitch shot3 - OK
subject embedded into panorama using Autostitch shot4 - OK
Dorothy is at the piano. These shots are slightly to the right and to the left of the original viewing position. Still works well, though the output looks a little wierd.

subject embedded into panorama using Autostitch shot5 - OK
subject embedded into panorama using Autostitch shot6 - OK
subject embedded into panorama using Autostitch shot7 - OK
Dorothy is on the sofa. A series of longer shots. Notice that in the longer shots, there is more distortion in the projected shot. In the last example, the result is quite 'torn' around the sofa area.

subject embedded into panorama using Autostitch shot8 - FAIL
Still on the sofa. This view is close, and from a lower view-point. Fails to match.

subject embedded into panorama using Autostitch shot9 - OK
Close shot down low. OK this time.

subject embedded into panorama using Autostitch shot10 - OK
Dorothy is on the couch again, this time later in the day under different lighting. Matches well.

subject embedded into panorama using Autostitch shot11 - FAIL
Close shot down low. Fails to match. Possibly, the view angle is too high, and/or the lighting too different from the background.

Conclusion

Overall it seems to work fairly well. As expected, there are problems when the camera moves or lighting changes. Check these posts for some ideas about how to use this: