A Different Tune

Autostitch is an automatic 2D image stitcher. Give it a collection of partly overlapping images, and it will position and blend them into a seamless panorama. A demo version of Autostitch is available for free download. It has a simple GUI which can be used to select input images, and adjust parameters such as output resolution, cropping, and image rotation. However, the GUI interface is a little inconvenient when you have a large number of repetitive stitching tasks to perform, for example : panoramic video.

A panoramic video can be made by stitching each frame of an input video against a background panorama, as described in this earlier post. To do this we need to run thousands of separate stitching operations, each with different sets of images. Its too much to do by hand via the GUI, which is why I wrote this little batch controller for Autostitch.

The batch controller is a simple program that pretends to be a user, sending sequences of events to the GUI to simulate the normal operation of the program:

1. In the Autostitch window, select menu File -> Open.
2. When the open dialog appears, enter a set of image names into the edit control, and then press the "Open" button
3. Wait for Autostitch to finish processing
4. When Autostitch opens the preview window, close it. This makes Autostitch ready to perform the next stitching operation

Fortunately, this is not too hard to do using the Win32 API. FindWindow(NULL, name) gets us a top-level window with a given name (eg. "autostitch" or "Open"). The controls for a window can be recursively enumerated using EnumChildWindows(window, callback, data). We can use our own callback function to search based on the name or class of a control. Windows uses messages to communicate events to controls. We can generate the same messages using SendMessage. For example, to click a button:

  SendMessage(button, BM_CLICK, 0, 0);

or to enter text into an Edit control:

  SendMessage(edit, WM_SETTEXT, 0, (LPARAM) text);

If you want more details, check out the source from the download links below.

Usage

To use the batch controller, you need to first start Autostitch in the usual way. Autostitch must be "pointing" at the directory containing your source images. You can set this using the "File->Open" menu. Just select one or more images and press "Open" - it doesn't matter whether the result is successful or not. From the command-line, you can then stitch any set of images from that directory like this:

  stitch file1.jpg file2.jpg file3.jpg

This will stitch the images as usual, opening the result "pano.jpg" in the default image previewer. To run multiple stitches in sequence, the preview window must be closed before the next stitch can start. Request this by specifying -wait -close. For example:

  stitch -wait -close file1.jpg file2.jpg file3.jpg

To stitch a sequence of images, use the -template option, like this:

  stitch -template frame%03d.jpg 20 50 file1.jpg file2.jpg file3.jpg

This specifies a file name template (using sprintf syntax), and a range of file numbers. The example generates a sequence of names "frame020.jpg", "frame021.jpg", ..., "frame050.jpg". Each stitch operation combines one of the sequence names with the remaining arguments, and renames the output panorama (by default, to "out0020.jpg" ... "out0050.jpg"). Template mode automatically implies -wait -close.

Here are some simple examples using the Autostitch demo images:

  stitch -wait -close 100-0023_img.jpg 100-0024_img.jpg 100-0025_img.jpg

This stitches the bottom 3 images of the "mountain" panorama.

  stitch -path s:/stewart/programs/autostitch/images/test/ 
             -template 100-%04d_img.jpg 38 40 
             100-0023_img.jpg 100-0024_img.jpg 100-0025_img.jpg

This stitches each of the top three images (38..40) against the bottom three images, producing three outputs, "out0038.jpg" .. "out0040.jpg". (change the path to reflect your Autostitch installation).

There are a bunch of settings that you might need to change. For example, the system looks for the preview window using one of the standard applications ("pano.jpg - Windows Picture and Fax Viewer" or "Microsoft Photo Editor - [pano.jpg]"). If you have a different preview app, you'll need to change the title of the preview window using the -output option.

In "template" mode, the system renames each "pano.jpg" to a different file name using the sequence number, but to find the resulting file you need to specify the result directory using -path. Alternatively, run the stitch command from the directory containing the source images.

Options:
-help      Show help message and exit
-version   Show version and exit
-wait      Wait for Autostitch to complete (single stitch only)
-close     Close preview opened by Autostitch (single stitch only)
-template  temp start end Specify template for batch operation
           A range of images is processed for start <= id <= end. Use sprintf
           formatting (eg. id=23, temp='file%04d.jpg' -> 'file0023.jpg')
-dryrun    Just output batch image lists without calling Autostitch
-path      Path to image directory (for batch stitch). INCLUDE TRAILING '/'

Stuff you probably don't need to change:
-window    Name of Autostitch window (default: autostitch)
-status    Name of Autostitch status window (default: Status)
-output    Name of preview window (default: 'pano.jpg - Windows Picture and Fax Viewer')
-fileOpen  Menu name for File->Open dialog (default: 'File:Open')
-openTime  Delay (in ms) for open dialog to populate controls (default: 500)
-closeTime Delay (in ms) to wait before closing preview window (default: 2000)
-outjpg    Template (dir,id) for output files (default: '%sout%04d.jpg')
-outtxt    Template (dir,id) for output files (default: '%sout%04d.txt')

Download

• Source code and Windows binary (gzipped tar file, 10K)
• C source on-line

This app is fairly minimalist, but it should not be hard to modify if required. I used mingw + gcc which are available for free from mingw.org.

Have fun! And if you manage to create something interesting, please post a comment.

Check here for more fun with Autostitch.

Autostitch is "the world's first fully-automatic 2D image stitcher". Developed by Matthew Brown at the University of British Columbia, Autostitch can automatically generate panoramas from a collection of overlapping images.

Autostitch uses a key-point detector known as SIFT (Scale Invariant Feature Transform) to locate interesting features in images. By finding the correspondence between key-points in different images, Autostitch can tell how the images are arranged with respect to each other, allowing them to be rendered into a seamless panorama. Scale invariance means that the features still look the same under scaling (rotation, translation, zoom, etc) which makes them robust to typical camera motion.

Click on the following images for a detailed version:

Swan River shore, Bicton, Western Australia

Playground, Palm Beach, Queensland

Lookout near Byron Bay

View of Tweed Valley from Montville

Sugar mill at Tully, view from atop the Golden Gumboot

Autostitch is clever, but its also fun! After playing with it a bit I got thinking. If you can build a panorama automatically, then you can "embed" any image taken from the same position into the panorama. This could be quite interesting as a creative tool, but also has applications in tools like media browsers where the relationships between images could be useful for organising and navigating media.

Check these posts for some ideas about how to use this:

• Embed images into a panorama
• Panorama Montage
• Panoramic Video
• A Batch Controller for Autostitch

In a previous experiment, we found that images can be embedded into a background panorama. Now its time to see if we can use this to make a stitched panoramic movie.

Step 1. Record a background panorama

I made a movie of my lounge room using a Panasonic NS-DV50A digital video (DV) camera. I captured this to DV-AVI format, then used VirtualDub to de-interlace and convert frames to JPEG images.

I took a couple of overlapping pans around the room. Nothing fancy here. Just think "painting with camera". Next I selected every 10th frame and made the background panorama using Autostitch:

Step 2. Record some action

Next, I took some video of the kids playing in the room. Again, I de-interlaced and converted the frames to JPEG images. You can see this video in the output below.

Step 3. Stitch frames against panorama

Here's the tricky part. We want to take each frame of the video, and stitch it together with all of the frames for the background panorama. This will generate one panorama per input frame, which we will then combine to produce the output.

The catch is that Autostitch is a GUI application. Doing this by hand using the GUI is way too time consuming, so I wrote some black-magic code to control Autostitch via a batch process.

This produces images with a fixed width but variable height. I resized these to a consistent size.

Step 4. Convert panoramas to movie

Next, I loaded the panorama image sequence into VirtualDub and converted it to a movie file. Additionally, I combined the panoramic image with the original movie, and exported these frames as a movie.

Results

Here's the stitched panoramic movie along side the input movie:

I tried to stretch the capabilities of the matching by doing some zooms and rotations with the camera. Towards the end of the movie is a section where the matching breaks down because there is probably too much transformation of the image. Although the SIFT should be able to handle this, there may be orientation-dependent differences in the characteristics of the features that cause it problems. This is due to the de-interlacing of the original image, which means that scan lines are duplicated so the detail in the horizontal and vertical directions are not the same.

Overall, I'm pretty happy with the result. Its not perfect, and could probably be improved with some specific support for this sort of thing. We are asking Autostitch to do a few things it was not designed to do. Basically, we don't want to re-stitch the entire panorama for each new foreground frame. We want the background to be stable, but can be more lenient when stitching the foreground frame. Also, with specific knowledge about which frame should go on top, it should be possible to smoothly blend the frames without losing the foreground.

Being able to embed video into a panorama suggests interesting possibilities for immersive media browsing environments based on 3D panoramas. See: "Browsing personal media archives with spatial context using panoramas".

Previously, we saw that subjects can be stitched into a background panorama. An extension of this idea is to stitch multiple subjects, or subjects in different positions into a single panorama.

Basically, the steps for doing this are the same except that we are now stitching more than one frame against the background panorama. The only real catch here is to make sure that the stitched frames don't overlap too much or they will obscure each other.

With some care, you can get a result like this. The panorama is generated automatically by Autostitch. I added a small white border around each image. Then I added the strip at the bottom to show the source images and the links to the embedded frames.

The ability to do this sort of thing offers interesting possibilities for media browsing. If we can automatically relate photos to the spaces they are taken in, this might be a useful way of organising collections. See: "Browsing personal media archives with spatial context using panoramas".

We know that Autostitch is fun for building panoramas. After playing with it a bit I got thinking. If you can build a panorama automatically, then you can "embed" any image taken from the same position into the panorama. This could be used creatively to build montages, or even panoramic video.

Does it work? Lets find out...
Continue reading ‘Fun with Autostitch : Panorama Embedding’ »