Automatic Image Stitching in Deep Zoom Composer

deep zoom composer

Deep Zoom is a terrific new feature that is available in Silverlight 2.0.  The best way to describe Deep Zoom is that it allows Silverlight to natively load arbitrarily large images.  To get an idea of what Deep Zoom can do, please take a look at the initial results of the Yosemite Extreme Panoramic Imaging Project.

But, what I really want to talk about today is the Deep Zoom authoring tool – Deep Zoom Composer (DZC).  The latest beta of DZC offers a feature that my group developed – automatic image stitching.  While we’ve shipped this in the past in other products like Windows Live Photo Gallery, this version of the stitcher has some significant improvements.  The first improvement is a better technique for automatically finding matching control points between images.  The second is better blending between images, especially when there are significant exposure differences.

Control points are very familiar to anyone who has used manually operated stitching software.  For those who haven’t used such software, the basic work flow is that you open one image, click on a point in that image that you think will likely appear in another overlapping image, then open that other image and click on the corresponding point.  You do this over and over again until all images that comprise your panorama have points matching into other images.  Recently computer vision algorithms have been invented that can automatically perform this function.  The most well known of these algorithms is the Scale-invariant feature transform (SIFT).  SIFT works so well that it has largely replaced manual control point editing and there are several automatic stitchers available, such as Autostitch, that take advantage of SIFT.

SIFT however isn’t perfect at matching images and sometimes misses matches or generates erroneous matches.  Last year, Simon Winder and Matthew Brown, two researchers in our group, tackled the problem of designing a better matching technique and published their work in a paper titled Learning Local Image Descriptors.  This paper is meant for a technical audience, so I will try to summarize the findings here.  SIFT was designed by smart people testing different ideas on a large number of test images.  The new technique instead uses a much much larger number of test images (produced by the Photosynth pipeline), and uses mathematical optimization techniques to automatically ‘learn’ a new control point matcher.  The net result is that the new technique produces 1/3 the number of false matches when compared to SIFT on our test data.  And ultimately for users of DZC this will lead to a higher success rate for your stitching projects.

It is personally very exciting for me to see Microsoft Research (MSR) results like these moving from academic journals to actual usable products so quickly.  The MSR computer vision researchers have long been active in academic circles, but now are finding more and more outlets in Microsoft products as well.  There are other examples such as Photosynth and HD View, where preview software was available very near to technical publication(1, 2).  I look forward to this trend continuing.

That said, please give the new Deep Zoom Composer a spin.  You can learn more about it and download it from:

http://blogs.msdn.com/expression/archive/2008/08/01/download-the-new-deep-zoom-composer-preview.aspx 

Note DZC is still Beta software and there are a few rough edges such as limits on output size.

-Matt Uyttendaele

 

(1) Photosynth was published at Siggraph 2006 as "Photo tourism: Exploring photo collections in 3D" in August and made available publicly in November.

(2) HD View was published at Siggraph 2007 as "Capturing and Viewing Gigapixel Images" and made available in conjunction with the conference.

This entry was posted in Uncategorized. Bookmark the permalink.

6 Responses to Automatic Image Stitching in Deep Zoom Composer

  1. Paul says:

    Sorry, but I can\’t see improved controlpoint stitching with the version I just downloaded today.5 pics are stitched worse together tahn with WL Photogallery.Is the improved  DZC version not yet uploaded?DZC "About" does not show a version, only 3.Aug 2008RegardsPaul

  2. Matt says:

    Yes the improved DZC is on the site I linked to.  Sorry to hear that it didn\’t work for you.  Over our test suite of many 100s of examples (sent in by users like you) this new code performs better than all previous versions.  If you\’d like us to take a look at your example, please post the source material somewhere that we can access it.-Matt  

  3. Paul says:

    @mattthanks, after a little jump and run adventure i found the right click to stitchon some pics stitching works fine, on others not (DZC aborts), on some it does not stitch all selected picsall tested pics have min. 30% overlap and are taken by a robot on a tripodi will upload some for youimho to improve in DZC:- show pics in exif-orientation- make preview- and crop windows sizeable-Paul

  4. Paul says:

    @matti uploaded example pics to http://hdview.spaces.live.com/blog/cns!1AD33AA162CE96C2!656.entry?action=post&wa=wsignin1.0is there a way to put them on live spaces in original resolution?thanksPaul

  5. Matt says:

    Hi Paul,  I would suggest putting them on a skydrive in their original resolution.-Matt

  6. Paul says:

    Hi Matt 13 Pics are on skydrive nowhttp://cid-bbae3e5b9e102512.skydrive.live.comPaul

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s