Deep Zoom is a terrific new feature that is available in Silverlight 2.0. The best way to describe Deep Zoom is that it allows Silverlight to natively load arbitrarily large images. To get an idea of what Deep Zoom can do, please take a look at the initial results of the Yosemite Extreme Panoramic Imaging Project.
But, what I really want to talk about today is the Deep Zoom authoring tool – Deep Zoom Composer (DZC). The latest beta of DZC offers a feature that my group developed – automatic image stitching. While we’ve shipped this in the past in other products like Windows Live Photo Gallery, this version of the stitcher has some significant improvements. The first improvement is a better technique for automatically finding matching control points between images. The second is better blending between images, especially when there are significant exposure differences.
Control points are very familiar to anyone who has used manually operated stitching software. For those who haven’t used such software, the basic work flow is that you open one image, click on a point in that image that you think will likely appear in another overlapping image, then open that other image and click on the corresponding point. You do this over and over again until all images that comprise your panorama have points matching into other images. Recently computer vision algorithms have been invented that can automatically perform this function. The most well known of these algorithms is the Scale-invariant feature transform (SIFT). SIFT works so well that it has largely replaced manual control point editing and there are several automatic stitchers available, such as Autostitch, that take advantage of SIFT.
SIFT however isn’t perfect at matching images and sometimes misses matches or generates erroneous matches. Last year, Simon Winder and Matthew Brown, two researchers in our group, tackled the problem of designing a better matching technique and published their work in a paper titled Learning Local Image Descriptors. This paper is meant for a technical audience, so I will try to summarize the findings here. SIFT was designed by smart people testing different ideas on a large number of test images. The new technique instead uses a much much larger number of test images (produced by the Photosynth pipeline), and uses mathematical optimization techniques to automatically ‘learn’ a new control point matcher. The net result is that the new technique produces 1/3 the number of false matches when compared to SIFT on our test data. And ultimately for users of DZC this will lead to a higher success rate for your stitching projects.
It is personally very exciting for me to see Microsoft Research (MSR) results like these moving from academic journals to actual usable products so quickly. The MSR computer vision researchers have long been active in academic circles, but now are finding more and more outlets in Microsoft products as well. There are other examples such as Photosynth and HD View, where preview software was available very near to technical publication(1, 2). I look forward to this trend continuing.
That said, please give the new Deep Zoom Composer a spin. You can learn more about it and download it from:
Note DZC is still Beta software and there are a few rough edges such as limits on output size.
(1) Photosynth was published at Siggraph 2006 as "Photo tourism: Exploring photo collections in 3D" in August and made available publicly in November.