Deep Zoom It All

Silverlight’s Deep Zoom, I think, is the best way* to interact with very large pictures over the web.  There are two main components of Deep Zoom.  The first is a file-format.  The second is a software component built into Silverlight that very efficiently handles this file-format.  It turns out that the Deep Zoom component is quite extensible.  This means that Deep Zoom can be used to not only view its native format, but also lots of other mega-images on the web.  To highlight this capability, MSR’s Eric Stollnitz has built a new web-app that lets users preview all of the major large picture formats. 

hdview-preview-annotate

His new HD View Preview page allows you to quickly try out Silverlight on any Deep Zoom, HD View, KML Photo Overlay, or Zoomify content.  For example, if you’ve previously used the Zoomify export in Photoshop, you can now use HD View Preview to quickly try out Silverlight directly on the Zoomify data.  You simply type the url of the Zoomify XML into the address link on the page (see 1 in figure to the right).  You can similarly enter the url of any KML file that contains photooverlay elements.  Eric has provided some quick example links, including some popular Zoomify data, Google Earth’s Gigapxl layer, Gigapan KMLs, and our favorite HD Views (3 in figure).  After trying out Silverlight on your content, if you like what you see, the page includes an embed code (5) so that you can easily embed what you are previewing into your own web-page or blog.  For developers, the source code for the Silverlight viewer being shown here is available on codeplex.

The page also allows you to experiment with our HD View research prototype.  The tabs on the page let you quickly swap between the Silverlight viewer and HD View (2).  HD View is where we continue to experiment with features that we hope to see in Silverlight someday, including high-dynamic range, color management, and turning flickr images into a QTVR-like experience.  As described in a previous post, if you have used Microsoft ICE or Windows Live Photo Gallery to generate a panorama, then you can get a surround video experience by simply pointing HD View at the resulting JPEG.  The HD View Preview page lets you quickly try this out.  To see this in action click on some of the example JPEG links (4).

After trying out Deep Zoom on your own content we think that you will agree that it is the best* way to experience very high resolution images on the web.  Now with the HD View Preview page it is super simple to try it out**.

Matt Uyttendaele

* by best I mean that the progressive download of imagery is near optimal and the application maintains an interactive frame-rate even when full-screen.

** don’t stop with just trying out the viewer, check out the Deep Zoom authoring tools, including  Deep Zoom Composer, the Photoshop Plugin, and command line tools.

Posted in Uncategorized | Leave a comment

Microsoft ICE tutorials

A few quick links to some great tutorials created by Image Composite Editor (ICE) users …

Posted in Uncategorized | Leave a comment

Best Viewed Large

This is the advice I see many people give in their comments when they post ICE panoramas on flickr, for example (1, 2, 3, 4, 5, etc).   I agree, these images are best viewed large, but web browsers do a poor job letting you explore large images.  This, after all, is why we created HD View.  But creating an HD View web page is cumbersome compared to the simplicity of uploading a jpeg to your favorite photo-sharing Web site.  ICE, Windows Live Photo Gallery (WLPG), and HDView have a cool feature that can help.  When you create a panorama in ICE or WLPG, those programs store metadata in the jpeg that describes the result.  For example, if the output is a 360-degree image, this is noted in the metadata.  On the viewing end, HDView interprets this metadata and can then provide a surround-video experience.

Here’s an example from Aleks Clark, one of my favorite ICE-using flickr photogs:

 

Notice that HDView reads the metadata out of this image and understands that it is actually a 360 full wraparound.  And to view it large you can just hit the full screen button on the top right.  All I had to do to embed this in the blog was to insert the following small embed code:

<iframe src=”http://research.microsoft.com/en-us/um/redmond/groups/ivm/HDViewSL/HDIFrame.htm?FileName=http%3A%2F%2Ffarm4.static.flickr.com%2F3394%2F3497207244_74d632775a_o_d.jpg&Viewer=0” style=”width: 400px; height: 300px; margin: 0;” frameborder=”0″ scrolling=”no”></iframe>

To use this, simply replace the URL of the jpg (underlined and in blue above) with your own.  This works with any Web photo service that preserves the original, like SkyDrive, SmugMug, or pro accounts on flickr.  A few caveats.  The first is that, in this case, unlike regular HDView pages, which show something almost immediately (thanks to the multiple resolutions stored on the server), the full jpg has to download before anything is shown.  On my home connection, Aleks’s image took about 17 seconds of buffering.  The second caveat is that unfortunately this feature doesn’t work in Silverlight yet, so it is HDView only for now.  Finally, the image can’t have been edited by another tool after ICE or WLPG, because these are likely to strip the relevant metadata.

So, next time you want to share your uploaded image LARGE, take a few extra minutes and provide an HDView embed on your blog!  If you try this out I’d love to see links in the comments.

-Matt

Posted in Uncategorized | 5 Comments

HD View 3.2 Released

Today we are releasing an update to HD View, HD View SL, and the associated creation tools (hdmake, and Photoshop plugin).  There are plenty of great new features in this update.

The Deep Zoom Format

HD View content had a slightly different format than Silverlight’s Deep Zoom.  As of this update that is no longer the case.  We have standardized on the Deep Zoom format.  This has a bunch of advantages.  The first is that in almost all cases(1) content that you generate for HDView will now also be viewable in the cross platform Silverlight and on your iPhone.  We have also started porting HD View completely to Silverlight, the name of that project is, surprisingly, HD View SL.  The SL version doesn’t have all of the HD View features, but we plan to close this gap in the future.  For now the default web pages that are generated by the updated hdmake and Photoshop plugin will use HD View on Windows and HD View SL on Mac.  This will allow everyone to see your content.  As before you can also use hdmake and the Photoshop plugin to create web pages that uniquely use Silverlight regardless of client platform. 

Eric Stollnitz, the developer working on HD View SL, has created a nice side-by-side comparison page of the two projects as they stand today.  Notice as you pan and zoom in one pane the view is synchronized in the other pane.  This is a good way to see the benefits of the advanced HD View features like tone-mapping and color management (we hope to see these features in Silverlight in the future).  It’s also a good example of how to use the HD View (SL) javascript API.

Embedding

We’ve updated our embed codes so that you can now embed HD View in spaces.live.com, blogger.com, or any other blogging service that accepts <iframe> tags from the live.com domain.  When embedded in a blog your content shows up as described above – using HD View on Windows and SL on Mac.  Here’s an example embed …

http://silverlight.services.live.com/invoke/63450/HDView/iframe.html?FileName=%3Fxmlversion%3D1.0%3Frootimageseturl%3Dhttp://research.microsoft.com/en-us/um/redmond/groups/IVM/HDViewSL/Rainier360_files/l/c_r.jpglevels%3D16width%3D21462height%3D2760projection%3DcylindricalthetaMin%3D0thetaMax%3D360phiMin%3D73.84phiMax%3D116.83rollPitchYaw%3D0.0000000.0000000.000000maxZoom%3D2.000000tileWidth%3D254tileHeight%3D254tileOverlap%3D1//root&FOV=37.9&Yaw=155.671&Pitch=88.977&Viewer=2

For more info on how to use embedding  please see our instructions page.  Included there are detailed instructions for a variety of ways to use HD View on your blog, including how to show KML photo-overlays, Zoomify content, and even Windows Live Photo Gallery panoramas stored on Flickr.

New Toolbar including Full Screen Mode

image

We’ve added a new vertical toolbar in the top-right corner that is common between HD View and HD View SL.  Included in this toolbar are buttons for full-screen mode, reset view, zoom in and zoom out.  The old HD View toolbar is also still present and as always has buttons to control panning mode, tone-mapping mode and lens mode. 

 

 

.zip tile archives

A big pain when generating HD View or Deep Zoom content is the many thousands on tiny image files that get generated.  These are particularly cumbersome when they need to be copied one-by-one up to a web server.  We found that many people were creating zip archives of these tile sets in order to facilitate transfer to their servers.  In the latest tools we’ve added the feature to directly create a zip file for the tiles.  This eliminates the step of having to generate a zip file yourself, and eliminates those thousands of individual files on your disk.  In addition, for local preview(2), HD View can read the tiles directly out of the zip archive.

Matt Uyttendaele

the fine print:

(1) HDR and 16bit images are still HDView only.  Also wide-gamut images will be converted to sRGB by our utilities for cases where HD View SL is selected. 

(2) HD View SL cannot do local preview when the tiles are stored in a zip file.

Posted in Uncategorized | 2 Comments

Astrometry

Last week I was chatting with Jonathan Fay, the architect of Microsoft’s World Wide Telescope (WWT). Over the years Jonathan has been a great friend to the HD View project.  He provided software early on that helped bootstrap HD View.  He also introduced us to the good folks at Meade Telescope, who provided our camera control platform (see pic1, pic2, video).  HD View has certainly benefited from and taken inspiration from work done in the astronomy community.  So I always pay close attention when Jonathan shows me the latest goings-on in astronomy.  Last week he demoed for me the astrometry flickr group.Image of the Jellyfish Nebula, annotated by Astrometry.net robot

At first glance this seems like any other group on flickr.  However, behind the scenes something pretty amazing is happening.  When a photographer posts an image to this group, a program called the blind astrometry server is put to work.  What this program does is automatically determine what part of the sky the image represents.  It works by extracting features, in this case quadruples of stars, from the image and comparing these against similar features extracted from a whole sky survey.  According to a technical description, this comparison runs in less than a second thanks to sophisticated indexing techniques.

Positioning the image enables some cool new features unique to the astrometry group.  The first is that objects in the picture are automatically tagged.  See these examples of the Jellyfish Nebula and the International Space Station flying over Seattle.  Notice that the notes in these pictures indicating star names were all placed there automatically by the blind astrometry server.  The second is that a link is added to allow the image to be browsed within the World Wide Telescope.  This lets users see the image in the context of the entire night sky.  For further reading, I recommend an interview with Christopher Stumm on the code.flickr blog.

So how close are we to a similar service for terrestrial photos?  Even though the universe is a vast place, surprisingly I think that auto-locating earth-based photos is a harder problem.  First there is far more data.  The portion of the Sloan Digital Survey used for flickr astrometry contains about 1 terabyte of imagery.  Last year the VE team added 36 terabytes of data in just one release.  I’m sure that Google Street View similarly contains many terabytes of data.  The second challenge is that there are more degrees of freedom for positioning an image on earth.  For space there are 4 variables to solve when placing an image, 3 for rotation and 1 for scale.  On earth there are at least 7 variables – 3 for position, 3 for rotation and 1 for focal length, and this ignores other terrestrial factors like variation over time and lighting changes.

These difficult problems are slowly being overcome though.  You already see Microsoft and Google systematically surveying the whole earth in a manner similar to the whole sky survey mentioned above.  The computer vision community has developed fast image recognition techniques for non-sky pictures.  There are tools for manually (not yet automatically) placing images within Google Earth or Virtual Earth 3D.  And, the Photo Tourism project (which became Photosynth) demonstrated auto-annotation of flickr images.  Over the coming years I think that we’ll see these technologies packaged up in an easy-to-use web service.

-Matt

Posted in Uncategorized | Leave a comment

Alexa, age 7, creates a panoramic photo

I saw a new “I’m a PC” commercial running during American Idol this week.  I’m pretty biased, but I really like this commercial.  Having been very involved in the development of this technology, I know that under the hood it is quite complex.  What this commercial brings home for me is that through the power of software, we have managed to hide all this complexity from the user and boil it down to, in Alexa’s words, “squishing all my little pictures into one big one.”

 http://images.video.msn.com/flash/soapbox1_1.swf  Video: The Rookies: Alexa, Age 7

And here are some real-world examples of novices creating panoramic photos using Windows Live Photo Gallery:

All of this reminds me of why I work at Microsoft.  While it’s been great to get positive peer reviews on the underlying technology that we develop (for example, see the photography-related publications on Rick Szeliski’s page), what really excites me is putting this technology to use in Microsoft products and enabling millions of people to do something new with their PCs.

-Matt

Posted in Uncategorized | 2 Comments

My Favorite Slide Show of the Inauguration

TVSynth

On Tuesday I blogged about some of the great Photosynth images available through the CNN’s “The Moment” project.  The goal of the project was to have people attending the inauguration send pictures from the crowd and have Photosynth stitch them together.  When you crowd source pictures some cool unexpected things happen.  In this case a lot of people submitted pictures of themselves watching the event on TV.  The result is quite moving.  This isn’t the type of Synth that you explore, rather you should just hit the play button in the top right of the window, sit back, and watch.

http://photosynth.net/silverlight/photosynth.aspx?cid=44fbe543-600c-404f-8ceb-8af763ca9b00

-Matt

Posted in Uncategorized | Leave a comment