I emphasize high-quality and free because ICE has in fact supported RAW images since last year, when I blogged about it. RAW support in Windows, and thus ICE, was available via a “plug-in.” Windows allows any third-party software vendor to extend which image types are supported in the operating system. Based on my own observations of ICE users, the “plug-in” strategy confused many. 

One source of these plug-ins is the camera manufacturers, like Canon and Nikon.  Unfortunately, they mostly do such a poor job developing and supporting their software that it made Windows, and other applications that relied on it, look bad. We have updated our codecs page to reflect this and strongly discourage anyone from installing the Canon or Nikon software. Another source is vendors who sell high-quality and well-supported plug-ins that cost $15 and up. However, many of our users felt that they shouldn’t need to pay for a feature that they expect to be built into a modern operating system. Mostly though, users simply assumed RAW wasn’t supported in Windows. Judging by the reaction from tech journalists (see “Windows Finally Gets RAW Image Support” 1, 2, 3) this notion was pretty widespread.

We have updated the WIC Codecs for Image Composite Editor page and added the new Microsoft codec pack to our recommended list. We also still recommend Fast Picture Viewer (FPV) for RAW files that aren’t supported by the Microsoft codec pack. Additionally, FPV is recommended if you have a need to use less common file formats, such as HDR and OpenEXR, in ICE.

The new Microsoft codec pack is available here: http://www.microsoft.com/download/en/details.aspx?id=26829. There is some great technology built in. The Windows Live team has built a sophisticated lab for characterizing cameras to ensure high-quality output from the RAW decoding process, and the RAW codec is built to take advantage of multi-core PCs, so it should be very fast.

-Matt Uyttendaele

We have a new Microsoft ICE specific twitter feed:  http://twitter.com/#!/MicrosoftICE  Please follow us to hear the latest ICE related goings on.

The ICE forum has moved.  It is now at: http://social.microsoft.com/Forums/en-US/ice/threads

A minor update of ICE is available at: http://research.microsoft.com/en-us/um/redmond/groups/ivm/ice/  Current users should be prompted to update when starting ICE.

Of course, you can upload to Photosynth.net to get a full immersive viewing experience, but you can also upload a flat version of the image directly to Facebook.  In either case this app allows you to share your panorama just moments after capturing it.

The app nicely complements the rest of the panorama creation tools available for Photosynth.  At the high end, we offer the Photoshop plugin, which lets you create and edit a panorama with whatever tools you want.  For a more automated solution on the PC, we offer Microsoft ICE with direct uploads to Photosynth.  And now for the mobile phone, we’re making this app available.  In addition to creating panoramas, the app is also a great way to view them.  For any panorama that you’ve created using any of the tools above, you can now use the app to view them on the iPhone as a fully interactive surround-video experience.

The Photosynth app is available now for iOS (iPhone 4, iPhone 3GS, iPod Touch 4th generation, and iPad 2) in the iTunes App Store.

Matt Uyttendaele

After installing, you will find some exciting new features.

Stitching From Video

The first is that ICE can now automatically stitch a panorama directly from video.  One fun use of video panoramas is “motion summaries,” like this result that ICE produced:

In this video the photographer was panning the camera to follow the motion of the snowboarder.  I used ICE to indicate “Start” and “End” points, and I gave a few hints about which video frames were interesting. The motion tracking and final composition was then done automatically by ICE.  You can access this feature by selecting “New Video Panorama” from the File menu.  This will bring up the Video Panorama dialog (shown to the right), where you can play or single-step through videos in order to choose start and end points. You can also optionally draw regions of interest on individual video frames to ensure that certain elements are present in the final composition. ICE supports most common video formats (avi, mov, wmv, and more).  Of course the results can also be uploaded to Photosynth. Note that this particular feature is only available for ICE running on Windows 7 (for other versions of Windows this menu item will be disabled).

Video is also a great way to quickly capture a panoramic scene.  For example, here a two-minute video was automatically turned into a panorama (this was shot with a Samsung Focus Windows Phone).

Automatic Vignette Correction

Lens vignetting is the effect of the center of a photograph being brighter than the periphery. This effect can be especially apparent when stitching photos. For example, notice the repetitive pattern in the sky in the above panorama.  This is caused by lens vignetting in the constituent images. In this ICE update, lens vignetting is automatically detected and removed.  The result is shown below.

Improved Blending

The new version of ICE has a new blending engine. Previous versions did blending by applying a color adjustment to the panorama source images such that there would be minimal visible differences between them. In some cases, this strategy couldn’t adequately hide the seams. For example, in the panorama below, the seam in the  moving water could not be completely hidden by color adjustment alone. In the new version of ICE we still run the color adjustment step, but where this is inadequate, we also do an intelligent fade between the images. This helps a lot in cases like the one below as can be seen in the two breakout panes.  Click on the image below to explore it in full resolution on the Photosynth website; you should find it very difficult to discover the seams between images.

And More

In addition to the above main features, we have also made a few other enhancements:

• An options dialog to control scratch disk location and ICE memory use.
• Enable perspective projection for wide field-of-view panoramas
• 1/3 less disk usage when stitching large data sets
• ICE is now more robust to corrupt metadata in source photos

Matt Uyttendaele ICE lead

with video stitching by: Neel Joshi,  Eric Stollnitz, Paramjit Sandhu; lens vignette removal by: Sing Bing Kang, Howard Good; and improved blending by: Rick Szeliski, Drew Steedly

Photosynth added the panorama feature back in March, but up until now, Microsoft ICE was the only way to generate panoramas.  A few clever people figured out how to bypass this restriction (see users xRez_Studio and OrasulMemorabil), but their solution takes a lot of effort. With the Photoshop plug-in, you can now use any stitching program (AutoPano, PTGui, Hugin, ICE, GigaPan Stitch, Photoshop, etc.), edit the results in Photoshop, and simply select “Export” from the Photoshop file menu to upload full-resolution, interactive panoramas to Photosynth.

Under most circumstances when using the plug-in, you will see the following dialog box displayed. This is necessary in order for you to indicate the properties of your panorama. At a minimum, you will need to enter the “Projection” and “Field of View” of your image (see list of available projections in second screenshot). For some panoramas, the horizon is not exactly in the center of the image.  In this case, you will need to uncheck “Center Image” and indicate the “Position of Horizon” (for example, 0 = top of image, 0.25 = one quarter of the way down from top, 1 = bottom of image).  Once you’ve done this, simply click “Publish,” and in a short time, your panorama will be available for the world to see on the Photosynth website.

If you used AutoPano or ICE to stitch your panorama and you didn’t somehow change the size of the panorama (for example, by cropping), then it’s even simpler.  You will not see this dialog box, and the process will continue directly to uploading. This is thanks to the metadata that those two programs store in panoramas that they create. All of the necessary information is already stored in the image.

One final feature to highlight is that the plugin also lets you create an immersive Photosynth panorama without stitching at all. The last projection type in the list above is ‘Fisheye’. This lets you upload photos shot with a fisheye lens.  The plug-in will undo the fisheye distortion and present the result as a wide field of view interactive experience. To demonstrate this I walked around Seattle’s University District Farmers Market and captured some of my favorite vendors. These were all shot with a Canon 1Ds Mark III and Sigma 8mm lens.

I also want to thank a few people who have tested a prerelease version of the plug-in and provided valuable early feedback:

• Larry Robinson stitched in PTGui, then used Photoshop to create this beautiful annotated panorama of Mt. Rainier.
• Bernhard Vogl used the plug-in to upload a variety of full 360s and gigapixel images

Please use the following links for more information:

• Download
• Forum – feedback, report problems, discuss, etc.
• Plugin website

The Photosynth Export Plug-in team – Matt Uyttendaele, Howard Good

An update to ICE is available at: http://research.microsoft.com/en-us/um/redmond/groups/ivm/ice/

Most of the bug fixes and changes in this version were based on feedback from our users:

• Photoshop layers are back as an output option. We have also added support for the Adobe Large Document Format (PSB), so there is effectively no longer a resolution limit when exporting to Photoshop.  Finally we’ve given users more control over the compatibility mode for PSD/PSB.
• We’ve fixed most of the top issues reported on the forum. Thanks to everyone who shared their problematic photos with us.
• We’ve enabled more of the stitching process to use multi-core, so your stitching projects should now run even faster.
• The ‘jaggies’ on the panorama borders are gone.
• We’ve added support for WIC codecs installed on your PC. This means that if you’ve installed a raw codec from your camera manufacturer, then ICE can directly stitch your raw images. I’m excited about this feature, but your mileage may vary depending on what codec you are using.  Most of my personal raw files are Canon CR2. So, I initially tried Canon’s raw decoder. Unfortunately it is very slow to open files, and because Canon doesn’t provide a 64-bit decoder, users of 64-bit ICE can’t open CR2s. I switched to using the FastPictureViewer Codec Pack and so far that has worked very well. Below is a table of raw codecs that we will try to keep up to date on the ICE web site.  We haven’t tested all of these with ICE, so it would be interesting to get feedback on your own experiences with ICE plus raw.

We’ve just released a long overdue update to the HD View Utilities.  This package includes the hdmake command line utility and the HD View Photoshop Plugin.  The important changes in the 1.11 update are:

• Fixes export issues on Windows 7/Vista SP2.
• Support for 64bit version of Photoshop
• A 64bit version of hdmake
• Multi-core processing
• Specification of source imagery in hdmake via a text file as requested on the forum by scrambler.  See the help page for the srcgridfile option.
• The Photoshop plugin and hdmake support Autopano and ICE panorama metadata.

I’ve mentioned on this blog in the past that Windows Live Photo Gallery (WLPG) and Microsoft ICE stores custom metadata that describes the projection  surface and field-of-view of a panorama.  Unfortunately there is no industry standard for storing this information in image files.  It turns out that Autopano also stores this information (in a different custom tag) within rendered panoramas.  When post processing panoramas (e.g in Photoshop) that are destined for an interactive viewer it is very useful to maintain this metadata.  So, in the latest Photoshop plugin if your source imagery contains Autopano, ICE, or WLPG metadata then you will see the “Metadata Projection/Extents found” message appear in the plugin.  This will then be used to auto-populate the fields in the ‘Projection’ section of the plugin.

If you perform any resizing operation in Photoshop, such as crop or scaling, then these values will no longer be valid and the message will change to “… Projection Extents don’t match image size…”.  In this case you’ll be left to fill the boxes in manually.

The metadata workflow is a test for the next major feature that we release, which is “Upload to Photosynth” from Photoshop.  It is important to have the extents correct for the Photosynth viewing experience. It would be great to get feedback, in the comments below or on the forum, about how best to maintain panorama projection and FOV information through post-processing.

The 64bit version installer is available here and 32bit here.

The HD View Team – Howard Good, Matt Uyttendaele

We are super-excited to announce a new release of Microsoft ICE.  A lot of hard work went into this update of ICE, and it has some great new features, which I’ll talk about shortly.  But, the most exciting thing — the thing that’s  probably going to make you stop reading and hit this download link — is that ICE now has a “Publish to Photosynth” button.  This lets you share your full-resolution ICE panoramas on the web.

Photosynth

Photosynth is Microsoft’s unique technology for reconstructing 3D spaces from a collection of photos.  These 3D reconstructions (known as synths) will continue to be an integral part of the Photosynth experience.  As of today, panoramas will also become integral to Photosynth.  Check out the Photosynth team blog for details on this new feature.

For ICE users, what this means is that you can now share your full-resolution panoramic photos on the web.  The Photosynth web site has been enhanced to let users interact with massive, fully immersive panoramas.  Photosynth uses intelligent streaming technology that allows for smooth pan and zoom of even the largest images.  This is a huge step forward compared to how most ICE users have been sharing their panoramic creations on the web.  The best way to understand this is to check out a few examples uploaded from ICE this morning.

Maps

Often the inspiration for capturing a panorama is the photographer’s desire to document a special place on Earth — be it the view from your office, a sunrise over Mt. Rainier, or your climbing gym.  Photosynth has some great features (see screenshot to the right) to help you share your unique knowledge of such places.  You can “Geotag” where your panorama was taken, and what direction you were looking.  You can add “General” tags and a text description to your photos.  Under “Highlights”, you can annotate all the details, big and small, within your images — for example, see the highlights on this 4-gigapixel panorama of Yosemite Valley.  ICE panoramas with geotags will be visible on Bing Maps alongside traditional synths. The Photosynth team is working with Bing Maps to make this happen soon.

Gigapixels

At Microsoft Research we’ve been working on creating super high resolution images for a few years now.  Some of this technology was in the first version of ICE, but it wasn’t really possible to create multi-gigapixel images with ICE — until now!  We’ve added a new type of project in ICE called a structured panorama.  This is accessed via the “File: New Structured Panorama” menu, which allows you to give ICE some hints about the order of your input images.  The ICE output size is now effectively unlimited – for example see this 10gigapixel result.  It is an ideal way to stitch images captured by robotic capture devices that are now available to consumers, like the Gigapan unit.  It’s been a kick for us to go back and use ICE on the early imagery that we captured when we were inventing this technology.  What took weeks of labor to process back then is now automatically handled by ICE — from import, through seamless composition, to upload to the web.  Those weeks of effort are now down to hours, and in some cases minutes if you have a fast multi-core computer.

Multi-core

Many of the processing-intensive tasks* within ICE have been enhanced to effectively use multi-core PCs.  Below is a screenshot of a computer’s CPU Usage while ICE is processing a Photosynth upload.  This particular computer has 8 cores, represented by the 8 graphs.  Ideally, what we want to see here is these graphs pegged at the top.  This means all the CPUs are working together to finish a job much faster.  While 8-core machines are not yet common, most new consumer PCs do have 4 cores.  If you’ve bought a new PC recently, you should now see ICE work up to 4X faster!

And More …

That’s all of the new stuff we’ll mention for now. Actually, one more cool thing is that ICE can now process 16-bit images.  And ICE is still free, as is Photosynth storage. 

To get more details and to download, visit the ICE web page.

The ICE Team – Matt Uyttendaele, Eric Stollnitz, Howard Good

Microsoft Research’s Michael Cohen will be presenting the inner workings of HD View at the High Dynamic Range Imaging Symposium and Workshop being held at Stanford University on Sept. 10-11.  A paper describing the work, HD View: A Software Camera for the Web, has just been made available as an MSR tech. report.  The event looks to have a great line-up of presentations, if your interested in HDR technology it would be well worth attending.  

-Matt

Silverlight’s Deep Zoom, I think, is the best way* to interact with very large pictures over the web.  There are two main components of Deep Zoom.  The first is a file-format.  The second is a software component built into Silverlight that very efficiently handles this file-format.  It turns out that the Deep Zoom component is quite extensible.  This means that Deep Zoom can be used to not only view its native format, but also lots of other mega-images on the web.  To highlight this capability, MSR’s Eric Stollnitz has built a new web-app that lets users preview all of the major large picture formats. 

His new HD View Preview page allows you to quickly try out Silverlight on any Deep Zoom, HD View, KML Photo Overlay, or Zoomify content.  For example, if you’ve previously used the Zoomify export in Photoshop, you can now use HD View Preview to quickly try out Silverlight directly on the Zoomify data.  You simply type the url of the Zoomify XML into the address link on the page (see 1 in figure to the right).  You can similarly enter the url of any KML file that contains photooverlay elements.  Eric has provided some quick example links, including some popular Zoomify data, Google Earth’s Gigapxl layer, Gigapan KMLs, and our favorite HD Views (3 in figure).  After trying out Silverlight on your content, if you like what you see, the page includes an embed code (5) so that you can easily embed what you are previewing into your own web-page or blog.  For developers, the source code for the Silverlight viewer being shown here is available on codeplex.

The page also allows you to experiment with our HD View research prototype.  The tabs on the page let you quickly swap between the Silverlight viewer and HD View (2).  HD View is where we continue to experiment with features that we hope to see in Silverlight someday, including high-dynamic range, color management, and turning flickr images into a QTVR-like experience.  As described in a previous post, if you have used Microsoft ICE or Windows Live Photo Gallery to generate a panorama, then you can get a surround video experience by simply pointing HD View at the resulting JPEG.  The HD View Preview page lets you quickly try this out.  To see this in action click on some of the example JPEG links (4).

After trying out Deep Zoom on your own content we think that you will agree that it is the best* way to experience very high resolution images on the web.  Now with the HD View Preview page it is super simple to try it out**.

Matt Uyttendaele

* by best I mean that the progressive download of imagery is near optimal and the application maintains an interactive frame-rate even when full-screen.

** don’t stop with just trying out the viewer, check out the Deep Zoom authoring tools, including  Deep Zoom Composer, the Photoshop Plugin, and command line tools.