ICE twitter feed, new forum, minor update

A few quick ICE announcements.

We have a new Microsoft ICE specific twitter feed:  http://twitter.com/#!/MicrosoftICE  Please follow us to hear the latest ICE related goings on.

The ICE forum has moved.  It is now at: http://social.microsoft.com/Forums/en-US/ice/threads

A minor update of ICE is available at: http://research.microsoft.com/en-us/um/redmond/groups/ivm/ice/  Current users should be prompted to update when starting ICE.

Posted in Uncategorized | Leave a comment

Panorama Stitching on your iPhone

The Photosynth team today released a beautiful new app for the iPhone.  This lets you interactively create panoramas directly on your iPhone.  When you enter the app it will show you an interactive preview of what you are capturing – very useful to make sure that you’ve captured everything you want.  Then you hit “Finish”, and it renders out a seamlessly stitched result that you can upload to a variety of web services.

Of course, you can upload to Photosynth.net to get a full immersive viewing experience, but you can also upload a flat version of the image directly to Facebook.  In either case this app allows you to share your panorama just moments after capturing it.

The app nicely complements the rest of the panorama creation tools available for Photosynth.  At the high end, we offer the Photoshop plugin, which lets you create and edit a panorama with whatever tools you want.  For a more automated solution on the PC, we offer Microsoft ICE with direct uploads to Photosynth.  And now for the mobile phone, we’re making this app available.  In addition to creating panoramas, the app is also a great way to view them.  For any panorama that you’ve created using any of the tools above, you can now use the app to view them on the iPhone as a fully interactive surround-video experience.

The Photosynth app is available now for iOS (iPhone 4, iPhone 3GS, iPod Touch 4th generation, and iPad 2) in the iTunes App Store.

Matt Uyttendaele

Posted in Uncategorized | 2 Comments

Microsoft ICE update–video to panorama, lens vignette, improved blending

We are pleased to announce our latest update to Microsoft ICE.  The download is available by following these links:  ICE for 32 bit Windows  -or-  ICE for 64 bit Windows

After installing, you will find some exciting new features.

Stitching From Video

The first is that ICE can now automatically stitch a panorama directly from video.  One fun use of video panoramas is “motion summaries,” like this result that ICE produced:

Ski jump 4

In this video the photographer was panning the camera to follow the motion of the snowboarder.  I used ICE to indicate “Start” and “End” points, and I gave a few hints about which video frames wvideodlgere interesting. The motion tracking and final composition was then done automatically by ICE.  You can access this feature by selecting “New Video Panorama” from the File menu.  This will bring up the Video Panorama dialog (shown to the right), where you can play or single-step through videos in order to choose start and end points. You can also optionally draw regions of interest on individual video frames to ensure that certain elements are present in the final composition. ICE supports most common video formats (avi, mov, wmv, and more).  Of course the results can also be uploaded to Photosynth. Note that this particular feature is only available for ICE running on Windows 7 (for other versions of Windows this menu item will be disabled).

Video is also a great way to quickly capture a panoramic scene.  For example, here a two-minute video was automatically turned into a panorama (this was shot with a Samsung Focus Windows Phone).

Automatic Vignette Correction

YosemiteFallsCropOrig

Lens vignetting is the effect of the center of a photograph being brighter than the periphery. This effect can be especially apparent when stitching photos. For example, notice the repetitive pattern in the sky in the above panorama.  This is caused by lens vignetting in the constituent images. In this ICE update, lens vignetting is automatically detected and removed.  The result is shown below.

YosemiteFallsCropDevig

Improved Blending

The new version of ICE has a new blending engine. Previous versions did blending by applying a color adjustment to the panorama source images such that there would be minimal visible differences between them. In some cases, this strategy couldn’t adequately hide the seams. For example, in the panorama below, the seam in the  moving water could not be completely hidden by color adjustment alone. In the new version of ICE we still run the color adjustment step, but where this is inadequate, we also do an intelligent fade between the images. This helps a lot in cases like the one below as can be seen in the two breakout panes.  Click on the image below to explore it in full resolution on the Photosynth website; you should find it very difficult to discover the seams between images.

blending

And More

In addition to the above main features, we have also made a few other enhancements:

    • An options dialog to control scratch disk location and ICE memory use.
    • Enable perspective projection for wide field-of-view panoramas
    • 1/3 less disk usage when stitching large data sets
    • ICE is now more robust to corrupt metadata in source photos

Matt Uyttendaele ICE lead

with video stitching by: Neel JoshiEric Stollnitz, Paramjit Sandhu; lens vignette removal by: Sing Bing Kang, Howard Good; and improved blending by: Rick Szeliski, Drew Steedly

Posted in Uncategorized | 31 Comments

Panorama Uploads From Photoshop

Today we are pleased to announce that the Photosynth Export Plug-in for Photoshop is available on the Microsoft Research Downloads page. This plug-in lets you export panoramas from Photoshop or Photoshop Elements directly to the Photosynth web service.

Photosynth added the panorama feature back in March, but up until now, Microsoft ICE was the only way to generate panoramas.  A few clever people figured out how to bypass this restriction (see users xRez_Studio and OrasulMemorabil), but their solution takes a lot of effort. With the Photoshop plug-in, you can now use any stitching program (AutoPano, PTGui, Hugin, ICE, GigaPan Stitch, Photoshop, etc.), edit the results in Photoshop, and simply select “Export” from the Photoshop file menu to upload full-resolution, interactive panoramas to Photosynth.

Under most circumstances when using the plug-in, you will see the following dialog box displayed. This is necessary in order for you to indicate the properties of your panorama. At a minimum, you will need to enter the “Projection” and “Field of View” of your image (see list of available projections in second screenshot). For some panoramas, the horizon is not exactly in the center of the image.  In this case, you will need to uncheck “Center Image” and indicate the “Position of Horizon” (for example, 0 = top of image, 0.25 = one quarter of the way down from top, 1 = bottom of image).  Once you’ve done this, simply click “Publish,” and in a short time, your panorama will be available for the world to see on the Photosynth website.

image

image

If you used AutoPano or ICE to stitch your panorama and you didn’t somehow change the size of the panorama (for example, by cropping), then it’s even simpler.  You will not see this dialog box, and the process will continue directly to uploading. This is thanks to the metadata that those two programs store in panoramas that they create. All of the necessary information is already stored in the image.

One final feature to highlight is that the plugin also lets you create an immersive Photosynth panorama without stitching at all. The last projection type in the list above is ‘Fisheye’. This lets you upload photos shot with a fisheye lens.  The plug-in will undo the fisheye distortion and present the result as a wide field of view interactive experience. To demonstrate this I walked around Seattle’s University District Farmers Market and captured some of my favorite vendors. These were all shot with a Canon 1Ds Mark III and Sigma 8mm lens.

I also want to thank a few people who have tested a prerelease version of the plug-in and provided valuable early feedback:

Please use the following links for more information:

The Photosynth Export Plug-in team – Matt Uyttendaele, Howard Good

Posted in Uncategorized | 18 Comments

New version of ICE available

An update to ICE is available at: http://research.microsoft.com/en-us/um/redmond/groups/ivm/ice/

Most of the bug fixes and changes in this version were based on feedback from our users:

  • Photoshop layers are back as an output option. We have also added support for the Adobe Large Document Format (PSB), so there is effectively no longer a resolution limit when exporting to Photoshop.  Finally we’ve given users more control over the compatibility mode for PSD/PSB.
  • We’ve fixed most of the top issues reported on the forum. Thanks to everyone who shared their problematic photos with us.
  • We’ve enabled more of the stitching process to use multi-core, so your stitching projects should now run even faster.
  • The ‘jaggies’ on the panorama borders are gone.
  • We’ve added support for WIC codecs installed on your PC. This means that if you’ve installed a raw codec from your camera manufacturer, then ICE can directly stitch your raw images. I’m excited about this feature, but your mileage may vary depending on what codec you are using.  Most of my personal raw files are Canon CR2. So, I initially tried Canon’s raw decoder. Unfortunately it is very slow to open files, and because Canon doesn’t provide a 64-bit decoder, users of 64-bit ICE can’t open CR2s. I switched to using the FastPictureViewer Codec Pack and so far that has worked very well. Below is a table of raw codecs that we will try to keep up to date on the ICE web site.  We haven’t tested all of these with ICE, so it would be interesting to get feedback on your own experiences with ICE plus raw.
Name
Supported formats
Cost
Works with 32-bit ICE
Works with 64-bit ICE
Adobe DNG Codec
Adobe Digital Negative
Free
Yes
No
ArdFry DNG Codec
Adobe Digital Negative
$29.95
Yes
Yes
Canon Raw Codec
Canon Raw
Free
Yes
No
FastPictureViewer  Codec Pack
Adobe Digital Negative, Canon Raw, Epson Raw, Fuji Raw, Hasselblad Raw, Kodak Raw, Leica Raw, Mamiya Raw, Minolta Raw, Nikon Raw, Olympus Raw, Panasonic Raw, Pentax Raw, Sigma Raw, Sinar Raw, Sony Raw
$9.99
Yes
Yes
Nikon Raw Codec
Nikon Raw
Free
Yes
No
Olympus Raw Codec
Olympus Raw
Free
Yes
?
Panasonic Raw Codec
Panasonic Raw
Free
Yes
No
Pentax Raw Codec
Pentax Raw
Free
Yes
No
Sony Raw Codec
Sony Raw
Free
Yes
?

 

 

 

Posted in Uncategorized | 1 Comment

HD View Utilities Update 1.11

We’ve just released a long overdue update to the HD View Utilities.  This package includes the hdmake command line utility and the HD View Photoshop Plugin.  The important changes in the 1.11 update are:

  • Fixes export issues on Windows 7/Vista SP2.
  • Support for 64bit version of Photoshop
  • A 64bit version of hdmake
  • Multi-core processing
  • Specification of source imagery in hdmake via a text file as requested on the forum by scrambler.  See the help page for the srcgridfile option.
  • The Photoshop plugin and hdmake support Autopano and ICE panorama metadata.

 

PSPlugHighlight

I’ve mentioned on this blog in the past that Windows Live Photo Gallery (WLPG) and Microsoft ICE stores custom metadata that describes the projection  surface and field-of-view of a panorama.  Unfortunately there is no industry standard for storing this information in image files.  It turns out that Autopano also stores this information (in a different custom tag) within rendered panoramas.  When post processing panoramas (e.g in Photoshop) that are destined for an interactive viewer it is very useful to maintain this metadata.  So, in the latest Photoshop plugin if your source imagery contains Autopano, ICE, or WLPG metadata then you will see the “Metadata Projection/Extents found” message appear in the plugin.  This will then be used to auto-populate the fields in the ‘Projection’ section of the plugin.

If you perform any resizing operation in Photoshop, such as crop or scaling, then these values will no longer be valid and the message will change to “… Projection Extents don’t match image size…”.  In this case you’ll be left to fill the boxes in manually.

The metadata workflow is a test for the next major feature that we release, which is “Upload to Photosynth” from Photoshop.  It is important to have the extents correct for the Photosynth viewing experience. It would be great to get feedback, in the comments below or on the forum, about how best to maintain panorama projection and FOV information through post-processing.

The 64bit version installer is available here and 32bit here.

 

The HD View Team – Howard Good, Matt Uyttendaele

Posted in Uncategorized | Leave a comment

ICE Is Now Synthy

We are super-excited to announce a new release of Microsoft ICE.  A lot of hard work went into this update of ICE, and it has some great new features, which I’ll talk about shortly.  But, the most exciting thing — the thing that’s  probably going to make you stop reading and hit this download link — is that ICE now has a “Publish to Photosynth” button.  This lets you share your full-resolution ICE panoramas on the web.

Photosynth

Photosynth is Microsoft’s unique technology for reconstructing 3D spaces from a collection of photos.  These 3D reconstructions (known as synths) will continue to be an integral part of the Photosynth experience.  As of today, panoramas will also become integral to Photosynth.  Check out the Photosynth team blog for details on this new feature.image

For ICE users, what this means is that you can now share your full-resolution panoramic photos on the web.  The Photosynth web site has been enhanced to let users interact with massive, fully immersive panoramas.  Photosynth uses intelligent streaming technology that allows for smooth pan and zoom of even the largest images.  This is a huge step forward compared to how most ICE users have been sharing their panoramic creations on the web.  The best way to understand this is to check out a few examples uploaded from ICE this morning.

Maps

Often the inspiration for capturing a panorama is the photographer’s desire to document a special place on Earth — be it the view from your office, a sunrise over Mt. Rainier, or your climbing gym.  Photosynth has some great features (see screenshot to the right) to help you share your unique knowledge of such places.  You can “Geotag” where your panorama was taken, and what direction you were looking.  You can add “General” tags and a text description to your photos.  Under “Highlights”, you can annotate all the details, big and small, within your images — for example, see the highlights on this 4-gigapixel panorama of Yosemite Valley.  ICE panoramas with geotags will be visible on Bing Maps alongside traditional synths. The Photosynth team is working with Bing Maps to make this happen soon.

 

Gigapixels

At Microsoft Research we’ve been working on creating super high resolution images for a few years now.  Some of this technology was in the first version of ICE, but it wasn’t really possible to create multi-gigapixel images with ICE — until now!  We’ve added a new type of project in ICE called a structured panorama.  This is accessed via the “File: New Structured Panorama” menu, which allows you to give ICE some hints about the order of your input images.  The ICE output size is now effectively unlimited – for example see this 10gigapixel result.  It is an ideal way to stitch images captured by robotic capture devices that are now available to consumers, like the Gigapan unit.  It’s been a kick for us to go back and use ICE on the early imagery that we captured when we were inventing this technology.  What took weeks of labor to process back then is now automatically handled by ICE — from import, through seamless composition, to upload to the web.  Those weeks of effort are now down to hours, and in some cases minutes if you have a fast multi-core computer.

image

 

Multi-core

Many of the processing-intensive tasks* within ICE have been enhanced to effectively use multi-core PCs.  Below is a screenshot of a computer’s CPU Usage while ICE is processing a Photosynth upload.  This particular computer has 8 cores, represented by the 8 graphs.  Ideally, what we want to see here is these graphs pegged at the top.  This means all the CPUs are working together to finish a job much faster.  While 8-core machines are not yet common, most new consumer PCs do have 4 cores.  If you’ve bought a new PC recently, you should now see ICE work up to 4X faster!

image

 

And More …

That’s all of the new stuff we’ll mention for now. Actually, one more cool thing is that ICE can now process 16-bit images.  And ICE is still free, as is Photosynth storage. 

To get more details and to download, visit the ICE web page.

The ICE Team – Matt Uyttendaele, Eric Stollnitz, Howard Good

Posted in Uncategorized | 5 Comments