Jay Taylor's notes

back to listing index

Improved Seam Carving for Video Retargeting

[web search]
Original source (www.faculty.idc.ac.il)
Tags: image-processing seam-carving improved-seam-carving www.faculty.idc.ac.il
Clipped on: 2018-05-27

Improved Seam Carving for Video Retargeting

Image (Asset 1/9) alt= Video, like images, should support content aware resizing. We present video retargeting using an improved seam carving operator. Instead of removing 1D seams from 2D images we remove 2D seam manifolds from 3D space-time volumes. To achieve this we replace the dynamic programming method of seam carving with graph cuts that are suitable for 3D volumes. In the new formulation, a seam is given by a minimal cut in the graph and we show how to construct a graph such that the resulting cut is a valid seam. That is, the cut is monotonic and connected. In addition, we present a novel energy criterion that improves the visual quality of the retargeted images and videos. The original seam carving operator is focused on removing seams with the least amount of energy, ignoring energy that is introduced into the images and video by applying the operator. To counter this, the new criterion is looking forward in time - removing seams that introduce the least amount of energy into the retargeted result. We show how to encode the improved criterion into graph cuts (for images and video) as well as dynamic programming (for images). We apply our technique to images and videos and present results of various applications.
   author = {Michael Rubinstein and Ariel Shamir and Shai Avidan},
   title = {Improved Seam Carving for Video Retargeting},
   journal = {ACM Transactions on Graphics (SIGGRAPH)},
   volume = {27},
   number = {3},
   year = {2008},
   pages = {1--9}

Image (Asset 5/9) alt= Play or download
Image (Asset 6/9) alt= Comparison of forward and backward energy
Image (Asset 7/9) alt= Comparison
Image (Asset 8/9) alt= Download. Run. Resize.


We thank Mammoth HD library (www.mammothhd.com) for allowing us to use their royalty free demo reel (road ski, water ski, kayak, fish, bicycle, nature). We thankWolf et al. for letting us use their video samples (basketball, football) and saliency maps (football). We thank the members of the following communities for publicly sharing their media: youtube (www.youtube.com): Nmbr5 (golf). blip.tv (http://blip.tv): Detroit Free Press (cheerleaders), aaron (cheerleader shaky camera), mindcaster (ape animation), Mike Krumlauf(highway), cuecast (interview). stage6 (stage6.com): dancers, Osaka hall image. flickr (www.flickr.com): Ben McLeod (bench), Thomas Hawk (rain). Other images were borrowed from Avidan and Shamir (waterfall, car, vase, umbrella, matches, snow). The footage from RATATOUILLE is courtesy of Disney/Pixar. The SIGGRAPH evolve sample was taken from the ACM SIGGRAPH 2008 demo video.

This work is patent pending, owned by Mitsubishi Electric Research Labs, Cambridge MA