Improved Seam Carving for Video Retargeting Presented at Project Page Abstract Effective resizing of images should not only use geometric constraints, but consider the image content as well. We present a simple image operator called seam carving that supports content-aware image resizing for both reduction and expansion. A seam is an optimal 8-connected path of pixels on a single Video, like images, should support content aware resizing. We present video retargeting using an improved seam carving operator. Instead of removing 1D seams from 2D images we remove 2D seam manifolds from 3D space-time volumes. To achieve this we replace the dynamic programming method of seam carving with graph cuts that are suitable for 3D volumes.
|Published (Last):||7 August 2007|
|PDF File Size:||11.8 Mb|
|ePub File Size:||2.15 Mb|
|Price:||Free* [*Free Regsitration Required]|
A vertical seam in an image is a path of pixels connected from the top to the bottom with one pixel in each row. Unlike standard content-agnostic resizing techniques such as cropping and scaling , seam carving preserves the most interesting features aspect ratio, set of objects present, etc. Above left is the original by pixel image; above middle are the vertical seams; above right is the result after removing vertical seams.
But these classes neglect to mention the improved seam carving algorithm, published the year after the original. The difference between the original and the improved can be seen below after carving seams on a by bench image. The left is the original seam carving algorithm and the right is the improved algorithm: It took 3 seconds using the original seam carving method and 8 seconds using the improved method, tested on a cheap Windows laptop using Anaconda Python 3.
The improved method preserves the gradient of the water and the bench supports much better, and the result is similar to what happens in Adobe Photoshop. In this notebook, I will be implementing both methods using numpy and scikit-image. Implementation Details For more details, read the original seam carving paper and the Improved Seam Carving paper, where the above photo is from.
Both papers introduce different ways to "calculate the energy of a pixel, which is a measure of its importance—the higher the energy, the less likely that the pixel will be included as part of a seam. The second paper determined the energy of a pixel by looking forward at the resulting image instead of backward at the image before removing the seam. At each step, we search for the seam whose removal inserts the minimal amount of energy into the image. These are seams that are not necessarily minimal in their energy, but will leave less artifacts in the resulting image, after removal.
This coincides with the assumption that natural images are piece-wise smooth intensity surfaces, which is a popular assumption in the literature. Calculating the three possible vertical seam step costs for pixel pi,j using forward energy. After removing the seam, new neighbors in gray and new pixel edges in red are created. In each case the cost is defined by the forward difference in the newly created pixel edges. The two resulting energy maps of the bench image using the two methods look like this, where higher energy pixels are brighter: As you can see, seams will not want to pass through the bench using backwards energy, which causes severe artifacts.
Here is an image of the resulting seams and accumulated energy matrices from the paper: Examples.