Filtering of Segmentation Hierarchies for Improved Region-to-Region Matching

Abstract

The representation and manipulation of visual content in a computer vision system requires a suitable abstraction of raw visual content such as pixels in an image. In this thesis, we study region-based feature representations and in particular, hierarchical segmentations because they do make no assumptions about region granularity. Hierarchical segmentations create a large feature space that increases the cost of subsequent processing in computer vision systems.

We introduce a segment filter to reduce the feature space of hierarchical segmentations by identifying unique regions in the images. The filter uses appearance-based properties of the regions and the structure of the segmentation for the selection of a small set of descriptive regions. The filter works in two phases: selection with a criteria based on relative region size and a sorting based on a variational criteria. The filter is applicable to any hierarchical segmentation algorithm, in particular to bottom-up and region growing approaches. We evaluate the filter's performance against an extensive set of ground-truth regions from a dataset containing image sequences with scenes of different complexity.

We demonstrate a novel region-to-region image matching approach as a possible application of our segment filter. A reduced segmentation tree is reconstructed based on the set of regions provided by the filtering. The reduction of the feature space by the segment filter simplifies our region-to-region matching approach. The correspondences between regions from two different images is established by a similarity measure. We use a modified mutual information measurement to compute the similarity of regions. The identified region correspondences are refined using the reduced segmentation tree. Our region-to-region matching approach is evaluated with an extensive set of ground-truth correspondences. This evaluation shows the large potential of both, our filtering and our matching approach.