An in-depth study of graph partitioning measures for perceptual organization

In recent years, one of the effective engines for perceptual organization of low-level image features is based on the partitioning of a graph representation that captures Gestalt inspired local structures, such as similarity, proximity, continuity, parallelism, and perpendicularity, over the low-level image features. Mainly motivated by computational efficiency considerations, this graph partitioning process is usually implemented as a recursive bipartitioning process, where, at each step, the graph is broken into two parts based on a partitioning measure. We concentrate on three such measures, namely, the minimum [41], average [28], and normalized [32] cuts. The minimum cut partition seeks to minimize the total link weights cut. The average cut measure is proportional to the total link weight cut, normalized by the sizes of the partitions. The normalized cut measure is normalized by the product of the total connectivity (valencies) of the nodes in each partition. We provide theoretical and empirical insight into the nature of the three partitioning measures in terms of the underlying image statistics. In particular, we consider for what kinds of image statistics would optimizing a measure, irrespective of the particular algorithm used, result in correct partitioning. Are the quality of the groups significantly different for each cut measure? Are there classes of images for which grouping by partitioning does not work well? Another question of interest is if the recursive bipartitioning strategy can separate out groups corresponding to K objects from each other. In the analysis, we draw from probability theory and the rich body of work on stochastic ordering of random variables. Our major conclusion is that optimization of none of the three measures is guaranteed to, result in the correct partitioning of K objects, in the strict stochastic order sense, for all image statistics. Qualitatively speaking, under very restrictive conditions, when the average interobject feature affinity is very weak when compared to the average intraobject feature affinity, the minimum cut measure is optimal. The average cut measure is optimal for graphs whose partition width is less than the mode of distribution of all possible partition widths. The normalized cut measure is optimal for a more restrictive subclass of graphs whose partition width is less than the mode of the partition width distributions and the strength of interobject links is six times less than the intraobject links. Rigorous empirical evaluation on 50 real images indicates that, in practice, the quality of the groups generated using minimum or average or normalized cuts are statistically equivalent for object recognition, i.e., the best, the mean, and the variation of the qualities are statistically equivalent. We also find that, for certain image classes, such as aerial and scenes with man-made objects, in man-made surroundings, the performance of grouping by partitioning is the worst, irrespective of the cut measure.