Method for selecting representative videos for change detection datasets

In evaluating the change detection algorithms, the algorithm evaluated must show a superior performance than the state-of-the-art algorithms. The evaluation process steps comprise executing a new algorithm to segment a set of videos from a dataset and compare the results regarding a ground truth. In this paper, we propose using additional information in evaluating change detection algorithms: the level of difficulty in classifying a pixel. First, for each video frame used in the evaluation, we created a difficulty map structure, which stores values representing the level of difficulty required by an algorithm to classify each pixel of that frame. Second, we developed a metric to estimate each dataset video's difficulty based on our difficulty maps. Third, we applied the metric to selecting the more representative videos from the dataset based on their difficulty level. Finally, to demonstrate the method's contribution, we evaluated it using all videos from the CDNet 2014 dataset. The results showed that a subset of videos selected by our method has the same potential as the original CDNet 2014 dataset. Hence, a new change detection algorithm can be evaluated more quickly using our subset of videos selected.