Two-dimensional scatter-difference discriminant analysis not only essentially avoids the small sample size problem which occurred in traditional Fisher discriminant analysis, but also saves much computational time for feature extraction. In this paper, by analyzing the equivalence and the intrinsic essence between two-dimensional scatter-difference discriminant analysis and traditional one-dimensional scatter-difference discriminant analysis, a new method, called block-based two-dimensional scatter difference discriminant analysis, is proposed. It constructs the image matrix under the mode of blocked matrices. As a result, the final discriminant features extracted are more effective for classification. Finally, extensive experiments performed on ORL and AR face database verify the effectiveness of the proposed method.