In recent years, breast cancer is one of the most death causing reasons worldwide whosetreatment involves early detection and diagnosis. A Computer-aided Diagnosis (CAD) system is used for the diagnosis of abnormalities present in the human breast in order to reach a decision. Hence, it is considered to be an effective means of reducing the mortality rate. In the proposed scheme, initially, a contourlet-based feature extraction technique is employed. Further, a filter approach, namely, the two-sample t-test is utilized to find out the most relevant features. Then, the mammograms with the reduced feature set are classified with SVM and k-NN classifiers. For validation of the proposed work, mammograms from the standard dataset, namely, Mammographic Image Analysis Society (MIAS) are considered. From the result analysis, it is observed that SVM produces more accurate results than that of the k-NN classifier. The achieved classification accuracies for normal-abnormal, and benign-malignant are 99.3%, and 99.98% respectively for SVM classifier. The obtained results show that the presented CAD framework outperforms some of the existing systems.