The diagnosis of the whole slide images of breast cancer is highly dependent on the experience of the pathologist, and the diagnosis results are prone to large human errors. Along with the rapid develop of the technique of computer and image manipulation, the computer-aided diagnosis is gradually used in medicine. Based on visual perception, we propose a model for rapid detection and refined segmentation of breast cancer regions in this paper. We adopt a classification model ResNet101 or MobileNetV2 to quickly filter out obvious non-cancer areas. Then, the semantic segmentation model U-Net is used to refine the segmentation. Experimental results show that our approach can quickly classify the lesion, ResNet101 model patch classification accuracy rate reaches 98.3%, MobileNetV2 model in the case of relatively few parameters, the patch classification accuracy rate reached 97.2%. When the results are not much different, the MobileNetV2 model can quickly classify the patch image. In accurately locating the lesion area, the FROC score best reached 79.5%. The results are competitive compared to the results of other state-of-the-art methods.