This paper proposes a novel learning method of binary local features for recognition of the finger vein. The learning methods existing in local features for image recognition intend to maximize the data variance, reduce quantitative errors, exploit the contextual information within each binary code, or utilize the label information, which all ignore the local manifold structure of the original data. The manifold structure actually plays a very important role in binary code learning, but constructing a similarity matrix for large-scale datasets involves a lot of computational and storage cost. The study attempts to learn a map, which can preserve the manifold structure between the original data and the learned binary codes for large-scale situations. To achieve this goal, we present a learning method using an anchor-based manifold binary pattern(AMBP) for finger vein recognition. Specifically, we first extract the pixel difference vectors(PDVs) in the local patches by calculating the differences between each pixel and its neighbors. Second,we construct an asymmetric graph, on which each data point can be a linear combination of its K-nearest neighbor anchors, and the anchors are randomly selected from the training samples. Third, a feature map is learned to project these PDVs into low-dimensional binary codes in an unsupervised manner, where(i) the quantization loss between the original real-valued vectors and learned binary codes is minimized and(ii) the manifold structure of the training data is maintained in the binary space. Additionally, the study fuses the discriminative binary descriptor and AMBP methods at the image representation level to further boost the performance of the recognition system. Finally, experiments using the MLA and PolyU databases show the effectiveness of our proposed methods.