Existing binary vulnerability detection methods are mainly divided into static analysis and dynamic analysis. Compared with dynamic analysis, static analysis has higher code coverage and detection efficiency but is limited by accuracy. Recently, deep learning has achieved a significant improvement in static analysis for binary vulnerability detection. However, static analysis still suffers from two limitations: (i) the identification of function vulnerabilities relies on their patches; (ii) feature extraction either loses contextual dependency or cannot effectively extract local features. In this paper, we propose a new binary vulnerability detection method named VulDet-BC by combining bidirectional gated recurrent units (BiGRU) and convolutional neural networks (CNN). First, VulDet-BC preprocesses binary code text to make each function have the same text structure. Then, for each instruction, we perform BiGRU with word-attention to generate the vector representation which contains contextual dependency between words. Finally, another BiGRU and CNN are combined to train the prediction model and the eventual model is adopted to forecast whether target binary functions contain vulnerabilities or not. VulDet-BC not only preserves long-distance information but also effectively extracts local features related to vulnerabilities. We have applied VulDet-BC to the dataset with 15,954 non-vulnerable functions and 16,327 vulnerable functions. The experimental results show that VulDet-BC achieves 86.7% in terms of accuracy, 97.6% in terms of recall, 80.8% in terms of precision, 88.3% in terms of F1-score, and 86.3% in terms of area under curve (AUC) on average, and outperforms the state-of-the-art baseline.