Traditional data-driven diagnosis methods rely on manual feature extraction and it is difficult to adaptively extract effective features. Aiming at the characteristics of nonlinear, non-stationary, and strong noise of rolling bearing faults, a novel intelligent fault diagnosis framework is proposed. whick combines variational modal decomposition (VMD), convolution neural network (CNN) and long short term memory (LSTM) neural network Firstly, the original bearing vibration signal is decomposed by VMD into a series of modal components containing fault characteristics. Secondly, the instantaneous frequency mean value method is used to determine the number of local modal components. .And the two-dimensional feature matrix is composed of determined local feature components and the original data, which is the input of the CNN. Thirdly, the CNN is used to implicitly and adaptively extract the fault feature and its output is the input of LSTM layer. And the LSTM is used to extract time series information of fault signals. Finally, the output layer is used to realize the pattern recognition of multiple faults of the bearing using Softmax function. The experimental results show that the proposed method improves the accuracy of the diagnosis and overcome the shortcomings of the traditional diagnosis methods.