Non-intrusive load monitoring (NILM) is an elegant solution for monitoring energy consumption. Essentially, it only requires a set of voltage and current sensors to be installed at the electrical entry point for load disaggregation. However, the main challenge of NILM is to accurately analyze the aggregate load data and determine the electrical consumption of each appliance. Recently, there have been some deep learning (DL) techniques proposed for NILM. These include deep convolutional neural networks (DCNNs), gated linear unit and residual network (GLU-Res), bidirectional long short-term memory (BLSTM), and autoencoder (AE). Generally, they can outperform some of the existing NILM models such as factorial hidden Markov model. Nevertheless, some of these DL methods cannot handle well on multi-state appliances, appliances with sparse patterns, and appliances with rapid changing patterns. This article proposes a new NILM model, which involves parallel convolution neural networks and BLSTM layers. Moreover, a feature extractor is proposed to unmask useful statistical features from aggregate signals to improve the learning capability of the network. The benchmark dataset REDD was used for testing the proposed method and the state-of-the-arts such as DCNN, GLU-Res, BLSTM, and AE. The results indicate that the proposed method can successfully outperform those methods.