To overcome the problems of automated brain tumor classification, a novel approach is proposed based on long short-term memory (LSTM) model using magnetic resonance images (MRI). First, N4ITK and Gaussian filters having size 5 × 5 are used to boost the of multi-sequence MRI quality. The presented deep LSTM model having four layers is utilized for classification. In each layer, optimal hidden units (HU) are selected such as 200 HU, 225 HU, 200 HU and 225 HU, respectively. These hidden or concealed units are chosen after performing extensive experiments to acquire better results. The results are validated on different versions of BRATS datasets (BRATS 2012–15, 2018) and SISS-ISLES 2015 dataset. The presented method attained dice similarity coefficient (DSC) 1.00 on 2012 synthetic, 0.95 on 2013, 0.99 on 2013 Leader board, 0.99 on 2014, 0.98 on 2015, 0.99 on 2018 and 0.95 on SISS-ISLES 2015. The methodology is also checked on real patient’s cases of brain tumor collected from Pakistan ordinance factory and achieved 0.97 DSC. The results confirm that the presented method provides more help for radiologists to classify brain tumor precisely.