The neurological disorder named Epilepsy is diagnosed through visual inspection and manual understanding based on the Electroencephalogram (EEG) signals. Several improvements in the deep learning methods result in solving the complex problem by end to end learning. However, the current deep learning models have not utilized the EEG data for seizure detection. The main intention of this paper is to develop novel seizure detection in EEG signal using a new feature extraction and classification approaches. The first phase of the proposed model is pre-processing of signal. Further, the combination of Empirical Mode Decomposition (EMD) and Discrete Wavelet Transform (DWT) is performed for the signal decomposition. Then, the feature selection is done by the concatenation of Principal Component Analysis (PCA) and Linear Discriminant Analysis (LDA). With this selected features, an enhancement is introduced with the weighted feature selection approach, in which the weight is optimized by the hybrid meta-heuristic algorithm named Jaya-Cat Swarm Optimization (J-CSO). This hybrid meta-heuristic algorithm is easy to implement with no algorithm-specific parameters dependence. It is it easily applicable to real-world optimization problems. Due to these advantages, the performance of the proposed model is superior to the conventional models. Then, the J-CSO-based heuristically Improved Ensemble Learning Model (I-ELM) is introduced in the detection phase, which is proposed by three different classifiers like Fuzzy classifier, Deep Neural Network (DNN), and Long Short Term Memory (LSTM). This is comparable to the accuracy achieved by other contemporary seizure classification approaches, and this approach could serve as an effective tool in the hands of medical practitioners for analyzing bulk data and for speeding up seizure diagnosis.