Focus is regarded as critical for students to learn properly, retain knowledge, absorb tough content, and manage time effectively, especially while learning from online video resources. However, maintaining focus over long study periods can be challenging, thus a drop in concentration results in lower knowledge retention, and understanding. This research proposes an EEG-based technique to tackle this issue. The technique aims to improve student attention by proposing breaks at required intervals or when the concentration level starts to drop significantly. Our system uses a combined approach of conventional approaches like Power Ratio analysis and machine learning algorithms. We take a more comprehensive approach, by including statistical analysis such as the Kendall Tau coefficient, which is used to understand brain activity, also with focus on beta waves which are generated when the individual is concentrating and when performing complex tasks and Theta waves that are generated when the individual starts to fall asleep. Power Ratio (beta is to theta ratio) analysis was used to distinguish between alert and drowsy states. This technique provides a more detailed assessment of the student's mental state and concentration levels. To assess this method's effectiveness, we collected data from participants while they were wearing NeuroSky MindWave devices and watched online educational videos. The participants were also asked to watch educational videos on the subject topics that they are interested in and videos which did not kindle their interest much. This data was then labelled using K-means clustering into four categories: Alert, High Concentration, Low Concentration and Drowsy. It was then used to train the machine and deep learning models. The machine learning models gave an accuracy of 73% in SVM and 75% in Random Forest Algorithm. The RNN Deep Learning model outperformed the proposed machine learning models and the previous models by achieving an accuracy of 96%, implying that it has the likely to improve student attentiveness and learning outcomes. Overall, this study demonstrates the potential of the proposed system, particularly the deep learning model, to improve student attentiveness and online learning experiences.