Predicting Student Success Using Big Data and Machine Learning Algorithms

The prediction of student performance, allows teachers to track student results to react and make decisions that affect their learning and performance, given the importance of monitoring students to fight against academic failure. We realized a system of the prediction of academic success and failure of the students, which is the overall result and the goal of the educational system. We used the personal information of the students, the academic evaluation, the activities of the students in VLE, Psychological, the student environment, and we added practical work and homework, mini projects, and the number of student absences which gives a vision of the quality of the student. Then we applied the methods of artificial intelligence and educational Data mining such as KNN, C4.5 and SVM for the prediction of the academic success of students, but these methods are not sufficient given the progressive number of students, specialties, learning methods and the diversity of data sources as well as student data processing time. To solve this problem, Big Data technology was used to distribute the processing in order to minimize the execution time without losing the efficiency of the algorithms used. In this system we cleaned the data and then applied the property selection algorithms to find the useful properties in order to improve the algorithm prediction rate and also to reduce the execution time. Finally, we stored the data in HDFS and we applied the classification algorithms for the prediction of student success using MAPREDUCE. We compared the results before and after the use of big data and we found that the results after the use of Big Data are very good at execution time and we arrived at a recognition rate of 87.32% by the SVM algorithm.