How student behavior and reflective learning impact grades in online business courses

Purpose - Many universities now offer courses online using learning management systems (LMS). Numerous studies have been conducted to assess the effectiveness of the LMS but few studies have examined how student online behavior within the course, or what they think about the online course, are related to their actual learning outcomes. The paper aims to discuss this issue. Design/methodology/approach - In this study, student activity in an online business course was captured though learning analytics and assignments to determine if online activity and reflective learning impact final grade. A post-positivist ideology was employed. The dependent variable was the grade resulting from five assignments assessed using rubrics. Correlation, t-tests, multiple regression, surface response regression, General Linear Model (GLM)/F-tests, text analytics, analysis of means (ANOM), and cluster analysis were used to test the hypotheses. Findings - Four statistically significant predictors of online student learning performance were identified: course logins, lesson reading, lesson quiz activity, and lesson quiz scores. This four factor model captured 78 percent of variance on course grade which is a strong effect and larger than comparative studies using learning analytics with online courses. Text analytics and ANOM conducted on student essays identified 17 reflective learning keywords that were grouped into five clusters to explain online student behavior. Research limitations/implications - First, from a pedagogy standpoint, encouraging students to complete more online lessons including quizzes, generally promotes learning, resulting in higher grades, which is a win: win for students and for the university. Second, from an IT perspective, the student pre and post testing resulted in statistically significant increase of IT-course knowledge, which puts students on a solid foundation to begin an online business course. Additionally, the link between students voicing IT problems but nonetheless scoring very well on the course certainly implies the development of IT self-efficacy, developed partly through the pre and post testing process. A clear link was established between course learning objectives and student learning performance by using a unique text analytics procedure. Originality/value - The mixed-methods research design started with hypothesis testing using parametric and nonparametric techniques. Once a statistically significant predictive GLM was developed, qualitative data were collected from what the students thought as expressed in their last essay assignment. Text analytics was used to identify and statistically weight the 17 most frequent reflective learning keywords from student essays. A visual word cloud was presented. Parametric statistics were then used to partition the reflective learning keywords into grade boundaries. Nonparametric cluster analysis was used to group similar reflective keyword-grade associations together to form five clusters. The five clusters helped to explain student online behavior.