Evaluating college students' evaluations of a professor's teaching effectiveness across time and instruction mode (online vs. face-to-face) using a multilevel growth modeling approach

Aims: Do college students' ratings of a professor's teaching effectiveness suggest that a professor's teaching improves with time? Does anything predict which instructors receive the highest ratings or improve the fastest? And, importantly, do the correlates of change differ across face-to-face and online courses? Methods: I used data from 10,392 classes taught by 1120 instructors across three years and fit a taxonomy of multilevel growth models to examine whether students' ratings of teaching effectiveness (SETEs) changed across time, whether differences in average SETEs correlated with growth, and whether online vs. face-to-face, tenure, discipline, course level, sex, or minority status affected these estimates. Results: SETEs remained relatively stable across time and teachers, although analyses uncovered a statistically significant, negative correlation between initial status and growth. Instructors starting with lower SETEs improved the fastest. These findings held across online and face-to-face instruction modes. However, in face-to-face classes, minority instructors received significantly lower average SETEs. This difference did not occur in online classes. No other predictors showed statistically significant effects. Finally, considerable SETE variance remained unexplained even when including the full predictor set in the model. Discussion: These findings reveal that professors' SETEs can improve. Additionally, they indicate that patterns of change in teaching effectiveness do not differ generally across online and face-to-face instruction modes. However, the results showed that minority teachers in face-to-face but not online classes received lower evaluations than their majority counterparts. Additional research should seek to understand what leads to SETE differences across minority and majority groups in face-to-face classes but not online classes. (C) 2009 Elsevier Ltd. All rights reserved.