Evaluating instructional technology implementation in a higher education environment

For decades colleges and universities have experimented with various technological innovations to deliver instruction. Although not new in higher education, use of learning technology has greatly increased in recent years (Burnaska, 1998). This increased use brings an increased need for understanding the methodologies and approaches best suited to the evaluation of learning technologies in higher education. Here we first review the literature and describe the methods used in a myriad of evaluation studies in this area. Our primary purpose, however, is to describe our experiences evaluating a campus-wide learning technology effort (the SCALE Project) at the University of Illinois at Urbana-Champaign. This evaluation spanned three years and used multiple methods. More importantly, it provided some challenges from which others may learn and benefit.