Flipping a classroom involves a more interactive class model where students and instructors spend the majority of the class time on various interactive activities in engagement with class materials. Often, this pedagogy style involves taking advantage of the new interactive technologies. In this work, we describe an experiment in an introduction to programming class (CS1) in which we compared the outcomes of offering the same interactive classroom with and without computers. The first approach required students to bring computers to class to engage with the class and materials individually and in groups using the computer-enabled tools. The other approach was to ban computers and require students to interact in person and engage with materials using pen and paper. In both approaches the students' attempts are shared with the class and discussed, and in general we attempted to maintain the same class models. We found that the use of computers alone had no statistically significant effect on the students' learning outcomes, enjoyment of the material, self-assessment of their understanding, use of teaching assistant resources, or self-estimate of how many hours they invested outside of the classroom. We did find that a statistically significant number of students preferred in-class engagements and interactions using computers. We also found that the instructor had much more useful and detailed information about individual student's interaction in class when computers were used. We conclude that, although many instructors are wary of requiring computer use in large classes, there is evidence that students prefer it, it does not negatively affect learning outcomes, and with appropriate tools and pedagogy, it gives the instructor a much deeper and more nuanced view of student performance in the class.