Despite the improvements in water and land use that controlled environment agriculture offers, energy use is still an open issue, in part because it is difficult to measure. This paper provides a comprehensive analysis of energy use in controlled environment agriculture to outline the role of environmental control as the primary driver and to understand the extent to which energy use is necessary. With respect to lighting, the paper examines the multiple conversions that occur to convert energy into plant biomass and discusses the progress of lighting technologies toward the current biological limits of photosynthesis. With respect to climate control, it considers the gain of near independence from the external environment in plant factories and the greater importance of plant physiological activity in the energy balance. The paper identifies appropriate metrics and relevant contextual information needed to measure energy use to allow comparison between literature studies and existing facilities, or with traditional agricultural techniques. The work contributes to a better understanding of the energy implications associated with developments in controlled environment agriculture.