Software Operations Support at Gemini Observatory

Operating a modern telescope requires many software systems working together to maintain/monitor the optomechanical positions, sequence and control individual instruments and wave-front sensors and control the data transfer and quality monitoring. Supporting these complex interconnected systems can be a daunting task, especially when any single failure can cause a cascade effect which tends to hide the original problem. At Gemini, we have several indispensable tools which allow us to track the behavior and performance of all running systems and enable us to accurately investigate and isolate any problems which occur at night. These tools include: The Gemini Engineering Archive (GEA), VME console logs, operational software logging, circular buffers and high level tools like the observation logs. We'll go into some detail about each of these tools and how they enable us to accurately investigate problems and performance issues. Finally, the most important ingredient for successful operations support is the dedicated people from electronics, systems, mechanical, optical and software specializations who work together, sharing their varied expertise, utilizing the tools and data collected in order to solve issues as they occur.