Telepresence for in-situ experiment control

We present an approach for remote operation of instruments in the Internet environment. We have applied this approach to in-situ electron microscopy experiments that require dynamic interaction with the specimen under observation, as it is excited with external stimuli, i.e., temperature variation, EM field variation, etc. The dynamic operations include control of the sample's position and orientation under the electron beam, the illumination conditions and focus, etc. Remote control via wide area networks like the Internet that do not offer real time data and command delivery guarantees is not practical for the finely tuned adjustments that dynamic studies require. We suggest that an effective approach to remote operation for this class of dynamic control applications must involve automated control operations performed near the instrument in order to eliminate the wide area network real-time delivery requirement. Our approach to this problem is based on advanced computer vision algorithms that permit instrumentation adjustments to be made in response to information extracted from the video signal generated by the microscope imaging system. We have determined the type of servo loops needed to enable remote operation and collaboration, and have introduced a novel partitioning of the control architecture for implementing this approach. In this partitioning, the low frequency servo loop functions that require direct human interaction are performed over the wide area network, and those functions that require low latency control are performed locally using the automated techniques. This approach hides the latencies in the wide area network and permits effective remote operation. The result is telepresence that provides the illusion of close geographical proximity for in-situ studies. Our testbed is a 1.5 MeV transmission electron microscope, which can now be used on-line via the global Internet.