Situation awareness for teams of dismounted warfighters and unmanned vehicles

Changes in military operations in recent years underscore changes in the requirements of military units. One of the largest underlying changes is the transformation from large-scale battles to quick-reaction mobile forces. There is also pressure to reduce the number of warfighters at risk in operations. One resultant need of these two factors is the increased need for situation awareness (SA); another is the use of unmanned vehicles, which increases the difficulty for the dismounted warfighter to maintain SA. An augmented reality (AR) system is a type of synthetic vision system that mixes computer-generated graphics (or annotations) with the real world. Annotations provide information aimed at establishing SA and aiding decision making. The AR system must decide what annotations to show and how to show them to ensure that the display is intuitive and unambiguous. We analyze the problem domain of military operations in urban terrain. Our goal is to determine the utility a synthetic vision system like AR can provide to a dismounted warfighter. In particular, we study the types of information that a warfighter is likely to find useful when working with teams of other warfighters. The problem domain is challenging because teammates may be occluded by urban infrastructure and may include unmanned vehicles operating in the environment. We consider the tasks of dynamic planning and deconfliction, navigation, target identification, and identification of friend or foe. We discuss the issues involved in developing a synthetic vision system, the usability goals that will measure how successful a system will be, and the use cases driving our development of a prototype system.