Knowledge-based and data-driven behavioral modeling techniques in engagement simulation

As knowledge and data increase in scale and complexity, it is more difficult to apply these two key assets to achieve optimal effectiveness in engagement simulation. The aim of this study was to investigate the techniques of knowledge and data integration with respect to the development of smart agents to predict accurate behaviors in tactical engagements. To reduce the complexity of combat behavior representation, with respect to the functions, we represented subject matter expert operational knowledge by proposing multiple levels of cascaded hierarchical structure, namely, the function decision tree, to increase the readability and maintainability of the behavioral model. For decision points in a behavioral model, smart agents can be trained based on data samples collected from rounds of constructive simulations which provide validated physical models and tactical principles. As a proof of concept, we constructed a simulation testbed of multi-warhead ballistic missile penetration, which generated 129,600 constructive simulations over a total of 84 h. Thereafter, we selected 5817 data samples (i.e. similar to 4.5% of the simulations) using an operational metric of total rewards exceeding 100. The data samples are used to train an artificial neural network and then this network is used to develop a deep reinforcement learning agent. The results revealed that the training process iterated nearly 17,000 epochs until the policy loss decreased to an acceptable low value. The smart agent increased the ratio of ballistic missile target hits by 18.96%, a significant increase when compared with the traditional rule-based behavioral model.