Lightweight Transformer Design for Real-time Flight Control Data Prediction

This paper presents lightweight design methodologies for transformer-based deep learning models aimed at real-time flight control data prediction. While large-scale transformer models have shown remarkable performance in various fields, their use in embedded applications is limited by high memory requirements, computational overhead, and energy consumption. To address these issues, we propose methodologies involving network restructuring and pruning techniques to make the model lightweight. Network restructuring modifies the model's architecture to a bottleneck form to reduce data size, while pruning targets the multi-head attention block to reduce memory consumption. We conduct a comparative analysis on prediction performance and parameter complexity using these lightweight design methodologies. This approach is expected to alleviate hardware constraints and enhance aircraft safety through deep learning-based real-time anomaly detection.