With the development and widespread application of network information, new technologies led by 5G are emerging, resulting in an increasingly complex network security environment and more diverse attack methods. Unlike traditional networks, 5G networks feature higher connection density, faster data transmission speeds, and lower latency, which are widely applied in scenarios such as smart cities, the Internet of Things, and autonomous driving. The vast amounts of sensitive data generated by these applications become primary targets during the processes of collection and secure sharing, and unauthorized access or tampering could lead to severe data breaches and integrity issues. However, as 5G networks extensively employ encryption technologies to protect data transmission, attackers can hide malicious content within encrypted communication, rendering traditional content-based traffic detection methods ineffective for identifying malicious encrypted traffic. To address this challenge, this paper proposes a malicious encrypted traffic detection method based on reconstructive domain adaptation and adversarial hybrid neural networks. The proposed method integrates generative adversarial networks with ResNet, ResNeXt, and DenseNet to construct an adversarial hybrid neural network, aiming to tackle the challenges of encrypted traffic detection. On this basis, a reconstructive domain adaptation module is introduced to reduce the distribution discrepancy between the source domain and the target domain, thereby enhancing cross-domain detection capabilities. By preprocessing traffic data from public datasets, the proposed method is capable of extracting deep features from encrypted traffic without the need for decryption. The generator utilizes the adversarial hybrid neural network module to generate realistic malicious encrypted traffic samples, while the discriminator achieves sample classification through high-dimensional feature extraction. Additionally, the domain classifier within the reconstructive domain adaptation module further improves the model's stability and generalization across different network environments and time periods. Experimental results demonstrate that the proposed method significantly improves the accuracy and efficiency of malicious encrypted traffic detection in 5G network environments, effectively enhancing the detection performance of malicious traffic in 5G networks.