Fog computing, as a crucial component of the edge computing paradigm, offers significant advantages in terms of reduced latency and real-time processing. However, its distributed and heterogeneous nature presents distinct security challenges. This research introduces a novel adaptive encryption framework powered by machine learning to address these security concerns. The proposed system dynamically selects and adjusts encryption methods and key strengths based on data sensitivity and the communication context, ensuring a balance between security and performance. The system employs the K-Nearest Neighbors (KNN) classification algorithm to categorize data into sensitive and normal types. For sensitive data, a hybrid encryption approach combining Elliptic Curve Cryptography (ECC) and Advanced Encryption Standard (AES) is applied, ensuring secure key derivation and strong cryptographic protection. For normal data, the system uses standard AES encryption, achieving processing efficiency while maintaining adequate security levels. The effectiveness of the proposed system has been validated through a comprehensive set of experiments, including evaluations of Encryption Time, Decryption Time, Encryption Throughput, Decryption Throughput, and Histogram Analysis. Additionally, we assess security through the Number of Pixels Change Rate (NPCR = 99.349%) and Unified Average Changing Intensity (UACI = 33.079%). The scalability of the system has been thoroughly validated using datasets of varying sizes (1kB, 100kB, and 1000 kB) with variability achieved through the use of both text and image datasets. These evaluations demonstrate the system’s adaptability and performance consistency across diverse data transmission scenarios, making it suitable for large-scale fog environments. The results collectively demonstrate that the adaptive encryption methodology significantly enhances security while maintaining efficiency in data transmission and processing within fog computing environments.