Enhancing Real-Time Video Streaming with Joint Frame Size and Rate Adaptation

With advancing network technologies, real-time communication (RTC) scenarios like cloud gaming and video conferencing have gained more attention. However, when addressing the challenge of meeting users' high-quality demands while dealing with network fluctuations, existing adaptive bitrate (ABR) or adaptive framerate (AFR) algorithms encounter limitations in enhancing Quality of Experience (QoE). This paper introduces Adaptive Frame Size and Rate (AFSR), a joint adaption algorithm based on DRL. AFSR dynamically adjusts the frame rate and frame size (magnitude of bits), and improves QoE in RTC by accurately assessing inter-frame quality and its impact on latency and stall. AFSR uses non-linear bitrate-quality relationships and precise end-to-end latency measurements. Comparative evaluations confirm AFSR's superior performance in RTC video transmission.