Nowadays, with the growing of multimedia applications, requirement for embedded multimedia system becomes popular. The implementation of a Multi-Model video encoder is introduced, which is composed of DSP (Digital signal processor) and FPGA (Field Programmable Gate Array). The FPGA complete some auxiliary tasks, such as video acquisition and YUV separation, while the DSP is dedicated for video compression. Considering the features of the DSP's architecture and memory resource, as well as the requirements of compression algorithms, the system level optimization strategies and code level optimization techniques are discussed. A feasible solution is given to avoid the overflow and underflow while transmitting the compressed data. The experimental results indicate that the embedded video encoder can adapt to different channels and compress various gray/color pictures with different resolutions.