Nowadays, the traditional workflow of designers' completing complicated design tasks has undergone a profound transformation due to the pervasive intervention of generative artificial intelligence (AI) tools, especially when multi-stakeholder participation is getting involved in the design process. Yet we know little about the designers' perceptions and practices of collaborating with generative AI as a co-creative agent within the context of multi-stakeholder participation. To investigate these questions, we took the domain of avatar design as an example and conducted a qualitative interview study with 21 expert avatar designers who have got different levels of experience and expertise in utilizing generative AI tools in their design workflow. We found that designers not only would fall in a dilemma when deciding whether to consider AI as a co-creative agent according to different stakeholders' interests, but they also face many challenges in effectively co-creating with the current systems, including challenges in consistently adjusting AI outputs and getting design inspiration within the iterative generation process, etc. Based on our findings, we concluded both the epistemological and creative patterns of collaborating with generative AI and high-lighted several design opportunities from both technical and ethical perspectives to better support future designer-AI co-creation.