The mechanism of industrial agglomeration on ecological environments is profound and complex, and has been a hot research topic in recent years. Applying the Copeland–Taylor Model, the present study investigated the environmental pollution effects of the industrial specialized and diversified agglomerations in the Yangtze River Delta urban agglomerations, using the prefecture-level panel data for the period ranging from 2006 to 2016. The main results were as follows: (1) In regard to the entire group of subdivided industries, it was found that a U-shaped relationship existed between specialized agglomeration and the environmental pollution level. In addition, it was also indicated that the diversified agglomeration crossing threshold values had displayed an environmental pollution retardation effect. (2) It was observed that, for different factor-intensive industries, the choice of the specialized agglomeration mode for the labor-intensive and technology-intensive industries was more conducive to environmental protection. Meanwhile, the adoption of a diversified agglomeration mode by the capital-intensive and resource-intensive industries would result in lower levels of environmental pollution. Consequently, it is clear that government departments must control the agglomeration scale at a reasonable level, promote the industrial transformation of an intensive development mode, and encourage the adoption of differentiated agglomeration modes for various industries, so as to reduce environmental pollution in the Yangtze River Delta urban agglomerations.