Chemical fractions of heavy metals (Mo, Ni, Cu, Zn, Fe, Mn, Pb, Cd, and Cr) and compositions of bacteria and fungi in surface sediments from the Mo-Ni polymetallic mine area were analyzed. The results indicated that the mean concentrations of Mo, Ni, Cu, Zn, and Cd were higher than their background values. The mean percentage of Cr in residual fraction was much higher than that of other heavy metals. Mo, Cu, Zn, Fe, and Pb were mainly associated with oxidizable fraction. The dominant proportions of Mn and Cd were found in exchangeable fraction with mean percentages of 93.46% and 54.50%, respectively. According to RAC classification and potential ecological risk index (PERI), the Cd with high bioavailability had a very high environmental risk. The MisSeq sequencing results of bacteria and fungi revealed that microbial communities discrepantly respond to different sampling sites. The most abundant phylum of bacteria and fungi were Proteobacteria and Ascomycota, respectively. The bioavailable heavy metals including Mo-B, Pb-B, and Cd-B were recognized to have important influences on both dominant bacterial and fungal communities. The present study manifested that the bioavailability of heavy metal is very important to assess the potential environmental risk and plays a key role in shaping microbial structure.