In this work, the friction and wear behaviour of acrylonitrile butadiene rubber (NBR), hydrogenated acrylonitrile butadiene rubber (HNBR), acrylate rubber (ACM), and fluoroelastomer (FKM) against steel surfaces under unidirectional dry sliding conditions have been studied. The influence of surface roughness of the steel counterface on friction and wear was studied using a block-on-ring test configuration. At low load, the friction coefficient decreased after a running in period and the wear was insignificant, especially for the ACM and FKM. The running-in time in terms of achieving a stable dry friction for the different elastomers, from longest to shortest, is in the order HNBR, NBR, FKM, and ACM, with an exception in case of FKM sliding against a smooth steel counterface. At higher contact pressure, powdery worn particles on the ACM and a decrease in friction coefficient were observed, but for FKM and HNBR, worn particles with roll shapes were produced. The worn particles of FKM were significantly larger than those of the other tested materials, and a considerably higher wear in FKM was observed.