We investigate the velocity dependence of the friction between two rigid blocks limited by a self-affine surface such as the one generated by a crack. The upper solid is subjected either to gravity or to an external elastic stiffness, and is driven horizontally at constant velocity, V, while the lower solid is fixed. For low velocities, the apparent friction coefficient is constant. For high velocities, the apparent friction is shown to display a velocity weakening. The weakening can be related to the variation of the mean contact time due to the occurrence of jumps during the motions. The cross-over between these two regimes corresponds to a characteristic velocity which depends on the geometry of the surfaces and on the mean normal force. In the case of simple gravity loading, the velocity dependence of the apparent friction at high Velocities is proportional to 1/V-2 where V is the imposed tangential velocity. In the case of external elastic stiffness, two velocity weakening regimes can be identified, the first is identical to the gravity case with a 1/V-2 dependence, the second appears at higher velocities and is characterized by a 1/V variation. The characteristic velocity of this second cross-over depends on the roughness and the elastic stiffness. The statistical distribution of ballistic flight distances is analysed, and is shown to reveal in all cases the self-affinity of the contacting surfaces.