Pebble accretion has become a popular component to core accretion models of planet formation, and is especially relevant to the formation of compact, resonant terrestrial planetary systems. Pebbles initially form in the inner protoplanetary disc, sweeping outwards in a radially expanding front, potentially forming planetesimals and planetary cores via migration and the streaming instability. This pebble front appears at early times, in what is typically assumed to be a low-mass disc. We argue this picture is in conflict with the reality of young circumstellar discs, which aremassive and self-gravitating. We apply standard pebble accretion and streaming instability formulae to self-gravitating protostellar disc models. Fragments will open a gap in the pebble disc, but they will likely fail to open a gap in the gas, and continue rapid inward migration. If this does not strongly perturb the pebble disc, our results show that disc fragments will accrete pebbles efficiently. We find that in general the pebble-to-gas-density ratio fails to exceed 0.01, suggesting that the streaming instability will struggle to operate. It may be possible to activate the instability if 10 cm grains are available, and spiral structures can effectively concentrate them in regions of low gravito-turbulence. If this occurs, lunar mass cores might be assembled on time-scales of a few thousand years, but this is likely to be rare, and is far from proven. In any case, this work highlights the need for study of how self-gravitating protostellar discs define the distribution and properties of solid bodies, for future planet formation by core accretion.