An appealing approach for studying the reionization history of the Universe is to measure the redshift evolution of the Lyman alpha (Ly alpha) fraction, the percentage of Lyman-break selected galaxies that emit appreciably in the Ly alpha line. This fraction is expected to fall-off towards high redshift as the intergalactic medium becomes significantly neutral, and the galaxies' Ly alpha emission is progressively attenuated. Intriguingly, early measurements with this technique suggest a strong drop in the Ly alpha fraction near z similar to 7. Previous work concluded that this requires a surprisingly neutral intergalactic medium - with neutral hydrogen filling more than 50 per cent of the volume of the Universe - at this redshift. We model the evolving Ly alpha fraction using cosmological simulations of the reionization process. Before reionization completes, the simulated Ly alpha fraction has large spatial fluctuations owing to the inhomogeneity of reionization. Since existing measurements of the Ly alpha fraction span relatively small regions on the sky, and sample these regions only sparsely, they may by chance probe mostly galaxies with above average Ly alpha attenuation. We find that this sample variance is not exceedingly large for existing surveys, but that it does somewhat mitigate the required neutral fraction at z similar to 7. Quantitatively, in a fiducial model calibrated to match measurements after reionization, we find that current z = 7 observations require a volume-averaged neutral fraction of < x(HI)> >= 0.05 at 95 per cent confidence level. Hence, we find that the z similar to 7 Ly alpha fraction measurements do likely probe the Universe before reionization completes but that they do not require a very large neutral fraction.