Numerical simulations of mathematical models can suggest that the models are chaotic. For example, one can compute an orbit and its associated finite-time Lyapunov exponents, and these computed exponents can be positive. It is not clear how far these suggestions can be trusted, because, as is well known, numerical methods can introduce spurious chaos or even suppress actual chaos. This focused review examines the fidelty of numerical methods. We look at the didactic example of the Gauss map from the theory of continued fractions, which allows a simple examination of backward error analysis for discrete dynamical systems and gives a clear picture of the effects of floating-point arithmetic. A similar use of backward error analysis, in the form of defect control, gives a useful understanding in the case of continuous dynamical systems. Finally, we discuss limitations of this 'backward' point of view.