The heliospheric magnetic field, carried from the Sun by the solar wind, responds in a complex way to changes in coronal conditions through the 11-year solar activity cycle. The changes are particularly significant at high heliolatitudes where the most important changes occur in the corona between solar minimum and solar maximum. Using the magnetic field observations on the Ulysses mission which has just completed 11 years in a polar orbit around the Sun, the changes in the heliospheric magnetic field are described and linked to their origin in the corona. The solar minimum heliosphere has a relatively simple structure, with the two polar regions dominated by fast solar wind steams and unipolar magnetic fields; these are separated by the equatorial region in which fast and slow solar wind streams interact to produce a stable pattern of corotating interaction regions. Around solar maximum, the structure of the heliosphere becomes considerably more complex, as the sources of the solar wind become more variable, with a significant contribution by transient events such as Coronal Mass Ejections that inject complex magnetic structures, the remnants of coronal loops, into the heliosphere. The study of the Ulysses observations over a complete solar cycle has led to a much more comprehensive, three-dimensional view of heliospheric structure and dynamics as a function of solar activity.