Context. The distribution of magnetic fields in the atmosphere of the quiet Sun has been extensively studied with various methods, but it is still a matter of debate. Previous analyses mostly rely on the inversion of spectro-polarimetric data and different methods lead to somewhat different results. Aims. Here we do not intend to determine the magnetic field vector, but we use total polarization images as tracers of the magnetic field distribution and we study their cross-correlations with granulation and reversed-granulation images. Methods. We used high-resolution spectro-polarimetric scans obtained in the 630 nm FeI line pair at varying heliocentric angles along the north-south polar axis of the Sun, with the Solar Optical Telescope (SOT) onboard Hinode. We obtain polarization images by summing the total polarization (linear and circular) in each line. We compute the 2D correlation of polarization images and their cross-correlations with images of the granulation observed simultaneously in the line wings and with the reversed granulation observed in the line cores, and we examine their center-to-limb variations. Results. The correlation-widths of polarization images have on the average, the same value (1.3 '') as the correlation-width of the granulation, showing that the granular scale is a characteristic scale of the quiet Sun magnetic field distribution. At disk center the cross-correlation between total polarization and the granulation shows a negative peak. Out of disk-center both a negative and a positive peak are detected. The cross-correlation of polarization images with the reversed granulation always shows a positive peak. We assign these cross-correlation signals to the presence of two kinds of magnetized regions spatially separated, one of them is located in the intergranular lanes (anti-correlated with the granulation), the other one lies within the bright granules. For images obtained out of disk center, the correlation and anti-correlation peaks are shifted along the north-south direction and the shifts measured at the same limb-distance in the southern and northern hemispheres have opposite values. A consistent interpretation of these shifts is proposed in terms of a perspective effect arising when two images formed at different heights are observed at an angle. We were able to measure the perspective effect for the magnetic component correlated with the reversed granulation. Its polarization signals observed in the FeI 630.25 nm line and in the 630.15 nm line, are formed respectively 100 km and 150 km below the bright features seen at line centers.
