What are the fundamental limits of passive polarization imaging?

We compare four passive polarization imaging configurations by quantitatively assessing their target detection performance for different kinds of noise. Through closed-form expressions we determine the gain of these configurations compared to intensity imaging in the case of target/backrgound discrimination. For three of these configurations we show that a minimum amount of polarimetric contrast between the target and the background is required to outperform intensity imaging. We show that the only configuration that has always better performance requires to use a polarizing beamsplitter and assumes that the main source of perturbation is the background shot noise. This work has interesting perspectives for imaging architecture design.