Modulation Spaces of Symbols for Representations of Nilpotent Lie Groups

We investigate continuity properties of operators obtained as values of the Weyl correspondence constructed by Pedersen (Invent. Math. 118:1–36, 1994) for arbitrary irreducible representations of nilpotent Lie groups. To this end we introduce modulation spaces for such representations and establish some of their basic properties. The situation of square-integrable representations is particularly important and in the special case of the Schrödinger representation of the Heisenberg group we recover the classical modulation spaces used in the time-frequency analysis.