Informally, a public-key steganography protocol allows two parties, who have never met or exchanged a secret, to send hidden messages over a public channel so that an adversary cannot even detect that these hidden messages axe being sent. Unlike previous settings in which provable security has been applied to steganography, public-key steganography is information-theoretically impossible. In this work we introduce computational security conditions for public-key steganography similar to those introduced by Hopper, Langford and von Alm [7] for the private-key setting. We also give the first protocols for public-key steganography and steganographic key exchange that are provably secure under standard cryptographic assumptions. Additionally, in the random oracle model, we present a protocol that is secure against adversaries that have access to a decoding oracle (a steganographic analogue of Rackoff and Simon's attacker-specific adaptive chosen-ciphertext adversaries from CRYPTO 91 [10]).