A major concern with cyber-security for Smart Homes are attacks against the privacy of their residents. Unfortunately, as has been shown in recent years, traditional cyber-security techniques, such a authentication, firewalls, and encrypted communication, are not sufficient to protect against privacy-invasion attacks. Researchers have shown that an attacker can detect the activities of the occupants by passively monitoring the encrypted wireless network traffic of the smart devices. In this paper, we focus on the question how the smart devices in a Smart Home are physically arranged. More specifically, we want to identify devices that are located in the same room or at least in close proximity. Similar to other works, we will only rely on information obtained through passive network measurements. We present techniques to determine whether a smart lamp and an IP camera are located in the same room. We then extend those techniques to cameras that have an automatic infrared mode switch. Finally, we show how the relative position and orientation of multiple cameras can be estimated.