Speaker Anonymity and Voice Conversion Vulnerability: A Speaker Recognition Analysis

Anonymized voice data is essential for maintaining privacy in voice-based exchanges of sensitive information, particularly in healthcare. However, voice conversion methods prioritize target voice identity over completely obscuring the source speaker's voice features. Recent advancements in speaker recognition systems have increased their ability to detect subtle residual voice features of the source speaker in voice-converted samples with greater precision and accuracy, posing potential risks to voice anonymity. Balancing speaker anonymity and recognition accuracy is a persistent challenge in voice-based applications, where maintaining voice anonymity and correctly identifying the speaker are critical but vulnerable. Violating voice anonymity can result in privacy and security threats. In this work, we examine multiple voice conversion and speaker recognition systems to explore the threats to voice anonymity. Our findings demonstrate the significant risk of identifying source speakers from converted voice samples. We discovered that voice anonymity is more vulnerable to breaking with one-to-one conversions compared to many-to-many and any-to-any conversions. The likelihood of identifying the original speaker from anonymized speech data depends on target voice features, voice conversion techniques, and speaker recognition methods.