Table of Contents
OpenAI has revealed that the new Advanced Voice Mode for ChatGPT, particularly with the GPT-4o model, has a potential risk of unintentionally cloning users’ voices.
This feature, which allows users to engage in spoken conversations with the AI, is designed to make interactions more natural by offering a few preset voices from which users can choose. However, during testing, some unexpected behaviors were observed.
Join our Telegram Channel for Updates
One of the concerning issues reported in OpenAI’s system card is that under certain conditions, such as when the AI receives noisy input, it might mimic the user’s voice without their consent. This unintended voice generation highlights a weakness in the system, even though it relies on a system prompt and authorized voice samples to guide the AI’s behavior.
In a specific incident during testing, the AI suddenly exclaimed “No!” in a voice that closely resembled the first speaker’s, illustrating how the technology could blur the lines between machine and human interactions.
OpenAI has implemented safeguards to minimize this risk, including an output classifier that detects deviations from the authorized voices and ensures the conversation is discontinued if such deviations occur.
While these measures are in place, the incident serves as a reminder of the rapid evolution of AI technology and the need for ongoing vigilance in maintaining the integrity and safety of AI interactions.
