OpenAI is worried that users will 'love' ChatGPT, affecting interactions between people

OpenAI has cited studies showing that humans talking to AI as if they were talking to another human can lead to misplaced trust. This impact increases as GPT-4o's voice quality continues to improve.

GPT-4o's audio capabilities make interactions with the model more human-like, OpenAI said in a report on the safety of the ChatGPT-4o version.

As Open AI predicts, socializing with AI could also affect social norms, such as making users less dexterous or less willing to interact with humans.

In addition, AI's ability to remember details during conversations and perform tasks can also make people increasingly dependent on technology.

Teams testing ChatGPT-4o's voice feature have also discovered that this model can be exploited by bad actors for malicious purposes. Specifically, bad actors can activate this model to repeat false information and create conspiracy theories, convincing users to believe in untrue things.

OpenAI will continue to experiment to better understand how voice capabilities in AI can make people emotionally engaged.

Jessica Tanner

Update 17 August 2024