OpenAI considers adding a watermark to ChatGPT-generated text

But the company is debating internally whether to release it or not. On the one hand, it seems like the responsible thing to do; on the other hand, it can hurt the company's profits.

OpenAI considers adding a watermark to ChatGPT-generated text Picture 1

OpenAI's watermarking is described as adjusting how the model predicts the words and phrases that are most likely to occur after previous words and phrases, creating a detectable pattern.

Offering any way to detect documents written by AI is a potential boon for teachers trying to discourage students from using AI to do their homework. The magazine reports that the company found the watermark did not affect the quality of the chatbot's text output. In a survey the company commissioned, "people around the world supported the idea of ​​an AI detection tool by a margin of 4:1," the Journal wrote.

After the Journal published its story, OpenAI confirmed that it pushed for adding a text watermark in a blog post update today spotted by TechCrunch. In it, the company says its method is highly accurate ("99.9% effective," according to documents seen by the Journal) and resistant to "spoofing, such as paraphrases." The company also said it was concerned about stigma about the usefulness of AI tools for non-native speakers.

But it seems OpenAI is also worried that using a watermark might offend ChatGPT users. According to the survey, nearly 30% of them told the company that they would use the software less if the watermark feature was implemented.

Despite this, some employees still felt that the watermark feature would be effective. However, in response to users' unpleasant feelings, the Journal said some of the proposed testing methods "may be less controversial among users but are unproven." In today's blog post update, the company said it is 'in the early stages' of exploring embedding metadata, and it's still 'too early' to know how well the new feature will work does, but since it is cryptographically signed there will be no incorrect detection results.

Kareem Winters

Update 05 August 2024