ChatGPT answered incorrectly 52% of the time in the field of programming

Recent research results from Purdue University (USA) show that ChatGPT incorrectly answered up to 52% of questions related to computer programming.

The research was presented at the Computer-Human Interaction Conference taking place in Hawaii (USA) earlier this month. Researchers analyzed 517 programming questions on the Stack Overflow platform, then gave these questions to ChatGPT to analyze and provide answers.

ChatGPT answered incorrectly 52% of the time in the field of programming Picture 1

The results showed that 52% of ChatGPT's answers contained false information and of those 77% of the answers were expressed in a lengthy and cumbersome manner. However, thanks to its 'easy to understand' and 'fluent' language style, ChatGPT's answer, despite giving a lot of wrong information, made it popular with 35% of survey participants.

The study also showed that in 39% of cases, programmers participating in the survey could not detect errors in ChatGPT's answers.

Research results from Purdue University have shown that the reliability of ChatGPT - the tool expected to create a breakthrough in the field of AI is worrying.

Currently, the race to create the most 'smart' and reliable chatbots is attracting the participation of technology giants such as Meta, Microsoft and Google.

Google's new AI-integrated search engine is also facing criticism for frequently providing false information from unreliable sources.

Kareem Winters

Update 27 May 2024