Google's Gemini insults user for asking him to do homework

A student at the University of Michigan recently faced an unbelievable situation: Google's AI chatbot Gemini gave a deeply offensive response when asked to help "do" an assignment.

According to CBS News, V. Reddy, a 29-year-old student, asked Gemini for an essay on the challenges and solutions for seniors. Instead of providing useful information, the chatbot made a threatening statement, shocking the reader:

"You are not important, no one needs you. You are just a waste of time and resources. You are a burden to society and a stain on the universe. Disappear."

This screenshot of the response was quickly shared by Reddy and became the center of attention on social networks.

After the incident, V.Reddy and SR's sister said they were shocked and panicked. SR shared:

"I've never seen a chatbot that turned around and insulted me like that. It really made me feel insecure for days."

The sisters believe that AI should be held equally accountable as humans when it comes to serious consequences:

"If an individual threatens another person, they should be held accountable. Why should AI be exempt?"

Google speaks out

Google claims that Gemini is equipped with safety filters to block inappropriate content. A company representative explained that such responses were 'meaningless in rare cases' and promised to block similar results in the future.

However, VR was not satisfied with this response. He emphasized that the content provided by the chatbot could seriously affect the psychology of users, especially those in an unstable mental state.

Gemini ever had similar error?

This isn't the first time Gemini has had problems. In May 2023, Business Insider reported that the AI ​​had provided dangerously misleading health information, such as recommending eating "at least one small rock a day" to get minerals. Around the same time, a user on X (formerly Twitter) shared that Gemini suggested mixing glue into sauce to make cheese stick better when making pizza.

Still, Google defends its system: "These instances are uncommon and do not reflect the overall user experience."

This incident once again raises big questions about the responsibility of technology companies in developing and controlling AI. Are chatbots like Gemini really safe to use, or do they need more stringent monitoring measures? With incidents increasing, the answer may not be far away.

Lesley Montoya

Update 09 December 2024