Will AI completely change the iPhone experience?

Will AI completely change the iPhone experience? Picture 1

In a post on May 30, news site AppleInsider shared details of the artificial intelligence (AI) features that Apple is about to add to iOS 18, mainly related to the Siri assistant and the Notes application. ) and Messages.

Rumors about integrating many AI tools into iOS 18 have been around for a long time. If nothing changes, Apple will introduce these features at the WWDC 2024 programmer conference, taking place from June 10.

Series of improvements to Siri

The AI ​​features are developed internally under the name Project Greymatter. In the iOS 18 beta, Apple is researching a notification summary feature called Greymatter Catch Up. Users can ask Siri to provide an overview of recent notifications.

Siri's response capabilities are likely to be significantly improved through a new intelligent response framework, and a large language model (LLM) that runs directly on the device.

When responding or summarizing content, Siri can remember important information such as names, companies, calendar events, locations, dates and more, AppleInsider highlights.

The smart assistant on iOS 18 can answer questions, summarize long texts, and convert voice to text. Everything is processed on the device through the Ajax model developed by Apple, or transferred to the server for more complex tasks.

Improved photo editing capabilities

In addition to the Siri assistant, iOS 18 and macOS 15 are expected to integrate AI into the Photos application. Internally, Apple is developing Clean Up, a feature to remove objects from photos using AI-generated software.

Regarding Project Greymatter, Apple is developing an internal application called Generative Playground. Rumors say the software uses generative AI to compose and edit photos, which can be sent to others via iMessages.

Information about the application coincides with many recent rumors, claiming that iOS 18 can use AI to create unique emojis for each person.

AppleInsider 's source revealed that the Apple Notes test version also mentions Generation Tool, although it is unclear whether this tool is used to create images or text.

Apple Notes has been upgraded significantly

As for Apple Notes, the note-taking software can also integrate a series of AI-related features including recording, converting to text and then summarizing using LLM.

Recording and summary files can be inserted in parallel with other documents. That means a note can contain a recording of an entire lecture or meeting, with handwritten notes and even images.

A new mode will appear called Math Notes, allowing you to scan mathematical formulas and find solutions through a completely new calculator application, being developed internally under the name GreyParrot.

In addition to math formulas, the Notes app on iOS 18 can also support graphing, similar to the Grapher software on macOS.

Apple is said to be researching the ability to enter math formulas from the keyboard, called Keyboard Math Predictions. According to AppleInsider , this feature can automatically recognize and complete expressions entered by the user.

In other words, the Notes application will automatically complete mathematical formulas and suggest results to users, similar to the current keyboard word prediction system.

Vision Pro users can also benefit from Apple's Transformer language model (LM), with a redesigned voice control interface. Thanks to the Ajax model, Apple can add feedback suggestion features in messaging, mail and Siri applications.

What does Apple's AI have to compete with?

Recently, Apple has been actively developing AI to compete with Google or OpenAI. However, according to AppleInsider , Apple's AI does not seem to be as impressive as Google Gemini Advanced.

At the I/O conference taking place on May 14, Google laid out quite interesting AI use cases, when users can ask questions in video form, then receive responses generated by AI.

To demonstrate this feature, Google's AI was shown a video of a broken turntable, with the question why the device didn't work. The model then analyzes and responds that the spinning disk may not be in a balanced position so it is not working yet.

During the event, Google also announced AI video creation software called Veo. OpenAI owns a similar tool, called Sora.

Meanwhile, Project Greymatter and Apple's Ajax language model cannot create or process video. That's why Apple seeks to cooperate with Google and OpenAI to add many features for users.

Apple will also compete with products such as Rabbit R1 or Humane AI Battery. Apple's advantage comes from the fact that AI can work on existing devices, users do not have to buy specialized AI products.

AI-related projects will be introduced by Apple at WWDC 2024 on June 10, operating on new operating systems such as iOS 18, iPadOS 18 and macOS 15.

Micah Soto

Update 03 June 2024