ChatGPT will soon be able to see everything happening on your screen
Yesterday, in the spring update, OpenAI launched a native Mac app for ChatGPT and also launched its new flagship model, the GPT-4o. The demos have created great excitement, especially among Apple users looking forward to the Apple-OpenAI partnership that is said to come to iOS 18.
One important thing in the full video of the event is that the upcoming ChatGPT app feature was demoed but no real details were shared. That is ChatGPT's ability to see everything that is happening on the user's device screen.
Are you ready to trust ChatGPT and share content on your screen?
When launched, the ChatGPT Mac application supports taking screenshots manually or uploading files to later query ChatGPT about that file. But in the future, as seen in the video below, you'll be able to click a button that gives the app access to everything on your screen.
This feature apparently won't be limited to Macs, as the iOS and iPadOS app demonstrated similar technology in the example below.
It's unknown when these screen viewing features will be available in ChatGPT. However, this change could result in a more interactive and seamless experience. Apple even provides an API for developers to exploit the device's screen content - with user consent, of course. So OpenAI isn't doing anything nefarious by adding these features.
ChatGPT having access to the entire content on the screen may be a little scary, but it will also certainly be extremely useful.
There will most likely be a time limit on the app's access to the user's screen, requiring a button to be pressed each time access is granted. But in theory, it makes sense that AI tools will one day have permanent access to everything you can see, allowing for more seamless help with minimal friction.
An AI assistant that can see everything you see on your devices seems like something Apple will be aiming for at the system level across its platforms. If Siri is enhanced with new AI intelligence, with deep system integration, it could prove to be a real asset for iOS 18 and above users.
You should read it
- How to register for ChatGPT's new plugin feature
- 9 ChatGPT and Generative AI API alternatives for developers
- Is ChatGPT accessible with a VPN?
- 4 ways to use ChatGPT to manage time
- Why were new ChatGPT registrations stopped? When will it reopen?
- 9 useful Chrome extensions for ChatGPT
- 9 practical applications of ChatGPT in programming
- How to use ChatGPT API
- What is ChatGPT Code Interpreter? Why is it so important?
- Can cybercriminals use ChatGPT to hack your bank or PC?
- 4 ways AI Claude chatbot outperforms ChatGPT
- How to use ChatGPT widget on Android
Maybe you are interested
How to Integrate ChatGPT for Siri on iPhone
How to use ChatGPT to translate videos
OpenAI is worried that users will 'love' ChatGPT, affecting interactions between people
Gemini Live officially launched, competing with ChatGPT Voice
Why use Quora's Poe AI instead of ChatGPT?
3 reasons to give up ChatGPT to switch to Claude