Google has begun rolling out new artificial intelligence features to its Gemini Live platform, significantly expanding its capabilities. The new features—real-time screen-reading and live video interpretation—allow the AI assistant to analyze what’s displayed on a smartphone screen or through a camera feed and provide instant answers. These upgrades, developed under Google’s “Project Astra” initiative, were first demonstrated nearly a year ago and are now becoming available to users.
AI-Powered Screen-Reading and Live Video Analysis
The two major additions to Gemini Live are:
- Screen-Reading Feature – Users can share their phone screens with Gemini Live and receive AI-generated insights based on what’s displayed. Whether it’s summarizing an article, explaining complex topics, or answering specific questions, Gemini can provide contextual responses.
- Live Video Interpretation – Using a smartphone’s camera, Gemini can process real-time visual data and assist users with tasks like identifying objects, translating text, or making aesthetic choices. For example, in Google’s demo video, the AI helped a user choose a suitable paint color for freshly-glazed pottery.
These features are currently exclusive to Gemini Advanced subscribers under the Google One AI Premium plan. Google has not yet confirmed when, or if, they will become available to non-paying users.
Read More: Google cracks down on fake business listings with lawsuit
First Public Sightings and Demonstrations
The rollout was first reported by a Reddit user with a Xiaomi smartphone who noticed the new screen-sharing feature in action. The user shared a demonstration video showcasing Gemini Live’s ability to analyze and respond to content displayed on their screen. The news was later confirmed by Google spokesperson Alex Joseph in an email to The Verge.
Similarly, another Reddit user shared footage of the live video feature, highlighting how Gemini could analyze surroundings in real time through a phone’s camera. The AI assistant was able to recognize objects and answer contextual questions, marking a major step forward in real-time AI interaction.
How Gemini Live Stands Out Against Competitors
Google’s latest AI advancements position it ahead of its competitors in the AI assistant market. While Amazon is working on an upgraded version of Alexa (Alexa Plus) and Apple has delayed its next-gen Siri rollout, Gemini is already implementing the AI-driven capabilities that rival platforms are still developing.
Samsung continues to offer Bixby, but Gemini’s integration into Samsung smartphones gives Google a unique advantage in the Android ecosystem. With Gemini Live evolving into a more interactive AI assistant, it sets a new standard for real-time AI functionality.
Read More: Pakistan joins Google Wallet for digital transactions
These new features represent a significant milestone in Google’s AI roadmap. Project Astra aims to make AI assistants more context-aware, multimodal, and interactive, allowing users to communicate with them naturally and efficiently. With real-time screen-reading and video interpretation, Gemini Live is pushing the boundaries of what AI can do on mobile devices. Whether assisting users with reading comprehension, offering live translations, or making shopping recommendations based on camera input, Google is inching closer to a fully integrated AI companion.