Google has launched "Gemini Live Vision" — a feature that allows the AI assistant to see your screen in real-time and provide contextual help, troubleshooting, and guidance.
How It Works
Say "Hey Google, watch my screen" and Gemini begins analyzing what you're looking at. It can help with spreadsheet formulas, explain error messages, guide you through software, translate foreign text on screen, and even provide cooking instructions from a recipe photo.
Use Cases
- Debugging code: Gemini sees the error and suggests fixes
- Tax preparation: Walks you through forms in real-time
- Online shopping: Compares prices and finds coupons as you browse
- Learning: Explains any concept on screen in simpler terms
Privacy
Google says screen data is processed in real-time and not stored. Users can see exactly what Gemini is analyzing and pause sharing at any time. However, privacy advocates are concerned about the precedent of an AI constantly watching your screen.
Available on Android devices and Chrome browser. iOS and other browsers coming later this year.
