Google is set to introduce several new features to Gemini Live, its AI assistant that facilitates real-time conversations. Starting next week, Gemini Live will enable users to highlight items on their screens while sharing their camera. This functionality aims to assist users in identifying specific objects more effectively. For instance, if a user is searching for the correct tool among various options, they can point their smartphone camera at the collection, and Gemini Live will highlight the appropriate tool directly on the screen. This feature is expected to be available on the newly announced Pixel 10 devices when they launch on August 28. Subsequently, visual guidance will be rolled out to other Android devices, with plans to expand to iOS in the following weeks.
In addition to this visual capability, Google will also enhance Gemini Live’s integration with various applications, including Messages, Phone, and Clock. For example, if a user is discussing directions with Gemini and realizes they are running late, the assistant will allow interruptions. The user can request Gemini to text a friend with an update, and the chatbot will draft a message accordingly.
Google is also introducing an updated audio model for Gemini Live, which is said to improve the assistant’s use of human speech elements such as intonation, rhythm, and pitch. This update will enable Gemini to adjust its tone based on the context of the conversation. Users will have the option to modify the speaking speed too, similar to features available in other AI voice systems. Additionally, if prompted for a retelling of a story from a particular character’s perspective, Gemini may adopt an accent to enhance the narrative experience.
Source: https://www.theverge.com/news/763114/google-gemini-live-ai-visual-guidance-speech-update