Google AI Updates: Canvas, Search Live & New Features

Google Enhances AI Mode with New Features

On Tuesday, Google revealed updates to AI Mode, its experimental Search feature. These enhancements empower users to pose intricate queries and pursue in-depth exploration of topics directly within the Search interface.

Introducing Canvas for Organized Learning

A key addition is Canvas, a tool designed to facilitate the creation of study plans. It also allows for the organization of information across multiple sessions within a dedicated side panel.

To begin, users can initiate a study plan for an upcoming assessment by clicking the “Create Canvas” button. AI Mode then begins assembling content in the Canvas panel, which can be further refined through subsequent prompts.

Soon, the ability to upload files, such as class notes or a course syllabus, will be available. This will allow for customization of study guides. The Canvas feature will be accessible to users participating in the AI Mode Labs experiment in the U.S. in the coming weeks.

Search Live: Real-Time Assistance with Project Astra

Google is integrating capabilities from Project Astra into AI Mode through Search Live. This is directly linked with Google Lens, the company’s visual search technology.

Search Live functions as an on-demand expert, providing real-time explanations and access to relevant web links while analyzing the user’s camera view. This was explained by Robby Stein, VP of Product, Google Search, in a recent press release.

To utilize this feature, users should open Google Lens within the Google app, select the Live icon, and pose a question while directing the camera at the subject of interest. A conversational exchange with AI Mode is then possible, leveraging the visual context from the camera feed.

The rollout of Search Live with video input begins this week on mobile devices in the U.S. for participants in the AI Mode Labs experiment.

Expanding Lens Integration

Google also announced that Lens will soon be usable within AI Mode to inquire about elements displayed on a desktop screen.

For instance, if a user encounters a geometry problem, they can seek clarification on a specific diagram. By selecting “Ask Google about this page” from the address bar and highlighting the diagram, an AI Overview with key information will appear in the side panel.

Further questioning can be conducted through AI Mode by selecting it at the top of the Lens search results or by clicking the “Dive deeper” button at the bottom of the AI Overview. This functionality will be available this week.

Enhanced Document Support

Currently, users can utilize AI Mode in the Google app to ask questions about images. This capability is now extended to desktop use.

Furthermore, Google is introducing support for PDF uploads on desktop, enabling users to pose detailed questions about document content. For example, lecture slides in PDF format can be uploaded and analyzed with follow-up questions.

Google plans to broaden AI Mode’s file support beyond PDFs and images later this year, including integration with Google Drive files.