Nvidia Canvas: AI Painting Tool - Realistic Landscapes Instantly

AI-Assisted Artistic Creation: From Sketch to Photorealistic Landscape

Artificial intelligence has steadily been augmenting the capabilities of illustrators and photographers for years, intelligently completing visual content where gaps exist. Current advancements now focus on enabling AI to assist artists even at the initial stages of creation, beginning with a blank canvas.

Nvidia’s recently released Canvas tool empowers creators to roughly sketch a landscape using color-coded areas, similar to a paint-by-numbers approach. The AI then transforms these simple shapes into convincingly photorealistic imagery, though perhaps not yet at a gallery standard.

How Nvidia Canvas Works

Each distinct color applied to the canvas corresponds to a different element, such as mountains, water, vegetation, or ruins. When these colors are applied, the rudimentary sketch is processed by a generative adversarial network (GAN).

GANs function through a collaborative process, with a generator AI attempting to produce a realistic image and a detector AI assessing the realism of that image. This iterative feedback loop results in a depiction that the system deems reasonably realistic based on the initial input.

This represents a significantly refined version of the original GauGAN prototype showcased at CVPR in 2019. The current iteration boasts improved smoothness, enhanced image quality, and compatibility with any Windows computer equipped with a suitable Nvidia graphics card.

Applications and Benefits for Artists

The technology has proven effective in generating realistic representations of faces, animals, and landscapes, although subtle imperfections often remain detectable by the human eye. However, Canvas isn’t intended to create images indistinguishable from reality.

As concept artist Jama Jurabaev explains, the primary benefit lies in facilitating rapid experimentation with imagery that is more detailed than a simple sketch. It allows artists to quickly visualize concepts and iterate on ideas.

For example, envisioning a crumbling ruin within a field alongside a river requires a detailed sketch to convey the final vision. Traditional methods can lead to wasted effort if the initial concept proves flawed after hours of work.

Utilizing Canvas, artists can immediately visualize the impact of factors like sunlight direction and shadow placement, enabling them to refine their ideas efficiently. The tool also allows for quick adjustments to parameters like time of day and color palettes.

Image Credits: Nvidia

“I’m no longer hesitant to start with a blank canvas,” Jurabaev stated. “I’m comfortable making significant changes, knowing that the AI will assist with the details. This allows me to concentrate on the creative aspects, while Canvas handles the technical execution.”

Comparison to Google’s Chimera Painter

The functionality of Canvas bears resemblance to Google’s Chimera Painter, a project that employed a similar process to generate fantastical creatures. However, Chimera Painter dealt with more complex elements like limbs, fur, and teeth, making it more challenging to use and prone to errors.

Image Credits: Devin Coldewey / Google

While Chimera Painter proved complex, even basic blob painting could yield results beyond the capabilities of an amateur artist. Canvas offers a more accessible and user-friendly experience.

Unlike Chimera Painter, which relied on cloud processing, Canvas operates locally and necessitates a powerful Nvidia video card for optimal performance. GPUs are the preferred hardware for machine learning tasks, and a real-time GAN requires substantial processing power.

The application is available for free download here.