Fei-Fei Li's World Labs Launches Marble AI Model

World Labs Launches Marble, Its First Commercial World Model

World Labs, the company established by prominent AI researcher Fei-Fei Li, has announced the release of its inaugural commercial product: a world model named Marble. This new offering is accessible through both freemium and paid subscription options.

Marble empowers users to transform textual descriptions, images, videos, 3D designs, or panoramic views into editable and downloadable 3D environments.

Advancing the Field of World Models

The launch follows a limited beta preview period of two months. It also occurs just over a year after World Labs emerged from its initial stealth phase with $230 million in funding.

This positions the startup as a leader in the development of world models, which are AI systems designed to create internal representations of environments.

These models are capable of predicting future events and formulating plans for action.

How Marble Differs from Existing Solutions

While companies like Decart and Odyssey have released demonstration versions of their own world models, and Google’s Genie remains in limited research preview, Marble distinguishes itself through a key feature.

Unlike systems that generate worlds dynamically during exploration, Marble creates persistent, downloadable 3D environments.

This approach minimizes inconsistencies and allows users to export their creations as Gaussian splats, meshes, or videos.

AI-Native Editing and Hybrid 3D Design

Marble is also the first model of its kind to integrate AI-native editing tools.

It features a hybrid 3D editor that enables users to establish spatial structures before the AI populates them with visual details.

Insights from World Labs Co-founder

“This represents a completely new category of model focused on generating 3D worlds, and we anticipate continuous improvement over time,” stated Justin Johnson, co-founder of World Labs, in an interview with TechCrunch.

“We’ve already made significant advancements in this area.”

From Interactive Scenes to Realistic Environments

In December of the previous year, World Labs showcased its early models’ ability to generate interactive 3D scenes from a single image.

Although impressive, these initial scenes exhibited a somewhat cartoonish aesthetic and limited explorability due to rendering limitations.

Beta Testing and Initial Impressions

During the beta preview, the Marble model successfully generated compelling worlds from image prompts alone, ranging from game-like settings to photorealistic depictions of real-world spaces.

Initial scenes displayed some morphing at the edges, but this issue has reportedly been addressed in the current release.

However, a world generated during the beta phase using the same prompt appeared more refined and aligned with the user’s intent than the current output.

Expanding Creative Control

“One of our primary goals for Marble is to empower creative control,” Johnson explained.

“Users should have the option to quickly generate content, but also be able to delve deeper and exert precise control over the generated elements.

We want to avoid a scenario where the machine completely dictates the creative process.”

Input Flexibility and the Chisel Editor

Marble’s approach to creative control begins with its versatile input options.

The beta version accepted only single images, requiring the model to extrapolate unseen details for a complete 360-degree view.

The full launch now allows users to upload multiple images or short video clips, enabling the model to create more accurate digital twins of existing spaces.

Furthermore, the introduction of Chisel, an experimental 3D editor, allows users to define coarse spatial layouts – such as walls, boxes, or planes – and then use text prompts to guide the visual style.

This decoupling of structure and style mirrors the relationship between HTML and CSS in web design.

Unlike text-based editing, Chisel provides direct manipulation of objects within the 3D environment.

Expanding and Composing Worlds

A new feature allows users to expand existing worlds when they encounter areas that begin to degrade.

The model can then generate additional detail in the surrounding region.

For creating exceptionally large spaces, users can combine multiple worlds using a “composer mode.”

Johnson demonstrated this capability by merging a cheese-themed room with grape chairs and a futuristic meeting room located in space.

Exploring the Realm of Spatial Intelligence

Marble, a new offering, is accessible through a tiered subscription model. These levels include a Free tier (providing four generations from text, image, or panorama inputs), Standard ($20 monthly, offering 12 generations alongside multi-image/video input and enhanced editing capabilities), Pro ($35 monthly, with 25 generations, scene expansion features, and commercial usage rights), and Max ($95 monthly, encompassing all features and 75 generations.

Potential Applications and Industry Reactions

Johnson anticipates that the primary initial applications for Marble will be found within the gaming sector, visual effects for cinematic productions, and virtual reality experiences.

However, sentiment among game developers is varied. A recent survey conducted at the Game Developers Conference revealed that one-third of respondents perceive generative AI as having a detrimental effect on the games industry. This represents a 12% increase compared to the previous year’s survey results.

Marble's Role in Game Development

Concerns raised included intellectual property infringement, substantial energy consumption, and a potential decline in the quality of AI-generated content. Investigations, such as one conducted by Wired, have indicated that studios like Activision Blizzard are leveraging AI to streamline processes and address employee turnover.

Johnson envisions developers utilizing Marble to create background environments and ambient spaces, subsequently integrating these assets into game engines like Unity or Unreal Engine to implement interactive elements, logic, and coding.

He emphasizes that the tool isn’t intended to completely overhaul existing game development pipelines, but rather to provide readily usable assets for incorporation.

Advantages for VFX and VR

For visual effects, Marble addresses the issues of inconsistency and limited camera control often encountered with AI video generators, according to Johnson. The creation of 3D assets allows artists to meticulously stage scenes and manage camera movements with precise frame accuracy.

Although World Labs isn’t currently prioritizing virtual reality (VR) applications, Johnson acknowledges the industry’s significant need for content and expresses enthusiasm regarding the launch. Marble is presently compatible with both the Vision Pro and Quest 3 VR headsets, enabling users to experience generated worlds in VR immediately.

Expanding into Robotics and Beyond

The potential of Marble extends to robotics as well. Johnson points out that, unlike image and video generation, robotics lacks a vast collection of training data. Generators like Marble can facilitate the creation of simulated training environments.

A Step Towards Spatial Intelligence

As outlined in a recent manifesto by Fei-Fei Li, CEO and co-founder of World Labs, Marble signifies the initial stride towards establishing “a truly spatially intelligent world model.”

Li posits that the subsequent generation of world models will empower machines to attain spatial intelligence at an unprecedented level. She believes that, while large language models equip machines with reading and writing abilities, systems like Marble can impart the capacity to perceive and construct.

The ability to comprehend the existence and interaction of objects in three-dimensional space, she argues, could lead to advancements extending beyond gaming and robotics, impacting fields such as science and medicine.

“Truly intelligent machines necessitate spatial intelligence,” Li stated.

For confidential information or sensitive tips, we are dedicated to covering the AI industry’s internal operations – from the companies driving its evolution to the individuals affected by their choices. Contact Rebecca Bellan at rebecca.bellan@techcrunch.com or Russell Brandom at russell.brandom@techcrunch.com. Secure communication is available via Signal at @rebeccabellan.491 and russellbrandom.49.