Odyssey AI: Stream 3D Interactive Worlds

Odyssey's AI for Interactive Streaming Video

Odyssey, a startup established by leading figures in self-driving technology – Oliver Cameron and Jeff Hawke – has created an artificial intelligence model enabling users to actively “interact” with streaming video content.

Exploring Video with AI

Currently accessible via an “early demo” on the web, the model dynamically generates and streams video frames at a rate of 40 milliseconds each. This allows viewers to navigate and examine different areas within a video, functioning similarly to exploration within a 3D video game environment.

According to a blog post from Odyssey, the model functions by predicting the subsequent state of the visual world. It considers current conditions, incoming actions, and a record of past states and actions. This is achieved through a novel world model, which demonstrates abilities such as generating realistic-looking pixels, maintaining spatial coherence, learning actions directly from video footage, and producing consistent video streams lasting five minutes or longer.

The Rise of World Models

Numerous companies, ranging from startups to major technology corporations, are actively developing world models. These include DeepMind, World Labs led by AI researcher Fei-Fei Li, Microsoft, and Decart. The belief is that these models will eventually facilitate the creation of interactive media – like games and films – and power realistic simulations, such as robot training environments.

Concerns within the Creative Industries

However, the reception within creative fields is not universally positive. Investigations, such as a recent one by Wired, have revealed that game development studios, including Activision Blizzard, are utilizing AI to reduce costs and address workforce shortages. Furthermore, a 2024 study commissioned by the Animation Guild estimated that over 100,000 jobs in U.S. film, television, and animation could be impacted by AI in the near future.

Odyssey's Commitment to Collaboration

Odyssey emphasizes its intention to work alongside creative professionals, rather than replace them.

The company states that interactive video “opens the door to entirely new forms of entertainment, where stories can be generated and explored on demand, free from the constraints and costs of traditional production.” They envision a future where all video content – including entertainment, advertising, education, training, and travel experiences – will evolve into interactive formats powered by their technology.

Current Limitations and Future Development

The current demo of Odyssey’s technology is acknowledged by the company as being somewhat unrefined. The generated environments exhibit blurriness and distortion, and can be unstable, with layouts shifting unexpectedly during exploration.

Despite these limitations, Odyssey is committed to rapid improvement. The model currently streams video at up to 30 frames per second, utilizing clusters of Nvidia H100 GPUs, at a cost of $1 to $2 per “user-hour.”

Looking forward, the company is focusing research on developing more detailed world representations that accurately capture dynamics, while also enhancing temporal stability and maintaining consistent state. They are also expanding the range of actions the model can perform, moving beyond simple motion to include broader interactions with the virtual world.

A Unique Data Acquisition Approach

Odyssey is employing a distinct strategy compared to many AI labs in the world modeling field. They have designed a specialized 360-degree camera system, mounted on a backpack, to capture real-world landscapes. The company believes this approach will result in higher-quality models than those trained solely on publicly available datasets.

Investment and Leadership

To date, Odyssey has secured $27 million in funding from investors including EQT Ventures, GV, and Air Street Capital. Notably, Ed Catmull, a co-founder of Pixar and former president of Walt Disney Animation Studios, serves on the startup’s board of directors.

In December of last year, Odyssey announced development of software that will allow creators to import scenes generated by its models into industry-standard tools like Unreal Engine, Blender, and Adobe After Effects for further refinement and editing.