Chinese AI Video Startup Blocks Sensitive Images

Sand AI's Magi-1: Open-Source Video Generation with Restrictions

A startup originating in China, Sand AI, has recently launched an openly licensed AI model designed for video generation. This model has received positive feedback, including commendation from Kai-Fu Lee, the founding director of Microsoft Research Asia.

However, testing conducted by TechCrunch suggests that Sand AI is implementing censorship measures on its hosted model. These measures appear to prevent the creation of images that could potentially offend Chinese regulators.

Introducing Magi-1

Sand AI unveiled Magi-1 earlier this week. This model generates videos through an “autoregressive” process, predicting successive frame sequences. The company asserts that Magi-1 is capable of producing high-quality, controllable video footage with improved physics accuracy compared to other open-source models.

The model’s substantial size – 24 billion parameters – makes it largely inaccessible for direct use on typical consumer-level hardware. Running Magi-1 effectively requires between four and eight Nvidia H100 GPUs. Consequently, for many users, Sand AI’s platform represents the primary means of accessing and testing the model.

Censorship in Practice

The platform utilizes an initial “prompt” image to initiate video generation. TechCrunch’s investigation revealed that certain prompts are not accepted.

Sand AI actively blocks uploads depicting Xi Jinping, the Tiananmen Square protests, the iconic Tank Man image, the flag of Taiwan, and symbols associated with Hong Kong’s pro-democracy movement. This filtering operates at the image file level; simply renaming the files does not circumvent the restrictions.

Comparison with Other Chinese AI Startups

Sand AI is not alone in implementing image restrictions. Hailuo AI, the generative media platform from Shanghai-based MiniMax, also prevents uploads of images featuring Xi Jinping.

However, Sand AI’s filtering system appears more stringent. Notably, Hailuo AI permits images of Tiananmen Square, a distinction from Sand AI’s approach.

Regulatory Compliance and Information Control

As highlighted by Wired in a recent article, AI models operating within China are subject to rigorous information controls. A 2023 law prohibits the generation of content that could “damage the unity of the country and social harmony.”

This broad definition encompasses content that challenges the government’s established historical and political narratives. To ensure compliance, Chinese startups frequently employ censorship techniques, including prompt-level filtering and model fine-tuning.

A Contrast in Filtering Priorities

Interestingly, while Chinese AI models often restrict political expression, they frequently exhibit fewer safeguards against the generation of explicit content compared to their American counterparts. Recent reporting by 404 indicates that several video generators developed by Chinese companies lack fundamental protections against the creation of non-consensual intimate imagery.