this startup’s metal stacks could help solve ai’s massive heat problem

The Challenge of Cooling High-Density AI Servers

During its March announcement regarding the Rubin series GPUs, Nvidia revealed a significant development: server racks utilizing the Ultra version of the chip, anticipated for release in 2027, may require up to 600 kilowatts of electrical power. This demand surpasses the capacity of many currently available fast electric vehicle chargers.

As data center racks escalate in power consumption, maintaining adequate cooling presents a substantial obstacle. A new company believes a solution lies in utilizing layered metal structures.

Alloy Enterprises' Innovative Cooling Technology

Alloy Enterprises has engineered a technology that transforms copper sheets into robust cooling plates designed for GPUs and associated peripheral components. These supporting elements, including memory and networking hardware, contribute approximately 20% to a server’s overall cooling requirements.

“The importance of addressing this 20% was less critical when rack power consumption was around 120 kilowatts,” explains Ali Forsyth, co-founder and CEO of Alloy Enterprises, in an interview with TechCrunch. However, with racks now reaching 480 kilowatts and projected to hit 600 kilowatts, engineers are compelled to develop liquid cooling solutions for components like RAM and networking chips, areas where viable options are currently lacking.

Alloy’s method employs additive manufacturing techniques to create cold plates capable of fitting into confined spaces while enduring the high pressures inherent in liquid cooling systems.

A Unique Manufacturing Process

Notably, the startup deviates from conventional 3D printing. Instead, it employs a process of bonding metal sheets through a combination of heat and pressure. While more costly than traditional machining, this approach proves more economical than 3D printing.

The resulting cold plate functions as a unified block of metal. Unlike machined products, it lacks seams, and unlike 3D-printed alternatives, it isn’t porous. “We achieve the inherent material properties of the copper,” Forsyth states. “Its strength is comparable to that of a traditionally machined piece.”

Conventional cold plate manufacturing relies on machining, which involves using tools to create desired features. Due to the size of these tools, each plate half must be machined independently. These halves are subsequently sintered – a process fusing metal powders with heat – introducing a potential leak point under high pressure. Alloy’s process, termed “stack forging,” produces seamless cold plates.

Performance and Design Advantages

Stack forging also enables the creation of finer features, down to 50 microns – approximately half the diameter of a human hair – facilitating increased coolant flow. According to Forsyth, Alloy’s cold plates demonstrate 35% superior thermal performance compared to competitors.

Due to the intricacies of stack forging, Alloy manages most of the internal design work. Clients provide essential specifications and dimensions, which the company’s software then translates into a manufacturable shape.

Within Alloy’s facility, copper rolls are initially prepared and cut to the required size. Features are then laser-cut, and areas intended to remain unbonded are coated with an inhibitor. Each cold plate slice is registered and stacked before undergoing diffusion bonding, utilizing heat and pressure to consolidate the slices into a single metal component.

Forsyth confirms that her company is collaborating with “all the major players” in the data center industry, though specific details remain confidential.

Originally designed for use with a common aluminum alloy, the technology was adapted for copper upon receiving significant interest from data centers. Copper’s excellent thermal conductivity and corrosion resistance made it an ideal material. Following the product announcement in June, demand “increased dramatically,” Forsyth reports.