Koyeb Launches Tenstorrent AI Accelerator Support

Tenstorrent AI Accelerators Now Available on Koyeb

Following a substantial funding round of approximately $700 million, Tenstorrent’s AI accelerators are now accessible to developers through the Koyeb platform.

Tenstorrent specializes in the creation of AI processors based on the RISC-V instruction set architecture. They have also developed TT-NN, an open-source neural network library, and TT-Metalium, an open-source, low-level programming model.

Challenging the Dominance of Nvidia

Tenstorrent is actively working to establish alternatives to Nvidia GPUs and the widely used CUDA library. The company finds itself in competition with other innovators like Axelera, Etched, and Groq.

Koyeb: A Serverless Cloud Platform

Koyeb, established by former Scaleway executives, concentrates on providing a serverless cloud platform. This platform is designed to offer developers an abstraction layer above the complexities of cloud infrastructure.

Koyeb’s competitors include Fly.io, Railway, and Render.

Deployment and Scalability

Developers can deploy applications across multiple virtual machines using Koyeb’s command line interface or through integration with their code repository via a simple git push.

The platform readily supports Docker containers and a wide range of programming languages.

A key feature of Koyeb is its automatic scaling capability. Applications can be scaled to hundreds of servers when demand increases, and infrastructure is automatically scaled down during periods of lower traffic.

Focus on AI Applications

Recently, Koyeb has prioritized support for AI applications. The serverless architecture of the platform facilitates a low-latency experience crucial for AI workloads.

Koyeb has integrated Tenstorrent’s PCIe boards into its data centers, allowing developers access to the low-level TT-Metalium SDK for writing both host and kernel programs.

New Instance Types

Developers will discover two new instance types within Koyeb’s documentation and administrative panels:

• The TT-N300S instance provides 24GB of GDDR6 memory, 192MB of SRAM, and delivers up to 466 FP8 TFLOPS. It is paired with 64GB of RAM and 4 vCPUs.
• The TT-LoudBox instance incorporates four N300S units. This configuration offers 96GB of GDDR6, 768MB of SRAM, and a performance of up to 1,864 FP8 TFLOPS. It includes 256GB of RAM and 16 vCPUs.

Hardware Agnosticism and Future Outlook

Koyeb aims to establish itself as a hardware-agnostic cloud platform with this release. Yann Leger, Koyeb’s co-founder and CEO, likened this development to the initial introduction of ARM into the server market.

Leger emphasized Koyeb’s experience, gained from deploying customized servers with Scaleway in 2013-2014, in handling diverse architectures and hardware.

Tenstorrent, meanwhile, is actively seeking partners to cultivate a developer ecosystem around its open-source programming model. Building a viable alternative to Nvidia’s AI ecosystem requires collaborative effort.