French Startup ZML Unveils Free Tool to Speed Up Inference Across Various AI Chips

Nvidia continues to hold a strong position in the market, yet new competitors and alternatives are surfacing from various directions.

ZML, an innovative AI startup from France and backed by Turing Award winner Yann LeCun, has introduced inference-performance software that allows several open-source large language models to function on multiple chips, including Nvidia, AMD, Google’s TPU, Apple Metal, and Intel Arc.

With the launch of their LLM inference server, ZML/LLMD, the company seeks to break down existing barriers and enhance the capabilities of various chips for AI applications, sometimes achieving speeds that surpass typical performance, as noted by ZML founder Steeve Morin in an interview with TechCrunch.

As AI becomes more embedded in our everyday and professional lives, optimizing inference—the processing of prompts—has gained increased importance, often overshadowing model training. However, challenges related to underlying software and infrastructure can still result in vendor lock-in, according to Morin.

Delivering optimal performance across diverse chips is more than just a technical achievement; it has the potential to disrupt the market, especially in light of rising concerns regarding AI costs.

ZML aims to provide businesses and cloud providers with the flexibility to use a mix of chips, some of which may be more cost-effective or energy-efficient. “The objective is to enable users to build their own systems and realize true efficiency gains that encourage broad AI adoption,” Morin explained.

Such software could bolster innovative AI chipmakers, many located in Europe, as Morin mentioned companies like Axelera, Fractile, Kalray, OLIX, Q.ANT, SiPearl, SpiNNcloud, and VSORA. However, he emphasizes ZML’s ability to partner with these firms on new initiatives as its primary focus.

Morin is optimistic about Nvidia’s future prospects due to its established supply chain. He referenced to TechCrunch that ZML maintains a solid partnership with the leading AI chip manufacturer, which is gearing up for an increase in inference demand.

The surge in investment in inference has prompted some observers to describe it as an “inference gold rush.” As a result, ZML faces competition from companies like Baseten, which recently reached a $13 billion valuation; Inferact, launched by the creators of the open-source project vLLM; and RadixArk, the firm commercializing SGLang.

While both vLLM and SGLang compete with LLMD, Morin’s aspirations for ZML are broader. “We are at a point where we co-design silicon,” he stated. He also emphasized the agility of ZML’s compact team of just 20, which he considers a key factor in the startup’s ability to innovate swiftly, with more releases planned soon.

This efficient team is also well-funded relative to its size. Morin, who was previously the VP of engineering at Zenly—a company acquired by Snapchat for a significant amount in 2017—has secured $20 million from several venture capital firms, including Harry Stebbings’ 20VC, >commit, AALVC, Drysdale Ventures, Xavier Niel’s Kima Ventures, Kindred Capital, LocalGlobe, and Puzzle Ventures.

In contrast to ZML’s initial public offering, an inference-focused ML framework launched in 2024 and updated in March, ZML/LLMD is not an open-source product. However, it is being released as a free resource to gather usage insights. “I prefer to evaluate the landscape and generate revenue where it’s most beneficial, rather than impede my growth by being overly greedy prematurely,” Morin shared.

It’s unclear when ZML/LLMD will shift to a paid model or how its adoption will progress. However, the startup’s cap table suggests that other founders are taking notice, including Dagger and Docker founder Solomon Hykes, and Clément Delangue and Julien Chaumond from Hugging Face, as well as LeCun from AMI Labs. This reinforces the notion that European AI startups can indeed flourish locally. “I couldn’t have founded ZML anywhere but in Paris,” Morin remarked.