French Startup ZML Unveils Free Tool to Optimize Inference Across Multiple AI Chips

Nvidia continues to hold a strong market position despite the emergence of new competitors and alternatives across various sectors.

ZML, a cutting-edge AI startup from France backed by Turing Award winner Yann LeCun, has introduced software focused on inference performance, allowing a range of open-source large language models to function on different hardware, including Nvidia, AMD, Google’s TPU, Apple Metal, and Intel Arc.

With the launch of their LLM inference server, ZML/LLMD, the company seeks to overcome existing challenges and enhance chip performance for AI tasks, often achieving speeds that surpass conventional benchmarks, as noted by ZML founder Steeve Morin during an interview with TechCrunch.

As AI becomes more integrated into daily and professional life, the optimization of inference—the processing of prompts—has gained paramount importance, often overshadowing model training. Nonetheless, issues relating to software and infrastructure may lead to vendor lock-in, as Morin pointed out.

Improving performance across various chips poses not just a technical challenge but also the potential to disrupt the market, especially amid rising concerns over AI costs.

ZML aims to provide companies and cloud providers with the flexibility to utilize a combination of chips, some of which may be more economical or energy-efficient. “Our objective is to enable users to build their own systems and achieve real efficiency gains that promote widespread AI adoption,” Morin explained.

This software has the potential to empower innovative AI chip manufacturers, many of which are situated in Europe. Morin highlighted companies such as Axelera, Fractile, Kalray, OLIX, Q.ANT, SiPearl, SpiNNcloud, and VSORA. However, he reaffirmed ZML’s commitment to collaborating with these firms on new endeavors.

Morin remains optimistic about Nvidia’s future, crediting its success to a well-established supply chain. He shared with TechCrunch that ZML has developed a strong partnership with the top AI chip manufacturer, which is gearing up for a surge in inference demand.

The heightened emphasis on inference has led some analysts to label it an “inference gold rush.” As a result, ZML faces competition from companies like Baseten, which recently reached a $13 billion valuation; Inferact, founded by the creators of the open-source project vLLM; and RadixArk, which is commercializing SGLang.

While both vLLM and SGLang present competitive challenges for LLMD, Morin has larger aspirations for ZML. “We are at a phase where we co-design silicon,” he stated. He also emphasized the agility of ZML’s compact team of just 20, viewing this as vital to the startup’s ability to innovate quickly, with more releases anticipated soon.

This efficient team is also significantly well-funded for its size. Morin, a former VP of engineering at Zenly—a company acquired by Snapchat for a considerable sum in 2017—has secured $20 million from various venture capital sources, including Harry Stebbings’ 20VC, >commit, AALVC, Drysdale Ventures, Xavier Niel’s Kima Ventures, Kindred Capital, LocalGlobe, and Puzzle Ventures.

In contrast to ZML’s first public offering, an inference-centered ML framework launched in 2024 and updated in March, ZML/LLMD is not open-source. However, it is available as a free tool for gathering usage insights. “I prefer to assess the landscape and generate revenue where it matters most rather than hinder my growth by being overly greedy too soon,” Morin remarked.

It remains unclear when ZML/LLMD will shift to a paid model or how its adoption will progress. Nevertheless, the startup’s cap table indicates that prominent individuals are taking notice, including Dagger and Docker founder Solomon Hykes, Clément Delangue and Julien Chaumond from Hugging Face, and LeCun from AMI Labs. This underscores the potential for European AI startups to thrive locally. “I couldn’t have launched ZML anywhere but in Paris,” Morin noted.