Robot Brain Developers Are Advancing Past the GPT-2 Era

The field of Physical AI is rapidly becoming one of the most vibrant segments in venture capital, as companies attract billions of dollars to harness the technology underpinning Large Language Models within robotics.

This excitement culminated in a notable IPO for Unitree, the leading robotics manufacturer in China, which achieved an impressive valuation of $66 billion during its launch on a Chinese stock exchange similar to NASDAQ. However, this week, the company faced a steep decline, nearly halving its market value. Analysts express concerns that despite advancements in the physical capabilities of robots, they still fall short in executing tasks that generate value effectively.

At the recent Actuate conference—an event for developers creating AI systems for robotics—the atmosphere was charged with enthusiasm. Since its launch in 2023, attendance has surged threefold, reaching 1,500 participants, according to the event’s organizer, Foxglove, a company devoted to supporting builders of physical AI models in managing and visualizing their datasets.

Warning signs of the associated risks were also evident; a display at Avala’s booth, another company in the physical AI infrastructure sector, highlighted a solution for the “robotics data crisis.”

This crisis pertains to the deficit of high-quality training data essential for AI models. The ambition to develop generalized robots that can tackle various tasks is still a long way off, while the application of end-to-end learning for specific tasks has not yet produced consistently commercially viable products. Developers are urged to emulate the progress made in leading AI laboratories by acquiring or creating more diverse datasets, experimenting with various training techniques, and exploring enhanced reinforcement learning scenarios.

Harry Mellsop, co-founder of Antioch, a startup focusing on simulation tools for model builders, states that physical AI is in its “GPT 2 era,” likening it to the OpenAI model that preceded ChatGPT. He underscores the need for more data and computational power to transcend present constraints, particularly the necessity for GPUs optimized for ray tracing to produce high-fidelity simulations.

Currently, autonomous vehicles are at the forefront, partially due to their ability to collect relevant data from human-driven cars, and partly because their main goal is to avoid collisions rather than engage physically with their surroundings. Much of the tooling for model-building is adapted from autonomous vehicle developers; for example, Foxglove was established by former staff of Cruise, the self-driving project of General Motors.

Now, automobile manufacturers are increasingly banking on their investments in machine learning tools to enable them to compete with specialized humanoid robot developers. Tesla is already making progress with its Optimus robot, while both Wayve, focused on autonomous vehicles, and ride-sharing giant Uber have opened robotics labs dedicated to humanoid designs as part of their research and development initiatives.

“I believe you need to start with vehicles… manipulation robotics is similar to self-driving technology from five years ago,” said Alex Kendall, CEO of Wayve, in an interview with TechCrunch. “The data infrastructure, simulation, and ML operations will probably be interchangeable; however, the specific world model for the simulator will vary post-training. There will be significant similarities, but also necessary differences for various implementations.”

Kendall argues that it’s too early to commit to any specific hardware platform, given the rapid evolution of sensors and other components, and that a truly universal model should be more hardware-agnostic.

Théophile Gervet, CEO of Genesis AI—a vertically-integrated humanoid robotics firm that raised $105 million in seed funding this year—disagreed, telling TechCrunch that “we’re too early in this wave for a brain strategy to be effective; there are substantial opportunities for co-designing hardware and AI.”

Gervet also brought up another critical issue in the sector: the targeted focus of physical AI businesses. Companies concentrating on specific tasks are successfully deploying their robots—like Gritt, developing solar farms; Agility, utilizing robots in industrial settings; and Bedrock, autonomously operating excavators. In contrast, general-purpose humanoid robots remain confined to labs.

“Customers are not interested in a general-purpose robot that operates at an 80% success rate,” Gervet observed about the dilemma. “Many players are pursuing a general approach, but they offer little value due to the lack of vertical specialization… On the other hand, if you’re developing for a niche tailored to GPT 2, you risk being overshadowed by rivals leveraging GPT 4.”

Nonetheless, the temptation to invest in a specific vertical is strong as it not only yields revenue but also gathers invaluable real-world deployment data. While task-specific data may not have the variety needed to improve general-purpose models, it is crucial for creating robots that deliver real value. Kevin Peterson, CTO of Bedrock, highlighted that his company is initially focusing on excavation to better grasp the challenges of “manipulation in the wild,” while planning to develop an intelligence layer applicable across a range of construction machines.

Handling such data introduces its own challenges, particularly considering the richness of visual and LiDAR data. Foxglove recently launched a new product leveraging Nvidia’s Cosmos open weight world model, allowing engineers to navigate data through sophisticated natural language queries for evaluation and simulation purposes. The goal is to improve triage and debugging, enabling model builders to iterate more quickly.

So, what could be the anticipated ChatGPT moment for physical AI that Sam Altman suggested might materialize in just a few years? Kendall posits that the most widespread robot deployment currently consists of consumer vacuum bots. For him, a ChatGPT moment would be something that ignites consumer excitement, rather than just attracting investor interest, which seems plentiful already.

“One potential example would be achieving eyes-off autonomy for under $1,000 worth of hardware in a vehicle,” Kendall explains; not by coincidence, his company is licensing models to automotive manufacturers to achieve precisely that. He perceives this venture as a multi-billion dollar opportunity that will enable the development of a truly general embodied AI model.

For Gervet, the moment when physical AI genuinely becomes a reality is typified by “manipulation that functions seamlessly out of the box. You should be able to communicate with a robot in natural language to carry out any basic manipulation task—like pushing, pulling, closing a laptop, or cleaning a table—and have it work reliably to an extent of around 80% or better right from the outset—that’s similar to your ChatGPT experience.”

Adrian Macneil, CEO of Foxglove, offers a different perspective on the matter.

“There won’t be a singular ChatGPT moment for robotics,” he shared with TechCrunch. “The factor that made ChatGPT a significant event was distribution—they soared from zero to a million active users in just a week… achieving distribution in the real world is far more complex. I would be delighted for the emergence of an Apple II moment in robotics or an IBM PC moment—when can we purchase a home robot that starts performing useful and entertaining tasks?”