OTHER

Can Safeworld Ensure the Safety of Generative AI Robots?

A new movement in robotics is shifting control to generative AI models, introducing a key challenge: this method often lacks the reliability associated with conventional algorithms. What steps can you take to ensure the safety of your humanoid robot?

Dr. Ding Zhao, the director of the Safe AI lab at Carnegie Mellon University, has dedicated his career to tackling this pressing issue. Together with seasoned startup executive Kyle Wong and machine learning engineer Simo Rachidi, he co-founded Safeworld, a company focused on addressing these safety concerns.

“The safety challenge we face focuses on complex probabilistic evaluations of generative AI—how do we evaluate risks within a system utilizing probabilities?” Zhao explained. “Trust is the second complicating factor, as both elements are essential for the effective deployment of robots.”

Emerging from its stealth stage, Safeworld has successfully secured over $12 million in seed funding, led by Shine Capital and a16z Speedrun, with additional contributions from Box Group, the Carnegie Mellon University Endowment, Innovation Endeavors, and SV Angel.

“This is the ideal time to establish safety standards in the industry as robots are being designed and released,” remarked Jonathan Lai, a partner at a16z Speedrun, in an interview with TechCrunch. “It will be far too late if we wait until robots are in homes, interacting with children, and creating safety risks.”

Safeworld is focusing on evaluating robotic control systems through realistic simulations that incorporate human models. This effort reflects the difficulties faced by companies like Tesla and Wayve, which must ensure that their vehicles can react appropriately to unexpected road situations. However, Zhao emphasizes that robots encounter even greater complexity as they navigate unstructured environments and comply with diverse safety regulations.

“A common issue is managing blind spots in factories,” Wong observed. “What speed or stopping distance is necessary to avoid a collision between a robot and a human? For example, if someone is carrying boxes, can the robot identify that person?”

To tackle this problem, Safeworld intends to create a digital model of a factory corner using tools like Genesis or MuJoCo, integrate a simulation of the robot with its actual software, and run thousands of scenarios that involve human interactions with the robot. Zhao points out that this project is more intricate than it appears due to the unpredictable nature of human behavior.

“We also conduct comprehensive tests on scenarios involving tripping and falling in our simulations,” Wong said. “Otherwise, consistently replicating human trips and falls would be extremely challenging.”

There are clear connections between Safeworld’s platform and the proprietary tools utilized by robotics manufacturers. Nevertheless, the founders believe that, in addition to their specialized knowledge, robot developers will need third-party validation to facilitate the sharing of safety case data among competing firms.

“Many people underestimate the complexities of these edge cases,” Zhao warned. “Our concern extends beyond the robot demonstrated in a showcase; it encompasses the robot functioning effectively at scale, especially among individuals who may have never interacted with a robot before.”

Vishal Dugar, CTO of Gritt Robotics, is developing AI systems for robots that assist workers in installing photovoltaic panels at large solar facilities, striving to overcome increasingly complex construction challenges. His company is collaborating with Safeworld to improve their safety simulations.

“The challenge with many of our systems is achieving formal safety verification through mathematical proofs or formulas,” Dugar explained. “Empirical testing remains vital.”

As robots collaborate with humans, ensuring that robotic arms do not unintentionally hit human workers is essential. Validating this in real-world environments necessitates accounting for numerous potential scenarios.

“Humans present a wide variety of appearances,” Dugar noted. “Their postures can vary—standing, kneeling, tripping, or running. It’s crucial to consider all possible human behaviors and variations in appearance, such as attire, size, shape, height, skin color, and more.”

As both Safeworld and generative AI in robotics continue to advance, the company is evaluating the best approach for its offerings—whether to develop a platform for external users or adopt a service-oriented model—but the team is confident they are tackling a major challenge.

“We are on course to become the first profitable company in this domain,” Zhao declared. “Any organization aiming to implement these systems will need to partner with us to address safety issues.”

When you purchase through links in our articles, we may earn a small commission. This does not affect our editorial independence.