Collaborative Real-Time AI Security: Engaging All Stakeholders, Including Google

I recently had the opportunity to engage in a thought-provoking conversation with Francis de Souza, COO of Google Cloud, in a lively backstage environment during an event in Los Angeles. With the charm of an academic, de Souza shared insightful views on the AI security challenges that organizations currently face, noting, “there’ll be a transition period, and then I think we get to this better place.”

Our conversation highlighted that his insights extend beyond Google, shedding light on the uncertainties even the tech giant is grappling with.

De Souza emphasized a crucial point reiterated by security experts: security cannot be ignored. “As organizations begin their AI journeys, they must adopt a platform-centric approach,” he remarked. “Security should not be an afterthought or entirely reliant on employees.” He specifically cautioned against “shadow AI,” where employees utilize consumer tools without sufficient oversight, urging companies to weave security, governance, and auditability into their strategies from the very beginning. “You cannot have an AI strategy without a data strategy and a security strategy; all must be interlinked.”

Importantly, he was not merely advocating for Google Cloud. When I pointed out that his comments seemed biased towards Google, he clarified that the company endorses a multicloud strategy, warning against a singular focus on one cloud. “Even if they rely on a single cloud, they are utilizing SaaS applications, and their partners might be operating across multiple clouds,” he clarified. “Organizations must uphold a consistent security posture across diverse cloud environments and models.”

De Souza also underscored the increasingly intricate threat landscape, stating that traditional defense strategies are rapidly becoming obsolete. He highlighted that the time from the initial breach to a full-blown attack has plummeted from eight hours to a mere 22 seconds, with attack surfaces now extending beyond classic network boundaries. “Beyond your standard environment, there are now models, data pipelines for training these models, agents, and prompts. All these components require rigorous protection.”

A lesser-known risk he pinpointed involves agents interacting with a company’s internal systems, revealing overlooked data storage locations. “Many organizations are still running outdated SharePoint servers and have neglected legacy access controls. However, agents within your organization can uncover those data assets and their contents.”

So, what solution does he propose? Aligning machine speed with machine speed. “We’re witnessing the rise of an AI-native, fully agent-driven defense mechanism that enables organizations to deploy agents to meet their security challenges,” he explained. “Rather than depending solely on human defenses, companies can now allow humans to supervise a fully agent-driven model.” He stressed that this issue has escalated into a matter of leadership, transcending tech boundaries. “This requires attention from the board and executive team, not just the security department.”

However, with AI taking on more security duties, the lack of qualified personnel to oversee these processes presents a significant hurdle, as AI-driven vulnerabilities are escalating faster than security teams can manage. “We will need skilled professionals to tackle the bug-pocalypse,” remarked Lea Kissner, LinkedIn’s chief information security officer, in a recent interview with The New York Times, indicating it may take years for the industry to fully grasp AI security.

This brings us back to platform providers. The Register reported that several Google Cloud developers faced unexpected five-figure charges due to unauthorized API calls to Gemini models—services many had not intentionally utilized or activated. These incidents followed a familiar pattern: API keys initially meant for Google Maps were granted access to Gemini because Google expanded their usage without adequate notice.

Rod Danan, CEO of the interview preparation platform Prentus, revealed he received a bill of $10,138 in just half an hour after attackers compromised his API key. Similarly, Isuru Fonseka, a developer in Sydney, found charges nearing AUD $17,000 despite having set his limit at $250. Unbeknownst to them, Google’s automated systems had escalated their billing tiers based on account activity, raising their effective limits to as much as $100,000 without explicit approval.

Following The Register’s initial report, Google issued refunds to both individuals. However, the company informed The Register that it does not intend to change its automatic tier-upgrade policy, prioritizing the prevention of service outages over user financial considerations.

Moreover, concerns have emerged regarding the procedures available to developers wishing to cease operations. The Register highlighted that even developers who quickly delete a compromised key may still be at risk. Research from the security firm Aikido revealed that attackers could continue exploiting that key for up to 23 minutes due to Google’s gradual key revocation process. Aikido researcher Joseph Leon explained that during this interval, success rates remain unpredictable—at times, over 90% of requests still authenticate—allowing attackers to extract files and cached conversation data from Gemini.

Leon noted that newer credential formats from Google seem to address this issue: service account API credentials revoke in about five seconds, while Gemini’s recent AQ-prefixed key format takes roughly a minute. “Both operate at Google scale,” he stated in Aikido’s relevant paper. “Both indicate that this issue can technically be resolved for Google API keys as well.” Essentially, Leon argues that the 23-minute delay is not a technical flaw but a matter of company priority.

This consideration takes on particular importance when we reflect on de Souza’s significant observations, which deserve earnest consideration. While he is undoubtedly correct, there exists a discernible gap between the platform’s recommendations and their implementation rate—an essential factor to remember.