Collaborative Real-Time AI Security: Engaging All Stakeholders, Including Google

I recently had the opportunity to engage in a thought-provoking discussion with Francis de Souza, COO of Google Cloud, in an animated backstage environment during a Los Angeles event. Displaying an aura akin to that of a university professor, de Souza offered valuable perspectives on the AI security issues organizations face today, remarking, “there’ll be a transition period, and then I think we get to this better place.”

Our conversation made it clear that his insights went beyond Google, capturing the uncertainty that even Google experiences.

De Souza highlighted an essential point long stressed by security professionals: neglecting security is not a feasible strategy. “As businesses begin their AI journeys, they must embrace a platform-centric approach,” he asserted. “Security should not be an afterthought, nor should it rely solely on employees.” He specifically cautioned against the dangers of “shadow AI,” where employees utilize consumer tools without proper oversight, urging organizations to incorporate security, governance, and auditability right from the start. “There’s no AI strategy without a data strategy and a security strategy; they must be intertwined.”

Importantly, he wasn’t just promoting Google Cloud. When I suggested that his comments seemed to favor Google, he clarified that Google supports a multicloud strategy, warning that organizations too focused on a singular cloud may be misled. “Even if they rely on one cloud, they’re using SaaS applications, and their partners might operate on multiple clouds,” he explained. “Organizations need to maintain a consistent security posture across various cloud environments and models.”

De Souza also pointed to the increasingly intricate threat landscape, noting that traditional defense strategies are becoming obsolete. He remarked that the time from initial breach to actual attack has significantly decreased from eight hours to just 22 seconds, with the attack surface now extending beyond traditional network boundaries. “Beyond your usual environment, there are now models, data pipelines for training these models, agents, and prompts. All these elements demand robust protection.”

A lesser-known risk he highlighted involves agents that interact with an organization’s internal systems, exposing overlooked locations of data storage. “Many organizations still operate outdated SharePoint servers and legacy access controls that have been neglected for too long. Yet, agents within your organization will identify those data assets and their contents.”

His solution? Align machine pace with machine pace. “We are witnessing the emergence of an AI-native, fully agent-driven defense mechanism that enables organizations to deploy agents to meet their security needs,” he explained. “Instead of depending solely on human-led defenses, businesses can now have humans supervising a fully agent-driven model.” He stressed that this issue has escalated into a leadership concern, not merely a technical one. “This matter requires focus from the board and executive team, not just the security department.”

However, as AI takes on more security duties, the shortage of qualified professionals to manage these processes remains a concern, with AI-driven vulnerabilities outpacing the capacity of security teams. “We will need skilled professionals to tackle the bug-pocalypse,” remarked Lea Kissner, LinkedIn’s chief information security officer, in a recent interview with The New York Times, noting that it may take years for the industry to fully understand AI security.

This leads us back to platform providers. The Register recently reported an uptick in Google Cloud developers facing unexpected five-figure charges due to unauthorized API calls to Gemini models—services many had not intentionally used or activated. These incidents followed a well-established pattern: API keys initially designated for Google Maps had quietly been granted access to Gemini due to Google expanding their usage without sufficient notice.

Rod Danan, CEO of the interview-preparation platform Prentus, shared that he received a bill of $10,138 in just 30 minutes after attackers compromised his API key. Similarly, Isuru Fonseka, a developer in Sydney, discovered charges nearing AUD $17,000 despite believing his limit was set at $250. Unbeknownst to them, Google’s automated systems had raised their billing tiers based on account activity, increasing their effective limits to as much as $100,000 without explicit consent.

Following The Register’s initial report, Google issued refunds to both individuals. However, the company informed The Register that it does not plan to modify its automatic tier-upgrade policy, prioritizing the prevention of service outages over user budget considerations.

Additionally, concerns have arisen regarding the processes for developers wishing to halt operations. The Register highlighted that even developers who quickly delete a compromised key might still be at risk. Research from security firm Aikido indicated that attackers could continue to exploit that key for up to 23 minutes due to Google’s gradual key revocation process. Aikido researcher Joseph Leon explained that during this period, success rates remain unpredictable—at certain times, over 90% of requests still authenticate—allowing attackers a chance to extract files and cached conversation data from Gemini.

Leon noted that newer credential formats from Google seem to address this issue: service account API credentials revoke in around five seconds, while Gemini’s recent AQ-prefixed key format takes about a minute. “Both operate at Google scale,” he stated in Aikido’s relevant paper. “Both indicate that this issue can technically be rectified for Google API keys as well.” Essentially, Leon argues that the 23-minute delay is not a technical limitation but rather a question of prioritization for the company.

This consideration becomes particularly significant when contemplating de Souza’s poignant observations, which warrant serious attention. While he is undoubtedly correct, there is a noticeable gap between the platforms’ recommendations and their adaptation rate—an important element to consider.