How AI Could Add Complexity to Government Utilization of Hacking Tools

In early August, cryptography expert Matthew Green sparked intense debate on X with a provocative thread and a detailed blog post that quickly resonated within the cybersecurity community.

Green, who has been closely following the ongoing dialogue about government hacking tools aimed at preventing crime versus the imperative of strong encryption to protect the privacy of law-abiding citizens, posed a compelling question: What if AI advances to the point where there is such a reduction in bugs that law enforcement and intelligence agencies can no longer legally exploit criminals?

“I’m concerned that AI might make software overly secure,” Green remarked, warning that the U.S. government could lose access to the security vulnerabilities necessary for monitoring targets as companies fix an unprecedented amount of bugs.

Traditionally, law enforcement has claimed that encryption hinders their ability to capture criminals and terrorists. The phrase “going dark” gained traction in 2014 when then-FBI director James Comey cautioned that encryption could prevent authorities from intercepting conversations or accessing data on various devices.

During this time, apps like Signal, WhatsApp, and Apple’s iMessage introduced end-to-end encryption to the public, rendering traditional real-time wiretaps of calls and texts nearly impossible. Furthermore, tech giants such as Apple began to encrypt data on their devices by default, complicating access to iPhones safeguarded by strong PIN codes or passphrases.

Since then, law enforcement has continued to apprehend criminals — including utilizing vulnerabilities in their devices — while innocent civilians have benefited from enhanced privacy due to encryption. Green notes that this has been facilitated by what he describes as “an uneasy kind of truce.” Rather than embedding backdoors into devices for authorities, governments have opted to invest in purchasing hacking tools and spyware that can bypass device and user security.

For Green, this fragile agreement is on the verge of being upended by AI, as advocates suggest and early data supports that LLMs are advancing in their ability to efficiently identify security vulnerabilities on a large scale. This indicates that companies may soon reach a point where their software and systems are much less prone to bugs and attacks.

Green warns that this outcome may prompt governments to once again demand backdoors, intentionally compromising the security of all devices.

A gold rush of bugs

We engaged several individuals to gather their views on Green’s statements, including privacy and cybersecurity experts, as well as hackers who have experience creating hacking tools for governments. Some echoed Green’s concerns, while others presented opposing opinions or recognized the validity of both perspectives.

Luna Tong, a researcher who has worked with two notable companies that focus on bug discovery and exploit creation to help governments infiltrate systems, agreed with Green, asserting that we are currently witnessing a “gold rush of bugs,” but this will be a short-lived phenomenon as bugs will soon become rare.

Another researcher, who has spent over a decade in offensive security firms, voiced concerns that AI could make human security researchers obsolete as bug identification becomes increasingly difficult, potentially giving defenders an advantage over offensive researchers. This individual chose to remain anonymous to speak freely.

“It’s clear that no state will give up the possibility of surveillance,” stated Paolo Stagno, the chief technology officer at Crowdfense, a leading firm that develops, acquires, and sells undisclosed vulnerabilities — or zero-days — to governments. Stagno expressed that the current landscape, which requires governments to exploit security weaknesses to access devices, is the “most democratic system we have,” but this may change if bugs become too hard to find.

Three other professionals from the offensive cybersecurity realm, along with one former member, disagreed. Their arguments suggest that while readily available bugs may be easier to discover, more complex and valuable vulnerabilities that are particularly advantageous for governments will remain; and that AI can actively assist researchers in selling vulnerabilities to government entities.

Hamid Kashfi, founder of the offensive security company DarkCell and an employee at the AI security startup Xbow, insisted, “For every AI-discovered and reported bug, there are probably 20 that go unreported.” Kashfi explained that researchers who opt not to disclose bugs to vendors can still uncover intricate and valuable vulnerabilities.

Two researchers currently engaged in discovering bugs for zero-day firms told TechCrunch that they are more concerned about new security measures in modern devices that complicate hacking than they are about the rise of AI.

Eva Galperin, director of cybersecurity at the Electronic Frontier Foundation and a specialist in government spyware, contended that offense currently has the edge due to a mix of AI’s powerful ability to unveil vulnerabilities and an uptick in vulnerabilities from “vibe-code” development using AI tools.

However, Galperin argued that finding more bugs does not necessarily equate to faster or even impactful patching, given the complexities tied to the patching process. She also pointed out that there will always be renewed demands for backdoors, as authoritarian regimes consistently seek “exceptional access.”

Katie Moussouris, who has helped both large and small organizations address and patch reported bugs for years, stated, “We still have a considerable way to go before the latest smartphones and laptops are entirely free of bugs.”

“There will be a point when finding bugs becomes significantly more difficult, which may lead to pressure for the implementation of backdoors,” Moussouris, the founder and CEO of Luta Security, observed.

“I believe we have at least until after the next presidential election before the intelligence community is significantly hindered enough to push for backdoors seriously,” Moussouris concluded.