TL;DR: AI models are now capable of autonomously discovering and exploiting software vulnerabilities at scale. Organisations that fail to implement strong governance and modern security practices risk falling behind both attackers, regulators, and more importantly customer expectations.
Table of Contents
What happens when AI can discover and exploit zero-day vulnerabilities overnight without human expertise or intervention?
That future isn’t theoretical anymore.
Recent advancements in AI capability are dramatically accelerating the pace of cyber threats. Models are no longer just assisting engineers; they are actively identifying and weaponising vulnerabilities faster than most organisations can respond.
In this post, you’ll learn what this shift means for your organisation, why governance is now a requirement (not a best practice), and what you should do to prepare.
The breakthrough: AI capability has changed the game
A new generation of AI models has demonstrated a step-change in cyber security capability.
In benchmark testing against real-world systems:
- Hundreds of vulnerabilities were analysed autonomously
- Over 180 working exploits were successfully generated
- Dozens of cases achieved deep system-level control
More concerning is how this was achieved. These results did not require expert security engineers. Non-specialists were able to prompt the model and receive working exploits within hours.
In one case, an AI system identified a 27-year-old vulnerability in OpenBSD, an operating system widely regarded as one of the most secure in the world.
With offensive capability now this accessible, the threat model for every organisation has fundamentally changed.
Why this matters: A new class of cyber threat
The implications for organisations are significant
AI-driven exploitation introduces:
- Speed at scale – vulnerabilities can be discovered and exploited faster than patch cycles
- Lower barrier to entry – advanced attacks no longer require deep expertise
- Increased attack surface – legacy systems and overlooked vulnerabilities become immediate targets
The result is an AI threat landscape where:
- Attackers can automate discovery and exploitation
- Zero-day vulnerabilities become more common
- Defensive teams are constantly reacting rather than preventing
- Capabilities that once required state-level resources are becoming increasingly accessible to individuals.
For many organisations, existing security practices were not designed for this level of speed or autonomy.
The governance gap: Organisations are not ready
While AI capability is accelerating, organisational readiness is not keeping pace.
Many organisations are constrained by legacy ways of working that slow coordination, delay remediation, and make it harder to respond to AI-driven threats at the speed required.
Structural debt under pressure
Structural debt reveals itself in common patterns across organisations.
- Siloed teams and fragmented ownership – security, engineering, and data functions often operate independently, slowing decision-making and response
- Manual, time-intensive processes – vulnerability management and incident response workflows are not designed for real-time or near-real-time threats
- Limited visibility across systems – gaps across cloud, data, and AI environments make it difficult to assess exposure accurately
- Inconsistent governance models – policies and controls are often applied unevenly, particularly for emerging AI use cases
These are not new problems, but the pressure of AI significantly amplifies their impact.
When vulnerabilities can be discovered and exploited in hours, delays caused by coordination overhead, unclear ownership, or manual processes become critical risk factors.
The widening response gap
This structural debt, coupled with the forcing function of AI, widens the gap between emerging threats and the speed at which organisations can respond.
Without changes to how teams operate, not just the tools they use, even well-resourced organisations will struggle to:
- Prioritise and remediate vulnerabilities effectively
- Manage AI-related risks in a consistent way
- Respond to emerging threats before they escalate
The challenge is now fundamentally organisational, not just technical.
In an AI-driven threat landscape, slow organisations become vulnerable organisations.
Regulators have recognised the same risk and are beginning to act.
What happens next: Regulation and industry response
Governments and industry leaders are already responding. In April 2026, APRA called for a step-change in AI-related risk management and governance, warning that “governance, risk management, assurance and operational resilience practices are not keeping pace with the scale, speed, and complexity of AI adoption.”
APRA has also stated it would “continue to assess the implications of these technological advancements to ensure the ongoing safety and resilience of the financial system.”
In the US, Treasury Secretary Bessent and Fed Chair Powell summoned Wall Street leaders to an urgent meeting on AI-driven cyber risks, signalling that regulators now treat these capabilities as a systemic financial stability concern.
Key developments include:
- Project Glasswing a coalition including AWS, Microsoft, CrowdStrike, and Palo Alto Networks, formed to proactively identify and patch vulnerabilities using advanced AI with restricted early access enabling controlled security testing before broader release.
- Growing regulatory pressure on organisations to demonstrate AI risk management, including upcoming EU regulations
As with early Cloud, regulation will catch up, and the window to act proactively before compliance becomes compulsory is narrowing.
What you should do now
Organisations need to act proactively to address this new risk landscape.
- Assess your exposure
- Identify where AI is used across your organisation
- Evaluate your vulnerability management maturity
- Understand dependencies on legacy systems
- Strengthen governance and ownership
- Define clear ownership of risk across security, engineering, and data.
- Implement policies that address AI usage alongside broader operational risk
- Align governance with emerging regulatory frameworks
- Modernise security practices
- Adopt DevSecOps approaches for faster remediation
- Improve visibility across cloud and data environments
- Automate detection and response where possible
- Strengthen vulnerability management
- Reduce patching cycles
- Prioritise critical systems and internet-facing assets
- Integrate threat intelligence into workflows
- Prepare for regulation
- Monitor upcoming AI and cybersecurity regulations
- Ensure auditability of AI systems
- Build incident response processes specific to AI risks
These steps are necessary, but they are not sufficient on their own. In our experience working with APRA-regulated organisations, the difference between governance that works and governance that stalls comes down to three things: bringing people along the journey, making the right way frictionless, and providing visibility into how the organisation actually operates.
When any of these break down, structural debt accumulates. When they work together, organisations build the adaptive capacity that AI-speed threats demand.
In a follow-up post, we will explore what paying down structural debt looks like in practice, and why the organisations that treat governance as an enabler rather than a constraint are better positioned to respond at pace.
Key takeaways
- AI can now autonomously identify and exploit vulnerabilities at scale
- Traditional security models cannot respond fast enough to AI-driven threats
- Organisational governance is becoming a regulatory requirement across major jurisdictions
- DevSecOps and automation are now critical capabilities for effective cyber resilience
- Organisations must modernise security operations urgently to close the growing response gap
Conclusion
AI-driven cyber capability is no longer a future risk; it is already here.
Organisations do not need to predict every AI-driven threat, but they do need governance and operating models capable of responding at AI speed. The sooner governance, security, and engineering teams align; the better positioned organisations will be to manage the next wave of AI-enabled cyber risk.
Frequently asked questions
What are AI-driven cyber threats?
AI-driven cyber threats use large language models and agentic AI systems to autonomously identify and exploit software vulnerabilities faster than traditional attackers. Unlike conventional threats, they can operate at scale without requiring deep human expertise.
Why is governance readiness a cybersecurity concern?
Governance determines how quickly an organisation can coordinate, decide, and act. Without effective governance – clear ownership, streamlined processes, and cross-team visibility – organisations lack the structure to respond to AI-enabled threats at the speed required, and face growing regulatory exposure.
How can organisations reduce AI security risks?
Organisations can reduce AI security risks by adopting DevSecOps practices, automating vulnerability management, strengthening governance frameworks, improving cross-team visibility, and modernising security operations to match the pace of AI-driven threats.
