Agentic AI is no longer just about chatbots. Autonomous AI agents now work across different workflows and systems, changing how businesses operate. These AI systems have persistent memory and can act with little human control, using non-human identities (NHIs).
This creates new security risks, increasing the likelihood of financial losses, legal problems, reputational damage, and operational disruption. To protect against these risks, companies use AI red teaming and penetration testing. These tests simulate attacks to find weaknesses like prompt injection, memory poisoning, and tool misuse, helping keep AI agents safe.
What is agentic AI?
Agentic AI refers to autonomous or semi-autonomous AI agents. LLMs power these agents and have both memory and tooling/ API access, making them both incredibly useful and high-risk as potential vectors for cyberattacks.
Agentic AI vs classic LLMs: What’s the Difference?
Whilst traditional Large Language Models (LLMs) excel at generating text responses to single prompts, agentic AI represents a fundamental shift in how artificial intelligence operates. Agentic AI systems can autonomously plan, execute multi-step tasks, use tools, and adapt their approach based on results—functioning more like a virtual assistant than a simple question-answering system.
Key Capabilities Comparison
| Capability | Classic LLMs | Agentic AI |
|---|---|---|
| Task Execution | Single-turn responses to prompts | Multi-step task completion with autonomous planning |
| Tool Usage | Cannot interact with external systems | Can use APIs, databases, search engines, and software tools |
| Reasoning Approach | Immediate response generation | Deliberative reasoning with reflection and course correction |
| Workflow Management | Requires human orchestration between steps | Autonomous workflow orchestration and task decomposition |
| Error Handling | Provides an incorrect answer or stops | Detects errors, adapts strategy, and retries with different approaches |
| Memory & Context | Limited to a single conversation context | Maintains persistent memory and learns from previous interactions |
| Decision Making | Responds to direct questions | Makes autonomous decisions about next steps and resource allocation |
| Task Complexity | Simple, well-defined queries | Complex, ambiguous problems requiring planning and iteration |
How businesses are using agentic AI today
With 23% of businesses scaling agentic AI systems within their enterprises, it’s clear that agentic AI has been widely embraced across various organisational functions, including operations, customer interactions, compliance, and AI-driven security controls.
AI models support human users by expediting mundane tasks that can be swiftly performed with LLM-based machine learning. What would take a human employee hours of research, scrolling and hand-picking quotes can be rapidly pieced together into good quality by many agentic AI models.
If you’re thinking of adopting agentic AI- or perhaps already have- you aren’t alone. 88% say their team or business function plans to increase AI-related budgets in the next 12 months due to agentic AI, clearly indicating its value in an organisational setting.
However, as with any autonomous system, use cases can amplify risk. Statefulness, autonomy, and integration depth all contribute to the level of exposure you may face, and it’s important to have a thorough understanding of how it’s being used within your organisation to ensure defensive measures are well-targeted.
Internal operations
Internal operations encompass tasks such as DevOps copilots, ticket triage, pull requests (PRs), automated tests, and deployments, where agentic AI systems assist in streamlining and automating these processes.
Customer-facing services
Customer-facing services include autonomous support agents that can modify accounts and issue refunds, highlighting the need for careful security oversight in these interactions.
Risk and compliance
Anti-money laundering (AML) and Know Your Customer (KYC) agents are employed to perform entity resolution and conduct regulatory checks, playing a critical role in ensuring compliance for businesses within the financial and fintech sectors.
Security operations
Security operations involve agents ingesting logs, enriching security alerts, and suggesting or executing response actions to potential threats.
Unsanctioned Deployments
Unsanctioned deployments by teams in email, CRM, and development tools represent an ongoing and growing security risk for businesses.
Agentic AI security risks
Agentic AI presents unique security challenges regarding vulnerability management. Security leaders will certainly be feeling increasing pressure to minimise agentic AI risks within their organisation and mitigate the potential of compromised agents.
To be successful in securing agents and safely championing AI adoption, it’s important to first know the associated risks related to agentic AI, so that your security team can take proactive steps in tightening operational controls and agentic systems.
Memory poisoning and long-term state attacks
Persistence across sessions leads to a slow manipulation of agent actions and behaviour, with the hopes of poisoning training data and autonomous AI systems to access sensitive data.
Tool misuse and over-permissive actions
Tool misuse is unfortunately all too common. When agents are entrusted to scripts, transactions, and system changes with minimal human oversight, they are left open to vulnerabiulity all vulnerable to prompt injection.
Privilege compromise and non‑human identity (NHI) sprawl
Unnecessary privilege escalation, overly privileged access controls, and poor lifecycle management all pose major risks to agentic AI security, allowing malicious actors to manipulate agent behaviour with minimal preventative security barriers or multi-factor authentication.
Weak segregation of duties and insufficient isolation between agentic AI components also increase the risk of unauthorised access and lateral movement within your business’s systems, making it easier for attackers to exploit vulnerabilities and escalate privileges.
Autonomous Malware
Autonomous malware powered by AI can adapt rapidly and blend seamlessly into network traffic, making detection increasingly difficult. Additionally, AI-assisted phishing campaigns can be executed at scale, significantly amplifying the threat landscape for organisations.
Supply chain and model-level vulnerabilities
Supply chain attacks pose a notable risk to businesses deploying agents. Attackers can poison model data to manipulate enterprise systems, allowing them unethical data access.
Backdoored plugins and complex agent ecosystems can introduce hidden vulnerabilities and malicious code that compromise the integrity, security, and reliability of autonomous AI agents, potentially leading to unauthorised actions and system-wide failures.
Governance, monitoring, and explainability gaps
Limited logging, no agent inventory, and difficulty auditing or tracing agent actions negatively impact the ability to maintain effective oversight, quickly identify security incidents, and ensure accountability within agentic AI systems, increasing your organisational risk and complicating compliance efforts.
How to secure AI agents
| Area | Safeguard Measures |
|---|---|
| Models | Evaluate model provenance, assess training data sources, review model cards and documentation, test for bias and failure modes |
| Plugins & Extensions | Verify publisher reputation, review permissions and access requirements, assess update frequency and maintenance, and test in isolated environments |
| Frameworks & Libraries | Audit open-source dependencies, monitor for known vulnerabilities (CVEs), track deprecation notices and security advisories, evaluate community support and activity |
| Code Review | Implement mandatory peer review for agent code, automated security scanning in CI/CD pipeline, test coverage requirements, and approval gates before production deployment |
| Dependency Management | Maintain software bill of materials (SBOM), automated vulnerability scanning, pin specific versions to prevent supply chain attacks, and regular dependency updates with testing |
| Vendor Contracts | Define data handling and privacy requirements, establish SLAs for security patching, clarify liability and incident response obligations, include audit rights and compliance attestations, specify data retention and deletion policies |
Meeting Regulations through Securing Your Agentic AI
All proactive security approaches must consider any compliance frameworks- both general standards and those relevant to your industry. Regulatory frameworks can be an excellent supplement for security professionals looking to ensure their critical infrastructure and agents remain aligned with industry standards, and understanding these is central in building a robust organisational security model.
Most businesses will be concerned with meeting the EU AI Act, NIST AI RMF, and ISO AI governance initiatives. Security testing your agentic AI is a critical component of being proactive in meeting compliance, and helps more clearly outline accountability and the necessity of educated human oversight.
Why reactive testing and LLM red teaming are vital to AI security
Traditional pentesting alone just isn’t sufficient in securing agentic AI. In fact, due to the agile nature of agentic AI systems, it barely scratches the surface of the range of unique potential threats your organisation could be facing. Here’s why reactive testing and LLM red teaming are so important:
- Behaviour and workflow risks only emerge under adversarial scenarios in AI systems.
- Red teaming tests can identify AI-specific threats, including prompt injection, memory poisoning, tool abuse, and NHI misuse.
- Conducting end-to-end flow testing and prioritising detailed reporting are crucial steps in thoroughly assessing and strengthening the security posture of agentic AI systems.
- LLM red teaming and reactive testing can more aptly give an oversight of your business impact, including the financial, regulatory, and operational implications of any AI-specific vulnerabilities identified during testing, helping you to optimally prioritise remediations.
Agentic AI can be business-transforming but incredibly high-risk for your organisation if misused. Maintaining control of your agentic AI systems requires a well-orchestrated blend of proactive security testing, governance, and monitoring.
Stay safe in deploying AI agents and ensure your security controls stay aligned with OnSecurity’s platform-based pentesting services. Get a free, instant quote today.
