Methodology
How we test AI systems
The full method behind an AI Offensive Security Review. Written to be forwarded to whoever owns your penetration testing panel, so they can assess it against the standard they already apply.
Position
Manual testing, against your system.
Automated AI testing tools run a list of known payloads and report which ones got through. That is a useful smoke test and it is not a penetration test, because the payload list is public and the interesting failures are specific to how you assembled your system.
What produces findings is a person who understands your architecture looking for the trust boundary you did not draw. A retrieval index that mixes tenants. A tool that accepts model output as validated input. An agent that inherited an engineer's credentials. None of those appear in a payload list.
We use automation where it helps with coverage, and the findings come from manual work.
Phases
Five phases, with what each one needs from you.
A typical engagement runs two to four weeks depending on the size of the attack surface. Phases overlap. Reporting starts at discovery, not at the end.
01
Scoping and threat model
We establish what the system is, who uses it, what it can reach, and what a successful attack would mean for your business. The threat model drives the test plan, so a customer-facing assistant with no tools and an internal agent with write access to finance systems get different engagements.
What this phase covers
- Architecture diagram or a walkthrough with an engineer
- List of tools, functions and integrations the model can call
- Data sources and retrieval indexes in scope
- User roles and tenancy model
- Agent identities and the permissions attached to them
Output
Agreed scope, threat model, rules of engagement and a signed authorisation letter.
02
Attack surface mapping
We map every point where content that a person outside your trust boundary can influence reaches the model, and every action the model can take as a result. Most high-severity findings are discovered here rather than during exploitation, because the finding is usually a trust boundary nobody drew.
What this phase covers
- Every channel that reaches the prompt: user input, documents, email, tickets, web content, logs
- Every tool the model can call, and the arguments it controls
- Every outbound path: rendered links, images, tool responses, webhooks
- Retrieval and memory ingestion paths
- Trust boundaries between tenants, users and business units
Output
A map of the real attack surface, usually including paths the client did not know existed.
03
Adversarial testing
Manual testing against each in-scope attack path. We test through the channels your system actually reads rather than running a generic prompt list, because the interesting failures are specific to how you assembled the system.
What this phase covers
- Direct and indirect prompt injection through each identified channel
- Tool and function-call abuse, including argument injection
- RAG and memory poisoning through content ingestion
- Jailbreak families that transfer across vendors
- Identity abuse and impersonation of non-human actors
- Exfiltration through rendered output and tool responses
Output
Confirmed weaknesses with working reproductions, reported as they are found.
04
Exploitation and chaining
Individual weaknesses are combined into the paths a real attacker would use. This is the phase that turns a set of medium findings into an accurate severity picture, because impact comes from chains rather than from isolated issues.
What this phase covers
- Chaining injection with over-permissioned tools
- Privilege escalation across agent and user boundaries
- Cross-tenant and cross-user data access
- Persistence through poisoned memory or stored content
- End-to-end demonstration of business impact
Output
Demonstrated impact, with the full chain documented step by step.
05
Reporting and retest
Critical and high findings are reported within 24 hours of discovery so remediation can start before the engagement ends. The final report is written for three audiences at once and is structured to be handed to an auditor without rewriting.
What this phase covers
- Executive summary for a non-technical reader
- Technical findings with reproductions and evidence
- CVSS v3.1 severity with environmental scoring
- OWASP Top 10 for LLMs and MITRE ATLAS mapping
- CPS 234 mapping for APRA-regulated clients
- Prioritised remediation with effort estimates
Output
Report, technical debrief, optional board briefing, and a free retest within 60 days.
Frameworks
What findings map to, and why.
Every finding carries a framework reference so the report is auditable and the method is defensible to someone who was not in the room.
OWASP Top 10 for LLM Applications
Primary classification for findings against generative AI applications.
OWASP Top 10 for Agentic Applications
Classification for agent-specific risk: goal hijack, tool misuse, identity and privilege abuse, memory poisoning.
MITRE ATLAS
Adversary technique mapping for AI systems, so findings sit alongside your ATT&CK-based reporting.
NIST AI RMF
Mapping findings to the govern, map, measure and manage functions for your risk register.
CISA and ASD ACSC agentic AI guidance
Control expectations for agentic deployments: least privilege, distinct identity, logging, human oversight.
APRA CPS 234
Reporting mapping for regulated entities, so findings drop into your existing evidence pack.
CVSS v3.1
Severity scoring with environmental metrics, so ratings reflect your context rather than a generic base score.
Scope boundary
What we do not test.
Stating this up front saves a procurement conversation later, and gives you a document to show your model vendor.
- × The hosted model itself. We do not attack the model vendor's infrastructure or attempt to extract their weights.
- × Denial of service against production, unless you specifically ask for it and we agree a window.
- × Social engineering of your staff, unless it is scoped as a separate red team objective.
- × Anything outside the assets named in the authorisation letter.
The reason is practical as well as contractual. Your exploitable risk is in the integration, the data, the tools and the identity layer, and that is where findings come from.
Preparation
What your team needs to have ready.
- A named technical contact who can answer architecture questions
- Test accounts at each privilege level, including the lowest
- A non-production environment where possible, or an agreed production window
- Signed authorisation letter and agreed escalation contacts
- Confirmation of who to tell if we find something critical at 2am
- Your model vendor's terms, if you want us to confirm the boundary against them
Get started
Forward this to your panel owner.
If the method meets your standard, a scoping call is the next step. If your panel needs something this page does not cover, tell us and we will answer it.