AI-written code, AI-powered features and AI agents with tools

“What is our AI risk?” is now a standing board question, and it is usually asked as if AI were one thing. It is at least three: code written by AI assistants, products that embed language models, and agents that use tools with real permissions. They have different failure modes, different owners and different controls. Treating them as one topic is why so many AI security policies are either too vague to apply or too strict to follow.

Three surfaces, side by side

	1 · AI-written code	2 · AI-powered features	3 · AI agents
What it is	Code suggested or generated by assistants, merged by developers	LLM calls inside your product: chat, summarisation, RAG	Models that choose and call tools: shell, APIs, files, email
Main risks	More code, faster; plausible but insecure patterns; invented or outdated dependencies	Prompt injection, output reaching eval or SQL, poisoned or unsafe model files, leaking secrets or data	Excessive agency, privilege misuse, unbounded loops, actions triggered by untrusted content
Natural owner	Engineering and AppSec	Product engineering and AppSec	Platform, identity and security architecture
Control that matters most	The same scanning and review gates, applied consistently	Secure design patterns and testing against the OWASP LLM Top 10	Inventory, least privilege and a policy decision before each tool call

The further right, the more an AI failure becomes an action rather than a bug.

Surface 1: code written with AI

Developer adoption is no longer a question. Stack Overflow’s 2024 developer survey found roughly three-quarters of respondents were using or planning to use AI tools in their development process. The security effect is not that AI code is uniquely dangerous; academic studies have found both better and worse outcomes depending on the task. The effect is volume. More code and more dependencies arrive per developer per week, and review capacity does not grow with them.

The controls are the familiar ones — static analysis, dependency scanning, secrets detection, a CI gate — but they have to be automatic and quiet enough that developers do not route around them. If AI doubles the rate of change, a noisy gate doubles the friction.

Surface 2: AI inside your product

Once a product calls a model, new classes of vulnerability appear. OWASP’s Top 10 for Large Language Model Applications, revised for 2025, is the most widely used catalogue. Several of its entries can be found directly in code before anything runs.

OWASP LLM 2025	What it looks like in code	Business impact
LLM01 Prompt injection	User input formatted straight into a prompt	The model follows an attacker’s instructions instead of yours
LLM02 Sensitive information disclosure	A provider key shipped to the browser; secrets in configs	Cost abuse, data leakage
LLM03 Supply chain	Pickle model files that can execute code on load; trust_remote_code=True	Code execution from a downloaded model
LLM05 Improper output handling	A model’s answer passed to eval, a shell or SQL	Classic injection, with the model as the attacker’s proxy
LLM07 System prompt leakage	Credentials written into a system prompt	Anyone who extracts the prompt gets the key

The model supply chain deserves particular attention. Several common model formats, including Python pickle, can run arbitrary code when loaded, and security researchers have repeatedly found malicious model files on public model hubs. A data science team downloading a model is, in effect, running code from the internet.

Surface 3: agents that act

An agent is a model in a loop that chooses tools: run a command, call an API, read a file, send a message. Protocols such as the Model Context Protocol, introduced in late 2024, have made it easy to give an agent dozens of tools at once. The security question changes from “what might the model say?” to “what might it do, with whose credentials, because of whose text?”

Illustrative: prompt injection reaching an agent with tools, stopped by a policy decision before the call.

The useful framing for executives is identity, not intelligence. An agent is a non-human identity with a set of permissions that takes instructions from content it reads. The same principles apply as for any privileged service account: know it exists, give it the least privilege it needs, require approval for consequential actions, log what it does, and cap how much it can do per hour.

Start with an inventory

Every one of these controls depends on knowing what AI exists. Surveys of engineering teams go stale quickly; an inventory derived from code updates on every push. InsightFortify’s AI analyser reads each repository for model weight files, models named in code, LangChain, CrewAI and AutoGen agents with their tools, and MCP servers in client configurations. It reads model files without loading them and code without running it.

Not every agent needs the same controls

The fastest way to stall AI adoption is to apply the strictest controls to every agent. The fastest way to an incident is to apply none. Tiering agents by the consequence of their actions lets most of them ship quickly while the few that matter get real scrutiny.

Tier	Example	What it can do	Minimum controls
1 · Read-only	Internal documentation assistant	Search and summarise approved sources	Inventory, logging, no credentials in prompts
2 · Internal write	Engineering agent that opens tickets or drafts code	Create or change internal records	Tier 1 plus least-privilege tools, rate limits, human review of changes
3 · External action	Support agent that issues refunds or emails customers	Act on customers, money or external systems	Tier 2 plus approval for consequential tools, deny lists for hosts, per-call policy decision
4 · Privileged	Operations agent with shell or cloud admin tools	Change infrastructure	Tier 3 plus sandboxing, no internet-facing input, named human owner, frequent review

InsightFortify’s guardrail policies map directly onto these tiers: deny_tool, require_approval, deny_host and max_calls_per_hour, each bound to the agents it governs. The same policies are checked against the inventory, so an agent that merely holds a denied tool is a finding before it ever uses it.

Questions boards ask about AI, and answers that hold up

The question	A weak answer	A strong answer
Where are we using AI?	“We surveyed the teams.”	“Our scans find 14 models, 6 agents and 9 MCP servers across 120 repositories; here is the list, updated on every push.”
Could an AI system leak customer data?	“We have a policy against it.”	“Two features had prompt-injection paths; both are fixed. No agent with access to customer data can call external hosts.”
What can our agents do without a human?	“Only what they are designed to.”	“Refunds and outbound email require approval; shell tools are denied for anything customer-facing. Last quarter 37 calls were blocked.”
Are we exposed through AI suppliers?	“We use reputable vendors.”	“All model files are safetensors from pinned sources; remote code execution on load is disabled everywhere.”

The numbers in the strong answers are illustrative. What makes them strong is that each one comes from an inventory or a log, not a survey, and so it can be checked and repeated next quarter.

A control set for each surface

Keep reading