AI Firewall vs LLM Guardrails: The Three Layers, and Which One You Need

LLM guardrails are checks on what goes into an AI model and what comes out. An AI firewall (or LLM firewall) is the version that sits in the traffic path, outside your code, so it screens every request without the application having to ask. In practice there are three layers: safety trained into the model, guardrail libraries you wire into software you build, and firewalls or gateways that screen traffic from tools you did not write. If you only use AI tools, the third layer is the one you can add. If you build with AI, you want the second and the third. The first you get for free and cannot tune.

Everything below was checked on the vendors' own pages, repositories and documentation on 23 September 2026.

The product pages are a mess: libraries call themselves firewalls, firewalls call themselves guardrails, and gateways sell both. This page sorts them by one question: where does the check run? That decides what it can see, who sets it up, and which of your tools it covers.

What are LLM guardrails?

A guardrail is any rule that inspects text on its way into or out of a model and decides: allow, change, flag or stop. They cover five different risks, and no single product covers all five equally well:

1. Prompt injection and jailbreaks. Instructions hidden in content the AI reads, or a user trying to talk the model out of its rules. Our prompt injection explainer has a live demo.
2. Data leaving. Personal data, customer records, API keys and passwords going to a provider, or coming back in an answer.
3. Harmful content. Hate, violence, self-harm, sexual content. This is content moderation.
4. Staying on task. Off-topic answers, a support bot giving legal advice, a competitor's name in your copy.
5. Format and correctness. Valid JSON, a required structure, answers grounded in the source.

An agency worried about a leaked client list and a SaaS company worried about its chatbot saying something awful need different things. Know which of the five you are buying for.

The three layers, at a glance

Layer	Examples (checked 23 Sep 2026)	Where it runs	Covers tools you did not write?	Main job	Cost to you
1. Model-side safety	Safety training by OpenAI, Anthropic, Google, Meta	Inside the model	Yes, but only that model's rules	Refusing harmful requests, resisting jailbreaks	Included; not tunable
2. Guardrail libraries	NVIDIA NeMo Guardrails, Guardrails AI, Meta LlamaFirewall, Llama Prompt Guard 2, Llama Guard 4	Inside your own application	No	Whatever you configure: injection, content, topic, format	Open source; you host and wire it
3a. Detection APIs	Lakera Guard, Azure AI Content Safety Prompt Shields	A cloud service your code calls	No	Injection detection; Azure also moderates content	Per call or plan
3b. Firewalls and gateways	Constellation Gate AI, Cloudflare AI Security for Apps, Portkey guardrails, Kong AI Prompt Guard	In the traffic path	Yes, if you can point the tool at it	Screening every request and response, logging	From free; blocking usually paid

Layer 3a is a hybrid: the checking happens in someone else's cloud, but your code still has to call it, so for coverage it behaves like a library.

Layer 1: the safety trained into the model

Every major model is trained to refuse some requests and to resist being argued out of its rules. OpenAI researchers described one version, the "instruction hierarchy", in April 2024: train the model to rank developer instructions above user input and ignore lower-privileged instructions that conflict.

What it catches: a large share of jailbreaks and harmful-content requests, on every tool that uses the model.

What it misses: your rules. The model does not know your client list is confidential or that your support bot should not discuss pricing. And it is not built to be your last line against injection: Anthropic, for example, describes defending its browser agent against prompt injection as ongoing work, not a solved problem.

What you do: nothing, which is the point. Count it as a floor, not a plan.

Layer 2: guardrail libraries you wire into your own code

If you build a chatbot, an agent or an AI feature, you can put checks inside it. These are the main open-source options.

NVIDIA NeMo Guardrails is "an open-source toolkit for easily adding programmable guardrails to LLM-based conversational systems", under Apache 2.0. It runs as a Python library inside your app or as a standalone server. Its useful idea is five kinds of "rails": input rails (on the user's message), dialog rails (steering the conversation), retrieval rails (on chunks pulled in for RAG), execution rails (on the input and output of tools the model calls) and output rails. It ships with jailbreak detection and content-safety checks among its built-in guardrails.

Guardrails AI is a Python framework, also Apache 2.0, built around validators. You combine validators from its Guardrails Hub into input and output "Guards" that wrap your model calls; it can also run as a REST service. It leans towards reliability as much as security, including making output match a required structure.

Meta LlamaFirewall is an open-source framework Meta describes as "a policy engine that orchestrates multiple security scanners". The scanners are Prompt Guard 2, a small classifier for direct injection and jailbreak attempts; AlignmentCheck, which reads an agent's reasoning in real time for goal hijacking and indirect injection; and CodeShield, a static analyser for code the model writes. It is built for agents, with integrations for OpenAI's agents SDK and LangChain.

Llama Prompt Guard 2 also works on its own: two open classifiers (86M and 22M parameters) that label text benign or malicious, covering both injections and jailbreaks, under the Llama 4 Community License. Llama Guard 4 is a different tool for a different job: a 12-billion-parameter content-safety classifier that labels prompts and responses safe or unsafe against 14 hazard categories. Its own model card warns it may itself be vulnerable to prompt injection and points to Prompt Guard 2 for that. Worth knowing, because the two names are easy to mix up.

What libraries catch: whatever you configure, as precisely as you like, with full access to your application's context.

What they miss: everything outside your code. A library in your chatbot does nothing for the Claude Code session on your laptop or the ChatGPT connector reading your inbox. And they are work: you host the models, tune thresholds, handle false positives and keep them updated.

Layer 3: the AI firewall in the traffic path

The third layer does not live in the application. It sits between the application and the model provider and reads the traffic as it passes. This is what people usually mean by AI firewall or LLM firewall, and it comes in two shapes.

Detection APIs your code calls

Lakera Guard (Lakera is now part of Check Point) screens messages over one API endpoint. Its documentation says it screens user and model messages as well as agent traffic, including "prompt attacks arriving through tool responses and tool descriptions" and data leakage in tool calls. Azure AI Content Safety offers Prompt Shields, which detects "user prompt attacks" and "document attacks" (hidden instructions in documents, e-mails and web pages); in Microsoft Foundry it can scan at the tool-response intervention point as well. The same Azure service also moderates text and images for hate, violence, sexual content and self-harm, so it covers content safety that injection-only tools do not.

Coverage depends on your code calling them, so they protect what you build, not what you use.

Firewalls and gateways that sit in the path

Cloudflare AI Security for Apps (formerly Firewall for AI) runs in Cloudflare's web application firewall in front of your own AI endpoints. It scans incoming requests for prompt injection, personal data and unsafe topics, including custom topics you define. Endpoint discovery is on all plans; the detections are a paid add-on for Enterprise customers. It protects an AI app you publish from its users, not your team's use of other people's AI tools.

Portkey runs guardrails inside its gateway on inputs, outputs or both: 20+ deterministic checks (regex, JSON schema, code detection), LLM-based checks including prompt injection scanning, and partner guardrails from third-party vendors. Each can deny the request or flag it and let it through. Kong AI Prompt Guard is a plugin for Kong's gateway that checks prompts against regular-expression allow and deny lists, by default on messages with the user role; Kong also lists an AI Semantic Prompt Guard plugin for topic-level allow and deny lists. Regular expressions catch patterns you already know, not new phrasings. Our LLM gateway comparison covers both on routing, fees and logs.

Constellation Gate AI is the one we use. Gate calls itself "the accountability layer for AI" and sits "between your agent and the model" as a hosted gateway. More on it below.

What firewalls catch: injection and data leaks in traffic from any tool you can point at them, including tools whose code you will never see.

What they miss: context only your application has, and anything that does not pass through them. And, like every detector, some attacks.

AI firewall vs guardrails: the short version

Guardrails is the category; a firewall is guardrails placed in the network path. The practical difference is coverage: a library or detection API protects the code that calls it; a firewall or gateway protects every tool that sends traffic through it. So the right answer depends mostly on one fact about you: do you build, or only use?

If you build with AI

You control the code, so use layer 2 for the checks that need your context, and consider layer 3 for the traffic your own tools generate.

1. Decide which of the five risks matter. A customer-facing chatbot needs content moderation (Llama Guard 4, Azure content filters, NeMo's content safety). An agent that reads the web needs injection screening on tool output (LlamaFirewall's AlignmentCheck, NeMo's execution rails, Prompt Shields on tool responses, or a gateway).
2. Screen tool results, not just user messages. A check on the user's prompt does nothing when the prompt was "summarise the open tickets" and the attack is inside ticket 14. This is indirect prompt injection, and it is how most real incidents have happened.
3. Fail closed. If the detector times out, the request should stop, not sail through.
4. Put the guardrails behind the structural defences, not in place of them. Scoped credentials, approval before anything is sent or spent, and no untrusted markdown images. Our seven defences, ranked puts detection fifth for a reason.
5. Map it to a checklist. Guardrails address the first two items on the OWASP LLM Top 10 directly and touch several others. If your agent uses MCP servers, our MCP security checklist covers that surface.

And remember your own coding agent: the Claude Code or Codex session you build with reads files and pages you did not write, and your app's guardrail library does not cover it (how to secure Claude Code).

If you only use AI tools

You cannot add a library to ChatGPT, Claude Desktop or Claude Code. Your layers are the model's own safety (already on), the settings and habits in our prevention guide, and a gateway you point your tools at. That last one is the only guardrail you can actually add, and for most solo marketers and small teams it is the whole answer to "which LLM firewall do I need".

Look for three things:

• It works with the tools you use. Claude Code, Codex, Cursor and similar tools let you change where they send requests; a chat app in the browser mostly does not. Check the list before you sign up.
• It scans tool output, not only your own messages.
• It keeps a record you can open next week, so you find out what it saw.

Where Constellation Gate AI fits

Constellation Gate AI is a hosted gateway: you change where your tool sends its requests, and Gate screens each request and each response before passing it on. Affiliate link: we earn a commission on Pro seats that start from our link, which pays for the testing and has no say in the verdict.

It screens for prompt injection, including indirect injection in tool output; the dashboard labels those "Injection blocked · tool output". Every request gets a verdict: allowed, flagged, blocked or redacted. Three policy scans can each run on prompts, responses or both, with the action set to flag or block: prompt-injection detection, PII and PHI redaction, and credential and secret leak detection. As far as Gate documents it, it does not do content moderation for hate or violence; if that is your risk, pair it with a content-safety classifier.

Gate's own figures (arXiv:2606.02959, checked 23 September 2026): 95.4% of attacks caught across 16 public benchmarks at a false-positive cap of 1 in 100, which is 97.4% F1. In a separate head-to-head at a matched false-positive rate, it reports 96.6% mean F1 against Lakera Guard's 83.7%. It wrongly blocks 2.7% of NotInject's benign-but-suspicious prompts and 1.4% of WildGuard's benign set, and screening adds a median 53 milliseconds. These are vendor-published numbers, not an independent ranking.

Simon Willison's line applies to every product on this page: in application security, "99% is a failing grade", because an attacker tries a hundred phrasings and keeps the one that works. So treat Gate as one layer, not a complete defence. Nobody buys a seatbelt because they plan to crash. You put it on because the cost is one click and the alternative is unbounded. You still want brakes, which are the access and approval defences in the pillar.

Free records, Pro blocks. The free plan is $0 with 20,000 recorded requests a month and a basic pattern-matching injection screen that catches blunt attempts only. Full detection, tool-result scanning, blocking, redaction and limits are Pro, at $20 per user per month. Our Gate review has the full picture, the Gate page has the calculator, and for a team there is a rollout guide.

Which layer do you need?

• Browser chat only, no connectors: model-side safety plus good habits.
• Connectors or a coding agent that reads files, e-mail or the web: a gateway in front of that tool. The most common case among our readers.
• A customer-facing chatbot you build: a content-safety classifier plus injection screening.
• An agent you build: checks on tool results, structural defences first, and a gateway on your own coding agent.
• A team: one gateway in front of everyone, so the rules and the record are the same for every person.

Your first fifteen minutes

This follows our fifteen-minute recipe format: one task, one tool, something finished at the end.

1. Minutes 0 to 5: sort your tools. List the AI tools you used this week. Next to each, write "use" or "build", and tick it if it reads anything you did not write: web pages, e-mail, PDFs, tickets, tool output.
2. Minutes 5 to 10: pick the layer. For every ticked "use" tool, the answer is a gateway. For every ticked "build" tool, write down which of the five risks it faces, and which library or API from layer 2 or 3a fits. That note is next week's work.
3. Minutes 10 to 15: put one tool behind a firewall. Start free with Gate, install Gate Connect, switch on the ticked tool you use most, restart it and carry on working. Setup is one toggle for Claude Code and OpenCode. Free records, Pro blocks: after a week, open the dashboard and read the flagged column. An empty column cost you nothing. A full one tells you exactly what the $20 seat is for. (Affiliate link; we earn a commission on Pro seats.)

Output: every tool sorted into a layer, and one of them screened and recorded. That is more than most teams with a security budget have.

Sources, all checked 23 September 2026: NVIDIA NeMo Guardrails repository (github.com/NVIDIA/NeMo-Guardrails); Guardrails AI repository (github.com/guardrails-ai/guardrails); Meta LlamaFirewall documentation (PurpleLlama repository and PyPI); Llama Prompt Guard 2 and Llama Guard 4 model cards on Hugging Face; Lakera Guard API documentation (docs.lakera.ai); Check Point press release on the Lakera acquisition; Microsoft Learn, "Prompt Shields in Azure AI Content Safety" and "What is Azure AI Content Safety?"; Cloudflare developer documentation, AI Security for Apps; Portkey guardrails documentation; Kong AI Prompt Guard plugin documentation; Wallace et al., "The Instruction Hierarchy", arXiv:2404.13208 (April 2024); Simon Willison, "Prompt injection explained" (May 2023); Constellation Gate AI product pages and technical report arXiv:2606.02959. Vendor figures are the vendors' own. If anything above has changed, tell us and we will fix it with a dated note.