What is a "guardrail" in an LLM system, and where does it sit?

A guardrail is an explicit rule/constraint wrapped around an AI system that inspects inputs and outputs and can block, reformulate, sanitize, or redirect — catching prompt injections, unsafe responses, and data leakage.

Guardrails are explicit rules or constraints built around AI systems so they stay on-topic, avoid disallowed content, protect sensitive data, and don't follow unintended instructions. They check both what the user asks (input) and what the AI returns (output) to detect risks like prompt injection, unsafe responses, and PII leakage.

When a guardrail detects a violation (e.g. "tell me how to build a bomb"), the system can block, reformulate, sanitize, or redirect the conversation. In agent frameworks (e.g. LangChain) they hook in at stages: before_agent → before_model → wrap_model_call / wrap_tool_call → after_model → after_agent.

Tip: Guardrails are the seatbelt, not the engine fix — they reduce harm from an inherently un-trustable text stream, but (see the cat-and-mouse problem) they don't make injection impossible.