The Prompt Asks, the Code Enforces: Guardrails for an LLM Agent in Production

The first week an LLM agent meets real users, it embarrasses you in three predictable ways.

I learned this running a Telegram AI agent for a logistics business (Django, PostgreSQL, Gemini on Vertex AI). The agent answers staff questions, looks things up through tools, and fills in records. Functionally it worked. But the replies had problems:

1. It talked too much. A yes/no question got four paragraphs.
2. It leaked its internals. Users occasionally saw tool names and labels that were meant for the model, not for them.
3. It surfaced raw errors. When a tool failed, fragments of Python exceptions ended up in the chat.

None of these are "the model is dumb" problems. They're "you trusted one layer to do everything" problems. The fix was to split responsibility in two.

Layer 1: The prompt asks

The system prompt sets the style. What worked for me was short, concrete, testable rules instead of vibes:

Reply style:
- Answer in 1–3 sentences unless the user asks for detail.
- Lead with the answer. No preamble, no restating the question.
- Use plain language. Never mention tools, functions, IDs, or system internals.
- If something failed, say what the user can do next. Never show error text.
- Numbers: always include the currency (USD or UZS) and the date they refer to.

"Be concise" does almost nothing. "1–3 sentences unless asked" does a lot.

But a prompt is a request, not a guarantee. Models drift, edge cases slip through, and a new tool with a weird return format can undo your careful wording overnight. So there's a second layer.

Layer 2: The code enforces

Every reply passes through a sanitizer before it reaches Telegram. It's deliberately boring:

# agent/sanitizer.py
import logging
import re

logger = logging.getLogger(__name__)

TELEGRAM_LIMIT = 4096
FALLBACK = "Sorry, something went wrong on my side. Please try again in a moment."

# Things that must never reach a user, whatever the model says.
_PATTERNS: list[re.Pattern] = [
    re.compile(r"Traceback \(most recent call last\):.*", re.S),
    re.compile(r"^\s*\w+(?:Error|Exception)\b.*$", re.M),
    re.compile(r"^\s*\[?(?:tool|function)[ _](?:call|result|output)\]?:?.*$", re.M | re.I),
    re.compile(r"`?\b(?:get|search|create|update)_[a-z_]+\([^)]*\)`?"),
]


def sanitize_reply(text: str, *, chat_id: int | None = None) -> str:
    original = text or ""
    cleaned = original

    for pattern in _PATTERNS:
        cleaned = pattern.sub("", cleaned)

    cleaned = re.sub(r"\n{3,}", "\n\n", cleaned).strip()

    if cleaned != original.strip():
        # Keep the evidence for us, not for the user.
        logger.warning("sanitizer_modified_reply", extra={"chat_id": chat_id})

    if not cleaned:
        return FALLBACK

    if len(cleaned) > TELEGRAM_LIMIT:
        cleaned = cleaned[: TELEGRAM_LIMIT - 1].rsplit(" ", 1)[0] + "…"

    return cleaned

Three details matter here:

• It logs when it changes something. If the sanitizer fires often, that's a signal the prompt or a tool needs fixing. A guardrail that silently cleans up forever hides the real problem.
• It has a fallback. If everything gets stripped, the user still gets a human sentence instead of an empty message.
• It respects the platform. Telegram rejects messages over 4096 characters. Better to trim gracefully than to throw an API error at the very last step.

Layer 0: Exceptions never become text

The sanitizer is the last line of defense. The first line is simply never handing exception text to the model or the user in the first place:

# agent/handlers.py
async def run_tool(tool, **kwargs) -> dict:
    try:
        return {"ok": True, "data": await tool(**kwargs)}
    except PermissionError:
        return {"ok": False, "reason": "not_allowed"}
    except LookupError:
        return {"ok": False, "reason": "not_found"}
    except Exception:
        logger.exception("tool_failed", extra={"tool": tool.__name__})
        return {"ok": False, "reason": "temporary_failure"}

The model only ever sees a short, safe reason code. It can say "I couldn't find that truck" — it can't paste a stack trace, because it never had one.

Testing guardrails

These are the cheapest tests you'll ever write, and they pay for themselves the first time a new tool misbehaves:

from agent.sanitizer import FALLBACK, sanitize_reply


def test_strips_traceback():
    reply = "Here you go.\nTraceback (most recent call last):\n  File ...\nKeyError: 'x'"
    assert sanitize_reply(reply) == "Here you go."


def test_strips_tool_labels():
    assert "get_truck" not in sanitize_reply("Calling get_truck(id=12) now. Truck 12 is in Almaty.")


def test_empty_after_cleaning_uses_fallback():
    assert sanitize_reply("ValueError: bad input") == FALLBACK


def test_respects_telegram_limit():
    assert len(sanitize_reply("word " * 2000)) <= 4096

The pattern

• Prompt for style and intent — it handles the 95%.
• Code for anything that must never happen — it handles the 5% that would end up in a screenshot.
• Logs on every intervention — so the 5% shrinks over time.

The prompt asks. The code enforces. Don't ship an agent with only one of them.