Guardrails at Input, Output and Tool Call Level

Guardrails are automated controls that constrain what a model can input, output or invoke. They are the difference between a model that can do anything and a model that can only do what is intended. Guardrails are not optional extras; they are a regulatory requirement for high-risk systems and a best practice for all systems.

Input guardrails filter, sanitise or reject user input before it reaches the model. The simplest input guardrail is checking for suspicious patterns: does the input contain keywords associated with prompt injection attacks? Does it contain profanity or threats?

Does it reference topics outside the model’s scope? These checks can reject or sanitise input before the model sees it.

More sophisticated input guardrails use the model itself to classify input. You can run a smaller model or classifier to determine whether the input is a legitimate question or a potential attack. If the classifier detects an injection attempt, the guardrail rejects the input and returns an error message to the user.

This requires careful tuning to avoid false positives where legitimate input is rejected.

Input guardrails can also enforce schema validation. If your API expects structured input (a JSON object with specific fields), parse and validate the structure before passing it to the model. Reject input that does not match the schema.

This prevents an attacker from sending data in unexpected formats that might confuse the model.

Output guardrails filter, transform or reject the model’s output before it reaches the user. The most common output guardrail is checking for harmful content. Does the output contain instructions for violence, self-harm, illegal activity or sexual exploitation?

Does it contain profanity or slurs? If so, the guardrail rejects the output and returns a default message to the user like “I cannot answer that question.”

Output guardrails can also check for factual errors and refusals. If the model outputs something that contradicts known facts in your system (for example, a discount percentage that does not exist), the guardrail can reject or correct it. If the model was supposed to refuse a request but generated a response anyway, the guardrail detects and blocks it.

Tool call guardrails control which tools a model can invoke and with what parameters. When a model decides it needs to call an external tool (like a database query or email), the guardrail intercepts the call and checks: is this tool available to this model? Are the parameters within expected ranges?

Has the tool been called too many times recently (rate limiting)? If any check fails, the guardrail prevents the call and asks the model to try a different approach.

Implement tool call guardrails by creating an abstraction layer between the model and tools. The model receives a list of available tools and their schemas. When the model decides to call a tool, it produces a structured request (like JSON).

Your guardrail layer receives this request, validates it against the tool schema, checks access control, applies rate limits, and only then invokes the tool. If validation fails, return an error to the model instead of invoking the tool.

Guardrails should be layered. Do not rely on a single guardrail to stop all attacks. An effective system has input validation, output filtering, and tool call controls all working together.

If an attack gets past one, the others catch it.

Guardrails have a cost: false positives. A guardrail that is too strict rejects legitimate requests. A customer support model with an output guardrail that blocks any message containing “not” would reject legitimate statements like “This is not the product I ordered.”

Tune guardrails carefully, measure false positive rates, and adjust thresholds based on real traffic. In practice, this means you should run your test set through the guardrails and measure what fraction of legitimate outputs are wrongly rejected. If that fraction is above an acceptable threshold (e.g., more than 5 percent), adjust the guardrail thresholds or rules.

Some guardrails are statistical: they measure properties of the output like sentiment, toxicity or semantic similarity to a reference corpus. Others are keyword-based or rule-based. The best approach depends on your use case.

For high-risk systems, use multiple methods because each has different strengths and weaknesses. A keyword filter is fast and deterministic but has high false positive rates on natural language. A statistical toxicity classifier is slower but more nuanced.

A combination catches attacks that exploit weaknesses in either method alone.