Termination: On the Verifier, Never on Confidence

Trusting a model’s self-reported confidence score to terminate an agent loop is a structural error that guarantees silent data corruption. The fundamental mistake in most agent architectures is treating the language model as both the worker and the quality inspector. When you ask the model to generate code and then ask it whether that code is correct, you are asking the same stochastic process to validate its own output. The correlation between the model’s stated confidence and the actual correctness of the output is often negligible. In production systems, this manifests as agents that run indefinitely because they never feel confident enough to stop, or agents that stop prematurely because they feel confident about a hallucination. The only reliable termination signal comes from a deterministic verifier. This verifier must be external to the generation loop. It must be a script, a compiler, a test suite, or a parser that returns a binary truth value. If the verifier says the output is invalid, the loop continues. If the verifier says the output is valid, the loop terminates. There is no middle ground. There is no “maybe”. The loop structure in the pipeline relies on three distinct exit conditions. The primary condition is verifier success. The secondary condition is a hard iteration cap. The tertiary condition is oscillation detection. Each serves a specific purpose in maintaining system stability. The iteration cap prevents infinite loops caused by a broken verifier or a model stuck in a local minima. The oscillation detection prevents wasted compute when the model is cycling between two invalid states. The verifier success is the only condition that produces a usable result. All other conditions represent failure modes that must be handled explicitly.

Why Confidence Scores Are Noise

Language models output logits. These logits are often converted into probabilities or confidence scores. Engineers frequently interpret high confidence as high accuracy. This interpretation is wrong. Confidence measures the model’s certainty about its next token, not the truth of the statement. A model can be 99 per cent confident that 2 plus 2 equals 5. It can be 99 per cent confident that a library function exists when it does not. The distribution of confidence scores across correct and incorrect outputs often overlaps significantly. Relying on this signal introduces non-deterministic behaviour into the termination logic. One run might terminate after three iterations because the model felt confident. Another run might run for twenty iterations because the model felt uncertain, even if the output was correct in both cases. This variance makes latency unpredictable and debugging impossible. The solution is to remove the confidence signal from the termination logic entirely. The loop should not inspect the model’s internal state. It should inspect the external state of the artifact. If the artifact compiles, the loop checks the tests. If the tests pass, the loop terminates. The model’s opinion on whether the tests passed is irrelevant. The test runner provides the truth. This approach aligns with the principle that agents are just loops. The loop generates, verifies, and repairs. The verification step must be deterministic. If you cannot write a deterministic verifier, you do not have an agent. You have a chatbot. The distinction matters. Chatbots are fine for conversation. They are unacceptable for code generation, data transformation, or any task where correctness is binary.

Iteration Caps and Cost Control

Every loop needs a maximum iteration count. This cap is not a suggestion. It is a circuit breaker. Without it, a single edge case can cause the agent to run until the cloud bill arrives. The cap should be set based on the expected complexity of the task and the cost of the model calls. For simple tasks, three to five iterations are often sufficient. For complex tasks, ten to fifteen might be needed. The number is not universal. It must be tuned per task type. The pipeline should expose this cap as a configuration parameter. It should not be hardcoded. When the cap is hit, the loop must terminate loudly. It should not return the last generated output as if it were correct. It should return an error state. This error state must be distinct from a success state. Downstream systems need to know that the agent gave up. They need to know that the output is unverified. Returning unverified output as verified output is the most dangerous failure mode in agent systems. It creates false positives that propagate through the system. The error should include the number of iterations attempted and the last error message from the verifier. This information is crucial for debugging. It tells you whether the cap was too low or whether the task is inherently difficult. The cost of each iteration must be tracked. This includes the token count for the generation and the compute cost for the verifier. The loop should abort if the cumulative cost exceeds a budget. This budget is separate from the iteration cap. It provides a financial safety net. The iteration cap prevents infinite loops. The budget prevents expensive loops. Both are necessary. The pipeline should log these metrics. They provide visibility into the efficiency of the loop. If most tasks hit the iteration cap, the cap is too low. If most tasks hit the budget, the task is too complex or the model is too expensive. These metrics drive optimization.

Detected Oscillation via Artifact Hashing

Oscillation occurs when the model generates the same invalid output repeatedly. This often happens when the repair prompt is not providing enough new information. The model reads the error, tries to fix it, introduces the same error, and repeats. Detecting this pattern saves compute. The loop should store a hash of the generated artifact at each iteration. If the current hash matches a previous hash, the loop has entered a cycle. The cycle length should be configurable. A cycle of two iterations is often noise. A cycle of three or more is likely a trap. The hash should be computed on the canonical form of the artifact. For code, this means stripping whitespace and comments. For JSON, this means sorting keys. The goal is to detect semantic equivalence, not string equality. Two outputs might differ in whitespace but be functionally identical. The hash must reflect this. When oscillation is detected, the loop should terminate. It should not continue hoping for a breakout. The breakout is unlikely without a change in strategy. The error message should indicate that oscillation was detected. This allows the system to trigger a fallback strategy. The fallback might be a different model, a different prompt, or human intervention. The limitation of this approach is that it requires storage. The loop must keep a history of hashes. This adds memory overhead. For long-running tasks, this history can grow. The history should be bounded. Only the last N hashes need to be kept. N should be larger than the expected cycle length. If N is too small, the loop might miss long cycles. If N is too large, the memory cost increases. The tradeoff is between detection accuracy and resource usage. In practice, a history of ten hashes is sufficient for most tasks. Longer cycles are rare. If they occur, the iteration cap will eventually terminate the loop anyway. The next lesson, Repair Prompting: Located Violations and Minimal Diffs, covers how to construct the repair signal that prevents these oscillations in the first place.

Apply it to your work