Repair Prompting: Located Violations and Minimal Diffs
Regenerating an entire artifact after a single verification failure is not just wasteful; it is a structural error that actively degrades quality by discarding correct logic. When the deterministic verifier in the loop flags a violation, the instinctive response is often to send the whole output back to the model with a generic instruction to fix it. This approach treats the generation process as a black box. It assumes the model will remember what was right and only change what was wrong. It rarely does. Large language models are probabilistic next-token predictors. They do not have a mental model of the code or text they just produced. When you ask them to rewrite a fifty-line function because of a type error on line forty-two, they often rewrite lines one through ten again. Those lines were already verified. Now they are unverified again. You have introduced new risk into regions that were previously safe. This is the primary reason agent loops fail to converge. The state space expands with every regeneration instead of contracting. The alternative is targeted repair. This requires the verifier to be precise. It must not just return a boolean or a generic error message. It must return the location of the violation and the nature of the violation. The repair prompt then becomes a surgical instruction. It asks the model to produce a minimal diff or a replacement for a specific span. This keeps the verified regions intact. It reduces the token count of the repair step. It focuses the model’s attention on the actual problem. The tradeoff is that the verifier becomes more complex. It must parse the output, understand the structure, and pinpoint the error. But this complexity is deterministic. It is cheap. It is reliable. The cost of building a good verifier is far lower than the cost of infinite regeneration loops.
Anatomising the Violation The first step in targeted repair is forcing the verifier to speak in coordinates. A message like invalid json is useless for repair. The model does not know where the invalidity lies. It will guess. A message like invalid json at path users.0.email is actionable. It tells the model exactly where to look. The verifier must inspect the output against the schema or rules and return a structured violation object. This object needs three fields. The path or span identifies the location. The rule identifies the constraint that was broken. The expected value or pattern provides the target. Consider a JSON generation task. The schema requires a string for the name field. The model outputs a number. The verifier catches this. Instead of rejecting the whole JSON blob, it returns a violation at path name with rule type_mismatch. The repair prompt then receives this violation. It does not receive the whole JSON. It receives the violation and the surrounding context. The context is crucial. The model needs to see the lines around the error to understand the structure. But it does not need to see the whole document. This reduction in context window usage is significant. It lowers latency. It lowers cost. More importantly, it lowers the chance of hallucination. The model is not asked to re-invent the wheel. It is asked to fix the tyre. The verifier must be deterministic. It cannot rely on the model to judge correctness. It must use code. Regex. Schema validation. Type checking. This ensures that the violation is real. It prevents the loop from chasing ghosts. If the verifier is probabilistic, the repair loop becomes unstable. The model might fix the violation in one way, but the verifier might reject it in another run. This leads to oscillation. The loop never terminates. Deterministic verification is the anchor. It provides the ground truth that the probabilistic generator tries to hit.
Constructing the Repair Instruction The repair prompt is not a conversation. It is a command. It must be concise. It must be unambiguous. It must specify the input, the violation, and the expected output format. The input is the relevant portion of the artifact. The violation is the structured object from the verifier. The expected output is the corrected portion. The prompt should not ask for explanations. It should not ask for apologies. It should ask for the fixed code or text. A good repair prompt template looks like this. Here is the current value at path name. It is 123. The rule requires a string. Return only the corrected value. Do not return the whole object. Do not return markdown. Return just the string. This constraint on the output format is critical. It prevents the model from adding noise. It makes parsing the repair easy. The pipeline can then apply the repair directly. It replaces the old value with the new value. It does not need to parse the whole artifact again. It just patches the hole. The tone of the prompt matters. Imperative verbs work best. Replace. Fix. Correct. Avoid soft language. Please fix. Try to correct. These phrases introduce uncertainty. The model may interpret them as suggestions. It may choose not to follow them. It may add commentary. The repair step is a mechanical operation. The prompt should reflect that. It should be a function call in natural language. The input is the error. The output is the fix. Nothing else. The context window for the repair prompt should be minimal. Include only the lines affected by the violation and their immediate neighbours. If the violation is at line 42, include lines 40 to 44. This gives the model enough structure to understand the syntax. It prevents it from losing track of indentation or nesting. It also prevents it from seeing unrelated code that might confuse it. The model is sensitive to distraction. Reducing the noise improves the signal. It makes the repair more likely to be correct. It reduces the number of repair iterations needed.
Minimal Diffs and State Preservation The goal of the repair is to change as little as possible. This is the principle of minimal diffs. When the model replaces a value, it should not reformat the surrounding code. It should not change variable names. It should not reorder imports. Any change outside the violation is a risk. It may introduce new errors. It may break existing tests. It may change the semantics of the code. The verifier will catch these new errors. But this extends the loop. It adds cost. It adds latency. To enforce minimal diffs, the repair prompt must explicitly forbid changes outside the violation. State clearly that only the specified path may be modified. Any other change will be rejected. This constraint forces the model to focus. It prevents it from taking the opportunity to refactor. Refactoring is not the job of the repair loop. It is the job of a separate optimization step. The repair loop is for correctness. It is not for style. Mixing these concerns leads to instability. The model may prioritize style over correctness. It may fix a formatting issue but leave a type error. The verifier will catch the type error. The loop will run again. This is inefficient. The application of the repair must be atomic. The pipeline should take the repaired value and replace the old value in the artifact. It should not re-parse the whole artifact unless necessary. If the artifact is JSON, update the field. If it is code, replace the line. This preserves the structure. It ensures that the verified regions remain verified. The verifier does not need to re-check the whole artifact. It only needs to re-check the repaired region. This is a significant optimization. It reduces the verification cost. It speeds up the loop. It allows for faster convergence. There is a limitation to this approach. It assumes that violations are local. Sometimes a violation in one place requires a change in another. For example, adding a field may require updating a schema definition elsewhere. The minimal diff approach may fail here. The repair may be invalid because it does not update the dependent parts. The verifier will catch this. The loop will run again. But the repair prompt may need to be smarter. It may need to include more context. It may need to identify dependent violations. This adds complexity to the verifier. It requires it to understand the dependencies in the artifact. This is hard. It often requires domain-specific logic. For simple artifacts, minimal diffs work well. For complex artifacts, they may require more sophisticated repair strategies. You should finish with Repair strategy comparison. This document should detail the tradeoffs between full regeneration and targeted repair. It should include examples of violation objects. It should show the repair prompts. It should measure the reduction in token usage. It should estimate the reduction in loop iterations. This comparison provides the evidence needed to justify the complexity of the verifier. It shows that the upfront cost pays off in the long run. It demonstrates that targeted repair is not just a optimization. It is a necessity for scalable agents. The next lesson, Observability: Violation Logs, Iteration Counts and Cost per Artifact, examines how to measure the performance of this loop.
