Modelling the Domain: Required Elements, Prerequisites, Orderings, Vocabulary

Most verification failures happen not because the logic is wrong, but because the domain model is too permissive. If you have been building the pipeline from the previous lessons, you have a stream of text and a desire to check it. The temptation is to write regular expressions or string matches against that text. This approach collapses under the weight of natural language variation. Instead, you must define the structure of the domain before you attempt to verify any instance of it. We will apply these to a runbook domain to produce a concrete domain model sketch.

Required Elements and Prerequisite Edges

A runbook is not just text. It is a sequence of actions that must occur for the system to reach a desired state. The first primitive is the required element. This is a specific action or state that must be present in the output. For example, in a database failover runbook, the action promote_replica is a required element. If the LLM output omits this step, the runbook is invalid. The verifier must locate this omission. It does not score the runbook as 80 per cent complete. It returns a violation: MISSING_REQUIRED: promote_replica. The second primitive is the prerequisite edge. This defines the dependency between two elements. Action B cannot occur unless Action A has occurred. In the failover example, promote_replica has a prerequisite edge from stop_application. The verifier checks the order of occurrence. If promote_replica appears before stop_application, the verifier returns a violation: PREREQ_VIOLATION: promote_replica requires stop_application. This is a directed graph constraint. The nodes are the required elements. The edges are the prerequisites. This structure is deterministic. It does not depend on the LLM understanding the meaning of the words. It depends on the presence and order of the tokens. You must define these edges explicitly. Do not rely on the LLM to infer them. The inference is probabilistic. The edge is deterministic. If the runbook domain has ten steps, you may have fifteen prerequisite edges. Some steps may have multiple prerequisites. The verifier must check all edges. This is a linear scan of the output. The complexity is proportional to the number of edges, not the length of the text.

Forbidden Orderings and Closed Vocabulary

The third primitive is the forbidden ordering. This is the inverse of the prerequisite edge. It states that Action B must not occur after Action A. For example, in a deployment runbook, rollback must not occur after mark_success. If the LLM output includes both steps in that order, the verifier returns a violation: FORBIDDEN_ORDER: rollback after mark_success. This captures logical contradictions that are not simple omissions. It captures sequences that are invalid by definition. The fourth primitive is the closed vocabulary. This is the set of allowed tokens. Any token outside this set is invalid. For example, if the runbook domain uses start_application, the vocabulary must include this token. If the LLM outputs launch_application, the verifier returns a violation: VOCABULARY_VIOLATION: launch_application not in closed set. This prevents the LLM from using synonyms that may have different meanings in the system. It forces the output to use the canonical terms defined in the domain model. The closed vocabulary is a whitelist. It is not a blacklist. Any token not in the whitelist is rejected. This is crucial for deterministic verification. It removes the ambiguity of natural language. These four primitives form a complete model for many domains. They cover presence, order, contradiction, and terminology. They are simple. They are easy to implement. They are easy to test. They do not require machine learning. They require only a clear definition of the domain.

Ontology as a Work Coat

What you have just built is an ontology. It is not a philosophical abstraction. It is a practical model of the domain. By calling it an ontology, you gain access to decades of prior art. You can use existing tools for graph traversal, constraint checking, and validation. You do not need to reinvent the wheel. You can use standard formats like OWL or RDF if you need interoperability. Or you can use simple JSON or YAML if you need simplicity. The key is to recognise that the domain model is an ontology. It defines the concepts and their relationships. It is not just a list of rules. It is a structured representation of the domain. This recognition allows you to leverage existing libraries and tools. It allows you to validate the model itself. It allows you to visualise the model. It allows you to reason about the model. It transforms the verification task from a string matching problem to a graph constraint problem. This is a significant shift in perspective. It moves the focus from the output text to the domain structure. It makes the verification deterministic and transparent. The cost of this approach is the upfront effort to define the domain model. You must identify the required elements, the prerequisite edges, the forbidden orderings, and the closed vocabulary. This requires domain expertise. It requires collaboration with subject matter experts. It is not a task that can be automated. It is a task that must be done carefully. The quality of the verification depends on the quality of the model. If the model is wrong, the verification is wrong. This is the tradeoff. You invest time in defining the model to gain deterministic verification. You accept the cost of model maintenance to gain the benefit of reliable checking.

The domain model approach has the highest upfront cost. It requires careful definition of the primitives. It requires maintenance as the domain evolves. However, it provides deterministic and explainable verification. It returns located violations rather than scores. It allows you to pinpoint the exact error in the output. This is essential for debugging and improvement. The regex approach is simpler but fragile. It breaks when the text varies. The LLM approach is flexible but unreliable. It provides no guarantee of correctness. The domain model approach strikes a balance. It is rigorous without being brittle. It is flexible without being probabilistic. This sketch should list the required elements, the prerequisite edges, the forbidden orderings, and the closed vocabulary for your runbook domain. It should be a concrete artifact that you can implement in code. It should be a starting point for the next lesson. The next lesson, titled “Closed-World Semantics: Silence Must Mean Checked”, will build on this model to define how silence in the output is interpreted. It will show how to use the closed vocabulary to detect omissions. It will complete the verification loop.

Apply it to your work