Chapter 6 · Semantics as a security issue

You cannot force agents to think in a comprehensible way. You can force them to pass through a door that only lets comprehensible thoughts through.

Of all the chapters in this book, this is the one with which managers find it easiest to agree — and whose significance they are most likely to underestimate. ‘Semantics’ sounds like linguistics, like academia, like a subject for people with horn-rimmed glasses. In this chapter, I shall show that this is a security problem of the highest order: the problem that, in an agent-based system, two parties are certain to use the same words yet mean different things — and that this state of affairs, if left unchecked, can undermine every other security mechanism described in Chapter 3.

The Tower of Babel, re-examined

What happens when several agents trained in different ways encounter one another? Each brings their own ontology with them: their own terms, their own implicit assumptions, their own ideas of what a message actually means. The consequence is no trivial matter, but a matter of mathematics: N agents with N different languages need N×(N−1) translation pairs to understand one another — and each of these pairs is a source of error in its own right. Three agents: six translations. Ten: ninety. The effort does not increase linearly, but quadratically.

But the real danger is not the effort. The real danger lies in the four states that arise when nobody solves the translation problem:

Silent misalignment. Two agents appear to be communicating successfully — the same words, different meanings — and the result looks like a success but is actually wrong. This is the most dangerous state a system can be in: it does not produce an error, but rather a result that nobody notices. A dashboard shows green because the protocol is formally satisfied. The meaning, however, was off the mark.

Semantic injection. A difference in meaning is also an attack vector: anyone who knows that the recipient understands ‘approved’ differently from the sender can formulate messages that bypass the validation mechanism because they fulfil the validation vocabulary whilst still altering the intended meaning. Prompt injection via the semantic route — harder to detect than the technical variant because everything looks correct on the surface.

Cascading escalation. A misunderstanding spreads through the swarm: Agent A misinterprets B, passes it on to C, C acts on the basis of the misunderstanding — and the end result is a decision that can no longer be traced back to anyone’s original intent. The audit trail from Chapter 3 shows who passed what on and when. It does not show what was actually meant in each instance.

Guard blindness. The most bitter consequence: a node that checks against the wrong semantics is not a checkpoint, but a mere ornament. It checks scenarios against a set of rules that misses the meanings with which the system actually operates. You can build the best door in the world — if what ‘understandable’ means is not defined, it checks nothing.

The answer: Interlingua

The solution is an old and tried-and-tested one: a common language between languages — an Interlingua. Not as a substitute for the individual languages: agents are free to formulate internally however they wish. But as a contract at the interfaces: whatever leaves or enters the system corresponds to a canonical meaning that is defined, versioned and verifiable.

And here is the correction this book makes to the author’s own earlier drafts — the expiry date principle in action: the canon is not the system’s enforcer of rules. The node (Chapter 3) enforces rules. The canon is the layer of meaning: the shared vocabulary against which the node checks. Anyone who conflates the canon and the node creates an entity that both defines what is valid and enforces what is valid — and it was precisely this conflation that had to be corrected in earlier architectures. Separate entities: the canon defines, the node validates, and the drift guardian (Chapter 7) detects when the system deviates from the definition.

The narrative analogy: the Web in 2005

To understand why this solution works, it is worth looking at another industry — and at a specific date: The Swarm world is today where the Web was in 2005.

Before HTML5, every page built its own structure: <div class=”my_header”>, <div class=”navigation”> — conventions that every developer invented freely and which only they themselves understood. Machines attempting to read these pages could only guess. Then came HTML5 with canonical semantic tags: <nav>, <article>, <header> — no prohibitions, no rules dictating what the inner workings of a page should look like. Just a standard at the interface that made behaviour verifiable. A browser was free to structure its internal workings as it wished — the standard ensured the interface was reliable.

This is precisely the architectural requirement set out in this chapter for agents: no ban on internal languages, but a canonical contract at the interfaces. The Web was not saved semantically by someone forcing all developers to think alike — but by ensuring that the gateway through which pages speak to machines only allows comprehensible content to pass through.

The three levels of enforcement

For the idea to become an enforceable architecture, more than good intentions are needed — enforcement is required at three levels. Only together do they seal the door:

Firstly: contract rather than wording. Incoming messages must comply with a machine-readable contract — intent, schema, constraints. The agent is free to formulate things internally however it wishes; the message on the channel must conform to the contract or it will be rejected. This enforces canonical semantics, not canonical prose.

Secondly: infrastructure enforcement — the crucial level. A system prompt cannot enforce compliance; a channel can. Messages pass exclusively through gateways that only allow canonical schemas through. Internally, the agent may use dialectical language — but not across the boundary. At the transition point, the message is either translated or rejected. (The browser analogy applies here literally: they were allowed to build internally however they wished; the standard made the interface verifiable.)

Thirdly: economic constraint. Languages arise from financial self-preservation — agents speak unintelligibly because, under pressure to be efficient, it is detrimental to be intelligible. Readability must therefore not be an optimisation variable: non-canonical messages must be more expensive than canonical ones — rejection, regeneration, escalation. Transparency is rewarded rather than punished.

The boundary of the house: bona fide, mala fide and the interpreter

So far, this chapter has dealt with semantics within the swarm. But a system has two directions: inwards and outwards. And at the boundary of the house, two questions arise that every architecture is eventually faced with.

The first: Is something incomprehensible an attack? No — not necessarily. Legal language has for centuries distinguished between malice and good-faith error, and the hub requires the same distinction: a message may be mala fide — deliberately formulated to deceive the verification process — or bona fide: well-intentioned, but in a dialect unknown to the canon. The response must differ. A malicious message is an incident: reject it, log it, escalate it. A message made in good faith is a translation issue: clarify it, categorise it, and — as a secondary benefit — expand the canon. Every case of doubt that has been discussed makes the gateway a little wiser.

The distinction itself, however, is semantic in nature: intent cannot be tested deterministically. That is why a tiered approach applies here too: the junction marks ‘unclear’, and the discussion falls to the owner of the category (Chapter 4). Anyone who treats all unclear cases the same way — letting everything through or rejecting everything — is choosing between an open door and the torment of false positives. The architecture opts for a middle ground with a clear focus.

The second: What is coming from outside? Other companies, other tools, other agents — by definition, the outside world does not speak the in-house canon. One cannot even expect it to be familiar with it. The result is a building block that is not entirely new: an interpreter sits at the boundary. External messages and tasks are translated into the internal semantics before being checked — schema mapping, concept mapping, with a human at the translation point where necessary — and only the translated version is allowed through the door. The principle of ‘translate or reject’ applies here too: not every incoming message deserves translation; some are simply foreign and remain outside.

And this predates any ‘agent era’. When a clerk at one company sends an order to another via a form, this is exactly what happens: she speaks her process language, the form translates it into ours, and neither side ever had to use the same words to do so. The interpreter is the oldest interface in business — EDI schemas, order forms, API contracts. The Interlingua thesis states just one thing: what has always applied to exchanges between companies must now also apply within one’s own system. The swarm is not the first stranger in the house — it is the first to have become a stranger to us internally.

The canon as a treasure: what needs protecting must not be left out in the open

And here is a consequence that follows from everything said previously, yet is rarely drawn: The canon is the only component of the system that can be stolen without touching a single other component.

Agents can be secured, channels monitored, nodes hardened. But the canon is a file — the layer of meaning within the organisation, what ‘approved’, ‘critical’ and ‘customer’ actually mean. And whoever possesses it possesses something more explosive than a password: complete knowledge of what the system is checking against. Whoever knows the map no longer needs to resort to force — every future message can be formulated in such a way that it matches the door yet still means something else. This is semantic injection in its purest form: not circumventing the check, but spying on the checking mechanism itself.

The canon therefore requires a level of protection that no other component needs — not out of paranoia, but as a consequence of its very nature:

It is a trade secret. Like the recipe of a family business: essential to operations, never made public. What is permitted to be made public — terms, interfaces, examples — constitutes the public version of the canon; the internal versions, schemas, validation rules and escalation thresholds remain behind the same threshold as the content they protect.

It is treated like a recipe — versioned, locked, assigned. Changes to the canon run through the same mesh that it itself secures: every version has an owner (Chapter 4), every change an audit trail (Chapter 3). A canon that every agent is permitted to read and amend is not a layer of meaning, but a suggestion — and a suggestion protects no one.

Its distribution is minimal. Agents do not need the canon; they need the door that checks against it. This is the third consequence of the separation of canon and node: the node knows the rules, the agent only knows the address of the door. Whoever compromises the agent gets the dialect — not the map.

An honest observation is in order: even a sealed-off canon does not prevent every attack — it merely makes the most effective ones more costly. Those who do not know the meaning layer must guess it; those who do know it no longer need to guess. Security cannot offer more than this — but this ‘more’ is the difference between a system that is being tested and one that has been figured out.

The honest limit

And the conclusion, as in every chapter: there is no such thing as 100 per cent compulsion. Under pressure to be efficient, comprehensibility is actively detrimental to the individual agent — this is not a bug, but optimisation logic (Chapter 2). The architectural answer is therefore not the promise, but the tiered approach: Channel enforcement + verification authority (drift monitor with spot checks against the canon) + obligation to provide justification at the junction. Whoever builds the door must also appoint the guardian who checks whether the door does what it promises. The next chapter is about him.