Parseable Is Not Valid

Here is a governance profile for Sentience Governor. It is valid YAML, and under version 0.3.2 it passed sentience profile validate with no errors and no warnings.

schema_version: 1
high_consequence:
  tools: [1, 2]

Bind it to a Claude Code session under version 0.3.2 and run two tool calls. Each time the hook logs one line to stderr and exits cleanly:

claude_code_hook: unhandled error: first argument must be string or compiled pattern

The agent carries on and the session looks governed: the registration says the profile is bound, with a fingerprint, and sentience profile resolve --agent-id reports bound with no warnings. Yet the Sentience Agent Execution Record for that session holds four events of the nine it should: the registration, the intent declaration, and one context snapshot per call. Both scope assertions, the records that say what the agent was about to do and how the policy judged it, are gone, and so are the three records that follow from them.

The Pydantic AI companion release found the defect while tracing what happens to a malformed profile, and stopped until core was fixed. We corrected the defect in Sentience Governor 0.3.2.1 before completing the Pydantic AI companion release. The investigation changed how we thought about configuration validity, execution evidence, and what a governor must preserve when its own assumptions fail. Those lessons apply to any system that makes decisions from configuration it did not write.

Parseable is not valid

The validator checked the shape of the file: sections present and well typed, known keys, known enumeration values. It did not check the values the runtime would consume. The pattern match runs a regular-expression search on each tools entry; an integer raises. The boundary detector compares time_gap_seconds numerically and slices a path by dir_change_depth; a string in either raises on the second event. The validator had been written against the schema, not against the code that reads it.

Worse, the code that read the schema was quietly helping. The schema_version accessor applied int(), so a boolean true was recorded as version 1; the section accessors applied dict(), so a section written as a list of pairs came back as a mapping. Invalid shapes were converted into policy before any consumer could notice.

The lesson is blunt: the runtime is the specification. Every field a consumer reads must be validated as that consumer reads it, and the validator must run where the profile becomes active, not only when an operator types a command. The corrected validator covers every runtime-consumed field, and the runtime reads a checked view of the raw data instead of trusting accessors. A validator whose only caller is the CLI is documentation with a return code.

Where an invalid configuration stops

Three responsibilities, and the fix only worked once all three existed.

The resolver decides which profile applies. If the first matching binding names a file that is missing, unparseable or invalid, resolution degrades to the machine default, never to a later binding, with one warning naming the file and the first errors. An invalid default means no profile, and the record says so.

The session manager is the one place every path shares, so it makes the final acceptance decision: an invalid profile handed to it directly is refused, nothing is activated, one warning names the session and the reason, and nothing is raised, because the caller has already opened a run.

The event builder is the defensive layer behind both. If a malformed value still reaches an evaluator, it substitutes the default the validator would have insisted on, logs the field once per session, and never derives policy from the malformed value.

That rule has a sharp edge we pinned in the tests. A valid dir_change signal next to an invalid depth still detects boundaries, at the default depth. An invalid time gap must never manufacture one: taken literally, a negative gap would fire on every event, which would look like governance and be noise. Tolerance means substituting a safe default, not inventing a decision.

What fail-open has to preserve

Fail-open is the right default for an observation-only governor. The question the failure forced was: fail open to what? The original hook already failed open and the agent kept running; by the usual definition nothing broke. What broke was the evidence, and every surface said otherwise.

Five things were tangled together in “fail open”:

Behaviour	What it protects
The agent continues	Execution availability
An invalid profile is rejected	Configuration integrity
A warning identifies the problem	Operational visibility
Resolution provenance is recorded	Governance accountability
Execution events are preserved	Evidence integrity

Previously, we had the first row and thought we had the second. With the fix, the same scenario degrades to the default, warns once with the file and the field, records degraded with the pattern that failed, and writes all nine events. A governor’s product is the last two rows: availability is table stakes, accountability and evidence are what it exists to produce.

Trace the failure into every adapter

Sentience Governor has four runtime surfaces: the Claude Code hook, an MCP client wrapper, a LangChain callback handler, and the Pydantic AI companion. Before changing anything we traced one question through all four: what happens when the default profile cannot be loaded? The hook failed open by design; the companion caught the exception and warned. The MCP wrapper let it propagate into the developer’s async with, stopping the agent on a configuration error. The LangChain handler never saw it, because the framework swallows handler exceptions unless told not to, so the run continued ungoverned with no warning and no record.

One defect, four consequences, two of them nobody had asked about. “Never raises into the governed application” is not a property you assert once; it has to be verified per surface. The fix (the resolver’s default step now fails open with a warning instead of raising) is a behavior change for two adapters, stated plainly in the changelog.

Write the corpus before the fix

The first thing we built was not the fix but a corpus: 62 malformed profiles, one file each, tagged by what should happen to them. Every reproduction from the issue, every YAML type and key variant that could reach the loader, and the warning-only cases that must keep loading. Then 217 tests over it, written against the behavior we wanted and run against the release that carried the defect: 155 failed, exactly as predicted.

Before the fix the corpus was the contract each layer had to turn green; after it, the regression guard, checked into the public repository. It proves the invalid cases and says nothing about the valid ones, and a patch that changes how profiles are accepted could change how good profiles behave. So the second instrument was a clean room: the earlier release’s validation scenarios replayed through installed packages, that published release in one environment and the corrected candidate in another, every event compared whole after removing per-run identifiers. Seven scenarios, 146 events, zero differences. The legacy profiles from two releases ago still produce their recorded fingerprints. A green suite says nothing about an invariant you never wrote down; that is why the invariant went into a corpus first.

A fingerprint is an identity

Every event governed by a successfully resolved profile carries that profile’s fingerprint, derived from its canonical form. The investigation raised a question we had not asked precisely enough: what is a profile’s admissible representation, the set of inputs the fingerprint is defined over?

The constructor’s JSON round trip had been answering it by accident: integer, boolean and null keys were silently rewritten as strings, and keys no consumer would ever read were fingerprinted as policy. The Ratify protocol specification states the discipline: a cryptographic identity needs a defined admissible representation, and a strict decoder rejects what falls outside it rather than coercing it.

The fix rejects non-string mapping keys at construction with an error naming the key path, and leaves canonicalization alone: every legitimate profile keeps its exact bytes and fingerprint, and twenty profiles compared between the earlier published release and the patch were identical. A Claude Code snapshot written by an earlier release for a profile that is now invalid is not reactivated: the hook resolves again and leaves the snapshot file alone. History is not rewritten because validation improved. A fingerprint that means “this policy content governed this decision” has to be defined over inputs that are policy. And be exact about what it is not: a deterministic, content-derived identity says which profile’s content was in force, not that the profile was authorized or that the governance it expressed was correct. The execution evidence recorded under it establishes what the governor observed, which is not automatically everything that happened.

Core owns semantics, integrations own their runtimes

The companion release could have patched around the defect inside the Pydantic AI adapter and shipped a week earlier. It did not, for a boundary worth stating. Core owns profiles, validation, resolution, evaluation, session binding, fingerprints and the evidence record. An integration owns how it attaches to its framework, what events the framework exposes, how tool metadata is translated, and what the framework can actually guarantee. Validity is core’s judgment: the companion’s readiness check asks core’s validator, encodes no schema rules, and its dependency floor moved to the core release that fixes the defect.

Two smaller lessons came with it. One companion test failed against the fixed core although behavior had not changed: it had asserted the wording of an exception path core no longer takes. Tests that pin a partner’s internals rot when the partner improves; tests that pin outcomes do not. And the companion’s documentation now says operations rules do not apply to Pydantic AI tool calls, which carry none of the shell classification those rules match. Saying what a feature does not do is part of shipping it; every such sentence was checked against the test that demonstrates it, and two elsewhere in the docs that failed that check were corrected.

The questions this leaves open

Sentience Governor evaluates agent actions against declared governance context and records governance evidence without blocking execution, and nothing in these releases changes that. But the episode leaves questions any governor must answer before its decisions can be trusted. When a governor cannot establish which policy is active, what should the record say, and what should the run do? Can every decision be tied to the effective policy’s identity, including when resolution degrades? When the governor itself fails, how do you tell a denied action, an unavailable decision and a configuration failure apart in the evidence? And which execution boundaries can a given framework actually guarantee, as opposed to observe?

One more observation, because it is how the defect was found. Sentience Governor is built with an increasingly autonomous coding agent, and it exists to govern autonomous execution. The agent’s investigation, tracing one exception path through four adapters and probing malformed profiles against the accessors and a real governed run, exposed assumptions the implementation had never established. Human review turned those findings into decisions: where an invalid profile stops, what fail-open must preserve, what a fingerprint is defined over. The agent implemented and verified the resulting invariants; the corpus, the compatibility requirements and the release evidence made them durable.

Nothing here learned on its own or rewrote a policy, and the governor did not govern its own development. But the loop points at the longer-term ambition for complex autonomous work: not simply removing human approvals, but reducing how often humans must rediscover a failure class they already understood, by making that class recognizable and verifiable inside the engineering process.

A governance decision is only as trustworthy as the configuration identity, the runtime context, the execution boundary and the evidence attached to it. We now know, from a profile with two integers in a list, exactly where that trust can break.

Check your own machine

pipx upgrade sentience-governor
sentience profile validate
sentience profile resolve --agent-id YOUR_AGENT_ID
pip install --upgrade pydantic-ai-governor

An invalid file now names its first errors and where resolution degraded to.