Prompt Inspection & Bundle Versions

Prompt bundles

A bundle is a governance descriptor (its prompt key, template key, prompt version, contract version, release status and rollback version, among others) together with the user prompt layout its code draws. Its prompt version is the bundle version and names its manifest on the classpath: prompt-bundles/manifests/<promptKey>/<version>.json. A build runs exactly two bundles of cortex.security-decision: the build default (v19 sections) and the v20 bundle (dictionary blocks). Older manifests stay as records of earlier bundles; they cannot be run.

• Element states — PRESENT hashes the element content; ABSENT, UNVERSIONED and INCLUDED hash a fixed marker, so the state itself is part of the identity. The bundle hash covers every element. The v19 bundle, for example, marks B2 (no dictionary) and B5 (no minimum action rules) ABSENT.
• Checked on read — a manifest whose bundle hash does not match its elements, or two manifests of one name with different hashes, fail fast.
• Manifest fields — besides the elements a manifest names its golden inputs, parent version, vocabulary version and compatible vocabulary versions, its rollback targets and its declared model profiles.

Choosing the bundle of a request

One selector chooses the bundle each request runs, picked by contexa.security.tiered.prompt-bundle.selector. The first prompt of an event fixes the bundle on the event, and its later prompts (Layer2 or a second pass) reuse it. A bundle version this build does not run falls back to the build default, reported once per version.

Registry states

PromptBundleState is the one lifecycle vocabulary of the registry. Older stored values are read through it, so gates, queries and screens classify a release the same way. Only ACTIVE and CANARY are enforced at runtime.

The small label under each state is the older release status name. The transition API takes either name and reads it through PromptBundleState; its receipt and the screens name the state canonically.

Moving a release to ACTIVE deprecates the other ACTIVE releases of its scope, prompt key and template, so one release is active. A release moves to DEPRECATED only while another release of its scope is active (NO_OTHER_ACTIVE_RELEASE).

Release gate

Every transition goes through PromptGovernanceReleaseGateService. A transition that is not allowed from the current state is blocked with ILLEGAL_STATE_TRANSITION; otherwise the gate checks the evidence the target state needs. A blocked transition changes nothing and is recorded with its blockers; an applied one sets the registry status, records the approval and rollback state, and tells the runtime caches.

EVALUATION_NOT_FOR_BUNDLE_HASH means the evaluation on record was not made with this bundle's hash: the evidence of a bundle is the evaluation of that bundle.

POST /contexa/admin/api/prompt-quality/releases/transition
Content-Type: application/json

{
  "registryScope": "PLATFORM_GLOBAL",
  "promptKey": "cortex.security-decision",
  "promptVersion": "<bundle version>",
  "targetReleaseStatus": "EVAL_ONLY",
  "rationale": "Open the shadow run of the candidate bundle"
}

The response is the transition receipt: 200 when applied, 422 with the blockers when blocked. A blank registryScope means PLATFORM_GLOBAL.

Shadow runs and canaries

Shadow run

A shadow run tries a candidate bundle on real requests without letting it decide anything. It is read from the registry's SHADOW release whichever selector chooses the decision's bundle, so a host that runs its configured bundle can still try a candidate.

• Sampling — a request is in the sample by a stable hash of its event ID below contexa.security.tiered.prompt-bundle.shadow.sample-percent (0 to 100; 0, the default, runs none). The shadow runner exists only when this property is set.
• Daily cap — at most shadow.daily-call-cap runs (default 100) per UTC day, one model call each.
• Apart from the decision — the event is copied before the decision, analysed after it on its own executor, and sealed as the SHADOW stage. The copy never reaches the enforcement handler, so nothing is enforced or learned, and it adds nothing to the session context.
• Only runnable bundles — a SHADOW release of a version this build does not run is skipped.

Canary

With selector=registry, a CANARY release runs for the requests inside its share; the others run the latest ACTIVE release. The share is the registry row's canary_percent (0 to 100; no value means the whole scope), applied to a stable hash of the requester (the user, else the session, else the event). Optional canary_tenants and canary_resource_families narrow the canary to those tenants and to requests whose resource address starts with, or whose resource ID equals, a listed family.

Rollback

PromptBundleRollbackService switches the executing bundle of a scope at once to a rollback target its manifest declares. The source becomes DEPRECATED, the target ACTIVE, the runtime caches are told, and the execution is recorded with the conditions it was checked against. A refused rollback is recorded with its reasons and changes nothing.

POST /contexa/admin/api/prompt-quality/releases/rollback
Content-Type: application/json

{
  "registryScope": "PLATFORM_GLOBAL",
  "promptKey": "cortex.security-decision",
  "fromVersion": "<executing version>",
  "toVersion": "<rollback target>",
  "reason": "Decision quality dropped after the release",
  "emergency": false
}

The result carries an execution key (RBX-...), its status and the checked conditions: EXECUTED, or REFUSED with HTTP 422. An emergency stop ("emergency": true) runs the same way and ends AWAITING_APPROVAL; another administrator then approves it with POST .../rollback/{executionKey}/approve. The operator who ran it cannot approve it.

Prompt inspector

PromptInspector reads the user prompt the model got from a sealed evidence package and runs each check over it, counting what matched, what did not and what could not be checked. It points at problems for a person to review; it never changes a prompt, a decision or a rule. The automatic round runs on the scheduler's thread under one lock across instances, inspecting the oldest uninspected packages up to the batch size; no decision thread runs it.

• Layouts — the v20 checks run only on packages drawn as v20-dictionary-blocks; a v20 package drawn with another bundle's dictionary or render is counted as unverifiable for those checks.
• Verdicts — PASS (every item matched), FAIL (at least one mismatch), INCOMPLETE (no mismatch, some items unverifiable), NOT_APPLICABLE (not a dictionary prompt; only the layout-free checks ran), ERROR (the package could not be inspected).
• Findings — MISMATCH or UNVERIFIABLE; an unverifiable item never counts as passing.
• Versions — every result names the inspector version (prompt-inspector/N) and the dictionary version (v20@ and the head of the dictionary digest). A new inspector version inspects the packages again.

Sealed evidence sampling and retention

In an OSS installation (contexa.enterprise.enabled not true) with capture on (contexa.pqa.oss.sealed-evidence.capture-enabled, default true), decisions are sealed into evidence packages: the system and user prompts as sent, the canonical context and the execution metadata. Enterprise keeps sealed packages in its own schema, so the OSS inspector beans are not created while Enterprise is enabled.

• Window after a change — the first full-capture-after-change decisions (default 1000) after a start, or after the active bundle changes, are all sealed.
• Sample — after the window, sample-percent (default 1, 1 to 100) of the requests are sealed, by a stable hash of the correlation ID.
• Retention — a package is kept for retention.period (default 90 days); one whose latest inspection found nothing wrong is removed after retention.clean-period (default 30 days). A package whose inspection found something keeps the default retention.
• Inspection sample — contexa.prompt-inspection.sample-percent (default 100) of the sealed packages are inspected, by a stable hash of the package ID.

Console and API

Every screen and endpoint requires the ADMIN role.

Configuration

contexa:
  security:
    tiered:
      prompt-bundle:
        selector: registry
        shadow:
          sample-percent: 5
          daily-call-cap: 100
  prompt-inspection:
    batch-size: 50
    sample-percent: 100
  pqa:
    oss:
      sealed-evidence:
        sample-percent: 1
        full-capture-after-change: 1000

The minimum action rule and prompt budget settings are listed in AI Engine Configuration.

Related documentation